Judge Pittman was born in Enterprise, Alabama. His ancestors had been among the white pioneers who settled Coffee County, and he picked cotton alongside both whites and blacks during the Great Depression. While at the University of Alabama, Pittman joined the United States Army Reserve, serving from 1938 to 1942. He received a Bachelor of Science degree in 1939 and a Bachelor of Laws from the University of Alabama School of Law in 1940.

Judge Pittman entered federal service as a special agent for the Federal Bureau of Investigation from 1940 to 1944. He served in the U.S. Naval Reserve and was called to duty as a lieutenant near the end of World War II, serving from 1944 to 1946. Upon discharge, he entered private practice in Gadsden, Alabama from 1946 to 1951, practicing as the law firm of Pittman & Miller. Pittman became a Circuit Judge of the 16th Judicial Circuit Court of Alabama from 1951 to 1953, and the circuit's Presiding Judge from 1953 to 1966. He also taught as a lecturer at the University of Alabama Center at Gadsden from 1948 to 1966.

President Lyndon B. Johnson nominated Judge Pittman on June 13, 1966, to a new joint seat on the United States District Court for the Middle District of Alabama and the U.S. District Court for the Southern District of Alabama. For several years he traveled in a circuit of federal courthouses in Montgomery, Mobile, Selma, Dothan, and Opelika. His service on the Middle District terminated in 1970, when he was reassigned to only the Southern District. Pittman served as Chief Judge of the Southern District from 1971 to 1981, and he became the first federal judge in Mobile to hire black and female law clerks. Judge Pittman assumed senior status on July 15, 1981, but continued with a reduced docket for decades. As of 2020, Pittman is the last judge appointed by a Democratic president to the Southern District of Alabama.

During the Civil Rights Era, Judge Pittman was assigned two complex cases brought by the NAACP involving at-large offices in Mobile County, Alabama. At the heart of both cases was the at-large election system begun following the 1911 revision of Alabama's constitution; no African Americans had thereafter won any county-wide office.

The Bolden case went to trial on July 12, 1976; and on October 21, 1976, Judge Pittman issued a decision for the plaintiffs which led to considerable controversy. The "Constitutional Crisis Committee" asked for Pittman's impeachment, and Mobile commissioner and then-mayor Lambert C. Mims offered to sign the impeachment petition, but the city attorney advised against it. While 5th U.S. Circuit Court of Appeals evaluated his decision, Judge Pittman postponed the 1977 city election, allowing the three committeemen elected in 1973 to remain in office, and the appellate court later affirmed Judge Pittman's decision.

Ultimately, the U.S. Supreme Court granted certiorari and in Mobile v. Bolden reversed the appellate judgment and vacated Judge Pittman's decision. To comply, Judge Pittman held a second hearing beginning May 1981, although this time elections were not postponed. Meanwhile a "smoking gun" letter was discovered and admitted into evidence—written by Mobile lawyer and Congressman Frederick G. Bromberg to the Alabama legislature in 1909, it clearly indicated the purpose of the at-large system was to prevent blacks from holding office. Both Commissioners Mims and Greenough promised not to appeal the second Bolden decision if the city lost, although Commissioner Robert Doyle avoided the issue. Ultimately, Doyle won re-election immediately, and both Mims and Greenough won re-election in runoffs. However, before the second trial Michael Donald was found beaten, strangled and with his throat slashed. Four Ku Klux Klan members were convicted for the crime in state court, and his mother won a $7 million settlement that effectively ended the Klan's operations in Alabama.

Judge Pittman's second Bolden decision, issued on April 15, 1982 also favored the plaintiffs. On January 31, 1983, rather than appeal, all parties agreed to a settlement whereby the next election for city office would be based on districts rather than at-large. The Alabama legislature passed appropriate legislation, introduced by state representative Mary Zoghby, and 72% of state voters on May 15, 1985 approved switching to a mayor-council form of government. Three African Americans were elected among the 7 new districts, the first blacks to serve in Mobile's government since Reconstruction.

Lila G. Brown's case alleging that at-large elections of members of the Mobile County school board diluted the voting strength of African Americans in violation of the Voting Rights Act and the Fourteenth and Fifteenth Amendments began in 1976. Thus it came more than a decade since the Birdie Mae Davis school desegregation case had begun, and after the November 1974 elections replaced long-time school board members Charles McNeil and William Crane with young local attorney Dan Alexander (who would become the school board's dominant force for decades, but after declining to run for election after resolution of the Brown legal case, would be convicted for extorting kickbacks from local architects as Mobile's schools finally began addressing infrastructure deficits in 1977) and Ruth Drago (a retired teacher and former president of the Mobile County Education Association and Alabama Education Association). In the 1976 election, Hiram Bosarge (a retired army veteran who would decades later be acquitted despite Alexander's conviction), replaced veteran school commissioner Robert Williams.

Judge Pittman had conducted a bench trial, and (before the November 1978 elections) had ordered five single-member school board districts, with Alexander made the non-voting board president, and the Fifth Circuit had affirmed. Retired air force officer Norman Cox and local dentist and NAACP president Robert Gilliard had been elected as the school board's first black members, each from a majority-black district. However, Alexander was dissatisfied by losing his vote, and the case was ultimately appealed to the U.S. Supreme Court and held pending the decision in Mobile v. Bolden, then vacated in light of that decision. When Alexander tried to strip votes from Gilliard and Cox, Judge PIttman affirmed their offices, as well as single-member districts for the upcoming September election, and again denied Alexander a vote in school board meetings. Alexander called for Judge Pittman's removal, and rhetoric heated in 1980. The Fifth Circuit allowed Alexander to remain on the board pending the new trial scheduled for March 1981. In January 1982 the school board approved a plan to end the Birdie Mae Davis litigation handled by Judge Hand, which plan included a committee to design a plan to create a unified school system and appointment of two sociologists as a professional observation team.

However, Judge Pittman's orders from the Bolden and Brown cases proved controversial and appeared to require a special school board election in 1983. When the new 11th Circuit affirmed his order and the Supreme Court denied the school board's' appeal, Alexander declined to seek election, as did long-time board member Berger, so their seats were won by Republican Howard (Chipper) Mathis III and Judy McCain (who soon was elected the board's president and promised like Mathis to direct money and energy toward school rooms rather than courtrooms).

Judge Pittman remained married to his first wife for 56 years, even becoming her main caregiver when she was stricken with Alzheimer's disease. They raised a son and a daughter. Several years after her death, he remarried, at age 86. Even while a federal judge, Pittman volunteered for a meals-on-wheels program, delivering meals to the elderly. He also was extremely scrupulous about court finances, going to a nearby pay phone rather than make personal phone calls on the line to his judicial chambers. He was also a life trustee of Samford University.

Judge Pittman fully retired in March 2006, aged 89. At the year's end, the Mobile and Baldwin County bar associations awarded him the 2007 Howell Heflin award for bringing honor to the legal profession. Although once a pariah in Mobile such that a cross was burned on his lawn and he stopped going to church for six months lest his presence lead to an attack on the congregation, by his death Judge had become much admired, and Mobile's city council passed a resolution honoring him.

Several of his law clerks became judges, including U.S. Magistrate Judge Sonja Bivens, and Alabama Circuit Court Judge Craig Sorrell Pittman.