Elmo Williams was born James Elmo Williams in Lone Wolf, Oklahoma. Orphaned at 16, he attended schools in Oklahoma and New Mexico before moving to Los Angeles. In 1933 he struck up a relationship with film editor 'Merrill G. White' . In 1970 Williams was appointed Vice President in charge of Worldwide Production at Fox, a job he left in 1973 to go into independent production.
At the time of his death, Williams was the oldest living Oscar winner.