He was born in Liberty County, the son of J.D. and Bessie Hines Stafford. His siblings were Leta, Ethel, Hines, J.D. Jr., Ruth, Teresa, Robert Pryor, and Cathleen Stafford. His mother was a sister of Beulah Hines Fraser McCall. As a young man he played professional baseball for 12 years in Tennessee. He married Willie Mae Dawson, and their only child was Harold N. Stafford Jr.
He returned to Hinesville after his baseball career and became a businessman and member of the Hinesville City Council. He was chairman of the city improvement committee, and was elected Mayor of Hinesville in 1943. It was during his administration that the first streets of Hinesville were paved during World War II. Stafford Memorial Park in Hinesville was named in his honor.
From "Sweet Land of Liberty, A History of Liberty County, Georgia" by Robert Long Groover; Appendix Number 39, Page(s) 232; Used by the permission of the Liberty County Commissioners Office