Kirby compiled a career record of 182 wins and five defeats. He began boxing at the age of 12 and was the first boxer to fight out of St. Andrews Parish in Charleston. Kirby fought and won tournaments in both Carolinas, Virginia, Florida, and Georgia. He won the Carolinas Golden Gloves three times and represented the Carolinas team at Sunnyside Gardens, New York in 1961. Fighting as a lightweight, he lost the national finals of his weight class. Kirby won knockout awards in six different tournaments. He garnered outstanding boxer trophies in seven amateur tournaments. Kirby was also a winner in many service bouts during his stint in the U.S. Navy.
Harold worked for the Atlantic Coast Line Railroad for 25 years. He retired from the Railroad in 1987 due to an injury. He now sells real estate. He and his wife have two children.