Born in Flint, Michigan, Morgan presently lives in Wilmington, N.C. where he is the director of the Boxing and Physical Fitness Center for the City of Wilmington. His wife, Yvonne is the current president of the North Carolina Boxing Association. Morgan began boxing in 1947. He won the Michigan AAU Championship in 1952 and 1953. He served in the U.S. Army and was a member of the 1955 Pan American Boxing Team. Morgan helped train the U.S. Army team for eight years and was head coach from 1972-1974. He retired from the U.S. Army in 1947 and was recruited by Wilmington to begin a boxing program.
He served as a president of the North Carolina Boxing Association for four years, he was chairman of the Junior Olympics from 1975-1991 and again from 1995 until 2000. He was team manager for the U.S. Olympic Festival in 1987 and the USA vs. USSR in 1981. He served as head coach for the USA vs. England matches in 1987 and the USA female team vs. Canada in 2000.
Morgan has spent most of his life boxing or coaching and managing other boxers.