Scruggs fought to three Golden Gloves championships as a Welterweight in the early 1950s. He earned titles at Charleston, Raleigh and Durham. He also earned Most Valuable Boxer at the Charlotte YMCA in 1950. Scruggs also earned five runner-up honors during five tournaments. In 76 amateur fights, he posted a record of 62-12-2. His pro career was a perfect 22-0. Scruggs is one of the few boxers who can lay claim to never being knocked down during a bout.