Joe Diffie was born into a musical family in Tulsa, Oklahoma, in 1958. His first musical performance came at age 14, when he performed in his Aunt Dawn Anita's country music band. Diffie's father, Joe R. , played guitar and banjo, and his mother sang. Following in his mother's footsteps, Diffie began to sing at an early age, often listening to the albums in his father's record collection. Diffie has said that his "Mom and Dad claimed that [he] could sing harmony when [he] was three years old. " His family moved to San Antonio, Texas, while he was in the first grade, and subsequently to Washington, where he attended fourth and fifth grades. Later, he moved to Wisconsin for the years he was in sixth grade through his second year of high school, and back to Oklahoma, where he attended high school in Velma. In his last two years in high school, Diffie played football, baseball, and golf in addition to running track; in his senior year, he was recognized as Best All-Around Male Athlete.