Only getting speaker audio when headset connected

Problem just started today everything was working fine last night. This morning I turned on my xbox and I was getting no sound from the speakers. Checked with a different tv input everything sounded normal so it's not the TV. Tried restarting xbox and no luck. Finally tried plugging my headphones into the controller to see if xbox was outputting anything and as soon as I plugged them in I started getting speaker audio in addition to through the headphones. Unplugged headphones and lost audio from the speakers. Tried it a couple times and same result each time. Anyone got any ideas that could help me fix this?