To stop Windows from changing your default audio device, you should troubleshoot your audio device first. Run Windows built-in Playing Audio troubleshooter and see if it fixes the issue.

Haven't messed with Windows 11, but in 10 you can disable devices you don't want to use. I do this to my monitors as they are never going to produce sound. See if this link helps, the setting are ...