I use YouTube Music and this (usually) isn’t a problem if you have premium. The only times I’ll get the music video audio instead of the studio track is if I’ve liked the music video on YouTube. In that instance it’ll sometimes show up in my automatically generated Playlist. For me, it’s worth it for the huge music library that YouTube Music has.
I just did a bit of research and it looks like if you have a free account it will almost always play the Youtube video instead of the album version (possibly as an attempted foil for ad-blockers?).