This whole subject feels like a can of worms! I don’t think the role of cognitive biases is factored in enough.
Two major forms can influence this - confirmation bias, where we see/hear what we expect to see/hear, and cognitive dissonance, where we twist reality into our preconceptions/misconceptions. (This is very informal definition, BTW)
We all do it, and what is more, we don’t realise that we are doing it. Scott Adams’ book ‘Loserthink’ is one of the best informal discussions of this subject I have read.
When I worked as a professional photographer I saw it a lot - a person buys a mega-expensive lens and claims it is an order of magnitude better than his previous consumer level lens. Lab results show the pro lens may be a tiny bit better optically, but you’d hardly notice without examining photos under a microscope or massively enlarged on screen. But he’s paid over £1,000 for it, so it has to be better… (Pro lenses usually cost more because they are weatherproof, are built like tanks, and so on. Most lenses on the market, including consumer ones, are optically very, very good.)
I suspect something like this might occur in audio as well. For example, I have a CD that I like. I buy it again on Qobuz at the highest res and expect it to sound better. And it does! But is that because it sounds better or am I just listening to it more carefully, noticing things I haven’t noticed before? The same could be said about oversampling.
Without blind A/B testing there really is no way to tell through hearing alone. And @Jud has said A/B testing won’t work for 99% of us, and I don’t doubt he’s right… But that makes this untestable, unfalsifiable, and 100% subjective.
I could buy an Analog to Digital converter and then analyse the audio output from my DACs using the software @Jud mentioned. Assuming it could log its output to a file, I certainly have the programming skills to then analyse and identify the differences between oversampling, no oversampling, and fiddling with the oversampling parameters. But it seems a lot of effort to go to, for no particular reward, as it won’t make the music sound any better.