I find that with my setup, the music sounds better to my ears when I use upsampling. My setup is probably pretty average. I have high-quality headphones and when listening through them I find the sound is fine without any upsampling.
My DAC (CA DacMagic 200M) can play up to DSD 512, but in the upsampling options, the 512 option is greyed out. This is the case with either r8brain or sox. Just wondering why the 512 option is greyed out.