Yes, the sample rate drops as the Delay Multiplier increases, but in a slightly different way to how you're thinking about it because the Delay knob doesn't change the sample rate. 

You'll also be helped by making a distinction between the delay time you get (say, 1 second) and the delay knob setting.  If you are in 1x mode and have your Rose set to 1 second delay time, activating 2x mode would make the delay time 2 seconds by cutting the sample rate in half (slightly degrading the sound).  If you switched to 5x delay multiply, you would get a 5-second delay time with a fair amount of degradation in the audio quality.