Absolutely right, as you have mentioned you'll end up with a pile of 3 - 4 grade (mainly 4) which would then have to be downgraded in price.
I was with one of our order pickers the other day and he was getting out some 25 x 100mm 5th Redwood PAR for an order, out of a row of 10 boards there...