That is correct. I registered when VAT was introduced - I had been registered for Purchase Tax previously though.
There are advantages if your customer base are also registered. I'm still registered 51 years later and most quarters I now get a refund since I'm semi-retired so my turnover is often less than £100. Q2 this year I'll get a £1.36 refund (T/O £44) - - - Q1 I paid them £320.81. (T/O £3k)
I don't begrudge the few hours a quarter that I spend dealing with my accounts and the VAT Return takes me less than 5 minutes. Anyone who considers a businessman to be an "Unpaid Tax Collector" has no idea what it entails.