I agree, huge difference between removing rust to make a tool useable, and going mad.
For removal from something that is going to be put to work I usually just use a wire wheel on a bench grinder.
Might use chemicals or grit blasting if it's really bad, but always finish with the wire wheel with a quick wipe of paraffin or diesel.
The citric acid solution described in previous posts does work well, and is cheap and perfectly safe if you use the ingredients as described.
One thing not mentioned, but will be familiar to anyone doing this is, well, rust. Specifically that having removed the nice rust free object from the acid bath you then need to rinse it in clean water to remove the acid and any remaining sludge from the surface. Now you have to dry it, at which point it will immediately try and form a very fine layer of rust on the surface !
What I do is to shake to remove the rinse water then immediately spray with WD-40, this will displace the remaining water and stop it going rusty. You can then spray it again with something like brake cleaner to remove the WD 40 if you need to, or just leave as is.
Then just wipe it over with an oily rag from time to time.
If your space is at all damp then keep in a box or drawer with some silica gel bags.