A lot of good advice above. Unless it's something REALLY special (most unusual in my case!) I do the following:
Go skip (& similar) diving - e.g. 1: you can often see a bit of electrical gear or a dead PC/laptop which will yield sheet metal, often very accurately punched and folded; e.g. 2: old printers, photocopiers, etc (beautifully ground hardened steel rod (I think often it's silver steel), etc, etc. Just keep your eyes open.
Local small metal working business/es - e.g. my local jobbing welding/fabrication shop (for cheap offcuts of rod, angle, plate, etc);
Local metal stockholder (ditto the comment posted above about selling offcuts of just about anything at "sensible" prices);
Keep an eye out for discarded electrical plugs & sockets, etc - especially the "large" brass pins on UK items (compared to us foreigners) are a good source of small bits of brass - even better if you ever see one around are the UK old 15 Amp round pin plugs;
Ask your family and friends to keep their eyes open too;
Go to WH Smith for latest copy of "Model "Engineer", or "Engineering in Miniature", or "Model Engineers' Workshop" mags. Study the small ads (mainly on the back pages) for a LOCAL model engineering specialist shop - as someone posted above, unless you're just buying a 6 inch length of thin ali rod, postage/shipping costs are far too high to go far afield for most metal stuff.
Similarly keep whatever mag (above) you buy and next time you take the good lady shopping somewhere, or visit Aunt Gladys, or w.h.y; see if there's a m.e. specialist on the way/nearby.
It all adds to the fun of owning a lathe and making/repairing stuff!