There is a good chance this is route cause as bearing wear could allow slight spindle angle or racking, I would also check the collet first for scoring or burrs, this would prevent cutting tool sitting square.
If you go for the bearing replacement, ensure you go for a good set rated for around 20,000 rpm, I would go to an engineering bearing supplier, like SKF, to get recommendations, ebay and amazon et al are rife with cheap bearing prices, none of which i suspect would be up to the job.
Also the high quality bearing from an engineering bearing specialist is likely less expensive than a Trend one.