It’ll be fine, I’ve used a jacobs chuck in my lathe for sanding and polishing a lot, probably more than actually drilling. Never had the Chuck work loose off the arbor. A while ago, i swapped the Chuck off my cheap pillar drill and it was quite difficult to remove it off the arbor.
By the way...