If they are ordinary drill bits (designed for metal) they won't produce a clean hole in timber - better to use 'lip and spur' drill bits for wood - Rob
Are you using a sharp wood drill - lip and spur? If you are drilling right through the wood then make sure you have a backing piece held tight up to the exit hole area.
If it quite a crucial hole that has to be very neat, you could try going through fist with a small drill bit, and then going from both sides with the correst side.
Lip & spur are usually best.
If the ones you've done are to be keept a small countersink bit may tidy them up, but will enlarge the opening which you may find unaccetable.