Rule of thumb, use a standard clearance for the bigger bearing at the pulley end and a C3 for the (sometimes smaller) bearing at the tail end. C3 isn't just about extra clearance for bearings that run at high speed, get hot and expand. It provides a little tolerance of imperfect axial alignment of the two bearing housings. You are going to clock and shim the two side castings that support the cutter block as precisely as you possibly can but they are separate parts bolted together not one casting bored through in a single operation so they will never be perfectly aligned. Motors likewise rely on the machining accuracy of 3 separate pieces.
Bearings on the sedgwicks - motor or block - don't run hot.