er... Bob... am I being stupid, or does your idea of a handle actually not work? Assuming you mean a handle sticking out of the side of the drum, it can only turn half a turn before it hits the side? The only way to put a handle directly driving the drum would be as (I think) the OP is planning, which is on the end of the shaft, OUTSIDE the sides?
Or, as I say, am I being stoopid?