Inspector
Nyuck, Nyuck, Nyuck!
A WAG but can it be that the surface oil dries quickly and then as the oil deeper in the wood, curing more slowly, releases the solvents or lighter oil compounds to migrate out, softening the surface as it dries? The only other thing I can think of is the resins present in the wood you use may be interfering with the finish. Maybe try a small piece of some other woods to see if they behave any differently. Maple for example. Does another oil finish behave the same?
Pete
Pete