Is it the first couple of inches and the last couple of inches exclusively or is it a gradual curve along the whole piece which the bevel deepens towards the end?
If it's the first and last couple inches it may be that your workpiece is "dropping in" on the start and end of the cut, which means it's falling into the cavity with the router bit before it clears the space between the fences.
If it's a gradual curve, it's either the workpiece is not straight, the fence is not straight, or the table is not flat. Since you ruled out the last two did you check the workpiece? For this, I think a bearing guided cutter is the solution.