Yes, you would need to knock off the end to gain access. I see they are blind mortises so I would make a matching template and use a top bearing mounted cutter to clean it all out. If you make an accurate template the mortise will not enlarge. I presume you will use the existing slats to go back...