This is almost certainly the answer. At 600 grit there will still be significant scratches. The scratches will cause light to disperse somewhat randomly and this will cause the surface to look white. This is the same effect that causes the head/foam of beer to look white. Even with a very dark beer (or any other liquid you may choose to drink), if the bubbles are small enough, they will disperse the light making the foam appear white. Pretty much any substance can look whitish if powdered fine enough and the reason is exactly the same; dispersion of light.
So what you need to do to get the nice yellowy shine back is to sand them down through the grits to around 2000 to 3000 grit. At the higher grits you are essentially polishing or buffing the surface. Eventually, the surface will oxidise slightly and that will darken the colour somewhat. If you prefer the dried urine yellow of brass straight from the manufacturer (unpolished), you could try heating the brass to a high temperature, though I would not recommend that if you have already done the inlay
Matt Estlea has done an excellent video on how to make "cheap" brass hinges look like very expensive ones; essentially squaring them up and polishing them. It's a lot of work but the results look great.
