I wonder if a claw hammer face is marginally harder than a club/lump hammer, I've seen mushroomed lump hammers and they're specifically designed for hitting something that's hard likes chisels and knocking bricks off so ideally, you want the hammer to absorb the impact and marr rather than chipping pieces of the face off because it's too hard. I've never seen a mushroomed claw hammer (Well, I did once but it was a very poor quality Stanley that had a hard life) and they're designed for driving soft fixings and knocking wood apart/together so in theory they can be made to a higher hardness because there's less likelihood of chipping the face in regular usage. Ergo, I think your brickie mate was worried you were either going to chip the face of the hammer, or chip off a piece of the chisel which in the bad old days could puncture someone quite easily since chips can travel very quickly, I think there are old case studies of people's femoral arteries being punctured by metal chips being shot off hammers and chisels.
That probably only applies to older hammers and chisels though, modern ones seem to have a beating and not chip at all due to better metal composition, or lesser hardening.