Yes, they are that hard. Typically the metal is inlet before final metal finish. Any small marks in the metal from the inletting process are given a final polish before bluing.

Metal finish is the last operation when building/stocking a gun. Blued parts can be inlet without damage to the finish but extreme care must be taken.