Dave Schiller,
I don't believe I would work on the barrels at all. The fit of the forearm "iron" often causes the problem you describe. It may help if you remove the "iron". You should see a square cut in the very forward end that bears on the lug the forearm locks into. If you slightly polish the inside of this notch, the rear end of the iron won't bear as hard on the front of the receiver. Don't remove too much, it is as easy to make it too loose as it is to make it too tight.
Mike