And some people go to the other extreme, wishing to add weight to the butt, which can cause problems as well.
I bought a G Grade Lefever a few years ago on Gunbroker. The auction photos showed a clean longitudinal crack in the butt stock, and it also appeared that the buttplate was warped outward. The gun sold pretty cheap because of the obvious defects. When I received it, I pulled the buttplate and found a 7/8" hole about 3 1/2" deep that was filled with lead shot which was held in place with a small wad of tightly packed cloth. Over time, the lead shot oxidized heavily causing it to swell, which split the stock and forced the buttplate outward. I dug out the shot and glued the clean split closed with Titebond II wood glue, and then filled the hole with a hardwood dowel also glued in place with epoxy. The warped buttplate was easily bent back into shape after heating it with boiling water. I was lucky that there was no missing wood and the repair was imperceptible.
And I also feel that guys who think a couple ounces of imbalance are the reason for poor shooting are probably fooling themselves.