If you had read my post about using a nail set or a pin punch to fit into the 4-40 hole, you would have read that it is only there to act as a fulcrum point for the notched "V" tool to push it over into the fit.
That spring is made for a 100lbs. pressure and there are not many things made .093 (3/32") that can handle that force without bending or breaking.

The workers to did this at Hunter Arms did it day in and day out and they knew how to do it. There are pictures of them doing so. Since I don't do this everyday I used a method that the late Russ Ruppel told me, and that is with the small hole drilled in the wood.
As to making the wood weak, no such thing in that area.

David