Larry,
I have had the exact same experience with 1 1/4oz of 4s through an IC choke. Makes you wonder where they all went. Put a single #4 pellet in center mass of a bird and it likely doesn't matter where the rest of them went.
I'm not one of them, but, I've hunted with guys who were very reliable at knocking down birds at 40+ yards. There were a lot of 12 gauges with modified chokes and store bought 1 1/4 oz loads of 5s and 6s along with those guys.
I like 5s on wild MN pheasants. I have a ton of old 1 1/4oz 12 gauge loads in 4 that I'd happily trade to someone for 5s, or even 6s.

Best,
Ted