I seriously doubt the shape of the case had much to do with your failures with smokeless. Much more likely your powder/primer combo were simply not compatible. Most of these cases are built for pistol primers & the ordinary pistol primer being intended to ignite a small charge of powder in a smaller container are simply not as hot as an average shotshell primer.
Alcan used to offer an all brass shell designed for the old Remington #57 size shotshell primer & they could be used even for magnum loads of smokeless. Their regular brass hulls which used a Berdan primer or the domestic Rem/Win brass hulls which used pistol primers were ordinarily confined to using black, bulk smokeless or a few fast, easily ignited smokeless powders. Red Dot usually worked OK but nothing much slower.