The other reason it happens is not having the butt placed firmly against your shoulder. If you have the butt planted well, your hand won't be doing much anyway. Like most of shooting, it's all boils down to operator error.