I am going to weigh in just because of recent events. A friend recovering from back injurie and has been shooting .410 instead of his usual 12 on clays and fall off is not much. I believe he could shoot .410 on doves and quail and not miss many more than with his 12.
On the flip side I just returned from Scotland and shot driven pheasant and partridge. Out of 17 different drives only 1 would have been ok for such small gauge work. We had high winds and steep terrain. This was my first such outing but I was shooting with very experienced guns that all felt some of our drives were the most difficult they had ever seen. Provided the keeper set you up in a good setting and the weather cooperated this is possible but I would say for experts only.