Jim, a good drill press with interchangeable pulleys speeds will do a nice job of getting rid of the 90% before your start with the little chisels. New, sharp, center cutting end mills in several sizes, at the highest speed the pulleys will go, work for me. Also, a decent cross slide drill press vice is so imperative to do the hogging out like this. I did 95+% of the inletting on this Schillinger lock with mine, then finished it up with chisels.
Shoot me a p.m. if you want to talk about it one evening.