I've never had any problem with damage to an oil finish by removing common adhesives left over from tape or stickers with mineral spirits. Once a linseed-based oil finish cures, mineral spirits aren't aggressive enough to do anything. Regardless, it's easy enough to touch-up with some 1200 grit w.o.d sandpaper and fresh b.l.o.

Acetone-based solvents like lacquer thinner are a whole different story.
JR

Be strong, be of good courage.
God bless America, long live the Republic.