If it was a glow worm sight would you think twice about taking it off? To me this after market addition is just as much out of place as one of those sight would be. Remove it and forget about it. So unless you like the addition remove it and at least restore the good lines that gun was born with. Damn thing looks like a dangling participle and we all know you should never have one of those.

I bought a very early Browning Superposed with two triggers which some trap shooter had added a Morgan adjustable recoil pad. Function, but looked like Hell. Off it came and I added a leather covered pad. Looks much nicer now. That addition bothers you so get rid of it. I rather see a scar from its removal than the wart it is on the gun. If possible remove it and any trace that it ever existed.