Aesthetically it would be better to use a wooden shim to fill the gap. Stained to a matching colour it will be barely noticeable.