Holland & Holland no longer have a presence in NYC (bad info posted earlier in this thread - imagine that). They moved their USA gunroom to Dallas, TX.

Beretta Gallery is still open though and would make for a great stop. See if you can take a look of Hemingway's Beretta 12 gauge O/U that is there.

I would second the American Museum of Natural History. There are also some gun collections to be found there.

As for restaurants there are so many great restaurants, some already mentioned, I would not know where to start.