Use AI-generated answers at your peril. As mentioned above, AI is more than happy to regurgitate widely believed falsehoods as ‘fact.’ Think of the times you’ve seen where people have written about guns with great certainty, but they were wrong. AI can’t distinguish between fact and fiction, especially if the fiction is widespread.
And it gets worse. If AI doesn’t have the facts you’re looking for, it makes them up. Newspapers are full of stories about lawyers presenting cases in court with AI-made-up citations; these are just the instances we hear about. Keeping the discussion on guns, I recently had a troubling experience when, after much prompting to use AI for research, I gave it a good workout.
I’m currently researching the Birmingham firm of J&W Tolley, for my book and an upcoming article. I wanted to know how far back the Tolley firm started making breech-loading game guns. Records haven’t survived, so a review of every known morsel of information was necessary. In an online, private, source of obscure gunmaker data I’ve grown to trust, was the following statement: “An 1862 advertisement stated ‘Gun rifle and pistol manufacturers, Contractor for Military Arms, Breech Loading Guns, Pistols, also all guns suitable for the African markets’.” This was the earliest date I could trace that mentioned breech-loaders, so it was a good starting point. I have also learned that every fact and statement about historical gunmaking needs to be checked and double-checked to avoid repeating errors, including published ones. A straightforward Google search did not turn up the 1862 advertisement or any Tolley advertisement along the same lines. Nor did a search using several other popular search engines. I subscribe to the British Newspaper Archive, and no such Tolley advertisement could be found in their voluminous database.
I then turned to AI, using the latest version of ChatGPT. I started a back-and-forth conversation with the language model, looking to identify the source of the advertisement. It started by quoting back to me my own writings on Tolley and Internet board posts influenced by the original online quote. By pushing harder for specifics, ChatGPT then informed me of the source of the Tolley advertisement, using those words:
“The 1862 advertisement for J. W. Tolley (James William Tolley) appeared in the South Australian Register, an Adelaide-based newspaper (National Library of Australia – NLA). In the November 20, 1862 edition, the firm was listed as “Gun, rifle, and pistol manufacturers” and explicitly described itself as: “Contractor for Military Arms, Breech Loading Guns, Pistols, also all guns suitable for the African markets.” This publication reflects the firm's early business model, which relied on exporting large quantities of “trade guns” to international markets, including Africa and Australia, before they became famous for their high-end “big bore” sporting rifles.”
Excellent, I thought; I couldn’t have asked for anything better (though a red flag appeared when it got the firm's name wrong). An Australian paper could explain why other databases didn't catch it. I could have stopped there, but I didn’t, as I double-check everything. So I was able to track down a scanned copy of that newspaper for that date, from the National Library of Australia. The Tolley advert wasn’t there. I checked issues from several days before and after that date, without success. So I went back to ChatGPT and challenged the model with this result. It basically said: “Correct, there is no such advertisement. The original published statement may have been in error,” without acknowledging that moments earlier it had presented its ‘findings’ as fact. This absurdity, of giving factual statements, then after challenging them, being informed that the earlier ‘facts’ were indeed wrong, was repeated several times. This was my first experience with large language models being biased towards pleasing you rather than being accurate/objective. I wanted an answer, pressed hard for one, so it gave me what I wanted, but it totally made it up.
The Wikipedia entry for AI hallucinations explains: “In the field of artificial intelligence (AI), a hallucination or artificial hallucination (also called bullshitting, confabulation, or delusion) is a response generated by AI that contains false or misleading information presented as fact.” Yeah. “For example, a chatbot powered by large language models (LLMs), like ChatGPT, may embed plausible-sounding random falsehoods within its generated content.” No kidding. I could have been satisfied with the first ChatGPT response, and written about that data point online. This would have been scraped up by bots, and further solidified the misinformation with a new reference. Others, or chatbots, might have quoted me as a reference, and there you have it: false data as fact, stronger with every repetition. As AI becomes further and further embedded in Google searches and academic research, how long before we can’t trust any ‘fact’ out there? It was an interesting and frightening learning experience.
Case in point, I just noticed a recent YouTube video about an SKB over/under made in...1854? I guess AI wrote that script...
As script writers cut corners to speed up money-generating online output, things are going to get much worse. At least with non-AI search engines, you could trace back to sources more easily. With AI, not so much. Welcome to the Misinformation Age.
