[email protected] writes: > On Sun, Mar 29, 2026 at 08:32:46AM +0000, Ihor Radchenko wrote: >> [email protected] writes: >> The problem with references is not new, and RAG-based systems have been >> developed to tackle it. Nowadays, LLMs are not bad citing resources, >> when asked (deep research). Normal chats should be treated with care though. > > Technically correct, yes, but the point I was making is meta-technical: > there are companies out there betting their farms (and those are pretty > BIG farms) on people adopting that stuff massively. So the stuff has > to suggest that "it knows what it is doing". And they are suceeding.
There’s a recent example about Schnapspralinen that made the rounds because it matched a prediction by a German Kinderbuchautor and Comedian. Someone created a Wikipedia page claiming that Schnapspralinen were made by Stängli. Wikipedia editors quickly flagged it and after 5 days of deletion discussion the article got deleted. Trolling attempt foiled. But Gemini still had it in its cache and happily said that multiple people created Schnapspralinen, including Stängli. And it provided the reference with a summary about Stängli. Ending with the assuring: "So Schnapspralinen were created by multiple different people". (this is the "it knows what it is doing" part) But there is no Stängli who created Schnapspralinen. And clicking on the Wikipedia page showed the deletion page. But who clicks there if the reference listing has a summary that supports the point? A day later Gemini had been fixed (I guess that there’s an anti-SEO-group at Google who fixes such stuff, when it starts making the rounds in the media), but now a week later, when I asked ChatGPT (I just checked), while Stängli is not mentioned, the summary remains changed: "Schnapspralinen were created by multiple different people". So while part of the error has been fixed, there are leftovers that survived the debunking of the troll attempt. In the worst location: in the concluding sentence "intended" to stay with you. Best wishes, Arne -- Unpolitisch sein heißt politisch sein, ohne es zu merken. https://www.draketo.de
signature.asc
Description: PGP signature
