On Wed, Jun 19, 2024 at 12:33 PM Jason Resch <[email protected]> wrote:
*> **Just the other day (on another list), I proposed that the problem > "hallucination" is not really a bug, but rather, it is what we have > designed LLMs to do (when we consider the training regime we subject them > to). We train these models to produce the most probable extrapolations of > text given some sample. Now consider if you were placed in a box and > rewarded or punished based on how accurately you guessed the next character > in a sequence.* > > *You are given the following sentence and asked to guess the next > character:* > *"Albert Einstein was born on March, "* > > *True, you could break the fourth wall and protest "But I don't know! Let > me out of here!"* > > *But that would only lead to your certain punishment. Or: you could take a > guess, there's a decent chance the first digit is a 1 or 2. You might guess > one of those and have at least a 1/3 chance of getting it right.* > *This is how we have trained the current crop of LLMs. We don't reward > them for telling us they don't know, we reward them for having the highest > accuracy possible in making educated guesses.* > Damn, I wish I'd said that! Very clever. John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis> mze -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv0%2BMkb0y%2B1fY-AryeOC6C-FwuR%2B654Ua_EjMb_%3D6CGNCQ%40mail.gmail.com.
