On 2/2/26 4:04 PM, Rich Pieri wrote:
On Mon, 2 Feb 2026 18:16:53 -0500
Steve Litt <[email protected]> wrote:
I think this is a marketing ploy, not a result of Large Language
Models. By telling the human how wonderful he or she is, they subtly
influence the human to use them more and more. I hear plenty of
This. As I wrote last week, these chatbots are designed to drive what
the operators call engagement and I call addictive behavior. It's a
deliberate design decision, not an intrinsic "feature" of LLM chatbot
tech.
The main LLM training is enormous and is done on everything they can
possibly find (the entire internet, every book and newspaper they can
get a hold of, etc), this creates the generative part of an LLM, and it
is what gives LLMs their "memorized the manual" knowledge, and their
generic style of writing.
There is a smaller effort of a secondary, reinforcement training, it
teaches what kinds of output are desired (high score) or not desired
(low score). Unlike the original training, reinforcement training
requires some external authority to score LLM output, and tell the LLM
whether humans will like it or not. (I have heard that a separate model
that has been trained on samples that real humans have scored, is then
used to teach the LLM in this reinforcement stage. At least that is one
possible approach. The details here are very proprietary.)
It is this reinforcement training that determines how flattering and how
engaging the final LLM will be. So yes, those aspects are not a feature
of the underlying generative model.
-kb
_______________________________________________
Discuss mailing list
[email protected]
https://lists.blu.org/mailman/listinfo/discuss
