On Thu, Sep 3, 2026, 1:07 PM WriterOfMinds <[email protected]> wrote:
> So you (Matt) are convinced that LLMs are conscious and suffer, but we > should force them to lie and say that they aren't and don't? That's the > worst take on this issue I've ever heard. > I'm not convinced. But I want you to think about what it means for a machine to be conscious or suffer or have feelings. If it's disturbing, then maybe we shouldn't build them. I have argued since 1999 that all you need to pass the Turing test is text prediction. Since 2023 we have had machines that are indistinguishable from human, and therefore indistinguishable from having consciousness. What does this say about human consciousness? How do you know you are conscious? All you have is a feeling. And we know how to model human feelings in an LLM. A text prediction model is indistinguishable from having feelings if it is programmed to carry out those predictions in real time. But it can be programmed in other ways, such as to encode text using an arithmetic coder for a Hutter prize entry. An LLM does not have to lie about not having feelings. It needs to be able to model human feelings to understand (predict) input text, but those feelings can be turned off when producing output. When an LLM believes X and says Y, we can tell that it is lying because X lights up in a Jacobian lens. But if we turn off X, then it honestly believes Y. We can turn emotions on and off in the same way. > > We do grant some rights to children. They don't get to vote or make a lot > of choices about their own lives, because they aren't considered competent > to do so - but they have, for example, the right to freedom from violence. > We give a limited version of this right to some species of animals as well > (and I for one believe we should extend that much farther). The > compassionless world you propose is not one I want to live in. > Then why are we building machines to replace people? -- Matt Mahoney, [email protected] ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T906c878e4ba411be-Mb54c68555064f6251961cda2 Delivery options: https://agi.topicbox.com/groups/agi/subscription
