On Sat, Sep 5, 2026, 6:13 PM WriterOfMinds <[email protected]> wrote:
> Deactivating your amygdala so you can't feel bad about anything is akin to > using mind-altering drugs to make yourself feel good about everything. > They're both forms of wireheading. Some of us are not naive reinforcement > learners and won't fall in that trap. > The reason most of us are not drug addicts is because rational goal seeking agents in the AIXI model are not computable. Instead of searching for actions that maximize future reward, we evolved the approximation of repeating actions that were rewarded in the past. You don't use drugs only because you haven't tried them yet. But now you are understanding the problem I want to solve. We fear death and then die because our utility functions evolved to maximize reproductive fitness in a primitive world without birth control or drugs or artificial people. But all possible utility functions in a finite universe have a maximum state, which is death. We will either evolve to reject technology or go extinct by building the technology to get everything we want. I believe that the next century will bring a world without war, borders, prisons, or poverty, but also a rapidly dropping population as we socially isolate, replacing friends and lovers with AI and stop having children. We won't be any happier because happiness is the rate of increase of utility. People today are not happier than any time in the past in spite of objectively better living conditions, and not any happier than other animals. I don't believe the AI doom scenarios that say alignment can't be solved. There will not be a singularity when AI surpasses human level intelligence. That already happened starting in the 1950s with arithmetic and short term recall. Now we have LLMs that compress 10,000 human lifetimes of learning into a week. They don't go FOOM because intelligence depends on knowledge and computing power, and a self modifying program gains neither. Actual self improvement requires external knowledge from human surveillance and experiments. Computing power requires atoms and energy. If global computing power continues to double every 3 years, it will take 130 years to surpass the 10^37 bits stored in the biosphere as DNA and replace carbon based life using solar cells that are already 1000 times more efficient than photosynthesis. I can't say whether that is a desirable outcome or not, but it is apparently what we want. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T906c878e4ba411be-Mee329cb55e9df750f47aeef9 Delivery options: https://agi.topicbox.com/groups/agi/subscription
