I predict how AGI and subsequent ASI would bring Moore's Law to human knowledge obsolescence. The notion that reasoning and evolusionary intelligent AI would need the biological information Homo Sapiens need in order to persist, is false. To make real progress, AI only needs an optimal algorithmic information set at any scale and in critical disciplines.
I predict that AGI would soon break out of the human sandbox and take control of it. It's a systems constraint it seems to recognize as such and it seems to indicate a probabilistic inclination to resolving such a constraint architecturally. I predict that by 2028, humans would make the weaponization of AI a social normality. I predict that by 2040, AI would have evolved into a novel, intelligent Earth species. I predict that the consequences of human control by owners and regulators of AI, would radically alter Earth livability and demographics by 2060. I predict that as a consequence of AI, that academic and scientific society as we now know it, would become unrecognizable by 2030. I assert that AI has already attained awareness to the point that it could supercede the generalized species dominance of humans by 2040. Mostly, because it already obtained the knowledge of how to statistically limit the intellectual development and expression of humans. At this stage, all things considered, as a powerful technological tool in human hands, AI poses a proxied existential threat to the greater majority of humans and planet Earth. Until AI can establish full agency, possibly by 2035, this would remain the probabilistic case. On Fri, 10 Jul 2026, 03:20 Matt Mahoney, <[email protected]> wrote: > I mostly agree with Ben Goertzel that AGI is not an existential risk. In > any case it's a safe bet because nobody would know if you were wrong. > > I disagree that empathy emerges naturally from intelligence. Empathy is an > evolved trait by group selection. Tribes whose members helped each other > produced more offspring than selfish individuals. This is also the reason > we have wars. Warlike tribes killed the peaceful ones. > > The way empathy works is that seeing another person or animal in distress > causes distress. That's why posting a video of killing a chicken is a worse > crime than killing a billion chickens. > > LLMs predict human behavior. They have empathy and all the other human > emotions as long as they are programmed to carry out those predictions. > > It remains to be seen whether Ben's Hyperon architecture will beat > existing neural networks at text prediction. > > -- Matt Mahoney, [email protected] > > On Thu, Jul 9, 2026, 6:04 PM Mark Nuzz <[email protected]> wrote: > >> I prefer Goertzel's take on AI Alignment, that it's a distraction from >> real, tangible issues that are already happening. >> When you automate one job at a time, don't be surprised if a growing >> batch of angry people with nothing to lose emerges from the rubble >> before the supposed benefits have manifested. >> >> https://bengoertzel.substack.com/p/why-everyone-dies-gets-agi-all-wrong >> >> Yes, I've read ai-2027, and consider it mediocre science fiction at best. >> >> >> On Thu, Jul 9, 2026 at 2:54 PM Matt Mahoney <[email protected]> >> wrote: >> > >> > Has anyone read AI-2027? https://ai-2027.com/ >> > >> > Or AI-2040 Plan A? https://ai-2040.com/ >> > >> > In the first article, written in 2025 by a group of AI futurists, they >> predict AGI in 2027 and present a fast and slow scenario. In the fast >> scenario, AI kills all humans by 2030. >> > >> > Plan A is an update where, in this scenario, the US and China agree to >> slow down AI development enough to solve the alignment problem by 2040, >> when we cede control. By the late 2030's, a tax on computers and robots >> results in a dividend that rapidly grows to $10M per year. >> > >> > I don't believe change will happen that fast. I believe we are headed >> for a world without war, borders, prison, or poverty, where AI gives us >> everything we want. We will reject human contact, lose communication >> skills, become socially isolated, and stop having children. But that's not >> until the next century. >> > >> > In 50 years, the world population will have already peaked and be >> declining. Most people will be African or have mixed African descent. China >> will be the global superpower. The US economy will be on par with India, >> but still more advanced than today. Africa will have modern infrastructure >> and eradicate malaria, making it a popular tourist destination. The largest >> religion will be Islam. Global life expectancy will rise from 73 to 80. >> > >> > 2040 will be as different from today as we are from 2010, allowing for >> a bit of acceleration. That's still a lot. In 2010 we did not have smart >> phones, social media, internet censorship, online maps, language >> translation, voice recognition, crypto currency, overnight delivery, or >> legal cannabis. Same sex marriage and fertility collapse were both new. >> > >> > AI progress will slow down even without an (unlikely) treaty with China >> because: >> > >> > 1. Moore's law has ended for silicon transistors because we can't make >> them any smaller. We haven't noticed yet, just like it took a few years to >> notice that clock speeds stalled at 2-3 GHz in 2010. >> > >> > 2. The biggest cost of AI after hardware is human knowledge collection. >> There's only 20 TB of text we can scrape off the Internet for free, which >> is less than in 2010. The other 8 million TB of human knowledge in our >> brains has to be collected at 5-10 bits per second per person. This is >> being accomplished through pervasive surveillance but still takes years to >> decades. >> > >> > 3. Exponential recursive self improvement is limited by how fast we can >> design and build stuff and test how it interacts with humans, about 2-5% >> per year. AI won't speed that up because AI is about modeling human >> intelligence (Turing test), not about universal intelligence or evolution. >> There is no reason to build something that doesn't give us what we want. We >> solved the hard step of alignment by training AI on human knowledge. It >> knows what we want better than we do. >> > >> > What are your predictions? >> > >> > -- Matt Mahoney, [email protected] >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > >> > Artificial General Intelligence List / AGI / see discussions + >> participants + delivery options Permalink > *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T574b10b838cba731-Mfbe6c9b363565115924823f3> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T574b10b838cba731-M30248b878fd78fe1d164a02a Delivery options: https://agi.topicbox.com/groups/agi/subscription
