The power problem is physics. The energy to switch states in transistors is V^2C/2 where Vcc = 1.3 volts and C is the capacitance of the input gates and wiring. A P-N junction in silicon has a forward bias of 0.6 V. Below that, no current flows. As you lower the voltage, it takes longer to charge the capacitor and the logic slows. 1.3 V is a compromise between speed and energy use.
You can lower the voltage by using germanium instead of silicon, which has a forward bias of 0.25 V. You can lower the capacitance by optimizing the routing to minimize signal distance, shrinking the transistors, using rounder wires, and using air instead of silicon dioxide insulation, which lowers the dielectric constant from 4.3 to 1. But you can't shrink transistor volume below a few million atoms and still have the required dopant density of 10^-5 to 10^-6, and we are already at that limit. Neuron axons fire up to 300 times per second in 1 ms pulses moving at 100 m/s, switching between -30 mV to +80 mV. So transistors use 250 times more energy per switch for the same size circuit, but can switch 10,000,000 times faster. But it seems that language models need far fewer neurons than the brain. The brain stores 10^9 bits using 600T synapses (86B neurons, 7000 synapses each). The top LLMs with 5-10T parameters are trained on at least 20 TB of common crawl text, which is 20,000 times human level knowledge. The difference is that synapses are binary but parameters are 8 or 16 bit floating point numbers which require a few hundred bit operations for a multiply and add operation. I think the reason that the brain stores less than 0.00001 bits per synapse is that it takes thousands of neurons to produce a continuous valued signal. When I hold my arm out, I can control my muscles within 0.5% of my range of motion even though all my muscle fibers are on/off. To do this takes (1/0.5%)^2 = 40,000 stochastic on/off signals added together. The Hutter prize has demonstrated training a 6M transformer with 4 bit weights on one human level training set (1 GB) using 10^18 operations including offline GPUs. I don't think this does human level inference yet because information theory suggests needing 50M to 100M parameters to represent the training data, which I estimate would need 10^19 operations consuming 10^10 J or a few thousand kilowatt hours of electricity. A human brain running at 20 watts for 75 years (2.5 x 10^9 seconds) uses 5 times more energy. So I think the power problem can be solved with good engineering and a few years of research. -- Matt Mahoney, [email protected] On Thu, Sep 24, 2026, 6:57 AM Quan Tesla <[email protected]> wrote: > I don't think it was known as such, but physics definitely dealt with it > in relation to atomic theory. My comment was generalized, not specific to > "20 watts". > > In its economic potential, the actual solution would transcend the value > of multinational GDPs. Who is not searching for those keys? > > John, if you had it, would you open source it, or lock it up "privately"? > > I might add, knowing the actual 20 watts probably would not be a > silver-bullet. Reason: I think the 20 watts constitute kinetic energy, not > flux. > > > On Thu, 24 Sept 2026, 11:21 John Rose via AGI, <[email protected]> > wrote: > >> From what I've gleaned I had guessed '46 - '48. But, who knows. Maybe we >> will soon. >> >> For AGI it's computational efficiency. What's going on in there to make >> it so efficient? Is there anything specific we could use that's not already >> known? >> >> On Thursday, September 24, 2026, at 1:06 AM, Quan Tesla wrote: >> >> I suspect, we already had it figured it out by 1945. Humanity just chose >> to learn the wrong lesson from it. If you had your 20 watts, what would you >> do with it? >> >> >> *Artificial General Intelligence List <https://agi.topicbox.com/latest>* > / AGI / see discussions <https://agi.topicbox.com/groups/agi> + > participants <https://agi.topicbox.com/groups/agi/members> + > delivery options <https://agi.topicbox.com/groups/agi/subscription> > Permalink > <https://agi.topicbox.com/groups/agi/T25fb5a030fcfa28e-Mf011b386bbd2f37c0fc3c926> > ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T25fb5a030fcfa28e-Mae2c3707dcc7e335d2869e75 Delivery options: https://agi.topicbox.com/groups/agi/subscription
