On Thu, Mar 28, 2024 at 9:27 PM Russell Standish <li...@hpcoders.com.au> wrote:
> * >"So to compare apples with apples - the human brain contains around > 700 trillion (7E14) synapses"* I believe 700 trillion is a more than generous estimate of the number of synapses in the human brain, but I'll let it go. *>"which would roughly correpond to an AI's parameter count* *NO! *Comparing the human brain's synapses to the number of parameters that an AI program like GPT-4 has is NOT comparing apples to apples, it's comparing apples to oranges because the brain is hardware but GPT-4 is software. So let's compare the brain hardware that human intelligence is running on with the brain hardware that GPT-4 is running on, that is to say let's compare synapses to transistors. I'll use your very generous estimate and say the human brain has 7*10^14 synapses, but the largest supercomputer in the world, the Frontier Computer at Oakridge, has about 2.5*10^15 transistors, over three times as many. And we know from experiments that a typical synapse in the human brain "fires" between 5 and 50 times per second, but a typical transistor in a computer "fires" about 4 billion times a second (4*10^9). That's why the Frontier Computer can perform 1.1 *10^18 floating point calculations per second and why the human brain can not. I should add that although there have been significant improvements in the field of AI in recent years, the most important being the "Attention Is All You Need" paper, I believe that even if transformers had never been discovered the AI explosion that we are currently observing would only have been delayed by a few years because the most important thing driving it forward is the brute force enormous increase in raw computing speed. > "*He [Ray Kurzweil] was predicting 2029 to be the time when AI will > attain human level intelligence.*" It now looks like Ray was being too conservative and 2024 or 2025 would be closer to the Mark, and 2029 would be the time when an AI is smarter than the entire human race combined. *> "I would still say that creativity (which is an essential prerequisite) > is still mysterious"* It doesn't matter if humans find creativity to be mysterious because we have an existence proof that a lack of understanding of creativity does not prevent humans from making a machine that is creative. Back in 2016 when a computer beat Lee Sedol, the top human champion at the game of GO, the thing that everybody was talking about was move 37 of the second game of the five game tournament. When the computer made that move the live expert commentators were shocked and described it as "practically nonsensical" and "something no human would do", and yet that crazy "nonsensical" move was the move that enabled the computer to win. Lee Sedol said move 37 was "*an incredible move*" and was completely unexpected and made it impossible for him to win, although it took him a few more moves before he realized that. If a human had made moves 37 every human GO expert on the planet would've said it was the most creative move they had ever seen. > *> "But singularity requires that machines design themselves"* Computers are already better at writing software than the average human, and major chip design and manufacturing companies like NVIDIA, AMD, Intel , Cerebras and TSMC are investing heavily in chip design software. > > * > Anyway my 2c - I know John is keen to promote the idea of > singularity this decade - but I don't see it myself.* One thing I know for certain, whenever the Singularity occurs most people will be surprised, otherwise it wouldn't be a Singularity. John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis> oib -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to everything-list+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv3HNhk6ufAiQjjeK419CqpSubiJp%3DnTpPefCSADYs0Osg%40mail.gmail.com.
