On Tue, Jul 2, 2024 at 12:52 PM Jason Resch <[email protected]> wrote:
*> I also see it as surprising that through hardware improvements alone, > and without specific breakthroughs in algorithms, we should see such great > strides in AI.* I was not surprised because the entire human genome only has the capacity to hold 750 MB of information; that's about the amount of information you could fit on an old-fashioned CD, not a DVD, just a CD. The true number must be considerably less than that because that's the recipe for building an entire human being, not just the brain, and the genome contains a huge amount of redundancy, 750 MB is just the upper bound. > *> Humans no longer write the algorithms these neural networks derive, the > training process comes up with them. And much like the algorithms > implemented in the human brain, they are in a representation so opaque and > that they escape our capacity to understand. So I would argue, there have > been massive breakthroughs in the algorithms that underlie the advances in > AI, we just don't know what those breakthroughs are.* That is a very interesting way to look at it, and I think you are basically correct. > *> I think the human brain, with its 600T connections might signal an > upper bound for how many are required, but the brain does a lot of other > things too, so the bound could be lower.* > The human brain has about 86 billion neurons with 7*10^14 synaptic connections (a more generous estimate than yours), but the largest supercomputer in the world, the Frontier Computer at Oak ridge, has 2.5*10^15 transistors, over three times as many. And we know from experiments that a typical synapse in the human brain "fires" between 1 and 50 times per second, but a typical transistor in a computer "fires" about 4 billion times a second (4*10^9). It also has 9.2* 10^15 bites of fast memory. That's why the Frontier Computer can perform 1.1 *10^18 double precision floating point calculations per second and why the human brain can not. By way of comparison, Ray Kurzweil estimates that the hardware needed to emulate a human mind would need to be able to perform 10^16 calculations per second and have 10^12 bytes of memory. And the calculations would not need to be 64 bit double precision floating point, 8 bit or perhaps even 4 bit precision would be sufficient. So in the quest to develop a superintelligence, insufficient hardware is no longer a barrier. John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis> bom -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv0FmETnmRQ2VK_EKxJ%2BmyBjkaetVY6swTT7QRoK_ofqOw%40mail.gmail.com.
