I think we need to be careful with considering LLM parameters as analogous to synapses. Biological neuronal systems have very significant differences in terms of structure, complexity, and operation compared to LLM parameters.
Personally, I don't believe it is a given that simply increasing the parameters of a LLM is going to result in AGI or parity with overall human potential. I think there is a lot more to figure out before we get there, and LLMs (assuming variations on current transformer based architectures) may end up a dead end without other AI breakthroughs combining them with other components, and inputs (as in sensory inputs).. We may find out that the singularity is a lot further away than it seems, but I guess time will tell. Personally, I would be very surprised to see it within the next decade. On Thu, Mar 28, 2024 at 9:27 PM Russell Standish <[email protected]> wrote: > > So to compare apples with apples - the human brain contains around 700 > trillion (7E14) synapses, which would roughly correpond to an AI's > parameter count. GPT5 (due to be released sometime next year) will > have around 2E12 parameters, still 2-3 orders of magnitude to > go. Assuming continuation of current rates of AI improvement > GPT3->GPT5 (4 years) is one order of magnitude increase in parameter > count, it will take to 2033 for AI to achieve human parity. > > > -- > You received this message because you are subscribed to the Google Groups > "Everything List" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/everything-list/20240329012651.GE2357%40zen > . > -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJrqPH9L6f%3Dc8%3DjjQAgXSP5WvHQ-k2dUvwS%2Btj-UWqw%2BaxUoZQ%40mail.gmail.com.
