On Tue, Mar 14, 2023 at 7:31 AM Telmo Menezes <[email protected]> wrote:
> One of the authors of the article says "It’s interesting that the >> computer-science field is converging onto what evolution has discovered", >> he said that because it turns out that 41% of the fly brain's neurons are >> in recurrent loops that provide feedback to other neurons that are upstream >> of the data processing path, and that's just what we see in modern AIs like >> ChatGPT. > > > > *> I do not think this is true. ChatGPT is a fine-tuned Large Language > Model (LLM), and LLMs use a transformer architecture, which is deep but > purely feed-forward, and uses attention heads. The attention mechanism was > the big breakthrough back in 2017, that finally enabled the training of > such big models:* > I was under the impression that transformers are superior to recurrent neural networks because recurrent processing of data was not necessary with transformers so more paralyzation is possible than with recursive neural networks; it can analyze an entire sentence at once and doesn't need to do so word by word. So Transformers learn faster and need less trading data. *> My intuition is that if we are going to successfully imitate biology we > must model the various neurotransmitters.* That is not my intuition. I see nothing sacred in hormones, I don't see the slightest reason why they or any neurotransmitter would be especially difficult to simulate through computation, because chemical messengers are not a sign of sophisticated design on nature's part, rather it's an example of Evolution's bungling. If you need to inhibit a nearby neuron there are better ways of sending that signal then launching a GABA molecule like a message in a bottle thrown into the sea and waiting ages for it to diffuse to its random target. I'm not interested in brain chemicals, only in the information they contain, if somebody wants information to get transmitted from one place to another as fast and reliablely as possible, nobody would send smoke signals if they had a fiber optic cable. The information content in each molecular message must be tiny, just a few bits because only about 60 neurotransmitters such as acetylcholine, norepinephrine and GABA are known, even if the true number is 100 times greater (or a million times for that matter) the information content of each signal must be tiny. Also, for the long range stuff, exactly which neuron receives the signal can not be specified because it relies on a random process, diffusion. The fact that it's slow as molasses in February does not add to its charm. If your job is delivering packages and all the packages are very small, and your boss doesn't care who you give them to as long as they're on the correct continent, and you have until the next ice age to get the work done, then you don't have a very difficult profession. Artificial neurons could be made to communicate as inefficiently as natural ones do by releasing chemical neurotransmitters if anybody really wanted to, but it would be pointless when there are much faster, and much more reliable, and much more specific ways of operating. John K Clark See what's on my new list at Extropolis <https://groups.google.com/g/extropolis> kuh -- You received this message because you are subscribed to the Google Groups "Everything List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/everything-list/CAJPayv089oC%3DAc-DswW5simNfWzQsGAZADjusaWOacE4M6kt9g%40mail.gmail.com.
