On Thu, Jul 6, 2023 at 11:30 AM <[email protected]> wrote: >... > Hold on. The Lossless Compression evaluation tests not just compression, but > expansion!
It's easy to get lost in word definitions. It sounds like you're using "expansion" in a sense of recovering an original from a compression. I'm using "expansion" in the sense of chaos. So more like generating a hurricane from butterfly wings. Seen in that light, the current obsession with "learning", or compression, might be the equivalent of starting from a hurricane and trying to squash it down into a butterfly's wings. "Compress" the hurricane. Which would be great if a hurricane were not a non-linear system and basically organic to the entire planet. So, organic to not just one butterfly, but all of them. In practice it means current LLMs might be something like trying to enumerate every hurricane ever seen. And maybe generalize across them. So, say, have "deep" hierarchy stacking all the sub-eddies within every hurricane, or an "attention" mechanism back along the storm track, to compare them. That might be how it places current tech in the context of explaining hurricanes. People do that with movement too. Transformer motion models? They capture zillions of frames, and reproduce anime simulations of motion. Of course, every new hurricane will actually be different. To really capture the relevant parameters of hurricane structure you would need to keep every butterfly on the planet, and every air molecule, and... I'm saying this kind of top down "learning", or compression, is a dumb way to explain hurricanes. Models which are just LARGE, is the wrong simplicity. It's the wrong way of dealing with what is actually chaos in the generation of hurricanes. Just allowing your data-bases to get bigger and bigger might reproduce hurricane like images. It might even predict the behaviour of a good number of them within statistical limits (LLM weather forecasting anyone?) With the very size which enumeration permits, it might be a way to reflect their puzzling variety. But it won't actually have anything to do with hurricanes. That you won't actually understand hurricanes until you accept that they "expand", from tiny causes. You can't capture the essence of hurricanes just by enumerating them and comparing them. You'll get an enormous data-base, and it will only ever get bigger, but it won't actually have anything to do with hurricanes. Any more than anime simulations of motion have anything to do with motion. All you will have done is capture something of the enormous variety of storms which can be generated, the sheer SIZE of the historical record, and ever expanding. Without capturing what is actually generating them. ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T42db51de471cbcb9-Mfd1bd907b0ca63f5a2fde0d7 Delivery options: https://agi.topicbox.com/groups/agi/subscription
