On Sun, Aug 2, 2020 at 1:58 AM Ben Goertzel <[email protected]> wrote:
> ... > ...I also think that the search for concise > abstract models is another part of what's needed... > It depends how you define "concise abstract model". Even maths has an aspect of contradiction. What does Chaitin call his measure of randomness in maths..? Chaitin's Omega. And he sees this random character correctly as a power. A feature not a bug. Chaitin: "Incompleteness is the first step toward a mathematical theory of creativity, a bridge between pure mathematics and biology." http://inference-review.com/article/an-algorithmic-god So maths has this contradictory truth aspect too. But once you resolve that with axioms, maths is concise and abstract, yes. You can build a concise abstract maths on top of that. And sure, it's desirable to do that. > And I don't think GPT3 is doing this "constantly unpacking structure > by combining observations, billions of features of it." in the right > way for AGI ... No, they're not doing it in the right way. And yes, they will need to relate it to sensory data to ground words in their turn, in the sensory world. That might be closer to the grounding you want. In particular I don't think string transformations will translate to a dynamical model. There might be some trick. But likely they'll need to change their relational principles again. Difficult to find energy minima of string transformations in real time. I think the right structuring model will turn out to be causal invariants, revealed by setting a sequence network oscillating. That should be dead simple. And tantalisingly suggesting a role for oscillations to match the observation of "binding by synchrony." But nobody has ears for that. String transformations are currently where it's at. > honestly Rob your own ideas feel way closer to the > mark... Thanks. And yes I don't think OpenAI are theoretically close to what I'm suggesting. You're right. At best I think they are backing into it, blindly, while looking the other way. By accident, without theory. They don't have a theory of meaning. They stumbled into string transformations while trying to enhance RNNs. Now they're stumbling into more and more parameters, without knowing why they need to. But if that's where they are, that's the context I have to talk to. I have to find what I can of value in that and try to show how I believe it can be better. What I can find of value is evidence that the more parameters you generate the better your model becomes - nobody knows why, I say resolution of contradiction in context. And secondly that those parameters can be easily generated from principled relationships between parts - even though they think you have to list them all in advance. My own theory is that meaning contradicts. That grasping this has been the problem for 50 years. (Caused Chomsky to reject learned grammar in the '50s for instance, and redirect linguistics in the direction of innate parameters for 60 years!) And to properly capture this contradictory meaning in context we must constantly generate meaning/parameters from simple meaningful relationships between parts: causal invariants, string permutations, transformations... But constantly generating it. A dynamical system. Unfortunately the gap between that and the current state-of-the-art is too great. Unless you can find 9/10 overlap with what people are already doing, they won't listen. So my best bet is to suggest how folks working on stuff like GPT-3 might improve what they've done. My answer is that they might think of calculating their "parameters" in real time. Not trying to enumerate all infinitely billions of them beforehand. But what they are doing is not "right". It's lame that they blow 12M trying to list every possible meaningful construction in advance. But currently GPT-3, with its 175 billion parameters and growing, is the best evidence I have that we need to embrace contradiction, and calculate meaningful parameters all the time, in a "symbolic network dynamics", as we've discussed before. -Rob ------------------------------------------ Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T21c073d3fe3faef0-M3eb3c66fee749efaf83a3994 Delivery options: https://agi.topicbox.com/groups/agi/subscription
