Re: [agi] Can symbolic approach entirely replace NN approach?

Yes, I seem to have missed where you showed how principal component analysis would apply to my simple example of ordering sets. Pity. I would have liked to have seen PCA applied to the simple task of ordering a set of people alternately by height or age. Anyway, the good thing is that you can

Re: [agi] Can symbolic approach entirely replace NN approach?

If you mean my focus is on encoding or representation as the key unsolved problem holding us back from AGI, then yes, you're probably right. On Mon, Jun 24, 2024 at 4:27 PM Quan Tesla wrote: > > Rob > > I applied my SSM method to your output here, like a broad AI might've done. > The resultant

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob, I already explained how it applies to your example, your just "unable" to comprehend it. Because your talk / think ratio is way too high.  -- Artificial General Intelligence List: AGI Permalink:

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob I applied my SSM method to your output here, like a broad AI might've done. The resultant context diagram was enlightening. You talk many things, but seemingly only evidence two, primary things. One of those has to do with constructors and embedding (the text-to-number transformations). I

Re: [agi] Can symbolic approach entirely replace NN approach?

Quan, Lots of words. None of which mean anything to me... OK "soft-systems ontology" turns up something: https://en.wikipedia.org/wiki/Soft_systems_methodology This British guy Checkland wrote some books on management techniques. Some kind of "seven step" process: 1) Enter situation in which

Re: [agi] Can symbolic approach entirely replace NN approach?

No, I don't believe I am talking about PCA. But anyway, you are unable to demonstrate how you implement PCA or anything else, because your algorithm is "far from complete". You are unable to apply your conception of the problem to my simple example of re-ordering a set. How about PCA itself? If

Re: [agi] Can symbolic approach entirely replace NN approach?

> What I mean by contradiction is different orderings of an entire set of data, not points of contrast within a set of data That's not what people usually mean by contradiction, definitely not in a general sense. You are talking about reframing dataset (subset) of multivariate items along the

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob. I'm referring to contextualization as general context management within complex systems management. As an ontology. The application of which has relevance for knowledge graphs, LLMs, and other knowledge-based representations. Your quotation: "Contextualization ... in LLM systemic

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sun, Jun 23, 2024 at 11:05 PM Boris Kazachenko wrote: > > There can be variance on any level of abstraction, be that between pixels or > between philosopical categories. And it could be in terms of any property / > attribute of compared elements / clusters / concepts: all these are derived

Re: [agi] Can symbolic approach entirely replace NN approach?

There can be variance on any level of abstraction, be that between pixels or between philosopical categories. And it could be in terms of any property / attribute of compared elements / clusters / concepts: all these are derived by lower-order comparisons. None of that falls from the sky, other

Re: [agi] Can symbolic approach entirely replace NN approach?

There were 5 or 6 totally mis-interpretations of my words in there, Boris. Mis-interpretations of my words was almost the whole content of your argument. I'll limit myself to the most important mis-interpretation below. On Sun, Jun 23, 2024 at 7:10 PM Boris Kazachenko wrote: > ... > Starting

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob, a lot of your disagreements stem from your language-first mindset. Which is perverse, you must agree that the language is a product of basic cognitive ability, possed by all mammals. Starting from your "contradiction": that's simply a linguistic equivalent of my variance.  I have no idea

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sat, Jun 22, 2024 at 7:50 PM Boris Kazachenko wrote: > > On Saturday, June 22, 2024, at 7:18 AM, Rob Freeman wrote: > > But I'm not sure that just sticking to some idea of learned hierarchy, which > is all I remember of your work, without exposing it to criticism, is > necessarily going to

Re: [agi] Can symbolic approach entirely replace NN approach?

On Saturday, June 22, 2024, at 7:18 AM, Rob Freeman wrote: > But I'm not sure that just sticking to some idea of learned hierarchy, which is all I remember of your work, without exposing it to criticism, is necessarily going to get you any further. It's perfectly exposed: 

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sat, Jun 22, 2024 at 6:05 PM Boris Kazachenko wrote: > ... > You both talk too much to get anything done... Ah well, you may be getting lots done, Boris. The difference is perhaps, I don't know everything yet. Though, after 35 years, it can be surprising what other people don't know. I like

Re: [agi] Can symbolic approach entirely replace NN approach?

>Wow. Lots of words. I don't mind detail, but words are slippery. He is marking a territory, like any dog. It's all about self-promotion: the more he talk about himself, the better he feels. You both talk too much to get anything done, it becomes an end in itself, substance is secondary.

Re: [agi] Can symbolic approach entirely replace NN approach?

Twenkid, Wow. Lots of words. I don't mind detail, but words are slippery. If you actually want to do stuff, it's better to keep the words to a minimum and start with concrete examples. At least until some minimum of consensus is agreed. Trying to focus on your concrete questions... On Sat, Jun

Re: [agi] Can symbolic approach entirely replace NN approach?

On Mon, Jun 17, 2024 at 3:22 PM Quan Tesla wrote: > > Rob, basically you're reiterating what I've been saying here all along. To > increase contextualization and instill robustness in the LLM systemic > hierarchies. Further, that it seems to be critically lacking within current > approaches. >

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob, basically you're reiterating what I've been saying here all along. To increase contextualization and instill robustness in the LLM systemic hierarchies. Further, that it seems to be critically lacking within current approaches. However, I think this is fast changing, and soon enough, I

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sunday, June 16, 2024, at 7:09 PM, Matt Mahoney wrote: > Not everything can be symbolized in words. I can't describe what a person > looks as well as showing you a picture. I can't describe what a novel > chemical smells like except to let you smell it. I can't tell you how to ride > a

Re: [agi] Can symbolic approach entirely replace NN approach?

Not everything can be symbolized in words. I can't describe what a person looks as well as showing you a picture. I can't describe what a novel chemical smells like except to let you smell it. I can't tell you how to ride a bicycle without you practicing. On Sun, Jun 16, 2024, 5:36 PM John Rose

Re: [agi] Can symbolic approach entirely replace NN approach?

On Friday, June 14, 2024, at 3:43 PM, James Bowery wrote: >> Etter: "Thing (n., singular): anything that can be distinguished from >> something else." I simply use “thing” as anything that can be symbolized and a unique case are qualia where from a first-person experiential viewpoint a qualia

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sat, Jun 15, 2024 at 1:29 AM twenkid wrote: > > ... > 2. Yes, the tokenization in current LLMs is usually "wrong", ... it should > be on concepts and world models: ... it should predict the *physical* future > of the virtual worlds Thanks for comments. I can see you've done a lot of

Re: [agi] Can symbolic approach entirely replace NN approach?

My point was that token boundaries are fuzzy. This causes problems because LLMs predict tokens, not characters or bits. There was a thread on Reddit about ChatGPT not being able to count the number of R's in "strawberry". The problem is that it sees the word but not the letters.

Re: [agi] Can symbolic approach entirely replace NN approach?

On Wed, May 29, 2024 at 11:24 AM Matt Mahoney wrote: > Natural language is ambiguous at every level including tokens. Is > "someone" one word or two? > Tom Etter 's tragically unfinished final paper "Membership and Identity

Re: [agi] Can symbolic approach entirely replace NN approach?

1. @Can symblic approach ... 2. @Rob Freeman LLMs, What's wrong with NLP (2009-2024),  Whisper *1*. *IMO the sharpness of the division "neat" and "scruffy",  NN and symbolic is confused: Neural Networks are also symbolic:*

Re: [agi] Can symbolic approach entirely replace NN approach?

Natural language is ambiguous at every level including tokens. Is "someone" one word or two? Language models handle this by mixing the predictions given by the contexts "some", "one", and "someone". Using fixed dictionaries is a compromise that reduces accuracy for reducing computation, like all

Re: [agi] Can symbolic approach entirely replace NN approach?

On Wed, May 29, 2024 at 9:37 AM Matt Mahoney wrote: > > On Tue, May 28, 2024 at 7:46 AM Rob Freeman > wrote: > > > Now, let's try to get some more detail. How do compressors handle the > > case where you get {A,C} on the basis of AB, CB, but you don't get, > > say AX, CX? Which is to say, the

Re: [agi] Can symbolic approach entirely replace NN approach?

On Tue, May 28, 2024 at 7:46 AM Rob Freeman wrote: > Now, let's try to get some more detail. How do compressors handle the > case where you get {A,C} on the basis of AB, CB, but you don't get, > say AX, CX? Which is to say, the rules contradict. Compressors handle contradictory predictions by

Re: [agi] Can symbolic approach entirely replace NN approach?

Matt, Nice break down. You've actually worked with language models, which makes it easier to bring it back to concrete examples. On Tue, May 28, 2024 at 2:36 AM Matt Mahoney wrote: > > ...For grammar, AB predicts AB (n-grams), Yes, this looks like what we call "words". Repeated structure. No

Re: [agi] Can symbolic approach entirely replace NN approach?

On Mon, May 27, 2024 at 9:34 AM Rob Freeman wrote: > James, > > I think you're saying: > > 1) Grammatical abstractions may not be real, but they can still be > useful abstractions to parameterize "learning". > And more generally, people pragmatically adopt different fictional abstract grammars

Re: [agi] Can symbolic approach entirely replace NN approach?

The top text compressors use simple models of semantics and grammar that group words into categories as fuzzy equivalence relations. For semantics, the rules are reflexive, A predicts A (but not too close. Probability peaks 50-100 bytes away), symmetric, A..B predicts A..B and B..A, and

Re: [agi] Can symbolic approach entirely replace NN approach?

James, I think you're saying: 1) Grammatical abstractions may not be real, but they can still be useful abstractions to parameterize "learning". 2) Even if after that there are "rules of thumb" which actually govern everything. Well, you might say why not just learn the "rules of thumb". But

Re: [agi] Can symbolic approach entirely replace NN approach?

It's also worth reiterating a point I made before about the confusion between abstract grammar as a

Re: [agi] Can symbolic approach entirely replace NN approach?

See the recent DeepMind paper "Neural Networks and the Chomsky Hierarchy " for the sense of "grammar" I'm using when talking about the HNet paper's connection to Granger's prior papers about "grammar", the most recent being "Toward the quantification of cognition

Re: [agi] Can symbolic approach entirely replace NN approach?

Thanks Matt. The funny thing is though, as I recall, finding semantic primitives was the stated goal of Marcus Hutter when he instigated his prize. That's fine. A negative experimental result is still a result. I really want to emphasize that this is a solution, not a problem, though. As the

Re: [agi] Can symbolic approach entirely replace NN approach?

I agree. The top ranked text compressors don't model grammar at all. On Fri, May 24, 2024, 11:47 PM Rob Freeman wrote: > Ah, I see. Yes, I saw that reference. But I interpreted it only to > mean the general forms of a grammar. Do you think he means the > mechanism must actually be a grammar? >

Re: [agi] Can symbolic approach entirely replace NN approach?

Ah, I see. Yes, I saw that reference. But I interpreted it only to mean the general forms of a grammar. Do you think he means the mechanism must actually be a grammar? In the earlier papers I interpret him to be saying, if language is a grammar, what kind of a grammar must it be? And, yes, it

Re: [agi] Can symbolic approach entirely replace NN approach?

On Thu, May 23, 2024 at 9:19 PM Rob Freeman wrote: > ...(Regarding the HNet paper) > The ideas of relational category in that paper might really shift the > needle for current language models. > > That as distinct from the older "grammar of mammalian brain capacity" > paper, which I frankly

Re: [agi] Can symbolic approach entirely replace NN approach?

James, Not sure whether all that means you think category theory might be useful for AI or not. Anyway, I was moved to post those examples by Rich Hickey and Bartoz Milewsky in my first post to this thread, by your comment that ideas of indeterminate categories might annoy what you called 'the

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob Yes, I understand the difference between a video and a paper. I did not think John criticized it either. If he had, it's helathy to render critique. You referenced the paper, and it remains relevant to the topic, regardless of the medium accessed. Indeed, I was referring to your decoupling

Re: [agi] Can symbolic approach entirely replace NN approach?

On Wed, May 22, 2024 at 10:34 PM Rob Freeman wrote: > On Wed, May 22, 2024 at 10:02 PM James Bowery wrote: > > ... > > You correctly perceive that the symbolic regression presentation is not > to the point regarding the HNet paper. A big failing of the symbolic > regression world is the same

Re: [agi] Can symbolic approach entirely replace NN approach?

On Wed, May 22, 2024 at 10:02 PM James Bowery wrote: > ... > You correctly perceive that the symbolic regression presentation is not to > the point regarding the HNet paper. A big failing of the symbolic regression > world is the same as it is in the rest of computerdom: Failure to recognize

Re: [agi] Can symbolic approach entirely replace NN approach?

On Thu, May 23, 2024 at 10:10 AM Quan Tesla wrote: > > The paper is specific to a novel and quantitative approach and method for > association in general and specifically. John was talking about the presentation James linked, not the paper, Quan. He may be right that in that presentation they

Re: [agi] Can symbolic approach entirely replace NN approach?

The paper is specific to a novel and quantitative approach and method for association in general and specifically. It emerges possible and statistical (most correct) relationships. This stands in stark contrast to the deterministic commitment to construct functional relationships. Hence, a

Re: [agi] Can symbolic approach entirely replace NN approach?

On Tue, May 21, 2024 at 9:35 PM Rob Freeman wrote: > > > Whereas the NN presentation is talking about NNs regressing to fixed > encodings. Not about an operator which "calculates energies" in real > time. > > Unless I've missed something in that presentation. Is there anywhere > in the hour

Re: [agi] Can symbolic approach entirely replace NN approach?

On Tuesday, May 21, 2024, at 10:34 PM, Rob Freeman wrote: > Unless I've missed something in that presentation. Is there anywhere in the hour long presentation where they address a decoupling of category from pattern, and the implications of this for novelty of structure? I didn’t watch the video

Re: [agi] Can symbolic approach entirely replace NN approach?

Thanks for sharing this paper. Positively brilliant! I think this is in-line with quantum thinking and holds great promise for quantum computing. It relates to a concept advanced by myself and my mentor, namely, gestalt management. Penultimately, we endeavor to most-correctly represent

Re: [agi] Can symbolic approach entirely replace NN approach?

James, The Hamiltonian paper was nice for identifying gap filler tasks as decoupling meaning from pattern: "not a category based on the features of the members of the category, let alone the similarity of such features". Here, for anyone else: A logical re-conception of neural networks:

Re: [agi] Can symbolic approach entirely replace NN approach?

Symbolic Regression is starting to catch on but, as usual, people aren't using the Algorithmic Information Criterion so they end up with unprincipled choices on the Pareto frontier between residuals and model complexity if not unprincipled choices about how to weight the complexity of various

Re: [agi] Can symbolic approach entirely replace NN approach?

"Importantly, the new entity ¢X is not a category based on the features of the members of the category, let alone the similarity of such features" Oh, nice. I hadn't seen anyone else making that point. This paper 2023? That's what I was saying. Nice. A vindication. Such categories decouple the

Re: [agi] Can symbolic approach entirely replace NN approach?

Tokens inside transformers are supervised internal symbols. -- Artificial General Intelligence List: AGI Permalink: https://agi.topicbox.com/groups/agi/T682a307a763c1ced-M102516027fd65ca8c1f90b8b Delivery options:

Re: [agi] Can symbolic approach entirely replace NN approach?

From *A logical re-conception of neural networks: Hamiltonian bitwise part-whole architecture* > *From hierarchical statistics to abduced symbols*It is perhaps useful to > envision some of the ongoing devel- > opments that are arising from enlarging and elaborating the > Hamiltonian logic net

Re: [agi] Can symbolic approach entirely replace NN approach?

On Mon, May 20, 2024 at 9:49 AM Rob Freeman wrote: > Well, I don't know number theory well, but what axiomatization of > maths are you basing the predictions in your series on? > > I have a hunch the distinction I am making is similar to a distinction > about the choice of axiomatization. Which

Re: [agi] Can symbolic approach entirely replace NN approach?

Well, I don't know number theory well, but what axiomatization of maths are you basing the predictions in your series on? I have a hunch the distinction I am making is similar to a distinction about the choice of axiomatization. Which will be random. (The randomness demonstrated by Goedel's

Re: [agi] Can symbolic approach entirely replace NN approach?

On Sun, May 19, 2024 at 11:32 PM Rob Freeman wrote: > James, > > My working definition of "truth" is a pattern that predicts. And I'm > tending away from compression for that. > 2, 4, 6, 8 does it mean 2n? or does it mean 10? Related to your sense of "meaning" in (Algorithmic Information)

Re: [agi] Can symbolic approach entirely replace NN approach?

On Saturday, May 18, 2024, at 6:53 PM, Matt Mahoney wrote: > Surely you are aware of the 100% failure rate of symbolic AI over the last 70 > years? It should work in theory, but we have a long history of > underestimating the cost, lured by the early false success of covering half > of the

Re: [agi] Can symbolic approach entirely replace NN approach?

James, My working definition of "truth" is a pattern that predicts. And I'm tending away from compression for that. Related to your sense of "meaning" in (Algorithmic Information) randomness. But perhaps not quite the same thing. I want to emphasise a sense in which "meaning" is an expansion of

Re: [agi] Can symbolic approach entirely replace NN approach?

It's not about who wins the battle of models, rather if the models employed would theoretically (symbolically) be a true representation of an AGI with potential for ASI. I think that LLMs on their own simply won't hack it. You may be satisfied with the tradeoffs in commercialized value, but there

Re: [agi] Can symbolic approach entirely replace NN approach?

On Thu, May 16, 2024, 11:27 AM wrote: > What should symbolic approach include to entirely replace neural networks > approach in creating true AI? Is that task even possible? What benefits and > drawbacks we could expect or hope for if it is possible? If it is not > possible, what would be the

Re: [agi] Can symbolic approach entirely replace NN approach?

Rob, the problem I have with things like "type theory" and "category theory" is that they almost always elide their foundation in HOL (high order logic) which means they don't *really* admit that they are syntactic sugars for second-order predicate calculus. The reason I describe this as

Re: [agi] Can symbolic approach entirely replace NN approach?

om: John Rose Sent: Friday, 17 May 2024 13:48 To: AGI Subject: Re: [agi] Can symbolic approach entirely replace NN approach? On Thursday, May 16, 2024, at 11:26 AM, ivan.moony wrote: What should symbolic approach include to entirely replace neural networks approach in creating true AI? Symbolo

Re: [agi] Can symbolic approach entirely replace NN approach?

On Thursday, May 16, 2024, at 11:26 AM, ivan.moony wrote: > What should symbolic approach include to entirely replace neural networks > approach in creating true AI? Symbology will compress NN monstrosities… right?  Or should say increasing efficiency via emerging symbolic activity for

Re: [agi] Can symbolic approach entirely replace NN approach?

James, For relevance to type theories in programming I like Bartosz Milewski's take on it here. An entire lecture series, but the part that resonates with me is in the introductory lecture: "maybe composability is not a property of nature" Cued up here: Category Theory 1.1: Motivation and

Re: [agi] Can symbolic approach entirely replace NN approach?

Historically the AGI community has not really embraced neural networks -- and the cost has been that the AI explosion has come from the mainstream more or less. On Thu, May 16, 2024 at 7:01 PM Quan Tesla wrote: > Without neural networks, a symbolic approach wouldn't be effective. My > view is

Re: [agi] Can symbolic approach entirely replace NN approach?

Without neural networks, a symbolic approach wouldn't be effective. My view is that, depending on the definition of what "symbolic approach" means in the context of AGI, in the least both such operational schemas would be required to achieve the level of systems abstraction that would satisfy a

Re: [agi] Can symbolic approach entirely replace NN approach?

First, fix quantum logic: https://web.archive.org/web/20061030044246/http://www.boundaryinstitute.org/articles/Dynamical_Markov.pdf Then realize that empirically true cases can occur not only in multiplicity (OR), but with structure that includes the simultaneous (AND) measurement dimensions of

Re: [agi] Can symbolic approach entirely replace NN approach?

It seems like most approaches including symbolic only could eventually lead to "true AI" if you mean ~ passing the Turing test, but it might take 100 years. There is a race to the finish aspect to AI though. On Thu, May 16, 2024 at 8:27 AM wrote: > What should symbolic approach include to

Re: [agi] Can symbolic approach entirely replace NN approach?

On 5/16/24 17:26, ivan.mo...@gmail.com wrote: What should symbolic approach include to entirely replace neural networks approach in creating true AI? Is that task even possible? What benefits and drawbacks we could expect or hope for if it is possible? If it is not possible, what would be the

Re: [agi] Can symbolic approach entirely replace NN approach?

Seems unlikely as the first approach. ANNs help us bridge things we have little understanding of via brute force and lots of data. Perhaps AFTER we get to ASI the ASI can figure out how to recode itself symbolically, at huge gain (likely) in performance. On Thu, May 16, 2024 at 11:27 AM wrote:

[agi] Can symbolic approach entirely replace NN approach?

What should symbolic approach include to entirely replace neural networks approach in creating true AI? Is that task even possible? What benefits and drawbacks we could expect or hope for if it is possible? If it is not possible, what would be the reasons? Thank you all for your time.