James,
Many of the solutions you describe can use information gathered from statistical models, which are opaque. I need to elaborate on this, because I think opaque models will be fundamental to solving AGI. We need to build models in a way that doesn't require access to the internals. This requires a different approach than traditional knowledge representation. It will require black box testing and performance metrics. It will be less of an engineering approach, and more of an experimental one.
Information retrieval is a good example. It is really simple. You type a question, and the system matches the words in your query to words in the document and ranks the documents by TF*IDF (term frequency times log inverse document frequency). This is an opaque model. We normally build an index, but this is really just an optimization. The language model is just the documents themselves. There is no good theory to explain why it works. It just does.
-- Matt Mahoney, [EMAIL PROTECTED]
Many of the solutions you describe can use information gathered from statistical models, which are opaque. I need to elaborate on this, because I think opaque models will be fundamental to solving AGI. We need to build models in a way that doesn't require access to the internals. This requires a different approach than traditional knowledge representation. It will require black box testing and performance metrics. It will be less of an engineering approach, and more of an experimental one.
Information retrieval is a good example. It is really simple. You type a question, and the system matches the words in your query to words in the document and ranks the documents by TF*IDF (term frequency times log inverse document frequency). This is an opaque model. We normally build an index, but this is really just an optimization. The language model is just the documents themselves. There is no good theory to explain why it works. It just does.
----- Original Message ----
From: James Ratcliff <[EMAIL PROTECTED]>
To: [email protected]
Sent: Wednesday, November 8, 2006 10:14:43 AM
Subject: Re: [agi] The crux of the problem
Matt: To parse English you have to know that pizzas have pepperoni, that demonstrators advocate violence, that cats chase mice, and so on. There is no neat, tidy algorithm that will generate all of this knowledge. You can't do any better than to just write down all of these facts. The data is not compressable.
James: You CAN actually, simply because there is patterns, anytime there are patterns, there is regularity, and the ability to compress things. And those things are limited, even if on a super-large scale.
The problem with that is the irregular parts, which have to be handled, and the amount of bad data, which has to be handled.
But a simple example is
ate a pepperoni pizza
ate a tuna pizza
ate a VEGAN SUPREME pizza
ate a Mexican pizza
ate a pineapple pizza
And we can see right off, that these are different types of pizza topping, and we can compress that into a frame easily
Frame Pizza:
can have Toppings: pepperoni, tuna, pineapple
can be Type: vegan supreme, mexican
This does take some work, and does require some good data, but can be done.
We can take that further to gather probabilities, and confidences about the Pizza frame, such that we can determine that a pepperoni pizza is the most likely if a random pizza is ordered.
This does not give a perfect collection of information, but alot can be garnered just from this. This does not solve the AI problem, but does give us a nice building block of Knowledge to start working with.
This is a much preferred method than hand-coding each piece as Cyc has seen, and they are currently coding and using many algorithms now that take advantage of statistical NLP and google to assist and suggest answers, and check the answers they have in place.
There is a simple pattern between Nouns and Verbs as well that can be taken out and extracted with relative ease, and also between Adj and Nouns, and Adv and Verbs.
Ex:
The dog eats, barks, growls, sniffs, attacks, alerts.
That gives us an initial store of information about a dog frame.
Then if given Rover barked at the mailmen. we can programmatically narrow the possibilities about what Actor can fulfill the "bark" role, and see that dogs bark, and are most likely to bark at the mailman, and give a probability, and confidence.
One problem I have with you task of text compression is the stricture that it retain exactly the same text, as opposed to exactly the same Information.
For a computer science data transmission issue the first is important, but for an AI issue the latter is more important.
The dog sniffed the shoes. and The dog smelled the shoes. Is so very close in meaning as to be acceptable representation of the event, and many things can be reduced to their component parts, or even use a more common synonym, or word root.
And it much more important that the system would be able to answer the question What did the dog sniff/smell? as opposed to keeping the data exactly the same.
As long as the answers come out the same, the internal representation could be in chinese or marks in the sand.
James Ratcliff
From: James Ratcliff <[EMAIL PROTECTED]>
To: [email protected]
Sent: Wednesday, November 8, 2006 10:14:43 AM
Subject: Re: [agi] The crux of the problem
Matt: To parse English you have to know that pizzas have pepperoni, that demonstrators advocate violence, that cats chase mice, and so on. There is no neat, tidy algorithm that will generate all of this knowledge. You can't do any better than to just write down all of these facts. The data is not compressable.
James: You CAN actually, simply because there is patterns, anytime there are patterns, there is regularity, and the ability to compress things. And those things are limited, even if on a super-large scale.
The problem with that is the irregular parts, which have to be handled, and the amount of bad data, which has to be handled.
But a simple example is
ate a pepperoni pizza
ate a tuna pizza
ate a VEGAN SUPREME pizza
ate a Mexican pizza
ate a pineapple pizza
And we can see right off, that these are different types of pizza topping, and we can compress that into a frame easily
Frame Pizza:
can have Toppings: pepperoni, tuna, pineapple
can be Type: vegan supreme, mexican
This does take some work, and does require some good data, but can be done.
We can take that further to gather probabilities, and confidences about the Pizza frame, such that we can determine that a pepperoni pizza is the most likely if a random pizza is ordered.
This does not give a perfect collection of information, but alot can be garnered just from this. This does not solve the AI problem, but does give us a nice building block of Knowledge to start working with.
This is a much preferred method than hand-coding each piece as Cyc has seen, and they are currently coding and using many algorithms now that take advantage of statistical NLP and google to assist and suggest answers, and check the answers they have in place.
There is a simple pattern between Nouns and Verbs as well that can be taken out and extracted with relative ease, and also between Adj and Nouns, and Adv and Verbs.
Ex:
The dog eats, barks, growls, sniffs, attacks, alerts.
That gives us an initial store of information about a dog frame.
Then if given Rover barked at the mailmen. we can programmatically narrow the possibilities about what Actor can fulfill the "bark" role, and see that dogs bark, and are most likely to bark at the mailman, and give a probability, and confidence.
One problem I have with you task of text compression is the stricture that it retain exactly the same text, as opposed to exactly the same Information.
For a computer science data transmission issue the first is important, but for an AI issue the latter is more important.
The dog sniffed the shoes. and The dog smelled the shoes. Is so very close in meaning as to be acceptable representation of the event, and many things can be reduced to their component parts, or even use a more common synonym, or word root.
And it much more important that the system would be able to answer the question What did the dog sniff/smell? as opposed to keeping the data exactly the same.
As long as the answers come out the same, the internal representation could be in chinese or marks in the sand.
James Ratcliff
This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?list_id=303
