On Thu, Mar 28, 2013 at 2:09 AM, Steve Richfield <[email protected]>wrote:
> Aaron, > > On Wed, Mar 27, 2013 at 11:21 AM, Aaron Hosford <[email protected]>wrote: > >> >> There are two layers of semantic information: the raw "literal" meaning, >> and the processed/integrated meaning. Semantic structure defines the raw >> meaning in a well-defined way. Idioms are remappings of literal meanings to >> other domains based on frequently used analogies. They exist because it's >> easier to reuse a familiar analogy than spend unnecessary minutes conveying >> a precise non-analogical meaning directly. >> > > This is true for around half or a little more of idioms, that either were > created to be idioms, e.g. "the best government that money can buy" or are > associated with now-obsolete practices like "don't throw the baby out with > the bathwater". The rest of the idioms are similes, e.g. "hoist by his own > petard", which in a military setting is NOT an idiom, or are "obvious" > exaggerations, e.g. "I worked my fingers to the bone" writing this posting. > Many of the idioms relating to now-obsolete practices are also similes. > With similes, you must disambiguate enough to decide whether the idiom is > really an idiom at all. > > A case might be made that a listener should be able to understand similes > and obvious exaggerations. This would be really nice, but is WAY beyond > what anyone now has any idea how to do. > True, but I consider these as well used similes or exaggerations, not idioms, and give them the separate treatment they deserve. Understanding similes and exaggerations is not way beyond what anyone has any idea how to do, because I already have ideas as to how to handle them. It comes back to the difference between intended meaning and literal meaning. If the system attempts to make sense of an exaggeration or simile, it need only recognize that a literal application leads to unsupportable conclusions -- in other words, the system needs a BS detector, in the form of reasoning and experience against which the system can compare to identify unlikely scenarios. Once such an unsupportable conclusion has been identified, the system can search the space of likely alternative meanings for the most reasonable one according to its own internal model of reality. This method on its own would not be difficult to implement; it's just that the internal model of reality has not yet been implemented. > And in the world of AGI. I'm not looking to make a chatbot, or a tool >> that pulls out only a handful of keywords and uses shrewd guesses to select >> from among a handful of well-defined, pre-programmed jobs like Siri. I'm >> trying to build something that actually understands and thinks about what >> is said to it, and generates original responses. >> > > What good is it to work on developing something, when there is little > understanding of the task, no apparent use for information outside of a > computer's understanding, no apparent need for the product, and no market > for it? We seem to be getting into the area of mental masturbation here. > Because it's really cool, for one. But putting that aside, I strongly suspect lucrative uses will pop out of the woodwork once the technology is available. Not everything is useful invented to serve an obvious pre-existing need. > I don't think so!!! Have you seen my own theory of language, based on >> placement and payload? It is in the patent application, or I'd be glad to >> post it (again) here. My theory is that semantics is only sensitive to >> structure within a payload, which is usually just a small part of a >> sentence. >> >>> >> No, I didn't see it. The short description you provide sounds familiar to >> my own theory, though: Most parts of a sentence serve only to identify the >> correct entities the sentence is about, because addressing is the hard part >> in a semantic memory. Subtracting out the information that is needed for >> addressing, you are typically only left with a small amount of information >> -- the relationship between the addressed entities that is asserted or >> denied. However, given the structure of human semantic memory, there is no >> way around this, and so human language is as efficient as it can be given >> the task at hand. One of the first signs that you have misunderstood >> someone is when they provide information that is seemingly superfluous, >> > > There MUST ALWAYS be superfluous information!!! You can't know exactly > what a listener knows and doesn't know. If you don't say enough extra to > make sure there is a solid frame of reference, which requires saying some > extra to cover things your listener might not know, then your listener > won't even know what you are talking about. > Yes, of course. But the guiding principle is that the listener will say things in the way that gives the shortest expected communication time. Sometimes that means adding in a few details that may not be needed; other times it means leaving things out that would take of wasted explanation if there's a good chance the listener already knows it. The listener can use small deviations to update their own model of the speaker's expectations about the listener's knowledge. Larger deviations can be used to detect misunderstandings. > and so it seems clear to me that maximal efficiency is a guideline used in >> actual human language generation. >> > > I think you should rethink this, as some redundancy is absolutely > necessary. > See above. > As I distantly recall, people were studying language structure in an >> attempt to understand how we think MANY years ago, back in the 1950-1960s. >> It didn't seem to go anywhere interesting. >> >>> >> Because analytical reasoning is more about efficient representation than >> simple logic. Humans think about the world in terms of objects, types, >> relationships, events, settings, attributes, and degrees. This is a rich >> representational framework, and consequently requires a rich set of >> semantic structures. The advantage human beings have over computers is that >> we are astoundingly good at identifying things; we are only mediocre at >> thinking about those things once we have identified them, but that's ok >> because most things don't require deep analysis once they are effectively >> identified. As a result, most semantic structures in language are devoted >> to the identification of things. These structures are not static, but >> rather culturally evolve to maximize utility over time, and so there is a >> divergence between different languages. However, I challenge you to find a >> language that doesn't refer to objects, types, relationships, events, >> settings, attributes, and degrees in some way or another, or that uses >> other kinds of semantic structures. (This list may not be complete, but I >> am confident it is at least nearly so.) >> > > There are some languages where the words appear more in the order of > importance than in complex relationships, leaving lots of ambiguities to > resolve. > These languages typically have more complex word forms to indicate grammatical roles, unlike English, which primarily uses word ordering for that task. The information is there, in one form or another. You are right that semantic processing slows things down. I think the best >>>> way to solve the problem is to process syntax and semantics separately, >>>> just as the brain does, rather than trying to accomplish both at once. >>>> (Much research has shown that the two faculties are indeed distinct, both >>>> in parsing and generation.) A small set of candidate syntactical structures >>>> can often be computed very quickly based on nothing more than word classes >>>> (verb, noun, preposition, etc.) together with special rules keyed off of >>>> special words -- this is not the bottleneck. Once a set of candidate >>>> syntactical structures has been identified, the semantic structure is often >>>> sufficiently constrained that only minimal processing and/or default >>>> assumptions based on familiarity is enough to rapidly and effectively >>>> reduce all combinations of legal syntax + semantics to only one or two >>>> choices, but AFAIK no one seems to be working on this other than myself. >>>> >>> >>> I also address the issue of language granularity in my patent, which I >>> can also re-post here. >>> >>> You can only say a finite number of things in any particular language. >>> If what you want to say isn't precisely one of those things (and it seldom >>> is), you MUST say something different than what you would rather say, but >>> close enough for a listener to get the essence of your message. >>> >>> There is no "simple" logic for either creating erroneous messages from >>> correct intentions, or for extracting the correct intentions from erroneous >>> messages. Everyone working on NL seems to make the "obvious" presumptions >>> that there is SO bidirectional way of translating back and forth between >>> intention and text. This does NOT exist. >>> >> >> The not-so-simple logic is to find the closest meaning to what was said >> that actually makes sense within the context of the listener's knowledge >> about the topic and the speaker's awareness thereof. Likewise, in language >> generation, we find the simplest way (to our knowledge) to say something >> for which the listener would correctly adjust the meaning to what was >> actually intended. This all fits into the efficiency framework; it often >> takes more time to say something precisely than it does for the listener to >> mentally repair an imprecise statement. It's all about using semantic >> distance (dissimilarity) to maximize efficiency. >> > > This is the way that present-day speech recognition like Dragon > NaturallySpeaking works. The problem with this approach is that there is no > reason to ever say the obvious, and MUCH reason to state when crazy things > happen. This turns probabilities on their head. > Well I'm not going to hear a sentence and expect it to mean something I already know. I'm going to expect it to mean something the speaker thinks I don't know, and that fits in with what I already know. Maybe dissimilarity is the wrong term. Dissonance probably fits better. > To "look under the hood" of this, I strongly recommend that you look at >>> a copy of one of the English translations of the Koran (pages are viewable >>> on Amazon). In these you will see the original Arabic, the nearest-meaning >>> English, and footnotes that often exceed the combined total length of the >>> Arabic and English that explain the differences between the nearest-meaning >>> English and what was apparently meant in the Arabic, which of course there >>> is continuing dispute over. >>> >>> For example, does the Koran say that "women should dress modestly >>> (whatever that means)", or that "women should cover their modesty (not run >>> around naked)"? Of course these are two VERY different interpretations. >>> Unfortunately, there were no dictionaries way back then, so there is no way >>> to resolve this argument, short of breaking out the firearms, which they do >>> all the time. >>> >>>> >>>> The key here is to recognize that disambiguation requires integration >>>> with previously acquired knowledge, and can not be computed for the >>>> language fragment in isolation. Thus, to make a system that can >>>> disambiguate language is to make a system that can understand language. >>>> >>> >>> That is a BIG jump. All that appears to be needed is to identify past >>> and present subject domains for a given author. >>> >> >> So if I tell you a story about two men, and in the middle of it, I say, >> "He gave him the key," you will be able to disambiguate who gave whom the >> key based on past and present subject domains for the author? Or if you >> don't see anaphora resolution as disambiguation, suppose the key opens the >> door to a much discussed computer room. How is a word frequency-based >> disambiguation going to identify the correct meaning for "key" as an object >> you hold in your hand, versus something associated with computers, like a >> PGP key? >> > > As I said before, this cannot work well until AGIs walk among us. > There are two ways to know the difference between a door key and a PGP key: One is to use them yourself, accumulating real world experience. The other is to use this amazingly powerful tool evolution has provided to we humans that allows us to transfer knowledge without actually accumulating experience, called language. > There is no substitute for understanding in language disambiguation, if >> you're going to do it reliably. >> > > That's the gold standard. However, after doing the work leading up to my > patent, even if you had the perfect algorithm and only needed to code it, > there won't be a computer for decades on which to run your program. > > Dragon NaturallySpeaking's "model" for "understanding" is just a lot of > sound and word pairing statistics - FAR short of the sort of the level of > understanding you are talking about. Still, it can't work acceptably well > at talking speed on gigahertz processors. Present processors are MANY > orders of magnitude too slow to process human-level understanding. > > So, unless you can find some tricks like I have found for my approach to > speed your approach WAY up, you will NOT be creating anything that works > until you are MUCH older. This might be a good project for your > grandchildren. > First I'm going to make it work. Then I'll work on making it fast. I *like *doing this, so I see no point in waiting. Even if I don't succeed, I will have learned something that I can pass on to those that come after me. > Statistical approaches that look for common word combinations for >>>> disambiguation are just naive attempts to summarize previously acquired >>>> knowledge, but they fail to capture that knowledge in sufficient depth to >>>> do the job reliably. >>>> >>> >>> Computers can only "compute" on what they have been programmed to >>> compute upon, whereupon selective word examination is adequate. >>> Alternatively, a computer can pass information through for a human reader >>> to do their own disambiguation, as in the case of automatic translation. >>> >>> >>>> There is no shortcut to understanding. >>>> >>> >>> I don't think that "understanding" can be accomplished, before machines >>> walk among us. Also, I don't think the "understanding" is needed for the >>> vast majority of PRACTICAL applications. >>> >> >> Practical applications are dependent both on what is needed and what is >> possible. Often, we don't recognize practical applications until they >> become possible. I think the ability to truly understand language will have >> many, many practical applications, once it becomes possible. >> > > The Russian Academy of Science has been pouring everything they have into > this, to create a better translator. This is a laudable goal, if and when > it can be achieved. > > >> You are probably right that machines must "walk among us" to fully >> understand what we say, but with resources like Wikipedia and the rest of >> the internet, there is a long way to go before that becomes a bottleneck. I >> can read about the Jabberwocky and still reason about it even though I have >> no idea what one is, so long as I stick to the things I'm told about it. >> > > Only because you can relate it to your real-world experiences, that a > computer confined to text will NOT have. > I'm of the opinion that, while knowledge gleaned through language cannot make up for real world experience, in sufficient quantity it can act as a substitute for many tasks. By the time I hit that wall, I will have begun working on integrating direct perception into my system. > Likewise, there is no reason a program could not reason about objects it >> has never seen, so long as it sticks to what it's told >> > > Enough of which is wrong to REALLY screw things up. > > I gave up on DrEliza carrying on back-and-forth conversations because > everything fell apart when someone made a false statement. It is damn near > impossible to accurately correct a misstatement with another statement. > Instead, I switched to having them edit their original statement to > eliminate their errors. > > There are enough errors in Wikipedia to brain-short ANY system. > Yes, this is definitely an issue. However, my system is designed to reason with contradictory evidence from the ground up, meaning that if it hears two different, conflicting facts, it will be able to weigh which one seems more reasonable, and seek out further evidence to bolster its conclusions. > -- and bothers to truly integrate that information rather than processing >> it superficially. >> > > This idea won't even survive desk checking. > Why do you believe that? > I think this discussion also gives some good insight into the >>>> combinatorial explosion problem that Jim keeps coming back to. If I have a >>>> problem that requires me to search 1,000,000 different combinations, but I >>>> can factor that search space into 3 (nearly) independent pieces, each with >>>> 100 combinations, then the total number of choices I need to search becomes >>>> only 300, plus minor processing to handle the sparse non-independent cases. >>>> This is why divide-and-conquer works. >>>> >>> >>> You are discussing this in a vacuum - like what application are you >>> looking at? I don't see how this discussion fits any present-day >>> application needs. >>> >> >> This is really directed at Jim, not at you. He frequently brings up >> combinatorial explosions as the fundamental bottleneck to AGI, and insists >> there must be some kind of breakthrough before we can get past it. I think >> the solution has been staring us in the face for ages, and we just need to >> bother using common sense techniques like divide-and-conquer in our >> problem-solving programs instead of just in our own direct problem solving. >> > > I don't think divide and conquer is enough to live with present-day > hardware. Also, divide and conquer does NOT mesh with best fit methods, but > rather demands that decisions are made during the "divide" phase that, when > wrong, dooms the process. > People are not always right, but we are often close enough. Sometimes we must give up guarantees of correctness to have a shot of being right at all. I think this is one of the main barriers holding back AGI. > In dividing syntactic parsing from semantic disambiguation, a similar >>>> effect applies. Sometimes it is better to temporarily work with *less* >>>> information, if that allows you to factor the problem. >>>> >>> >>> In most real-world applications (not including automatic translation), >>> only a tiny fraction of the text needs to be acted on. The rest can be >>> summarily ignored. >>> >> >> Not including automatic translation, and AGI -- if you're really going to >> do AGI and not just superficially fake it for a small problem domain or >> shallow stimulus/response. >> > > AGI is NOT going to be done on any sort of PC (because even supercomputers > are too slow), and is NOT going to be done during the next few years > (because there is nothing on the drawing boards that could conceivably work > in either hardware or software). > > When it becomes conceivable to run such programs, I may get more > interested in writing them. > Suit yourself. > Semantic knowledge does indeed occasionally determine which of several >>>> syntactic structures is the correct one, but those non-independent cases >>>> are sufficiently sparse that factoring still provides an enormous speedup. >>>> I expect the same principle to hold in most domains, and I believe the >>>> ability to effectively factor problems is core to any successful GI, >>>> artificial or not. >>>> >>> >>> Now, add that only the factors that actually have applicable words >>> present are even considered, and it will run VERY fast. Again, my point is >>> that looking at least frequently used elements gives you a TREMENDOUS speed >>> advantage, in addition to other advantages you may find. >>> >> >> I'm already using something similar to this technique in my system. Not >> so much in parsing, but in understanding/disambiguation once the parsing is >> complete and the base-level semantic structure has been identified. >> > > It appears that great minds think alike. > > Steve > > *AGI* | Archives <https://www.listbox.com/member/archive/303/=now> > <https://www.listbox.com/member/archive/rss/303/23050605-2da819ff> | > Modify<https://www.listbox.com/member/?&;>Your Subscription > <http://www.listbox.com> > ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
