Matt, On Sat, Mar 23, 2013 at 6:04 PM, Matt Mahoney <[email protected]>wrote:
> On Fri, Mar 22, 2013 at 9:35 PM, Steve Richfield > <[email protected]> wrote: > > There are two new tricks in my method (which need each other to work), > both of which appear to be entirely new. > > > > 1. Triggering analysis based on the presence of a least frequently used > word in a rule. > ... > > 2. Putting pointers to rules into prioritized queues, to put the mess > made from trying to make sense when only <1% of the rules are evaluated. > ... > > I agree you can probably patent these. That doesn't mean it hasn't > been done before and hasn't already been patented. After striking out trying to find anything like it on USPTO.com, I looked to ANTHING about parsing, and I ONLY found patents on speech recognition. Of course it is possible that one of those patents has some cover-all boilerplate at the end that might extend it to NL text. If it has been patented before, I am pretty sure it would be via extension of a speech recognition patent, and they have MUCH fuzzier input to process, so they are buried in Bayesian math for all their computation. I also claim a Bayesian method, but only at higher levels in the analysis, whereas speech recognition is fuzzy right from the bottom levels. > I'm just saying it > takes a great deal of effort to check, so the USPTO will probably take > the easy way out and issue the patent. > I agree. > > >> Blinding speed remains to be seen. > > > > I think it can be guesstimated with enough accuracy to make go/no-go > decisions. > > In theory, Cyc should be fast because it was designed to run on 1980's > computers. The issue is how many rules do you need? > Cyc wants rules for EVERYTHING, which might be reasonable in a primitive and disorganized world where anything is possible, but is EXTREMELY inefficient when understanding things in a modern world, e.g. do you REALLY need to understand how your TV works to use it? DrEliza seeks to understand failure cause-and-effect chains that wind themselves though our hypercomplex world, without touching the 99.999% that works right. That is a (thousands to one)^(cause-and-effect chain links) simplification, which is something like a grain of sand compared with the earth improvement in efficiency over Cyc. However, DrEliza was still falling into the exploding combinatorial complexity trap, which is what pushed me in the present direction. > > > I don't immediately see how data compression relates to understanding > text, though I DO see that some understanding might help with the > compression. > > A simple test to see if you understand some text is to see if you can > guess missing words. What happens if you take the presently best compression methods, search for symbols that if changed would result in the greatest improvement in compression, and change those symbols to improve the compression. Does this tend to improve or detract from the quality of the result? Has anyone tried this? There might be a surprise waiting. Compression measures the same thing. > That is where Shannon started out measuring information content e.g.byscrambling individual letters to see how much he could scramble, and still have people able to identify the words. > > > Lots of people have talked about "loss" in compression. I want to see > GAIN. In a gaining compressor, you might put in Wikipedia, and get out > Wikipedia with fewer misspelled words, better grammar, and some semantic > errors corrected. I believe that the competition should demand no NET loss, > i.e. it should gain at least as much as it looses. After all, what is the > value in more cheaply representing someone's typing errors?!!! > > Errors compress poorly because they represent unexpected symbols in > the text. So yes, you could probably correct text by substituting > symbols that compress more easily, if the model is good. > Yes, that is what I was thinking. > > >> Anyway, I would not be surprised if Google, Bing, Facebook, etc. are > >> already using similar techniques in their language models. > > > > Models - yes. Methods - I doubt it. In any case, if they haven't filed > for a patent on it, I will still own it, because "first to invent" is gone > EXCEPT for applications like mine that were filed before March 16, and is > now replaced by "first to file", and I have already filed. > > > > Hence, it isn't at all inconceivable that I could now own what they are > now working on!!! > > > > I wonder if I should send them a letter inquiring whether one of us is, > or will be, infringing on the other? > > Do you really think that these companies don't already own hundreds or > thousands of patents on the methods they invented and are using? I > suppose if your price is low, they will pay it to avoid the hassle. If > you ask for too much, they will point out prior art to invalidate your > patent. > This sounds like a game I can't lose!!! If the patent won't stand anyway, challenging them would get THEM to do the hard work of finding any prior art, before I spend any more in this direction. If they fail, then I will have vetted my approach and it WILL be worth lots of money. Hence, I should demand a fortune, and let the chips fall where they may. The odds would almost certainly be better than a lottery ticket. Do you see any flaws in this obviously non-conservative logic? > > >> You might > >> actually want to build something before making bold claims. > > > > > > Why? > > > > Most astute IP programs patent, build, and then patent again. I have > simply taken the first step and am gearing up for the second step, the > first task of which is to find partners, raise money, etc. THAT requires > making the bold claims needed to get people interested enough to > participate. Are you (or anyone else reading this posting) interested in > being a part of this? > > No, because I don't believe your bold claims. > The BIG question is whether it works, and whether others believe it. The thing missing in my (and other) parsing approach(es) is a canonical form to represent syntax and semantics, with plenty of hooks to attach new code to do new things. BNF would require a **LOT** of extension. My present plan is to get people from various areas to work on such a representation. I am now creating a paper for WORLDCOMP about the next steps, the initial submitted draft of which I will post here in a week or two for comment. In the process, all the right people will kick its tires, and the people who really count will have voted with their hands and/or feet. As you can see from my comments, I am of a mindset to see if this flies or flops as quickly as possible and go for the big bucks, rather than keep a low profile and go for the small bucks. > > If you want to make money, then patent something trivial and don't > tell anyone and wait for a big company to step on it. > This sounds like you have been listening to my attorney?!!! > http://en.wikipedia.org/wiki/JPEG#Patent_issues > The patent claim in this case was the "invention" of using a single > code to represent a run of zeros followed by a non-zero value. Who > would have thought it would be worth $105 million? The claim wasn't > even valid, due to prior art. > They screwed up. The trick is to just sue medium-sized companies for lunch money, until the patent lapses. THEN sue the big guys for past infringement. Also, look for settlements to avoid testing the patent. Further, make a deal with one of them to fund the suits, in return for a free ride. That way, you collect the money that can be collected now without a court hassle, and risk invalidation only after the patent lapses and you have already collected all that you can collect without a court hassle. > > > I think I posted that I was expecting it to take a linguist-decade. > > You might want to pay attention to what Kurzweil is doing at Google. > He has put together a team of several top researchers to tackle > exactly the natural language problem. He has access to a model with > 300 million concepts, and an awful lot of computing power. > Sounds like he is doing SOMETHING the hard way, which is a standard problem with people who think they have SO much money that they don't have to worry about scaling issues. Our hundred billion neurons is the result of a hundred million of years of optimization. People who think they can "throw iron" at AI problems are just wasting everyone's time. > > > Also note that Watson didn't need to understand what was happening, just > run like a mouse in a maze though the information. > > > >> It runs on a few thousand CPU cores. > > > > Of course, since their selected challenge is pretty close to the > traveling salesman problem, only each "trip" is a link in a gigantic > database. > > No, that's not how it works. They run hundreds of models in parallel > and combine the answers. > http://www.aaai.org/Magazine/Watson/watson.php > Thanks. I just printed this out, to be read during the next few days. > > > Cyc will never ever do anything useful. > > I agree. So why are you proposing a rule based system too? > The trick is to use rules to identify the exceptions and problems, and ignore the rest of the real world that bogs down Cyc. That seems to be what we do in our own brains, as even 100 billion neurons couldn't begin to be able to track all the things that are working just as they should. Steve ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
