On Fri, May 3, 2013 at 11:05 AM, Mike Tintner <tint...@blueyonder.co.uk> wrote: > > Matt: The idea is that for a sentence like "Put the green ball on the red > block.", a bag of words model is insufficient > > 1. "Bag of words model" is a good phrase - basically we're always talking > about some kind of semantic network/database, no?
What I mean by the bag of words model is that word order doesn't matter. If I want to build a question answering machine, then I load it with millions of sentences containing facts like "the world is round". In the bag of words model, I answer your question my matching words in the question like "what shape is the world?" to words in the answer. Since "world" matches, I give the correct answer. But you could have asked any other question containing the word "world" and I would have given the same answer. For example, it would give the same answer to "the world is flat" by correcting you. This techniques sometimes fails, for example, "Who in the world is John Q. Public?" This is one of many techniques that Google uses. This, and many other techniques would be used to assign points to each possible response, and then the result would be ranked with the highest scoring responses on top. Another technique is to match pairs of words, or to use a thesaurus to match "shape" or "flat" to "round". Another is to have users rank answers as more or less credible by counting links (the PageRank algorithm). And probably lots of others that are trade secrets. Combining lots of techniques is called an ensemble method. It is generally quite effective, producing a result better than any of its components. > 4. COGNITIVE SYNERGY (switching to other thread) is baloney. If you add three > progs together and get three bags of building blocks [or words] what you end > up with is three and only three kinds of structures. My criticism of cognitive synergy is that it is contrary to the known good performance of ensemble methods for machine learning. Ben claims that all of the parts have to be in place to show intelligence. But that isn't the case. If some parts are missing, you just get inferior results. There is a law of diminishing returns. Initially you get good progress as you add components, regardless of the order in which you add them. Adding more components gives you marginally better results. This has misled a lot of AI research down dead-end paths. It is interesting to compare SHRDLU with OpenCog. Both use virtual words that are simplified versions of reality. Both understand simple, grammatically correct sentences from a small subset of natural language that are relevant to those worlds. Note the similarity with SHRDLU's Blocks World. http://www.youtube.com/watch?v=ii-qdubNsx0 The video was made in 2009, but as far as I can tell, there has not been much progress since then. Ben can claim cognitive synergy as an excuse. There has been a lot of work, but all of the progress has been invisible so far. Once the software is finished, everything will come together. I don't buy it, and apparently investors don't either. No major software project has ever succeeded without showing progress along the way. It's not that they aren't trying. Ben obviously knows a lot about the problem and has made it his life's work to solve it. But he needs to be realistic about the difficulty of the problem and set the bar lower. He is far away from what he needs in computing hardware, software, training data, and dollars. Stop relying on wishful intuition and do the math. -- Matt Mahoney, mattmahone...@gmail.com ------------------------------------------- AGI Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/21088071-f452e424 Modify Your Subscription: https://www.listbox.com/member/?member_id=21088071&id_secret=21088071-58d57657 Powered by Listbox: http://www.listbox.com
