On Sunday 13 May 2007 08:14:43 am Kingma, D.P. wrote: > John, as I wrote earlier, I'm very interested in learning more about your > particular approach to: > - Concept and pattern representation. (i.e. types of concept, patterns, > relations?)
As I mentioned in another note, (about the tennis ball), a concept is a set of programs that embody your abilities to recognize, manipulate, and predict some thing, and to make inferences about its past. > - Concept creation. (searching for statistically signifcant spatiotemporal > correlations, genetic programming, neural networks, ?) I think that the brain has a lot of what Ben calls "pattern mining" in the hardware; and for our purposes, conventional (including some recent) work in ML/PR seems to be adequate. The main key to the process, though, is to have what programming language theorists call a "reflective tower": for the language to be able to represent itself, and thus reason about its own programs, and thus about the programs it uses to reason about programs, etc, etc. (BTW, this is where standard programming language theory fails us, in that they are very enamored of provably complete logics below the Gödel line. This is all very well for writing provably correct red-black tree implementations, but they are playing with pebbles on the beach, as it were, with a great ocean of truth lying undiscovered before them.) > - Concept revision/optimization. You mention you use search techniques, > could you be a little more specific? I try to represent the data as numeric vectors as much as possible so I can use standard scientific regression methods to create the functions that predict it. Discrete stuff is harder -- AI has tackled it for 50 years with modest results. However, it's my intuition and hope that continuous models underneath (and lots of brute force processing) will provide the "intuition" to conquer the combinatorial explosion. (Consider backprop: the finished neural net is not unlike the same function implemented in perceptrons, except that with a step function instead of a sigmoid you have no traction whatsoever for hill-climbing towards an optimum.) > Unfortunately you did not go into specifics yet. Since you wrote that your > ideas are "firm enough to start doing experiments", I was hoping you could > give a glimpse of the underlying idea's. I.e., what's your "You're speaking > of a "high-level functional reactive programming language" exactly? "If I were going to spend 8 hours chopping down a tree," said Abraham Lincoln, "I'd spend the first 6 sharpening my axe." When you're trying to write an AI, you don't need the distractions and extra work of worrying about storage allocation, process migration and communication, data formats, and so forth -- those wheels have already been invented (as have statistical analysis, regression, and so forth). Build a system with those easily available and interoperable, and worry about AI things thereafter. The easiest languages to "reflect" about are the functional ones -- ones where the semantics resemble math more than assembly language. This is all well and good, except that the pure functional paradigm leaves out the ability to deal with time. (You can't write X=X+1 in a functional language, because it isn't true!) There are 4 main ways people have tried to extend functional languages to deal with time. The first is ad hoc, as in Lisp: mix a functional language with an imperative one. Second, in what were called "applicative state transition" systems, model a big state machine and use the functional language to write the transition function. Third, the current favorite in the PLT crowd, is category theoretic monads: write a function that computes a list that is a trace of the behavior you want the program to enact. And finally, functional reactive programming, is to write a function that is interpreted as a circuit, where each value is actually a signal that varies in time. This, it turns out, is how the physicists and engineers have done it all along: a systems-and-signals circuit in control theory or cybernetics is isomorphic to a FR program. > One reason I'm interested is that there are many approaches to unsupervised > learning of physical (or just graphical) concepts, and I'm thinking of Serre > et al, Hawkins, neural network specialists (Geoffrey Hinton) and there could > be many more. But none of these theories I know of are strong enough to > extract high-level concepts such as 'gravity'. Neither is the average human being -- Newton was one of the great geniuses. What the average human does is expect things to fall down. Experiments with people who haven't studied physics show that they have a very poor ability to predict what will happen in simple experiments. (For example, more than half of undergrads and non-science faculty, when asked to draw the path that would be taken by a ball rolling off the edge of a table, don't draw anything resembling a parabola.) > Now when I imagine a hypothetical system capable of extracting such thing as > 'gravity', he would have to go through a process of many stages, one of the > first to learn about the general spatial phenomenon 'object', later the > temporal phenomenon 'velocity', and would eventually find out that the > vertical element of the velocity vector is ever decreasing with constant > amount. Now, a system that is capable of doing this without any prior > knowledge is pretty damn interesting, and you're promising it. Actually I haven't promised anybody anything, but my stated intentions are to take my ideas in this direction and experiment with them. But, as I pointed out, it's highly unrealistic to expect the system to derive the laws of physics in a recognizable form. Consider a baseball outfielder: he doesn't know ma = F = GmM/r^2, much less enough aerodynamics, to predict the fall of the ball even if he could estimate its initial velocity and angle. What he does is associate the quality of the sound of the crack of the bat, the feel of the wind on his face, the temperature, and heaven only knows what else, with the remembered fall from hundreds or thousands of trials. Tommy might do a little better because I'm likely to build in biases a lot more like the ontology of mathematical physics than the extremely complex set that evolution provided us (if for no other reason than that I know the one but not the other!). Even so, I'd be deliriously happy to see him separate that association to even one intermediate invariant. Josh ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=231415&user_secret=fabd7936
