Richard, It seems we both agree that systems, like Copycat's, that relatively successfully harness and control complexity for a desired purpose need to be explored on a much larger scale to better understand what, if any, problems result from such increases in scale. One would expect that such scale-related problems will occur, but how hard they will be to solve is the issue.
I would expect that most intelligently designed large Novamente-type systems would fall into this category. In my own ideas for a roughly Novamente-type system, I have been seeking a relatively uniform very very rough approximation of the cortico/basil-ganglia/thalamic architecture, all operating under the control of set of top level goals and a system for administering +- experiential related rewards. This architecture would basically be similar across most of the machine, to eliminate the number of design choices and/or non experientially set parameters. Much of the system's complexity would be experientially learned complexity, much of the learned goals would be behaviors or states that have been shown by learned experience to serve the top level goals. This strong experiential bias would be one of the guiding hands (actually it would be a set of millions of such guiding hands) that hopefully would tend to keep the system from suddenly going weird on us. As I said before, in my system most new thoughts and behaviors would be created by processes of recollection from similar contexts of various scopes, of generalizations of such recollections, of context specific instantiations of such generalizations, and of probabilistically favored mappings and stitching together of such generalizations or pieces of such recollections -- all with a certain amount of randomness thrown in, as in Copycat. Yes, there would be a tremendous number of degrees of freedom, but there would be a tremendous number of sources of guidance and review from the best matching prior experiences of the past successes and failures of the most similar perceptions, thoughts, or behaviors in the most similar contexts. With such guidance, there is reason to believe that even a system large enough to compute human-level world knowledge would stay largely within the realm of common sense and not freak out. It should have enough randomness to fairly often think strange new thoughts, but it should have enough common-sense from its vase experiences to judge roughly as well as a human when to, and when not to, act on such strange new ideas. It is my guess that there is a good chance the types guiding hands that make copycat work can be successfully extended and multiplied and applied to allow a Novamente-type system to successfully, usefully, and continuously compute from a human-level world knowledge. But I agree totally with what I think you are saying, i.e., that we should be seeking to constantly try such architecture in larger and larger projects to better understand the potential gotchas and to better understand the type of guiding hands such systems need to avoid the undesired effects of complexity. I would appreciate knowing what parts of the above you agree and disagree with. And if you have some particular suggestion for how the best extrapolate the Copycat approach not mentioned above, please tell me. Ed Porter -----Original Message----- From: Richard Loosemore [mailto:[EMAIL PROTECTED] Sent: Friday, December 07, 2007 10:18 AM To: [email protected] Subject: Re: [agi] Evidence complexity can be controlled by guiding hands Ed Porter wrote: >> RICHARD LOOSEMORE=====> At the cognitive level, on the other hand, there is > > a strong possibility that what happens when the mind builds a model of > some situation, it gets a large nummber of concepts to come together and > try to relax into a stable representation, and that relaxation process > is potentially sensitive to complex effects (some small parameter in the > design of the "concepts" could play a crucial role in ensuring that the > relaxation process goes properly, for example) > ED PORTER=====> Copycat uses a variant of simulated annealing to do its > relaxation process, except it is actually a much more chaotic relaxation > process than many (e.g., much more than Hecht-Neilsen's Confabulation), > because it involves millions of separate codlets being generated to score, > decide the value of, and to add or remove elements from a graph, that labels > grouping and relationships in the initial string, and between the example > initial string and the solution initial string, and between the example > initial string and the example changed string, and between the both the > solution initial string and the example changed string and the solution > changed string, as well as constructing the solution changed string itself > during this process. > > Each of the labelings and mapping links is made by a separate small program > called a codelet. Codelets are chosen in a weighted random manner. And one > codelet can clobber the work done by another. The ratio of importance > between some fitness weighting and pure randomness in the picking of codlets > varyies with temperature, which is a measure of overall labeling, mapping, > and solution fit, which tends to go down over time as the system moves > toward a coherent solution. But it can go up if the system starts settling > into a solution that creates a mapping or labeling flaw, at which time more > random codelets will be created and randomly change the system, but with the > changes being more likely in the parts of the graph or labeling that have > the least good fit, and thus requires the least energy to kick apart. > > Despite this very chaotic process, and the fact this process is sensitive to > complex dynamic effects that enable a slight change of state to causes it to > settle into different solutions, as Richard mentioned above, the weighting > of the system, which varies dynamically in a context sensitive way, causes > most of the solutions that it settles into to be appropriate, although they > may be quite different. > > For example, for the copycat problem where the goal is to change "ijkk" in a > manner similar to that in which aabc was changed to produce aabd, which > problem can be represented as > > ex aabc --> aabd > ijkk --> ? > > On one thousand runs the results were > # of occurrence result temperature > 1 612 were ijll 29 > 2 198 were ijkl 49 > 3 121 were jjkk 47 > 4 47 were hjkk 19 > 5 9 were jkkk 42 > 6 6 were ijkd 57 > 7 3 were ijdd 46 > 8 3 were ijkk 69 > 9 1 was djkk 58 > > ===EXPLANATION OF ANALOGY IN EACH SOLUTION=== > ex-last char in string has alphabet number incremented > 1-last set of the same chars in each string had alphabet number incremented > 2-last char in each string had alphabet number incremented > 3-one end char in each string had alphabet number incremented > 4-one end char in each string had alphabet number changed by one > 5-set of chars in string had alphabet numbers incremented > 6-last char in each string is changed to d > 7-last set of same chars in each initial string was changed to d > 8-last char in each string had alphabet number changed by a value of zero or > one > 9-one char on end of string was changed to d > > So you see that each of the changes except solution 8, which had the worst > temperature, meaning the system felt it was the worst "fit" actually > captured an analogous change. If temperature were used to filter out the > misfits, none of the runs would have produced a non-analogy. So despite > the chaotic nature of the system, it almost always settled on a labeling, > graphing, and solution that was appropriate, and when it didn't it knew it > didn't, because of the systems measure of analogical fit. > > Although this definitely is a toy problem, it might have as much potential > for "complexity" as the game of life, in terms of its number of components > (if you count its codlets), its computations, and its non-linearities. I > was told by somebody who worked with Hofstader that individual copycat > solutions running on unoptimized LISP code on roughly 1990s Sun work > stations normally took between about half hour to a major fraction of a day. > > > The difference between this and the game of life is that has been designed > to work. Despite its somewhat chaotic manner of approaching the problem, it > has weights, many of which are contextual, that guide the chaotic process, > in a very (very) roughly analogous way to that in which Adam Smith's > invisible hand guides the complexities of a market economy. > > Its processes of labeling, creating graphs of labels, graph matching, graph > extrapolating, and varying measures of similarity in a context sensitive way > are all processes that a Novamente type system would be using. > > All though this is a very very small example, it is a positive one. It > indicates that a collection of relatively free running non-linear > interactions (i.e., its codelets) can operate relatively reliably in an > intended manner -- despite the computational irreducibility of the operation > and the relatively large number of non-linear components -- given the > guiding hand provided by the systems goals, weights, and measures. > > Ed Porter > > (disclaimer. I am writing this late at night when I am very tired and > largely from my memory of having read the copycat chapter of "Fluid Concepts > and Creative Analogies" multiple times, but none more recently than several > years ago. Pei Wang may be able to correct me if I have made any gross > mis-descriptions.) In spite of that, this is one of the most insightful posts I have seen you make. This is *very* close to the strategy that I am proposing, and it illustrates well the basic idea that intelligence can be the result from a system that is complex. I very much approve of Hofstadter's work. The way it relates to my proposal is that it is a very simple model of one process, in a fairly restricted context, so what needs to happen is that this basic idea needs to be part of a much larger and more general system, if it is ever going to become an AGI (which by definition must be a complete intelligence). Now, in order to become a complete intelligence, this model is going to have to be expanded so hugely that the number of possible parameter and architecture choices (the number of degrees of freedom in the design) will expand greatly. As you know, as the number of degrees of freedom goes up, the number of possibilities to be explored goes up in power-law fashion (if there are two design choices for each degree of freedom then 1 degree implies 2 posibilities to explore, 2 degrees means 4, 3 degrees means 8, and so on). So when you present this beautiful piece of work by the Fargonauts, and wonder whether this perhaps contradicts my claim (as perhaps you might do), I say that this is exactly what we all should be doing, but as we scale up this approach we find that the possible design choices will force us all to be extremely organized about how we do the research, because there are potentially a massive number of combinations of design choices that have to be explored. If you want to understand my approach (as opposed to my criticisms of what I feel are bad approaches), then this Hofstadter example, together with my suggested extrapolation of it, is a *perfect* illustration of what I am doing, and what I am suggesting that we need to collectively get organized to do. Richard Loosemore ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?&; ----- This list is sponsored by AGIRI: http://www.agiri.org/email To unsubscribe or change your options, please go to: http://v2.listbox.com/member/?member_id=8660244&id_secret=73636676-ae3176
<<attachment: winmail.dat>>
