Thanks to Ben and Vlad for their help answering my question about how to estimate the number of node assemblies A(N,O,S) one can get from a total set of N nodes, where each assembly has a size of S, and a maximum overlap with any other set of O. I am sorry I did not response sooner but I spend a fair amount of time reviewing the tables recited in the below wikipedia article and in thinking about how one might obtain more relevant information, and I went away for the weekend.
It appears Ben and Vlad are right that the constant-weight code formula A(n,d,w) described at <http://en.wikipedia.org/wiki/Constant-weight_code> http://en.wikipedia.org/wiki/Constant-weight_code, is highly relevant to my question, if you take into account Vlad's suggestion that you fill in the variable slots in the constant-weight code formula A(n,d,w) with the parameters N,O, and S in my formula as follows; n = N d = 2(S-O+1) w = S I understand why you multiply (S-O+1) time 2 to get the hamming distance, i.e., because whenever comparing two sets, whatever non-overlap you had in one set, you would have an equal non-overlap from the other compared set to add to the hamming distance between the two sets being compared. I also don't understand whether A(n,d,w) is the number of sets where the hamming distance is exactly d (as it would seem from the text of <http://en.wikipedia.org/wiki/Constant-weight_code> http://en.wikipedia.org/wiki/Constant-weight_code ), or whether it is the number of set where the hamming distance is d or less. If the former case is true then the lower bounds given in the tables would actually be lower than the actual lower bounds for the question I asked, which would correspond to all cases where the hamming distance is d or less. IT WAS INTERESTING TO NOTE THAT THE WIKI ARTICLE SAID "APART FROM SOME TRIVIAL OBSERVATIONS, IT IS GENERALLY IMPOSSIBLE TO COMPUTE THESE NUMBERS IN A STRAIGHTFORWARD WAY." The tables at <http://www.research.att.com/~njas/codes/Andw/index.html#dist16> http://www.research.att.com/~njas/codes/Andw/index.html#dist16 indicates the number of cell assemblies would, in fact be much larger than the number of nodes, WHERE THE OVERLAP WAS RELATIVELY LARGE, which would be equivalent to node assemblies with undesirably high cross talk. It doesn't provide any information for cases where over lap is small, i.e, d is actually larger than w. It is clear A drops sharply as a percent of all the possible combinations from set N of size S as O increases, but were N and S are large the number of combinations C(N,S) would be very large, so even a very small percent of it might be much larger than N. But I can be sure. Some of the closest examples to what I was looking for in these tables were the following: In the case where n=32, d=16, w=16 and A = 62, solving for O 16 = 2*(16-O+1) = 32-2O+2 = 34-2O 2O = 34-16 = 18 O = 9, which is over half of w (or S) Near the end of the page under the label "Further lower bounds" In case where A(80,20,20) >= 53404, solving for O 20 = 2*(20-O+1) = 49-2O+2 = 42-2O 2O = 42-20 = 22 O = 1, which is over half of w (or S) In case where A(128,32,32)>=512064, solving for O 32 = 2*(32-O+1) = 64-2O+2 = 66-2O 2O = 66-32 = 34 O = 17, which is over half of w (or S) But you can see than in all of them the overlap O was over half the value of S, which means there would be very high cross talk.. In my next email I will suggest an simple search algorithm for exploring the lower bounds on A(N,O,S). Unfortunately, I haven't coded for so long that even writing code as simple as this algorithm would be hard for me, because if have forgotten the peculiarities of different languages and programming environments. But to any of you who are still in the coding groove, you should be able to write this program in less than half an hour, and it would be interested to see what results it would give. Ed Porter -----Original Message----- From: Ben Goertzel [mailto:[EMAIL PROTECTED] Sent: Thursday, October 16, 2008 10:38 PM To: [email protected] Subject: Re: [agi] Who is smart enough to answer this question? Well, coding theory does let you derive upper bounds on the memory capacity of Hopfield-net type memory models... But, the real issue for Hopfield nets is not theoretical memory capacity, it's tractable incremental learning algorithms Along those lines, this work is really nice... http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.33.817 I wonder how closely that method lets you achieve the theoretical upper bound. Unfortunately, current math seems inadequate to discover this, but empirics could tell us. If anyone wants to explore it, we have a Java implementation of Storkey's palimpsest learning scheme for Hopfield nets, specialized for simple experiments with character arrays. -- Ben G On Thu, Oct 16, 2008 at 10:30 PM, Vladimir Nesov <[EMAIL PROTECTED]> wrote: On Fri, Oct 17, 2008 at 6:26 AM, Ben Goertzel <[EMAIL PROTECTED]> wrote: > > Oh, you're right... > > I was mentally translating his problem into one that made more sense to me > biologically, as I see no reason why one would assume all cell assemblies to > have a fixed size ... but it makes slightly more sense to assume an upper > bound on their size... > Which is why I don't like this whole fuss about cell assemblies in the first place, and prefer free exploration of Hamming space. ;-) -- Vladimir Nesov [EMAIL PROTECTED] http://causalityrelay.wordpress.com/ ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/? <https://www.listbox.com/member/?&;> & Powered by Listbox: http://www.listbox.com -- Ben Goertzel, PhD CEO, Novamente LLC and Biomind LLC Director of Research, SIAI [EMAIL PROTECTED] "Nothing will ever be attempted if all possible objections must be first overcome " - Dr Samuel Johnson _____ agi | <https://www.listbox.com/member/archive/303/=now> Archives <https://www.listbox.com/member/archive/rss/303/> | <https://www.listbox.com/member/?&; 5> Modify Your Subscription <http://www.listbox.com> ------------------------------------------- agi Archives: https://www.listbox.com/member/archive/303/=now RSS Feed: https://www.listbox.com/member/archive/rss/303/ Modify Your Subscription: https://www.listbox.com/member/?member_id=8660244&id_secret=117534816-b15a34 Powered by Listbox: http://www.listbox.com
