> Could someone please let me know if I am barking up the wrong tree? No.. But you seem to be outlining what current Freenet and NGR is all about really. If you figured out some of these things for yourself then I think you should come and help us develop :)
I have snipped shamelessy from you original mail. Let me know if I cut out something that I shouldn't have. > Force nodes to specialise under load. Unobtanium routing > Route requests to nodes that specialise in an area of > keyspace, then use > estimators to choose between specialised nodes > > Use large routing tables to keep track of nodes specialising in small > areas of keyspace, If specialization where working well we wouldn't need to keep a complete keyspace coverage in our rt since we would only get requested for items that is somewhat within our own specialization.. Hmm.. When thinking about this.. If this where completely true then we would end up with a fragmented network.. Wherein lies the bridging? Possibly somewhat like your 2 rt:s idea... >so that each routing step comes closer to > where the > data is located or to nodes that specialise in finding certain keys. Isn't this what we are doing right now really? > Specialization routing > > A node should track its own specialization. > This would probably be done using 2 variables: the KEY (peak) of the > specialization and the RANGE of specialization. > Nodes should announce these variables whenever connecting to > another node. One main thing about uncensorability is that nodes don't trust other nodes.. If a node could select its own area of specialization (and other nodes trusted it) then it could be used to censor certain areas of the keyspace. That is why we are currently using the learning approach. > Nodes should keep a large routing table including each node's > specialization key and range. The routing table should also > keep track > of the speed at which previous keys were retrieved from each node so > that when more than 1 node with similar or overlapping > specialization is > available it can try to route to the fastest one. If the highly > specialized nodes are very slow/busy it can try some of the less > specialised ones encouraging them take up more of the load. NGR in a nutshell > As the load on the node increases it should start to QR requests that > are not in its RANGE. Unobtainium routing in a nutshell > but if a node gets a > request for a key, which is in its datastore AND that request > comes from > a node that has almost the same specialization it should give that > request priority above all others. In effect this will be > "sister" node, > which looks after the same area of keyspace. This is a part of specialization really.. Since a specialized node gets requests mostly around keyspace area X should keep similarly specialized nodes in its rt. See what I said above... > There may be other ways to "cluster" sister nodes together so > that they > maintain an index of what keys are (or are likely to be) in > each other's > datastores making it easier to find keys fast. Might be a good idea but then we would need some of the much discussed path-folding mechanism I think. > If a node appears to be under-utilised it should broaden its > range which > may cause it to shift its specialization left or right as > time goes on > and find areas of keyspace that are overloaded. Unobtainium routing > The routing table size will need to be very big. in fact it would > probably be good to have 2 routing tables. > The first large routing table should have a large and > accurate list of > nodes that specialise in the keyspace around our own area of > keyspace.. > The second, and probably smaller routing table should have a list of > nodes that service the rest of the keyspace. preferably > equally spaced.. > i.e. few nodes that service "a" keys, "b" keys etc. See that thing I said on this in the beginning of the mail.. > Inserts should work the same way as requests: nodes should > only insert > keys into nodes that specialise in that area of keyspace, and if they > are all slow/busy they should insert them into unspecialised > nodes which > should cause those nodes to specialise in that area. As we are doing now? > Obviously this still needs a lot of work, the more I think > about it the > more I come up with and the more problems I see, but I do > believe that > specialization, distribution and load balancing is the only > way it can work. True.. Those things are what Freenet is all about really /N _______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
