On Tue, Sep 16, 2003 at 02:57:48PM +0100, Gordan wrote: > Sorry for wasting bandwidth with this, but I have another theoretical question > about the Freenet design. Feel free to point me toward TFM to R if this is > answered elsewhere. :^) > > 1) What causes specialisation in nodes? Is the "specialisation" area "picked" > by a node using some method, e.g. most frequently used parts of key space, as > perceived by the node through incoming requests, or is it merely amplified > from random artefact clustering around certain parts of the key space?
Well, since 0.4, we have had an announcement protocol which picks a random key (which the node can't influence), for that node's initial seed specialization, adds a reference pointing to that key on each of a chain of nodes, and gives the node some initial keys to fetch near that key. On NGRouting, it produces estimators skewed to that key - but the basic principle is the same: the node is added to the routing table, and it is most likely to be routed to if the request is close to that key. Specialization occurs naturally from the node being fed queries near to previous references, and from the node answering those queries most effectively. Sadly most of the design documents have disappeared between the various website upgrades; some of them are on the site though. > > 2) If the node is started and never used, does it only ever acquire files > through inserts performed on other nodes? If a node is empty (and this would > imply lack of any specialisation), how/why does it ever get inserts/requests > routed to it? Announcements cause requests and inserts to be routed to the node. Requests and inserts that are successful also cause more requests and inserts to be routed to the node. > > 3) Is probabalistic caching is used to amplify specialisation? From previous > discussion, I seem to remember it being said that a node will cache > everything until the space runs out, at which point it will start to drop > data using "probabalistic caching". How is this traded off between > maintaining specialisation in parts of key space and caching the "most > popular" data? The two can contradict each other, e.g. the least > frequently/recently used file can be in the key space of strong, narrow > specialisation. Probabilistic caching does not directly depend on specialization, but it is intended to amplify it. If the datastore is full, we need to decide what to keep, because we will have to delete *something*. Probabilistic caching is based on the number of hops since the DataSource (which is randomly reset, but is nominally the node from whose store the data came from), so that the data is cached more closer to the source. Simulations suggest pcaching should improve routing; it also should improve the network's data capacity and it certainly makes flooding attacks more difficult. > > 4) If a very large data store is used, then the "cache everything" method > sounds like it may get in the way of reinforced specialisation of a node. > According to observations of size of my own data store, once the node is > integrated into the network (few days), the data growth seems to decay > exponentially, from a peak of about 1GB/day. A quick calculation, given the > data growth pattern I have seen, seems to imply a rate of about 100GB/year > (from empty, in the first year, obviously less thereafter if the pattern > holds). How are such "heavy" nodes treated by the network, given their lack > of specialisation? Is there a provision for "generalist" rather than > specialist nodes? Perhaps by being used as "good but slow, so don't route > there first" nodes? This could potentially slow the rate at which the data > falls out of the network, and make a compromise between fast and reliable > retrieval of popular data and slow and mostly reliable retrieval of old or > unpopular data. How would a big node gain specialisation without it's data > store getting completely filled up, other than by coincidence? How do > non-specialised nodes get requests routed to them? Is it only a matter of > last resort, i.e. "I have some routing requests left, and none of the other > more specialised nodes found the data, so I'll have a go asking this node > even though it doesn't look like it's area of specialisation"? Well, using NGRouting, yes, basically. Read the paper. > > Thanks. > > Gordan > _______________________________________________ > Devl mailing list > [EMAIL PROTECTED] > http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl -- Matthew J Toseland - [EMAIL PROTECTED] Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so.
signature.asc
Description: Digital signature
_______________________________________________ Devl mailing list [EMAIL PROTECTED] http://dodo.freenetproject.org/cgi-bin/mailman/listinfo/devl
