On December 7, 2007, Robert Hailey wrote: > > On Nov 30, 2007, at 6:40 AM, Ed Tomlinson wrote: > > > Hi, > > > > I think freenet is slow because its having problems finding data. > > The hit rates in 0.5 ranged around 5%. > > in 0.7 I see 2% on cache and 0% on store. We just are not finding > > data. My location shifts so fast that > > other nodes have no idea what is in my node's 8G store. I strongly > > suggest that we experiment with letting > > the store contents limit how far from the current location we can > > swap. This could easily be done with > > a tunable such that the effect could be turned off or could be made > > very strict. We could even use > > a feedback loop to find an optimal value with the store/cache hit > > rate being used to control just how strict > > we are. The idea would be to minimize the swapping limitation while > > maximizing the hit rate which > > would decrease the path length and hence the traffic. > > > > In my case the node usually uses a good chunk (80-90%) of the > > bandwidth I give to it. I think the > > bandwidth limits work. I would agree that backout is probably not > > needed and think it would be a > > good idea to try with it turn off - again this would improve routing > > which I think is the core of 0.7 > > performance problems. > > > > Thanks > > Ed > > > I have been musing on Ed's post for a while. Along with my > understandings: > > * The contents of the datastore(s) are not given weight when > considering a swap request. > * For a particular CHK/SSK insert, it finds one home node to be put > into the store (the rest in only cached). > * If the node targeted by the insert 'drifts' sufficiently far from > it's origin > ** that data in the store be unfindable by the network, and > ** is all-but guaranteed to be LOST forever, is it can only survive in > the networks caches (if it is un-popular). > > Consider the following patch (completed to the furthest point that my > current freenet understand can take it). > - If a CHKBlock is evicted from the datastore (not cache), AND we are > not (as best we can tell) "close" to where it should be, THEN re- > insert that block. > > This approximates the decision (is this eviction OLD/UNPOPULAR or > UNFINDABLE), and brings with it a rather odd perk for having small > datastores (that they would find these misplaced blocks sooner). > > The major risk, of course, is renewing old data. > > I do not have the means to test this on any sizable network. Would > this be a candidate for 'testnet' or simulations of which I hear? > > Also, I could not figure out how to revive a SSKBlock from the > datastore (or find out if it is even possible).
This is an interesting way to tackle the problem. Might I suggest that instead of doing this for data that we are about to discard we do it select a key from the DS at random and reinsert if its far from our current location. One thing that may be happening that somewant offsets this problem could be clustering of locations. e.g. even though my nodes location changes often it could be that it favours some subsets of the total location space. It would be worthing adding a metric to track how long a node spends in a given area of the location space. Ed
