I propose to implement the two level datastore in the near future. This means that: - We have two datastores: the cache and the store. The cache caches everything which passes through the node (modulo privacy concerns). The store only caches data for which the node is the best node by location on the insert chain so far. Both are LRU. - One in every X successful requests will turn into an insert. This will keep old data alive, rather than it stagnating when the locations change and it is suddenly stored in the wrong place. - The store is for long-term storage; to maximize the storage capacity of the network. The cache is for short term caching; optimizing the speed of fetching very popular documents and minimizing load on the network. - The store and the cache will be of equal size, for now. In future the store may be significantly larger than the cache. I will try to minimize the disruption caused on the network; I have already (in 901) made datastore shrinks preserve the most popular data, and I may code the changes in such a way as to slowly reduce the size of the cache as the store expands (the current store is the cache).
There are a few parameters we have to set, somewhat alchemically: - How often to turn a successful request into an insert (X). - The ratio of the size of the store to the size of the cache. (R) Is there any non-alchemical way to establish these parameters? If not, then what should they be? I suggest alchemical values: X = 1 in 200 (so if a typical request goes 10 hops, 1 in 20 of those starts an insert that goes 20 hops, adding on average 1 hop to a request; unfortunately we don't know either value with any confidence) R = 4:1 (assuming the cache shrinks gradually as the store grows) The above mechanism is the result of Oskar's simulations and theoretical work. -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20060726/d4b6324e/attachment.pgp>
