On Jun 18, 2009, at 9:07 PM, Matthew Toseland wrote: > CHUNK SIZE PROBLEM: > =================== > > Current plans call to split the keyspace up for each datastore, and > assign keys to a manageable sized bloom filter for a section of the > (hashed) keyspace. These can then be transferred separately, are > useful immediately after being transferred, can be juggled in > memory, etc. However, we cannot guarantee that the populaion of such > a chunk will be small enough for the filter to be effective. Solving > this requires either moving the boundaries dynamically (possibly > continually), or making the chunks significantly larger than they > would need to be in an ideal world.
Right... but I believe we prescribed that the split would be based on a hashed value of the key and not by logical keyspace location to avoid disproportionate chunks. That is to say... ideally a node is going to get a disporportionate amount of cache/store data about it's network location. STORE_SIZE/MAX_BLOOM_CHUNK -> N_CHUNKS H(key, N_CHUNKS) => n & (0 < n < N) CHUNK[n].add(key) Wouldn't the problem be reduced to finding a well-scattering hash function then? Surely even hashing the byte-wise double value of the location would be sufficient, I suppose that would look more like this... H(key) => location H(location)/N_CHUNKS => n > > LOCAL SECURITY ISSUES: AFTER-THE-FACT TRACING BY POWERFUL ATTACKERS > =================================================================== > > We have IMHO adequately addressed the network level security issues, > by not caching until HTL has reduced a bit. What about an evil peer sending a "full" filter to try and collect all traffic? Have we discussed that? Perhaps a backoff on a bloom filter false positive would kill such an attack, but may interfere with initial testing. -- Robert Hailey -------------- next part -------------- An HTML attachment was scrubbed... URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20090619/ff45127f/attachment.html>
