On Fri, May 1, 2009 at 7:01 PM, Matthew Toseland <toad at amphibian.dyndns.org> wrote: > On Friday 01 May 2009 22:43:50 Robert Hailey wrote: >> >> On May 1, 2009, at 3:46 PM, Evan Daniel wrote: >> >> > Yes, that's a question worth considering. ?There are both performance >> > and security issues involved, I think. ?Note that the partition could >> > be a set of contiguous regions (allowing performance optimization >> > around which piece of the keyspace you send info about), but it could >> > just as easily be determined by a hash function instead. >> > >> > You still check the same number of filters overall -- one per peer. >> > The difference is that for some peers you may have a partial filter >> > set, and therefore sometimes check their filters, instead of deciding >> > you don't have the memory for that peer's filter and never checking >> > it. >> >> Maybe if we partition it we can also get a free datastore histogram on >> the stats page. > > No, we cannot divide by actual keyspace, the keys must be hashed first, or the > middle bloom filter will be far too big.
Well, you could partition by actual keyspace as long as the partitions are (approximately) equal in population rather than fraction of the keyspace they cover. Doing that is only mildly nontrivial, and gives you a histogram with variable-width bars, but still an accurate histogram. Each bar would cover the same area; tall and skinny near the node's location, wide and short away from it. Evan Daniel
