On 28 December 2015 at 23:23, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote:
> On 12/28/2015 03:15 AM, David Rowley wrote: > >> Maybe it would be better to, once the filter is built, simply count the >> > number of 1 bits and only use the filter if there's less than >> <threshold> 1 bits compared to the size of the filter in bits. There's >> functionality in bms_num_members() to do this, and there's >> also __builtin_popcount() in newer version of GCC, which we could have >> some wrapper around, perhaps. >> > > I don't really see how this is relevant with the previous point. The > number of 1s in the bitmap can tell you the false positive rate for the > bloom filter, not what fraction of lookups will be answered with "true". > > So while this needs to be watched, so that we stop using the bloom filter > when it gets too full (e.g. due to under-estimate), it's completely > unrelated to the previous issue. Why is it not related? this has got me confused. If a bloom filter has all of the bits set to 1, then it will never filter any Tuples right? If so, then a filter with all 1 bits set should be thrown away, as it'll never help us, and the filter should generally become more worthwhile as it contains a higher ratio of 0 bits vs 1 bits. Of course we don't have a count of how many Tuples matched each bit, so this is based on the assumption that each bit matches an equal number of Tuples. Are you saying this is not an assumption that we should make? -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services
