Hi I recently modified FixedBitSet.iterator() to return its own iterator, instead of OpenBitSetIterator, and while I haven't done a thorough performance test, the results on LUCENE-5425 suggested this helped in general (even though we only tested facets). Also, last night's benchmark for WildcardQuery and PrefixQuery indicate a 20% jump in QPS.
This got me thinking (again) why do we keep both bitsets, and whether we couldn't live with just one. So clearly FBS is faster than OBS (perhaps unless you use fastSet/Get) since it doesn't need to do bounds checking. Also, FBS lets your grow itself by offering a convenient copy constructor which allows to expand/shrink the set. OpenBigSet however lets you manage 64 * 2^32 bits, while FixedBitSet only allows managing 2^32 bits. Is there any reason for this? Besides the fact that neither can return a DISI over more than 2^32-1 bits/docs, I don't see any real reason for that. We'll need to change FBS API to take/return long, but besides that? I did a quick search on the usage of OpenBitSet and eclipse found 272 mentions of it. I believe some, if not all, of these uses can be replaced by FixedBitSet. Definitely when the number of bits is known in advance, but I think also when it's not known, by growing the bitset. One place I found is DocValuesConsumer, which clearly knows the number of bits in advance (dv.getValueCount()), however this returns a long, so this prevents cutting over to FBS immediately, but if we allow FBS to handle longs, I think that we can? Anyway, wanted to get your thoughts before I open an issue and start this work. Shai
