richardstartin edited a comment on issue #6764: Consider using RoaringBitmapWriter for bitmap construction URL: https://github.com/apache/incubator-druid/pull/6764#issuecomment-450846465 @clintropolis You were right about where most of the performance came from. I cleaned up the commits a bit and ran against master. Here's the benchmark at `114a9fc38feda5f85799d24889007bc572d04dea` at 0.7.30 ``` Benchmark (bitmapAlgo) (prob) (size) Mode Cnt Score Error Units BitmapIterationBenchmark.constructAndIter roaring 0.0 1000000 avgt 5 130.624 ± 2.645 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.001 1000000 avgt 5 17553.925 ± 1177.041 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.1 1000000 avgt 5 1704213.394 ± 51487.534 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.5 1000000 avgt 5 6831889.531 ± 146377.716 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.99 1000000 avgt 5 13106844.584 ± 661339.555 ns/op BitmapIterationBenchmark.constructAndIter roaring 1.0 1000000 avgt 5 15204652.686 ± 1441562.179 ns/op ``` Here's a slight improvement on this branch at `1afb602de27d31367440b1cccc86ec799c59dc4c` owing to reduced construction times. ``` Benchmark (bitmapAlgo) (prob) (size) Mode Cnt Score Error Units BitmapIterationBenchmark.constructAndIter roaring 0.0 1000000 avgt 5 189.940 ± 3.313 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.001 1000000 avgt 5 13719.152 ± 42.376 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.1 1000000 avgt 5 1268587.758 ± 42864.087 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.5 1000000 avgt 5 4658899.187 ± 163463.751 ns/op BitmapIterationBenchmark.constructAndIter roaring 0.99 1000000 avgt 5 10556288.928 ± 212975.696 ns/op BitmapIterationBenchmark.constructAndIter roaring 1.0 1000000 avgt 5 11036729.972 ± 346125.258 ns/op ``` PS this has been squashed and force pushed so take another look.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
