richardstartin commented on pull request #6320: URL: https://github.com/apache/incubator-pinot/pull/6320#issuecomment-739154073
I added a benchmark which times the single shot time to seal an off heap index creator, with a range of cardinalities, numbers of docs, and different assignment strategies of document ids to dictionary ids. One is the best case for roaring (sorted), the other is the worst case (round robin). The benchmark takes quite a long time to run so I only ran one fork and reduced the number of iterations, but this PR demonstrates a significant improvement: before ``` Benchmark (assignment) (cardinality) (numDocs) Mode Cnt Score Error Units BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 1000000 ss 5 0.030 ± 0.003 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 10000000 ss 5 0.374 ± 0.265 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 100000000 ss 5 4.077 ± 0.191 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 1000000 ss 5 0.090 ± 0.012 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 10000000 ss 5 1.070 ± 0.304 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 100000000 ss 5 11.465 ± 0.308 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 1000000 ss 5 0.152 ± 0.040 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 10000000 ss 5 1.639 ± 0.067 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 100000000 ss 5 17.002 ± 0.217 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 1000000 ss 5 0.021 ± 0.002 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 10000000 ss 5 0.212 ± 0.004 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 100000000 ss 5 2.283 ± 0.109 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 1000000 ss 5 0.075 ± 0.002 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 10000000 ss 5 0.296 ± 0.189 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 100000000 ss 5 2.094 ± 0.284 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 1000000 ss 5 0.093 ± 0.014 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 10000000 ss 5 0.891 ± 0.327 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 100000000 ss 5 3.736 ± 0.067 s/op ``` after ``` Benchmark (assignment) (cardinality) (numDocs) Mode Cnt Score Error Units BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 1000000 ss 5 0.021 ± 0.011 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 10000000 ss 5 0.195 ± 0.026 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10 100000000 ss 5 1.966 ± 0.133 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 1000000 ss 5 0.032 ± 0.009 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 10000000 ss 5 0.287 ± 0.108 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 1000 100000000 ss 5 2.943 ± 0.176 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 1000000 ss 5 0.057 ± 0.036 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 10000000 ss 5 0.491 ± 0.131 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal ROUND_ROBIN 10000 100000000 ss 5 5.502 ± 0.225 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 1000000 ss 5 0.018 ± 0.005 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 10000000 ss 5 0.159 ± 0.012 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10 100000000 ss 5 1.554 ± 0.028 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 1000000 ss 5 0.021 ± 0.006 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 10000000 ss 5 0.181 ± 0.028 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 1000 100000000 ss 5 1.580 ± 0.028 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 1000000 ss 5 0.031 ± 0.003 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 10000000 ss 5 0.193 ± 0.020 s/op BenchmarkOffheapBitmapInvertedIndexCreator.seal SORTED_UNIFORM 10000 100000000 ss 5 1.752 ± 0.089 s/op ``` Please review the benchmark and let me know if you think the improvements look interesting - I am not going to be able to run cluster scale performance tests on this, but have prior experience of these APIs reducing resource cost in the system I wrote them for. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
