richardstartin opened a new pull request #8261: URL: https://github.com/apache/pinot/pull/8261
The profile below was taken from one of our customer's deployments - very high allocation rates are observed in no-index LIKE queries, because of matcher construction. <img width="1596" alt="Screenshot 2022-02-28 at 18 27 57" src="https://user-images.githubusercontent.com/16439049/156040801-956865d0-96b0-49fc-a74b-64f80cda5886.png";> This PR simply reuses the `Matcher` as it will never be used across threads at the segment level. This decreases allocation significantly (2.5x) and may slightly increase average query time. before: ``` Benchmark (_numRows) (_query) (_scenario) Mode Cnt Score Error Units BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 919.018 ± 14.737 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 656.552 ± 1412.756 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 806583564.800 ± 1735530730.138 B/op BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 758.236 ± 18.678 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 762.000 ± 1639.664 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 806583100.000 ± 1735530900.622 B/op BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 799.468 ± 17.653 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 733.147 ± 1577.529 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 806583204.000 ± 1735531021.166 B/op ``` after: ``` Benchmark (_numRows) (_query) (_scenario) Mode Cnt Score Error Units BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 863.085 ± 31.335 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 279.607 ± 601.640 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.001) avgt 5 326550961.600 ± 702576606.964 B/op BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 759.623 ± 77.455 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 307.396 ± 661.675 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.5) avgt 5 326583524.000 ± 702294555.871 B/op BenchmarkQueries.query 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 753.004 ± 79.038 ms/op BenchmarkQueries.query:·gc.alloc.rate 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 308.145 ± 663.030 MB/sec BenchmarkQueries.query:·gc.alloc.rate.norm 1500000 SELECT RAW_INT_COL FROM MyTable WHERE NO_INDEX_STRING_COL LIKE '%foo%' EXP(0.999) avgt 5 326583459.200 ± 702294795.237 B/op ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/16439049/156040801-956865d0-96b0-49fc-a74b-64f80cda5886.png"](https://user-images.githubusercontent.com/16439049/156040801-956865d0-96b0-49fc-a74b-64f80cda5886.png"){kind=link}