Jackie-Jiang commented on pull request #6593: URL: https://github.com/apache/incubator-pinot/pull/6593#issuecomment-781634450
Here are the performance number running the query: `SELECT COUNT(*) FROM groupBySample GROUP BY col5` against 10 segments of size 10M (100M documents total). `col5` is an `INT` dimension of cardinality 100k. Current master: ``` -------------------------------------------------------------------------------- Time Passed: 3225ms, Queries Executed: 5, Exeptions: 0, Average QPS: 1.5503875968992247, Average Broker Time: 642.0ms, Average Client Time: 644.2ms. Time Passed: 6430ms, Queries Executed: 10, Exeptions: 0, Average QPS: 1.5552099533437014, Average Broker Time: 640.3ms, Average Client Time: 642.5ms. Time Passed: 9636ms, Queries Executed: 15, Exeptions: 0, Average QPS: 1.5566625155666254, Average Broker Time: 639.8ms, Average Client Time: 642.0ms. Time Passed: 12854ms, Queries Executed: 20, Exeptions: 0, Average QPS: 1.555935895441108, Average Broker Time: 640.2ms, Average Client Time: 642.4ms. Time Passed: 16279ms, Queries Executed: 25, Exeptions: 0, Average QPS: 1.535720867375146, Average Broker Time: 648.72ms, Average Client Time: 650.88ms. Time Passed: 19474ms, Queries Executed: 30, Exeptions: 0, Average QPS: 1.540515559207148, Average Broker Time: 646.7333333333333ms, Average Client Time: 648.9ms. Time Passed: 22645ms, Queries Executed: 35, Exeptions: 0, Average QPS: 1.545595054095827, Average Broker Time: 644.6ms, Average Client Time: 646.7428571428571ms. Time Passed: 25852ms, Queries Executed: 40, Exeptions: 0, Average QPS: 1.5472690700912888, Average Broker Time: 643.925ms, Average Client Time: 646.025ms. Time Passed: 29043ms, Queries Executed: 45, Exeptions: 0, Average QPS: 1.5494267121165168, Average Broker Time: 643.0666666666667ms, Average Client Time: 645.1555555555556ms. Time Passed: 32251ms, Queries Executed: 50, Exeptions: 0, Average QPS: 1.550339524355834, Average Broker Time: 642.56ms, Average Client Time: 644.76ms. -------------------------------------------------------------------------------- CLIENT TIME STATISTICS: DescriptiveStatistics: n: 50 min: 626.0 max: 747.0 mean: 644.76 std dev: 21.05344026373779 median: 639.0 skewness: 3.126822147266319 kurtosis: 12.162479993963574 10th percentile: 628.0 25th percentile: 634.5 50th percentile: 639.0 90th percentile: 666.6999999999999 95th percentile: 694.0500000000001 99th percentile: 747.0 99.9th percentile: 747.0 -------------------------------------------------------------------------------- ``` PR branch: ``` -------------------------------------------------------------------------------- Time Passed: 3183ms, Queries Executed: 5, Exeptions: 0, Average QPS: 1.5708451146716935, Average Broker Time: 634.2ms, Average Client Time: 636.4ms. Time Passed: 6265ms, Queries Executed: 10, Exeptions: 0, Average QPS: 1.5961691939345573, Average Broker Time: 624.3ms, Average Client Time: 626.4ms. Time Passed: 9356ms, Queries Executed: 15, Exeptions: 0, Average QPS: 1.6032492518170158, Average Broker Time: 621.5333333333333ms, Average Client Time: 623.6666666666666ms. Time Passed: 12515ms, Queries Executed: 20, Exeptions: 0, Average QPS: 1.5980823012385137, Average Broker Time: 623.55ms, Average Client Time: 625.65ms. Time Passed: 15641ms, Queries Executed: 25, Exeptions: 0, Average QPS: 1.5983632760053705, Average Broker Time: 623.48ms, Average Client Time: 625.56ms. Time Passed: 18746ms, Queries Executed: 30, Exeptions: 0, Average QPS: 1.600341406166649, Average Broker Time: 622.6666666666666ms, Average Client Time: 624.7666666666667ms. Time Passed: 21810ms, Queries Executed: 35, Exeptions: 0, Average QPS: 1.6047684548372307, Average Broker Time: 621.0ms, Average Client Time: 623.0ms. Time Passed: 24933ms, Queries Executed: 40, Exeptions: 0, Average QPS: 1.604299522720892, Average Broker Time: 621.175ms, Average Client Time: 623.2ms. Time Passed: 28001ms, Queries Executed: 45, Exeptions: 0, Average QPS: 1.6070854612335272, Average Broker Time: 620.1111111111111ms, Average Client Time: 622.1111111111111ms. Time Passed: 31072ms, Queries Executed: 50, Exeptions: 0, Average QPS: 1.6091658084449023, Average Broker Time: 619.3ms, Average Client Time: 621.32ms. -------------------------------------------------------------------------------- CLIENT TIME STATISTICS: DescriptiveStatistics: n: 50 min: 603.0 max: 665.0 mean: 621.32 std dev: 12.362616459078787 median: 618.5 skewness: 1.3349071982821457 kurtosis: 2.3680776507610988 10th percentile: 608.2 25th percentile: 613.0 50th percentile: 618.5 90th percentile: 637.9 95th percentile: 647.25 99th percentile: 665.0 99.9th percentile: 665.0 -------------------------------------------------------------------------------- ``` We are able to reduce the latency by 20ms in average ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
