gianm commented on issue #9689: groupBy query: limit push down to segment scan is poor performance URL: https://github.com/apache/druid/issues/9689#issuecomment-613222019 It looks like the `reset` is happening on `init` of LimitedBufferHashGrouper. I'm guessing it's taking a long time because the AlternatingByteBufferHashTable starts out at the max size (`tableArenaSize / 2`) rather than growing like the regular ByteBufferHashTable does. The perf regression is probably because https://github.com/apache/druid/pull/8426 was added in 0.17.0, and before then, we wouldn't push down the limit all the way to the segment. I bet the pushdown to the merge buffer level isn't as bad because there's only one merge buffer per query, so the cost of init is only paid once, instead of per segment. I think to improve the performance we'd need AlternatingByteBufferHashTable to implement growability.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
