himanshug commented on issue #9689: groupBy query: limit push down to segment scan is poor performance URL: https://github.com/apache/druid/issues/9689#issuecomment-613632928 @xiangqiao123 that is simplest of queries in terms of aggregation with a largish limit, even with efforts to reduce overheads there would be a non-zero cost in pushing the limit to segment scan phase, for your use case I guess it would be advisable to disable limit pushdown to segments (in reality, you might need to measure before blindly believing in my claim though :) @gianm I agree with your observations. it sucks that we need to implement grow-ability to save cost of zeroing out where we already have allocated all the memory or is there any other advantage?. another consideration, I don't know/remember if we tried, a different layout for marking used/unused buckets. currently first byte of each bucket tells whether that bucket is used or not. Instead, if we reserved numBucket bytes at the start of buffer and used those for marking. Zeroing those out might be faster due to batching and even looking them up might be faster when most of the buckets were mark used.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
