leventov opened a new issue #8556: Pre-touch segments needed for queries on Historicals to not waste time in processing threads on disk I/O URL: https://github.com/apache/incubator-druid/issues/8556 When a Historical has much more segments on local disks than the memory available (typically, on "cold" Historicals with old data), a large part of query processing might be disk I/O pulling segment data into memory. This reduces the utilization of processing threads, which are currently recommended to be configured in a thread-per-core manner on Historicals. We don't preempt queries which are blocked in disk I/O because we have synchronous query processing architecture. We can disaggregate query processing and disk I/O by pre-touching the segments and the dimensions that are going to be needed for the query, and thus initiating disk I/O, while the query is still in the queue waiting for CPU resources for the query processing. If the query queue is so long that we risk that pre-touched memory will be evicted before the time comes for this query to be processed, we can initiate the pre-touch at a certain "checkpoint" in the queue, say, when there are just N (10, 100, etc.) queries ahead in the queue. It should be balanced with the I/O latency because the pre-touch just before starting the actual processing may not help much to disaggregate disk I/O latency from the processing. FYI @nishantmonu51
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
