Jackie-Jiang opened a new pull request #7492: URL: https://github.com/apache/pinot/pull/7492
## Description Add instance config and query option to limit the maximum execution threads used for a query. By default (same as existing behavior), pinot will use up to 10 threads (or half of the CPU cores is it is smaller than 10) for non-group-by queries; and all threads for group-by queries. This option can be used to: - Limit the thread usage for very expensive queries such as large group-by or full table scan - Fully utilize all the threads for non-group-by queries to get the smallest latency Example run of large group-by queries (10 segments, each with 10M records, grouping on a column of cardinality 10M): Without thread limit: - Latency: 4103ms - Total thread time: 38353324527ns With thread limit 1 (single thread): - Latency: 10495ms - Total thread time: 10487257767ns Profile of CPU usage (run without limit 3 times, then single thread 3 times)  ## Release Notes Added server instance config `pinot.server.query.executor.max.execution.threads` to configure the instance level thread limit Added query option `maxExecutionThreads` to configure the query level thread limit, which can be used to override the instance level limit if both exist -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
