kfaraz commented on code in PR #18163:
URL: https://github.com/apache/druid/pull/18163#discussion_r2158621245
##########
indexing-service/src/main/java/org/apache/druid/indexing/seekablestream/supervisor/SeekableStreamSupervisor.java:
##########
@@ -940,17 +941,8 @@ public SeekableStreamSupervisor(
spec.isSuspended()
);
- int workerThreads;
if (autoScalerConfig != null &&
autoScalerConfig.getEnableTaskAutoScaler()) {
log.info("Running Task autoscaler for supervisor[%s] for
datasource[%s]", supervisorId, dataSource);
-
- workerThreads = (this.tuningConfig.getWorkerThreads() != null
Review Comment:
I think the number of worker threads should be calculated something like
this:
- If `workerThreads` are specified, honor that config.
- If nothing is specified, use a minimum of 2 to 5 (1 seems too aggressive a
downscale from the old 10).
- If honoring the task count config (`ioConfig.taskCount` or
`autoScale.taskCountMax()`),
core pool size could be some factor of the task count e.g. `taskCount/4`
rather than a fixed value.
- For the non-core pool threads, we should perhaps also increase the timeout
to a couple of seconds rather than the default of 10 ms.
If these settings continue to do well over a period of time, we can consider
reducing them further.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
