kfaraz commented on PR #13339: URL: https://github.com/apache/druid/pull/13339#issuecomment-1322955593
> if you specify a taskCount of -1 then you align the number of tasks to # of partitions Thanks for your prompt response, @churromorales . I don't really like the idea of having a special meaning attached to a specific value of a config. Especially, passing a negative value to a config that is clearly supposed to be positive is not so great if we wanted to validate the parameters. That said, we could probably have this behaviour on passing task count as `0` (instead of `-1`). There are some other configs which do that, such as `maxSegmentsInNodeLoadingQueue`, where `0` means an unlimited load queue. So, here too `0` would mean maximum possible, which is nothing but the number of partitions. The idea of defaulting task count to number of partitions is a good one. We just need to implement it in the right way. Does the worker capacity based solution not seem viable to you? Even with passing the `0` value above, it would be safer if we just checked the worker capacity before launching a large number of tasks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
