churromorales opened a new pull request, #13339: URL: https://github.com/apache/druid/pull/13339
If you don't specify taskCount in your supervisor spec, it now defaults to the # of partitions you have in kinesis / kafka. This is useful for customers that are in a growth phase. Where they start with x number of partitions but then have to grow by some amount. Before you would have to update the supervisor spec and add middle managers (if needed). With the mm-less patch in druid, if a customer increased the number of partitions, all you have to do is restart the supervisor and it will pick up the new partitions. Even if they don't grow, it is best to have one task per partition as the default (unless others oppose this). Future work: this patch forces a manual restart of the supervisor, but no spec changes are needed, or knowledge of how many partitions your message bus has. In the future we could have a thread that detects a change and automatically restarts the supervisor, but that might be overkill. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
