kfaraz commented on PR #15976: URL: https://github.com/apache/druid/pull/15976#issuecomment-1968330003
__Summary__ IIUC, this PR seems to be implementing a task laning mechanism to better utilize available task slots. The lane that a particular task may be assigned to is determined by the new `label` field. Through configs, we may control the width and priority of each lane. __My current thoughts on the design__ 1. Since task laning may have applications outside of the contrib K8s extension too, the code changes should live in core Druid as suggested by @cryptoe . Another (less important) reason for this to be in core is the fact that we are adding a method to the `Task` API which currently has its only use in a contrib extension. 2. Instead of the new `getLabel` method in `Task`, we should have something more like `getTaskLaneInfo` that returns a `TaskLaneInfo` object which may be augmented later if needed. It should also be possible to manipulate the details of the lane info (i.e. the label in this case) using the task context. I guess the current problem in scheduling arises from the fact that we are limited to using immutable fields such as `type` and `datasource` to determine how a task should be scheduled. Powering this through the task context will allow users much more flexibility. 3. @YongGang , I suppose it would have been nice to put up a proposal for this design first before starting with the implementation. But since we already have the PR, please include details of the current design and any alternatives in the PR description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
