YongGang commented on PR #16041: URL: https://github.com/apache/druid/pull/16041#issuecomment-2021010161
@kfaraz , thanks for your input. The design of `TaskIdentitiesProvider` is to help identify the purpose of the task, as existing task type information falls short (for instance, MSQ task types like `query_controller` and `query_worker` offer limited clarity on the tasks' specific functions). Incorporating these tags into both metrics reporting and task reports enables us to address numerous monitoring and analysis questions, such as: - "how many async download task/Peon are running in the cluster?" - "how many tasks are reading from external data source (e.g. S3) vs internal Druid data transfer?" - "if we want prioritize/limit task running based on its purpose, which info should look at?" These are real questions we have and it's tightly coupled with the business requirements, that's why it's designed as an `ExtensionPoint` so it's not bound to a default implementation. This is the first step we tackle in this field, for instance to answer/solve the third question in the above list there are more works can do regarding to task tags/labels/laning, this interface will be setting the groundwork for subsequent enhancements. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
