Jackie-Jiang commented on PR #10451: URL: https://github.com/apache/pinot/pull/10451#issuecomment-1491025323
> Subbu has valid questions. Before we get to the znode creation. why are we not using the existing task framework for this? there is nothing stopping us from creating a task for each of these commands that need to run in background.. the only difference is controllers running the tasks instead of minions. > > Creating a task and managing fault tolerance etc is not trivial. Using the existing task framework allows us to leverage all the things we build for minion in terms of getting progress updates, the UI will have a tab to show the pending tasks etc. @kishoreg I think these are two very different questions. We didn't leverage the task framework for controller task in the first place because directly executing the controller task is much lighter weight for both coding perspective and execution perspective (no need to implement separate task generator/executor, no need to serialize task config etc.). We can consider moving it to task framework so that we get the fault tolerance (retry/cancel logic) from the task framework, but that involves very big change. We'll need to introduce the controller task executor concept, and ingest the resource manager into the controller task executor. That is out of the scope of this PR since this PR only splits the ZNode for all task types into one per task type. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
