ruanwenjun opened a new issue, #15882: URL: https://github.com/apache/dolphinscheduler/issues/15882
### Search before asking - [X] I had searched in the [DSIP](https://github.com/apache/dolphinscheduler/issues/14102) and found no similar DSIP. ### Motivation Once a TaskInstance is triggered, the status will be changed follow: submit -> dispatch -> running. Submit means the task instance is initialized, dispatch means the task has been dispatch to worker, running means the task is running in worker. <img width="1008" alt="image" src="https://github.com/apache/dolphinscheduler/assets/22415594/194ee04f-3307-4904-bd2c-ad86d08d69b8";> The dispatch status is useful in the history design, since `dispatch` is a async operation, but in the latest implementation, the dispatch operation is a sync operation, once a task has dispatch to worker we can directly set the task instance status to running. This means we will also deprecate the waiting queue in worker, since the waiting queue doesn't help to manage the task, it just make the task lifecycle more difficult and make the resource management difficult, we hope the waiting task is all manage by master, once the task has been sent to worker, it will get thread to execute. ### Design Detail 1. Remove the Dispatch status, once the task has been sent to worker then set task instance status to running. <img width="979" alt="image" src="https://github.com/apache/dolphinscheduler/assets/22415594/c870ab79-bad3-4278-88ad-f71e12eaf9cb";> 2. Deprecate the `TaskExecuteThreadsFullPolicy` in worker, the worker will reject the task once the worker thread pool is full. ### Compatibility, Deprecation, and Migration Plan This is compatibility, since the dispatch status will keep in code but only set it is depracated, will remove this after 3.3.0. ### Test Plan Tested by UT and E2E ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
