Radeity commented on issue #9337: URL: https://github.com/apache/dolphinscheduler/issues/9337#issuecomment-1356181289
> BTW, I think we may not put `ResourceManager` into master. We need more discussions on this point. Hi, @EricGao888, i think ds should not get involved in executors resource management. BTW, i think we should keep the concept of `Master` and `Worker`, still one for handling workflow, one for handling task, then introduce new concept of `Executor` in charge of actual task execution. Why separate `Executor` and `Worker`? - In current design, many types of task directly runs on worker. I think scheduler have to focus on scheduling, rather execution. So it's better not execute task on ds' `Worker` node. - `Worker` receives task from `Master` and is still in charge of listening to local/remote(corresponding to whether ds is deployed as standalone mode or ds on k8s/yarn) task execution. To be specific, if in standalone mode, `Worker` directly launch `Executor` at local, rather, launch remote executor in external k8s or yarn cluster, which is related to this issue. In a word, this could be a big architectural change. Anyway, just a humble suggestion, feel free to correct me, i'm interested in any further discussion! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
