lenboo commented on issue #4083: URL: https://github.com/apache/incubator-dolphinscheduler/issues/4083#issuecomment-730826151
- The distributed lock is used when the command is taken, that result in the bottleneck of concurrency another idea to solve this problem:  Scheduler(master/slave): 1. Calculate the distribution policy according to the master resource usage 2. Generate processinstance 3. Send processinstance to master according to the policy Master fault tolerance: 1. Other masters change PI to fault tolerant state 2. Insert fault tolerant command 3. The scheduler gets the commands and sends pi to the master according to policy (priority... ) ======================================================================= 另一个解决思路： Scheduler: 1. 根据master资源占用，计算分发策略 2. 获取command生成ProcessInstance 3. 根据策略(优先级。。。)，发送ProcessInstance给master master容错： 1. 其他master将PI改为容错状态 2. 插入容错command 3. scheduler获取command，发PI给master ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
