davidzollo commented on issue #15330: URL: https://github.com/apache/dolphinscheduler/issues/15330#issuecomment-2322792593
> * master fetch `t_ds_trigger_definition` table for trigger > * execute trigger, if it meets user-custom condition, insert a command into `t_ds_command`, note that this operation needs to be transactional > > Here's the question, currently our command fetching are based on id-based algorithm, the key point is when ProcessInstance is generated, we delete related command record. https://github.com/apache/dolphinscheduler/blob/dev/dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/process/ProcessServiceImpl.java#L299 but we should not delete `t_ds_trigger_definition` in this scenario. > > One solution is we add one more table to record offset for fetching, once master is alive, every time master will firstly acquire lock, then fetching `t_ds_trigger_definition` table for trigger and execute, finally we update offset into db with trigger_instance within one transaction. > > for more gracefully, we can do it in consistent-hash way. just like memcached & brpc. https://github.com/apache/brpc/blob/master/src/brpc/policy/dynpart_load_balancer.cpp#L83consider to its complexity, I will use method #1 for implementation. @ruanwenjun Can you help guide this design? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
