pegasas commented on issue #15330: URL: https://github.com/apache/dolphinscheduler/issues/15330#issuecomment-1895598640
> We need to define how to assign the trigger to Master, since we use `pull` mode, so all Master will pull the trigger from DB, we need to make sure one trigger will only be consumed by one Master.  Here's current quartz architecture. DolphinScheduler uses quartz scheduled by timer trigger and insert command into DB. For quartz scheduler, it use [exclusive lock](https://github.com/quartz-scheduler/quartz/blob/a5c4d27e963f51097f9b2777489d310a88897ca4/quartz/src/main/java/org/quartz/impl/jdbcjobstore/StdRowLockSemaphore.java#L42) while [acquiring Triggers](https://github.com/quartz-scheduler/quartz/blob/main/quartz/src/main/java/org/quartz/impl/jdbcjobstore/JobStoreSupport.java#L2793-L2827) and [fired them to get JobDetails for executing](https://github.com/quartz-scheduler/quartz/blob/main/quartz/src/main/java/org/quartz/impl/jdbcjobstore/JobStoreSupport.java#L2976-L3020) in hosted threadpool. see https://github.com/quartz-scheduler/quartz/blob/main/quartz/src/main/java/org/quartz/core/QuartzSchedulerThread.java#L291-L381 Currently quartz not share public interface for us for implementation our own trigger mechanism. Here's the high-level steps we implement this feature: - (Required) dolphinscheduler-ui & dolphinscheduler-api: add trigger controller: CURD trigger & push trigger implementation by inserting command directly - (Required) dolphinscheduler-scheduler-plugin: Implement new scheduler plugin - acquire distributed lock from `RegistryClient` - get triggers from db and wrap them into threadpool - if trigger success, callback will insert command into DB waiting for picking up to build workflow & processinstance. - (Optional) Consider unifying trigger and task or we just set them working individually. I am still considering a better solution that we can compatible with quart scheduler in first version, but it seems a bit hard. Luckily ds has few steps on it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
