liang-wenjie commented on issue #18454: URL: https://github.com/apache/dolphinscheduler/issues/18454#issuecomment-5173438130
Thanks for the question. We have a concrete production use case: some data ingestion and synchronization workflows have highly variable execution times. They usually finish within several minutes, but during source-side delays or backfills they can run much longer than their configured Cron period. For example, a workflow scheduled every 10 minutes may occasionally run for 30–40 minutes. The next Cron fires regardless of whether the previous workflow instance is still running, which can create overlapping instances, resource contention, and a backlog. The desired behavior is not merely another fixed-rate Quartz trigger. It is completion-aware fixed-delay scheduling: after the previous workflow instance reaches a terminal state, wait N minutes (or seconds), then create the next instance. This is similar to fixed-delay execution rather than Cron's fixed calendar schedule. I agree that the initial SimpleTrigger-based interval proposal does not by itself guarantee this behavior, because SimpleTrigger also advances by wall-clock time. I will refine the DSIP before proposing the interval part again, including: - explicit fixed-delay semantics based on workflow instance completion; - a no-overlap rule for the same schedule; - behavior for failure, pause, recovery, and scheduler restart; - how the next fire time is persisted and calculated. The missed-fire policy part is independent, so I will split it into a smaller proposal and PR. Would a completion-aware fixed-delay schedule be a meaningful direction for this use case? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
