CryoThrust commented on issue #12118: URL: https://github.com/apache/seatunnel/issues/12118#issuecomment-5556404113
A few contract points may help make the outbox design implementable without weakening the “never abandon a terminal fact” requirement: 1. Give every terminal event a stable identity such as `(jobId, taskGroupId, executionAttempt)` and make the master-side apply idempotent. A retry after an ambiguous RPC result must be safe, and a duplicate from before/after master failover must receive an acknowledgement rather than create a second transition. 2. Persist/enqueue the event before releasing the task completion callback. The dispatcher should be the only component that performs RPCs; completion threads should publish to the outbox and return, so a blocked master cannot consume one thread per event. 3. “Bounded outbox” and “never drop” need an explicit overflow policy. A bounded in-memory queue alone cannot satisfy both when the master is unavailable indefinitely. Either spill to a durable local store, or apply backpressure to completion publication and document the liveness trade-off; silently evicting the oldest terminal event is not safe. 4. Use a monotonic per-event state (`PENDING -> IN_FLIGHT -> ACKED`) and treat `JobNotFoundException` as an explicit terminal acknowledgement only when the master contract says the job is irrecoverable. Other failures, including an election/restore response, should return the event to `PENDING` with bounded exponential backoff and jitter. 5. Add tests for duplicate delivery, an ambiguous timeout after master acceptance, active-master handover while an event is in flight, and outbox overflow. The assertion should be that each terminal identity is applied at most once and eventually acknowledged when the master becomes reachable. This keeps the implementation transport-independent and makes the “exactly once” property an idempotent state-application contract rather than an assumption about the RPC transport. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
