Rangsh commented on issue #12492: URL: https://github.com/apache/seatunnel/issues/12492#issuecomment-5856601932
## Revised direction (after @DanielLeens feedback) Thanks @DanielLeens — agreed on the dependency and on rejecting a blanket `Exception` → `Throwable` change. ### Gate I will **not** start implementation until #12081 is green, merged, and this follow-up is rebased on that baseline. The revised design below is for alignment only. ### Failure-class behavior (APPEND write / response path) | Class | Examples | Behavior | | --- | --- | --- | | **Fatal VM / linkage** | `VirtualMachineError` (incl. `OutOfMemoryError`), `ThreadDeath`, `LinkageError` | Best-effort complete the affected request (`done(false)` if possible); trip sticky permanent fail-close so later APPENDs observe permanent failure immediately; **rethrow** so the failure propagates to the controlled node-failure path. **Do not** silently resume the sole consumer or reopen the writer in-process. | | **Other `Error`** | only if explicitly enumerated | Default: treat like fatal (permanent fail-close + rethrow). Any recoverable `Error` must be listed and justified before we catch-and-continue. | | **`Exception`** | current #12081 contract | Unchanged: sticky fail-close, complete future, keep consumer alive in fail-closed mode (no change to #12081). | So the previous “widen everything to `Throwable` and never rethrow” proposal is withdrawn for fatal errors. ### Disruptor exception-handler contract (to define before PR) Install a custom `ExceptionHandler` (not `FatalExceptionHandler` alone) with this contract: 1. **Permanent failure recording** — on fatal escape / handler entry for WAL APPEND, trip the same sticky fail-close flag used by #12081 (`appendBlockedAfterWriteFailure` / equivalent), so `IMapFileStorage` / `FileMapStore` can surface the existing permanent-failure path. 2. **Already-queued APPENDs** — best-effort complete with failure (`false` / permanent-block signal); do not leave them dependent only on wait timeouts. 3. **Later APPENDs** — observe permanent fail-close immediately (same loud surface as Exception fail-close), without writing. 4. **What must be rethrown** — fatal VM / linkage errors (table above) must be rethrown after best-effort completion + fail-close trip, so we do not silently keep processing in a corrupted JVM state. 5. **Out of scope** — no in-process worker restart, no writer reopen, no changes to #12081’s Exception / timed-wait behavior. ### Regression test Extend `WALWorkHandlerSurvivabilityTest` (or sibling) with a **safe synthetic `Error`** injected from `write(...)` (not real heap exhaustion): - affected future completes with failure / permanent-block signal (not timeout-only) - sticky fail-close trips - subsequent APPEND observes permanent failure without a second `write(...)` - fatal path does **not** silently leave the pipeline depending only on timed waits ### Ask Does this failure-class + Disruptor contract look right before coding (post-#12081 merge)? In particular: 1. OK that fatal `Error`s fail-close loudly + rethrow to node-failure, rather than keeping the consumer alive? 2. OK that recoverable `Error`s stay empty for v1 unless we explicitly enumerate any? Happy to adjust wording / split the handler contract further if you want it tighter. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
