hawk9821 commented on code in PR #13201:
URL: https://github.com/apache/seatunnel/pull/13201#discussion_r4546981789
##########
seatunnel-api/src/main/java/org/apache/seatunnel/api/sink/multitablesink/MultiTableSinkWriter.java:
##########
@@ -850,21 +872,24 @@ public void abortPrepare() {
/**
* Closes this writer and releases all resources.
*
- * <p>Drains remaining queues, then calls {@link
ExecutorService#shutdownNow()} to interrupt all
- * consumer threads. Each sub-writer is closed under its respective {@link
- * MultiTableWriterRunnable} lock to avoid concurrent access. Finally, the
shared {@link
+ * <p>Signals any active timer flush round to stop, drains pending writes,
and interrupts the
+ * executor threads. After any in-progress table isolation finishes, each
sub-writer is closed
+ * under its queue lock, which also protects any in-flight flush. Finally,
the shared {@link
* MultiTableResourceManager} is closed and this method waits for the
executor threads to stop.
*
* <p>Uses first-exception-wins error handling: if multiple sub-writers
throw during close, only
* the first exception is propagated. Resource manager close errors are
logged but not
* propagated.
*
- * @throws RuntimeException wrapping the first {@link Throwable} caught
from any sub-writer;
- * note that the method signature declares {@code throws IOException}
(inherited from the
- * interface), but the current implementation always wraps in {@code
RuntimeException}
+ * @throws IOException if shutdown reports failed tables for a batch job
+ * @throws RuntimeException if shutdown fails without a batch failed-table
report
*/
@Override
public void close() throws IOException {
+ CompletableFuture<Throwable> flush =
activeTimerFlush.getAndSet(TIMER_FLUSH_CLOSED);
Review Comment:
## `close()` has unbounded waits that `awaitTermination` cannot cover
**Location**: `MultiTableSinkWriter.close()`
There are two `synchronized` waits in `close()` that are neither
interruptible nor timeout-bounded, and the 60s
`awaitTermination(EXECUTOR_CLOSE_TIMEOUT_SECONDS)` runs only **after** both of
them. If a flush action hangs while holding a queue lock, `close()` never
reaches the timeout fallback.
### 1. `synchronized (this)` handshake
`handleTableFailure` is a `synchronized` method and holds the `this` monitor
for the entire isolation, which acquires every queue lock in
`removeTableWriters`. If a flush worker is stuck in `flushAction.run()` while
holding `runnable.get(i)`, an in-progress isolation on another thread blocks
forever waiting for that queue lock, so the `this` monitor is never released.
`close()` then blocks forever on `synchronized (this)` before it even reaches
the writer-closing loop.
### 2. `synchronized (runnable.get(i))` closing loop
Even if the handshake succeeds, the stuck worker still holds the queue lock,
so `close()` blocks forever in the closing loop.
### Root cause
`shutdownNow()` only sets the interrupt flag; it cannot release a
`synchronized` lock. When a flush action blocks on non-interruptible I/O (JDBC
socket read, HTTP request without timeout, `CompletableFuture.join()`), the
queue lock is held until the action returns on its own. The `synchronized
(this)` handshake solves the correctness problem (closing a writer twice) but
not the liveness problem.
### Why this PR matters here
On `dev`, flush runs serially on the task thread, so `close()` and flush
cannot run concurrently. This PR runs flush workers on the executor and
deliberately implements "close waits for in-flight flush to exit before closing
the writer." The cost is that `close()` duration now depends on when the
connector action returns.
### Test blind spot
`testAggregatedFlushReportsFailureAndClosesQueuesIndependently` blocks the
slow flush on `releaseSlowFlush.join()` and releases it manually in `finally`.
In production there is no such role to release a hung connector I/O.
### Impact
`close()` is called by `SinkFlowLifeCycle.close()` on the task shutdown
path. A hung task thread blocks job stop/restart, and a thread dump shows
`close()` stuck on `synchronized` — not obvious to diagnose.
**Severity**: Medium — non-blocking for merge (the same assumption already
exists on `dev` when a consumer thread hangs in `writeRow` while holding the
lock), but please state a position before merging: either document the
assumption ("flush actions must respond to interrupts and return within a
bounded time; callers do not time out and abandon the wait") or leave a
follow-up issue for a bounded close.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]