darinspivey opened a new issue, #38:
URL: https://github.com/apache/pulsar-connectors/issues/38
**What happened**
Under sustained catch-up (a backlog is present and message arrival outpaces
the flush rate), `JdbcAbstractSink.flush()` recurses instead of looping and
eventually throws `StackOverflowError`. The handler is `catch (Exception)`, so
the `Error` is not caught. The `ScheduledThreadPoolExecutor` running the flush
swallows it and cancels the periodic flush task. The sink then hangs silently:
1/1 Ready, no restart, no logged error, zero writes and zero acks, and the
consumer blocks at its unacked cap, stalling the whole Shared subscription.
**Where**
`jdbc/core/src/main/java/org/apache/pulsar/io/jdbc/JdbcAbstractSink.java`,
the `if (needAnotherRound) { flush(); }` tail call in `flush()`. Each queued
full batch adds a stack frame, so the recursion depth grows with the backlog.
**Affected versions**
Present in v4.1.3, branch-4.1, and current `master`.
**Impact**
Any JDBC sink that falls behind (backlog recovery, traffic spike) can wedge
permanently with no signal until throughput is noticed sitting at zero.
Restarting the instance clears it until the next time it falls behind.
**Fix**
Drain iteratively with a `do { ... } while (needAnotherRound)` loop instead
of recursing, and widen the handler to `catch (Throwable)` so an `Error` still
reaches `fatal()` and restarts the instance instead of vanishing. PR to follow.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
