darinspivey opened a new pull request, #39:
URL: https://github.com/apache/pulsar-connectors/pull/39
### Problem
Under sustained catch-up (a backlog is present and the arrival rate is
greater than the flush rate), `JdbcAbstractSink.flush()` recurses via `if
(needAnotherRound) { flush(); }` and never unwinds, eventually throwing
`StackOverflowError`. The handler is `catch (Exception)`, not `Throwable`, so
the `Error` bypasses the `fatal()` path added in PIP-297 (#25195). The
`ScheduledThreadPoolExecutor` swallows it and cancels the periodic flush, and
the sink hangs silently: 1/1 Ready, zero writes and acks, the consumer blocked
at its unacked cap, the whole Shared subscription stalled, and no logged error.
This reproduces in production JDBC sinks during backlog recovery and traffic
spikes. It is present in v4.1.3, branch-4.1, and current `master`.
### Fix
1. Convert the recursion to a `do { ... } while (needAnotherRound)` loop.
2. Widen `catch (Exception)` to `catch (Throwable)` and change
`fatal(Exception)` to `fatal(Throwable)` so an `Error` still triggers the
instance restart instead of vanishing.
3. Release `isFlushing` in a single `finally`, which also fixes a latent
stuck-flag path.
### Note on diff size
Most of the diff is re-indentation from wrapping the body in `try/finally`
and `do/while`. Reviewing with `git diff -w` shows the real delta is about 30
lines.
### Testing
Adds `SqliteJdbcSinkTest.testFlushDrainsLargeBacklogWithoutStackOverflow`,
which freezes flushing, queues a large backlog, then drives a single `flush()`.
With `batchSize=1` the drain performs one round per record, so the backlog size
equals the old recursion depth. The test throws `StackOverflowError` on the old
recursive code and passes on the iterative version, and asserts `fatal()` is
never invoked. The full `jdbc` test suite, `build`, `rat`, and `spotlessCheck`
pass locally.
Fixes #38
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
