The GitHub Actions job "Flink Connector JDBC CI" on 
flink-connector-jdbc.git/FLINK-40779-splitter-state-checkpoint-race has failed.
Run started by GitHub user Admaing (triggered by Admaing).

Head commit for run:
4cb6d9f12f594f39d6caa88733210d8bee59d9e5 / yongfu.gao <[email protected]>
[FLINK-40779][connector-jdbc] Snapshot splitter state together with its splits

JdbcSourceEnumerator enumerates splits through callAsync, so the splitter
cursor advances on a worker thread while the produced splits only reach
`unassigned` in the handler on the coordinator thread. Reading the splitter
state directly in snapshotState() could therefore record a state that had
already moved past splits that were still in flight; those splits were never
re-enumerated after a restore.

Capture the splits and the splitter state together inside the callable, and
commit the state from the handler. Enumerations are limited to one in flight,
so the committed state and the buffered splits stay in sync no matter how
callAsync schedules the callables; relying on their completion order would not
be safe. snapshotState() now uses that committed state.

For the setSql path the state is also seeded from the provider when no state is
configured. Stateful providers such as JdbcSlideTimingParameterProvider are
mutated in place while enumerating, so a first-batch checkpoint taken with a
null state could not roll the provider back on restore and the in-flight first
batch was still lost.

The fix sits in the enumerator rather than in the individual
SplitterEnumerator implementations so that every implementation is covered
(SlideTimingSplitterEnumerator is exposed the same way) and the
PublicEvolving interface stays unchanged.

On the error path the committed state lags behind, so a restore may
re-enumerate splits that were already produced. That is at-least-once rather
than data loss.

Generated-by: DeepSeek Harness (deepseek-v4.1-flash)

Report URL: 
https://github.com/apache/flink-connector-jdbc/actions/runs/38029587979

With regards,
GitHub Actions via GitBox

Reply via email to