The GitHub Actions job "Labeler-Trigger" on flink-cdc.git/FLINK-40560 has 
failed.
Run started by GitHub user tchivs (triggered by tchivs).

Head commit for run:
47ff606d5f30bcf7d3906200cff5f40b5fb5f347 / tchivs <[email protected]>
[FLINK-40560][postgres] Drop the replication slot when a snapshot-only source 
finishes

FLINK-38277 made a snapshot-only Postgres source drop its replication slot once 
the
stream split finishes, but gated it on

    streamSplit.getStartingOffset().isAtOrAfter(streamSplit.getEndingOffset())

which does not answer whether the split finished. 
HybridSplitAssigner#createStreamSplit
seeds those two offsets with the lowest and highest high watermark of the 
finished
snapshot splits, so they already differ whenever there is more than one split 
or any
concurrent write. Afterwards only 
IncrementalSourceRecordEmitter#updateStreamSplitState
advances the starting offset, and only for data-change records and heartbeats. 
A split
that reaches its ending offset without emitting such a record — a captured 
publication
with no traffic emits none — therefore left its slot behind on every run, 
pinning WAL
until an operator dropped it by hand.

IncrementalSourceReader#onSplitFinished already documents that a stream split 
finishes
for exactly two reasons: the enumerator suspended it so newly added tables can 
be
snapshotted, or it reached its ending offset. Only the second means the bounded 
read is
over, and the reader context already distinguishes them, so gate on
!isStreamSplitReaderSuspended() instead. That also removes the offset 
dereference and
with it a latent NPE when the stopping offset is null.

PostgresDialect#removeSlot swallows failures and returns false, so a failed 
cleanup was
only visible as an INFO line reading "false". Log it at WARN with the manual
pg_drop_replication_slot hint instead.

The reader keeps its own reference to IncrementalSourceReaderContext because 
the base
class holds its copy privately; that avoids widening the shared base class API.

Two tests in PostgresSourceReaderTest cover both finish reasons: a split whose 
starting
offset is behind its ending offset must release the slot, and a split suspended 
via
StreamSplitUpdateRequestEvent must keep it, since resuming needs the slot's 
position.
Reverting the guard turns the first one red.

Report URL: https://github.com/apache/flink-cdc/actions/runs/33944484898

With regards,
GitHub Actions via GitBox

Reply via email to