zhang-arvin opened a new pull request, #11864:
URL: https://github.com/apache/seatunnel/pull/11864
## Purpose
Fix three issues that cause Postgres-CDC to silently skip rows after
savepoint recovery, as described in issue #11847.
## Brief change log
1. **PostgresWalFetchTask.commitCurrentOffset used wrong LSN key**
- `commitCurrentOffset` was reading the LSN via `offset.getLsn()` which
uses `SourceInfo.LSN_KEY` ("lsn"), but Debezium's `commitOffset` expects the
`LAST_COMMIT_LSN_KEY` ("lsn_commit") field.
- Added `getLsnCommit()` to `LsnOffset` that reads from
`LAST_COMMIT_LSN_KEY` with fallback to `LSN_KEY`, and updated
`commitCurrentOffset` to use the committed LSN.
2. **IncrementalSourceStreamFetcher.shouldEmit used strict greater-than
comparison**
- In non-exactly-once mode, `shouldEmit` used
`isAfter(splitStartWatermark)` (strict greater-than), so after recovery the
first event with the same LSN as `splitStartWatermark` was filtered out.
- Changed to `isAtOrAfter(splitStartWatermark)` to include the
watermark-position event.
3. **PostgresWalFetchTask.execute missing error handler check**
- `execute()` did not check `errorHandler.getProducerThrowable()` after
the streaming loop completed, so streaming failures could be silently swallowed.
- Added error handler check to surface streaming errors.
## Verify this pull request
- [ ] This change is already covered by existing tests, such as
`LsnOffsetFactoryTest`.
- [ ] Manual verification: run Postgres-CDC with savepoint and verify no
rows are skipped after recovery.
## Does this PR potentially affect the following other parts of the project?
- [x] Connector-CDC
- [x] Connector-CDC-Postgres
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
