mattp5657 opened a new pull request, #3640:
URL: https://github.com/apache/iggy/pull/3640
## Which issue does this PR address?
Relates to #3582
## Rationale
Builtin CDC mode currently produces zero messages against a real
database - wrong plugin/read-path pairing, and a parser matching a
format no real Postgres output plugin emits. This PR is scoped to
getting CDC actually working and provably correct, nothing more.
Left out of scope here, and why:
- `pgoutput` as the long-term plugin - real binary decoder + streaming
connection is a much bigger lift; `test_decoding` unblocks correctness
today. Target once `cdc_backend = "pg_replicate"` gets picked up.
- At-least-once / crash safety (slot advances before send is confirmed,
no LSN tracked) - needs its own review since it touches the
state-persistence contract. Follow-up: #TODO.
- Cosmetic items (UUID vs PK as message id, processing time vs WAL
commit time, `old_data` unpopulated) - mechanical, separate PR.
Bundling all of the above into one PR would make it unreviewable -
splitting it this way keeps each piece independently checkable.
## What changed?
Before: wrong slot plugin, a parser matching a format no real Postgres
output emits, and invalid setup SQL - CDC produced zero messages, and
several related bugs failed silently too. After: parser rewritten
against real captured output, setup/poll SQL fixed and bounded, and
those failure modes now work correctly or fail loudly. Covered by
corpus-driven unit tests plus integration tests against a live
`wal_level=logical` container.
- Parser rewritten against real `test_decoding` output
- Slot/setup SQL fixed, including the upgrade path and table filtering
- Batch handling hardened (bad rows, unbounded backlog)
- Dead config fields removed
## Local Execution
- Passed
- Pre-commit hooks ran
## AI Usage
Claude was used to help generate and review this PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
