davidzollo commented on PR #10334: URL: https://github.com/apache/seatunnel/pull/10334#issuecomment-3794424438
Hi @rucciva, thanks for bringing up this feature request and working on the implementation. The proposed feature to allow disabling the mandatory `REPLICA IDENTITY FULL` check is very valuable, especially for "append-only" scenarios like the Outbox Pattern you mentioned. It provides a necessary flexibility to optimize PostgreSQL performance by reducing WAL volume and I/O overhead. I've reviewed the current state of PR #10334 and agree with the points raised by @zhangshenghang. To get this merged, we need to address the following: **1. Documentation (Critical)** The current documentation explicitly states that `REPLICA IDENTITY FULL` is required. We must update the `docs/en/connector-v2/source/Postgres-CDC.md` (and `docs/zh/...` if possible, otherwise we can start with English) to: - Add the new option `require-replica-identity-full` to the Options table. - **Add a Warning Block**: Explicitly warn users that setting this to `false` on tables that have `UPDATE` or `DELETE` operations might result in incomplete event data (missing "before" image), which can lead to data inconsistency in the sink. Emphasize that this should primarily be used for **Append-Only** tables. **2. Test Coverage** Please add unit tests in `PostgresDialectTest` or similar to verify: - Scenario A: `require-replica-identity-full = true` (default) throws exception when identity is not FULL. - Scenario B: `require-replica-identity-full = false` does **not** throw exception even if identity is DEFAULT or NONE. You can mock the `PostgresConnection` behavior to simulate different server-side Replica Identity settings without needing a real database instance for unit tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
