hladush commented on code in PR #14797:
URL: https://github.com/apache/iceberg/pull/14797#discussion_r2678294647
##########
docs/docs/kafka-connect.md:
##########
@@ -364,6 +368,39 @@ See above for creating two tables.
}
```
+### Change data capture
+This example applies inserts, updates, and deletes based on the value of a
field in the record.
+For example, if the `_cdc_op` field is set to `I` or `R` then the record is
inserted, if `U` then it is
+upserted, and if `D` then it is deleted. This requires that the table be in
Iceberg v2 format.
+The Iceberg identifier field(s) are used to identify a row, if that is not set
for the table,
+then the `iceberg.tables.default-id-columns` or `iceberg.table.\<table
name\>.id-columns`configuration
+can be set instead. CDC can be combined with multi-table fan-out. The property
`iceberg.tables.use-dv`
+can be set to `false` to disable delete vector (DV) mode for compatibility
with v2 tables. Note that DV mode
Review Comment:
Should it be set to false ? instead of can
Because it can be Must not use DVs for position deletes in V2:
at
org.apache.iceberg.relocated.com.google.common.base.Preconditions.checkArgument(Preconditions.java:217)
~[iceberg-bundled-guava-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.MergingSnapshotProducer.validateNewDeleteFile(MergingSnapshotProducer.java:286)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.MergingSnapshotProducer.addInternal(MergingSnapshotProducer.java:261)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.MergingSnapshotProducer.add(MergingSnapshotProducer.java:252)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at org.apache.iceberg.BaseRowDelta.addDeletes(BaseRowDelta.java:70)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ~[?:?]
at
org.apache.iceberg.connect.channel.Coordinator.commitToTable(Coordinator.java:283)
~[iceberg-kafka-connect-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.connect.channel.Coordinator.lambda$doCommit$1(Coordinator.java:167)
~[iceberg-kafka-connect-1.11.0-SNAPSHOT.jar:?]
at
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at org.apache.iceberg.util.Tasks$Builder$1.run(Tasks.java:315)
~[iceberg-core-1.11.0-SNAPSHOT.jar:?]
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
~[?:?]
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
~[?:?]
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
~[?:?]
at java.base/java.lang.Thread.run(Thread.java:840) ~[?:?]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
