huzk8 opened a new issue #4222:
URL: https://github.com/apache/iceberg/issues/4222
## desc
when i delete data in mysql，flink job will error.
## env dependence
flink version :1.13.5-2.11
Iceberg version 0.13.1
## flinksql info
```
CREATE CATALOG hadoop_catalog WITH (
'type'='iceberg',
'catalog-type'='hadoop',
'warehouse'='hdfs:///iceberg/warehouse/path',
'property-version'='1'
);
CREATE TABLE products (
id INT ,
name STRING,
description STRING,
weight float,
PRIMARY KEY (id) NOT ENFORCED
)WITH (
'connector' = 'mysql-cdc',
'hostname' = 'localhost',
'port' = '3306',
'username' = 'root',
'password' = '123456',
'database-name' = 'inventory',
'table-name' = 'products'
);
CREATE TABLE `hadoop_catalog`.`default`.`t_products`(
id INT ,
name STRING,
description STRING,
weight float,
PRIMARY KEY (id) NOT ENFORCED
);
SET execution.checkpointing.interval = 3s;
insert into `hadoop_catalog`.`default`.`t_products` select * from products;
```
## error msg
```
java.lang.IllegalArgumentException: Cannot write delete files in a v1 table
at
org.apache.iceberg.ManifestFiles.writeDeleteManifest(ManifestFiles.java:154)
at
org.apache.iceberg.SnapshotProducer.newDeleteManifestWriter(SnapshotProducer.java:374)
at
org.apache.iceberg.MergingSnapshotProducer.lambda$newDeleteFilesAsManifests$10(MergingSnapshotProducer.java:681)
at java.util.HashMap.forEach(HashMap.java:1288)
at
org.apache.iceberg.MergingSnapshotProducer.newDeleteFilesAsManifests(MergingSnapshotProducer.java:678)
at
org.apache.iceberg.MergingSnapshotProducer.prepareDeleteManifests(MergingSnapshotProducer.java:664)
at
org.apache.iceberg.MergingSnapshotProducer.apply(MergingSnapshotProducer.java:533)
at org.apache.iceberg.SnapshotProducer.apply(SnapshotProducer.java:164)
at
org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:283)
at
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:404)
at
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
at org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:282)
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitOperation(IcebergFilesCommitter.java:312)
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitDeltaTxn(IcebergFilesCommitter.java:299)
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitUpToCheckpoint(IcebergFilesCommitter.java:218)
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.initializeState(IcebergFilesCommitter.java:153)
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:118)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:290)
at
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:441)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:585)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:565)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:650)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:540)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
