Mask0407 opened a new issue #3280:
URL: https://github.com/apache/iceberg/issues/3280
In the Flink stream processing, I tried to insert the Iceberg table in
parallel using SQL, and the result was as follows:
```shell
2021-10-12 13:59:21,259 WARN org.apache.iceberg.util.Tasks
[] - Retrying task after failure: Base metadata location
'hdfs:///usr/hive/warehouse/iceberg/xxx/metadata/0010.metadata.json' is not
same as the current table metadata location
'hdfs:///usr/hive/warehouse/iceberg/xxx/metadata/0011.metadata.json' for xxx
org.apache.iceberg.exceptions.CommitFailedException: Base metadata location
'hdfs:///usr/hive/warehouse/iceberg/xxx/metadata/0010.metadata.json' is not
same as the current table metadata location
'hdfs:///usr/hive/warehouse/iceberg/xxx/metadata/0011.metadata.json' for xxx
at
org.apache.iceberg.hive.HiveTableOperations.doCommit(HiveTableOperations.java:240)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:126)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.SnapshotProducer.lambda$commit$2(SnapshotProducer.java:300)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:405)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:214)
~[dw-0.1.2.jar:?]
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:198)
~[dw-0.1.2.jar:?]
at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:190)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.SnapshotProducer.commit(SnapshotProducer.java:282)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitOperation(IcebergFilesCommitter.java:308)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitDeltaTxn(IcebergFilesCommitter.java:277)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.commitUpToCheckpoint(IcebergFilesCommitter.java:219)
~[dw-0.1.2.jar:?]
at
org.apache.iceberg.flink.sink.IcebergFilesCommitter.notifyCheckpointComplete(IcebergFilesCommitter.java:189)
~[dw-0.1.2.jar:?]
at
org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.notifyCheckpointComplete(StreamOperatorWrapper.java:99)
~[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.notifyCheckpointComplete(SubtaskCheckpointCoordinatorImpl.java:330)
~[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.notifyCheckpointComplete(StreamTask.java:1092)
~[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointCompleteAsync$11(StreamTask.java:1057)
~[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$notifyCheckpointOperation$13(StreamTask.java:1080)
~[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:50)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.mailbox.Mail.run(Mail.java:90)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.processMail(MailboxProcessor.java:317)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:189)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:619)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:583)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:758)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:573)
[flink-dist_2.12-1.12.5.jar:1.12.5]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
```
>My Iceberg Catalog uses Hive Catalog
I hope to get your help.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
