JTaky opened a new pull request, #9391:
URL: https://github.com/apache/paimon/pull/9391
### Purpose
When we run Paimon CDC Sink (aka Dynamic Paimon Sink) on the failures it
would be handy to see the table name, thus we can see how to tune the given
table.
Example of the stacktrace which is not practical:
```
java.lang.RuntimeException: There is no snapshot for commit user
44e8bd48-3cf4-4f51-98d8-70b56ae6a680 and identifier 71. This is unexpected.
at
org.apache.paimon.iceberg.IcebergCommitCallback.lambda$retry$1(IcebergCommitCallback.java:246)
at java.base/java.util.Optional.orElseThrow(Optional.java:403)
at
org.apache.paimon.iceberg.IcebergCommitCallback.retry(IcebergCommitCallback.java:240)
at
org.apache.paimon.operation.FileStoreCommitImpl.lambda$filterCommitted$1(FileStoreCommitImpl.java:277)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at
org.apache.paimon.operation.FileStoreCommitImpl.filterCommitted(FileStoreCommitImpl.java:277)
at
org.apache.paimon.table.sink.TableCommitImpl.filterAndCommitMultiple(TableCommitImpl.java:284)
at
org.apache.paimon.flink.sink.StoreCommitter.filterAndCommit(StoreCommitter.java:111)
at
org.apache.paimon.flink.sink.StoreMultiCommitter.filterAndCommit(StoreMultiCommitter.java:182)
at
org.apache.paimon.flink.sink.RestoreCommittableStateManager.recover(RestoreCommittableStateManager.java:81)
at
org.apache.paimon.flink.sink.RestoreAndFailCommittableStateManager.recover(RestoreAndFailCommittableStateManager.java:86)
at
org.apache.paimon.flink.sink.RestoreCommittableStateManager.initializeState(RestoreCommittableStateManager.java:76)
at
org.apache.paimon.flink.sink.CommitterOperator.initializeState(CommitterOperator.java:147)
at
org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.initializeOperatorState(StreamOperatorStateHandler.java:142)
at
org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:304)
at
org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:106)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreStateAndGates(StreamTask.java:858)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$restoreInternal$5(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:812)
at
org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:771)
at
org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:963)
at
org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:932)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:756)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:568)
```
Ideally you would like to track the metadata and troubleshoot the root cause
of missing snapshot, but since there is no table name reporter - it is harder
to act.
### Tests
- mvn -Ppaimon-iceberg -pl
paimon-core,paimon-iceberg,paimon-hive/paimon-hive-catalog -am compile passes
- mvn -Ppaimon-iceberg -pl
paimon-core,paimon-iceberg,paimon-hive/paimon-hive-catalog spotless:check passes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
