dixingxing0 opened a new issue #2470: URL: https://github.com/apache/iceberg/issues/2470
We've met an NullPointerException in our flink job, has anyone met this before? It seems like `rawWriter` is null which means `close` method was already invoked: https://github.com/apache/orc/blob/22828f79a526069d9629719c9476b7addad91ae6/java/core/src/java/org/apache/orc/impl/PhysicalFsWriter.java#L416 https://github.com/apache/orc/blob/22828f79a526069d9629719c9476b7addad91ae6/java/core/src/java/org/apache/orc/impl/PhysicalFsWriter.java#L486 ```java java.io.IOException: Could not perform checkpoint 1011 for operator SourceConversion(table=[runtime-catalog.fdm_realtime.fdm_realtime_rcm_feature, source: xxxxxxxxx -> IcebergStreamWriter (15/160). at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:892) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.io.CheckpointBarrierHandler.notifyCheckpoint(CheckpointBarrierHandler.java:113) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.io.CheckpointBarrierAligner.processBarrier(CheckpointBarrierAligner.java:137) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.io.CheckpointedInputGate.pollNext(CheckpointedInputGate.java:93) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.io.StreamTaskNetworkInput.emitNext(StreamTaskNetworkInput.java:158) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:67) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:351) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxStep(MailboxProcessor.java:191) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:181) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:566) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:536) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at java.lang.Thread.run(Thread.java:745) ~[?:1.8.0_92] Caused by: java.lang.NullPointerException at org.apache.iceberg.shaded.org.apache.orc.impl.PhysicalFsWriter.writeFileMetadata(PhysicalFsWriter.java:416) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.shaded.org.apache.orc.impl.WriterImpl.writeMetadata(WriterImpl.java:571) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.shaded.org.apache.orc.impl.WriterImpl.writeFooter(WriterImpl.java:636) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.shaded.org.apache.orc.impl.WriterImpl.close(WriterImpl.java:728) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.orc.OrcFileAppender.close(OrcFileAppender.java:127) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.io.DataWriter.close(DataWriter.java:63) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.closeCurrent(BaseTaskWriter.java:283) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.io.BaseTaskWriter$BaseRollingWriter.close(BaseTaskWriter.java:299) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.io.PartitionedFanoutWriter.close(PartitionedFanoutWriter.java:65) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.io.BaseTaskWriter.complete(BaseTaskWriter.java:84) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.iceberg.flink.sink.IcebergStreamWriter.prepareSnapshotPreBarrier(IcebergStreamWriter.java:65) ~[iceberg-flink-runtime-0.11.0-auto.jar-1617266335824.jar-1617267644514.jar:?] at org.apache.flink.streaming.runtime.tasks.OperatorChain.prepareSnapshotPreBarrier(OperatorChain.java:266) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.checkpointState(SubtaskCheckpointCoordinatorImpl.java:249) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$performCheckpoint$8(StreamTask.java:921) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.runThrowing(StreamTaskActionExecutor.java:47) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.performCheckpoint(StreamTask.java:911) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] at org.apache.flink.streaming.runtime.tasks.StreamTask.triggerCheckpointOnBarrier(StreamTask.java:879) ~[flink-dist_2.11-1.11.2-auto1.1-SNAPSHOT.jar:1.11.2-auto1.1-SNAPSHOT] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
