[
https://issues.apache.org/jira/browse/FLINK-22458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Flink Jira Bot updated FLINK-22458:
-----------------------------------
Labels: auto-deprioritized-major stale-minor (was:
auto-deprioritized-major)
I am the [Flink Jira Bot|https://github.com/apache/flink-jira-bot/] and I help
the community manage its development. I see this issues has been marked as
Minor but is unassigned and neither itself nor its Sub-Tasks have been updated
for 180 days. I have gone ahead and marked it "stale-minor". If this ticket is
still Minor, please either assign yourself or give an update. Afterwards,
please remove the label or in 7 days the issue will be deprioritized.
> Failed when operating empty catalog table
> -----------------------------------------
>
> Key: FLINK-22458
> URL: https://issues.apache.org/jira/browse/FLINK-22458
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Hive
> Affects Versions: 1.12.2
> Environment: Flink: 1.12.2
> Reporter: Spongebob
> Priority: Minor
> Labels: auto-deprioritized-major, stale-minor
>
> The pipline might like: HiveTable -> FlinkCatalogTable(might be empty) ->
> HiveTable
> It runs normally when the FlinkCatalogTable is not empty, But When
> FlinkCatalogTable is empty, Jobmanager throws this exception:
> {code:java}
> java.lang.Exception: Failed to finalize execution on master at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.vertexFinished(ExecutionGraph.java:1373)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionVertex.executionFinished(ExecutionVertex.java:877)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.Execution.markFinished(Execution.java:1241)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.updateStateInternal(ExecutionGraph.java:1610)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.updateState(ExecutionGraph.java:1584)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:663)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:447)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_251] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_251] at java.lang.reflect.Method.invoke(Method.java:498)
> ~[?:1.8.0_251] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.Actor.aroundReceive(Actor.scala:517)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.Actor.aroundReceive$(Actor.scala:515)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.ActorCell.invoke(ActorCell.scala:561)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.run(Mailbox.scala:225)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.exec(Mailbox.scala:235)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> [flownData-1.0-jar-with-dependencies.jar:?] Caused by:
> org.apache.flink.table.api.TableException: Exception in finalizeGlobal at
> org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:94)
> ~[flink-table-blink_2.12-1.12.2.jar:1.12.2] at
> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.finalizeOnMaster(InputOutputFormatVertex.java:148)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.vertexFinished(ExecutionGraph.java:1368)
> ~[flownData-1.0-jar-with-dependencies.jar:?] ... 35 more Caused by:
> java.io.FileNotFoundException: File
> hdfs://hadmaster:9898/data/hive_warehouse/ticket_data.db/dwd_flown_fin_segment/.staging_1619403238924
> does not exist. at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:1053)
> ~[hadoop-hdfs-client-3.1.3.jar:?] at
> org.apache.hadoop.hdfs.DistributedFileSystem.access$1000(DistributedFileSystem.java:131)
> ~[hadoop-hdfs-client-3.1.3.jar:?] at
> org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1113)
> ~[hadoop-hdfs-client-3.1.3.jar:?] at
> org.apache.hadoop.hdfs.DistributedFileSystem$24.doCall(DistributedFileSystem.java:1110)
> ~[hadoop-hdfs-client-3.1.3.jar:?] at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> ~[hadoop-common-3.1.3.jar:?] at
> org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:1120)
> ~[hadoop-hdfs-client-3.1.3.jar:?] at
> org.apache.flink.hive.shaded.fs.hdfs.HadoopFileSystem.listStatus(HadoopFileSystem.java:170)
> ~[flink-connector-hive_2.12-1.12.2.jar:1.12.2] at
> org.apache.flink.table.filesystem.PartitionTempFileManager.headCheckpoints(PartitionTempFileManager.java:137)
> ~[flink-table-blink_2.12-1.12.2.jar:1.12.2] at
> org.apache.flink.table.filesystem.FileSystemCommitter.commitUpToCheckpoint(FileSystemCommitter.java:93)
> ~[flink-table-blink_2.12-1.12.2.jar:1.12.2] at
> org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:92)
> ~[flink-table-blink_2.12-1.12.2.jar:1.12.2] at
> org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.finalizeOnMaster(InputOutputFormatVertex.java:148)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.vertexFinished(ExecutionGraph.java:1368)
> ~[flownData-1.0-jar-with-dependencies.jar:?] ... 35 more 2021-04-26
> 10:14:31,726 INFO org.apache.flink.runtime.executiongraph.ExecutionGraph [] -
> Job insert-into_hive.ticket_data.dwd_flown_fin_segment
> (5e2132be01c51bb5279ed535b730448b) switched from state RUNNING to FAILING.
> org.apache.flink.runtime.JobException: Recovery is suppressed by
> NoRestartBackoffTimeStrategy at
> org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:118)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getGlobalFailureHandlingResult(ExecutionFailureHandler.java:93)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.DefaultScheduler.handleGlobalFailure(DefaultScheduler.java:252)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.UpdateSchedulerNgOnInternalFailuresListener.notifyGlobalFailure(UpdateSchedulerNgOnInternalFailuresListener.java:62)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.failGlobal(ExecutionGraph.java:1116)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.vertexFinished(ExecutionGraph.java:1373)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionVertex.executionFinished(ExecutionVertex.java:877)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.Execution.markFinished(Execution.java:1241)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.updateStateInternal(ExecutionGraph.java:1610)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.executiongraph.ExecutionGraph.updateState(ExecutionGraph.java:1584)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:663)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.scheduler.SchedulerNG.updateTaskExecutionState(SchedulerNG.java:89)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:447)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_251] at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:1.8.0_251] at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:1.8.0_251] at java.lang.reflect.Method.invoke(Method.java:498)
> ~[?:1.8.0_251] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:305)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:212)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:77)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:158)
> ~[flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.Actor.aroundReceive(Actor.scala:517)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.Actor.aroundReceive$(Actor.scala:515)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.actor.ActorCell.invoke(ActorCell.scala:561)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.run(Mailbox.scala:225)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.Mailbox.exec(Mailbox.scala:235)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> [flownData-1.0-jar-with-dependencies.jar:?] at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> [flownData-1.0-jar-with-dependencies.jar:?]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
