macdoor615 created FLINK-25049:
----------------------------------
Summary: sql-client.sh execute batch job in async mode failed with
java.io.FileNotFoundException
Key: FLINK-25049
URL: https://issues.apache.org/jira/browse/FLINK-25049
Project: Flink
Issue Type: Bug
Components: Table SQL / Client
Affects Versions: 1.14.0
Environment: {code:java}
//代码占位符
{code}
Reporter: macdoor615
execute multi simple sql in a sql file, like
{code:java}
insert overwrite bnpmp.p_biz_hcd_5m select * from bnpmp.p_biz_hcd_5m where
dt='2021-11-22';
insert overwrite bnpmp.p_biz_hjr_5m select * from bnpmp.p_biz_hjr_5m where
dt='2021-11-22';
insert overwrite bnpmp.p_biz_hswtv_5m select * from bnpmp.p_biz_hswtv_5m where
dt='2021-11-22';
...
{code}
if
{code:java}
SET table.dml-sync = true;{code}
execute properly.
if
{code:java}
SET table.dml-sync = false;{code}
some SQL Job failed with the following error
{code:java}
Caused by: java.lang.Exception: Failed to finalize execution on master ... 37
more Caused by: org.apache.flink.table.api.TableException: Exception in
finalizeGlobal at
org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:91)
at
org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.finalizeOnMaster(InputOutputFormatVertex.java:148)
at
org.apache.flink.runtime.executiongraph.DefaultExecutionGraph.vertexFinished(DefaultExecutionGraph.java:1086)
... 36 more Caused by: java.io.FileNotFoundException: File
hdfs://service1/user/hive/warehouse/bnpmp.db/p_snmp_f5_traffic_5m/.staging_1637821441497
does not exist. at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:901)
at
org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:112)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961)
at
org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:958)
at
org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at
org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:958)
at
org.apache.flink.hive.shaded.fs.hdfs.HadoopFileSystem.listStatus(HadoopFileSystem.java:170)
at
org.apache.flink.table.filesystem.PartitionTempFileManager.listTaskTemporaryPaths(PartitionTempFileManager.java:104)
at
org.apache.flink.table.filesystem.FileSystemCommitter.commitPartitions(FileSystemCommitter.java:77)
at
org.apache.flink.table.filesystem.FileSystemOutputFormat.finalizeGlobal(FileSystemOutputFormat.java:89)
... 38 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
