xiaojin.wy created FLINK-15098:
----------------------------------
Summary: CsvTableSink has no properties of a OVERWRITE mode to
write, but the exception hint the user use it when a file or directory already
exists
Key: FLINK-15098
URL: https://issues.apache.org/jira/browse/FLINK-15098
Project: Flink
Issue Type: Improvement
Components: Table SQL / Client
Affects Versions: 1.10.0
Reporter: xiaojin.wy
When using sql-client to execute a sql , there will be a IOException just like
the content below if the file or directory already exists. But there is no
place for the user to set the overwrite mode. In another word, the hint tell
users to use overwrite mode, but flink has not support it now.
The exception:
Caused by: org.apache.flink.runtime.client.JobExecutionException: Cannot
initialize task 'HashAggregate(isMerge=[true], select=[Final_AVG(sum$0,
count$1) AS avg_1]) -> SinkConversionToRow -> Map -> Sink:
CsvTableSink(avg_1)': File or directory already exists. Existing files and
directories are not overwritten in NO_OVERWRITE mode. Use OVERWRITE mode to
overwrite existing files and directories.Caused by:
org.apache.flink.runtime.client.JobExecutionException: Cannot initialize task
'HashAggregate(isMerge=[true], select=[Final_AVG(sum$0, count$1) AS avg_1]) ->
SinkConversionToRow -> Map -> Sink: CsvTableSink(avg_1)': File or directory
already exists. Existing files and directories are not overwritten in
NO_OVERWRITE mode. Use OVERWRITE mode to overwrite existing files and
directories. at
org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:216)
at
org.apache.flink.runtime.scheduler.SchedulerBase.createExecutionGraph(SchedulerBase.java:243)
at
org.apache.flink.runtime.scheduler.SchedulerBase.createAndRestoreExecutionGraph(SchedulerBase.java:215)
at
org.apache.flink.runtime.scheduler.SchedulerBase.<init>(SchedulerBase.java:203)
at
org.apache.flink.runtime.scheduler.DefaultScheduler.<init>(DefaultScheduler.java:119)
at
org.apache.flink.runtime.scheduler.DefaultSchedulerFactory.createInstance(DefaultSchedulerFactory.java:104)
at
org.apache.flink.runtime.jobmaster.JobMaster.createScheduler(JobMaster.java:278)
at org.apache.flink.runtime.jobmaster.JobMaster.<init>(JobMaster.java:266) at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:98)
at
org.apache.flink.runtime.jobmaster.factories.DefaultJobMasterServiceFactory.createJobMasterService(DefaultJobMasterServiceFactory.java:40)
at
org.apache.flink.runtime.jobmaster.JobManagerRunnerImpl.<init>(JobManagerRunnerImpl.java:146)
... 10 moreCaused by: java.io.IOException: File or directory already exists.
Existing files and directories are not overwritten in NO_OVERWRITE mode. Use
OVERWRITE mode to overwrite existing files and directories. at
org.apache.flink.core.fs.FileSystem.initOutPathDistFS(FileSystem.java:937) at
org.apache.flink.api.common.io.FileOutputFormat.initializeGlobal(FileOutputFormat.java:286)
at
org.apache.flink.runtime.jobgraph.InputOutputFormatVertex.initializeOnMaster(InputOutputFormatVertex.java:100)
at
org.apache.flink.runtime.executiongraph.ExecutionGraphBuilder.buildGraph(ExecutionGraphBuilder.java:212)
... 20 more
The properties that a csvTableSink support now:
connector.type, connector.property-version, connector.path, format.type,
format.property-version, format.field-delimiter
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
