[
https://issues.apache.org/jira/browse/FLINK-15066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002754#comment-17002754
]
Jingsong Lee commented on FLINK-15066:
--------------------------------------
About writing CSV in SQL world:
Hive/spark behavior: the output path must be a directory. It can be checked
whether to append or overwrite according to whether "insert into" or "insert
overwrite".
Flink's current behavior:
* By default, whether a directory or a single file is based on parallelism.
Single parallelism is a file, and multiple parallelism is a directory. For
example, "insert... values..." must be a file, because there is only one
parallelism source of values.
* The default behavior is to throw exception when a file exists.
We can do:
* Consider the support of "insert overwrite". For overwrite that has been
supported in the output format layer, only one boolean should be passed in.
* It is hard to support append mode. Because the file output format layer does
not support it, we need to modify FileOutputFormat and other classes to add an
append policy of file name generation.
So my conclusion is that we can wait until 1.11, after we introduce filesystem
connector, to take a look again to this problem.
> Cannot run multiple `insert into csvTable values ()`
> ----------------------------------------------------
>
> Key: FLINK-15066
> URL: https://issues.apache.org/jira/browse/FLINK-15066
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
> Reporter: Kurt Young
> Assignee: Jingsong Lee
> Priority: Major
> Fix For: 1.10.0
>
>
> I created a csv table in sql client, and tried to insert some data into this
> table.
> The first insert into success, but the second one failed with exception:
> {code:java}
> // Caused by: java.io.IOException: File or directory /.../xxx.csv already
> exists. Existing files and directories are not overwritten in NO_OVERWRITE
> mode. Use OVERWRITE mode to overwrite existing files and directories. at
> org.apache.flink.core.fs.FileSystem.initOutPathLocalFS(FileSystem.java:817)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
