[
https://issues.apache.org/jira/browse/SPARK-13638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-13638:
---------------------------------
Description:
https://github.com/databricks/spark-csv/pull/254
tobithiel reported this.
{quote}
I'm dealing with some messy csv files and being able to just quote all fields
is very useful,
so that other applications don't misunderstand the file because of some sketchy
characters....
{quote}
When writing there are several quote modes in apache commons csv. (See
https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
This might have to be supported.
However, it looks univocity parser used for writing (it looks currently only
this library is supported) does not support this quote mode. I think we can
drop this backwards compatibility if we are not going to add apache commons csv.
This is a reminder that it might break backwards compatibility for the options,
{{quoteMode}}.
was:
https://github.com/databricks/spark-csv/pull/254
tobithiel reported this.
{quote}
I'm dealing with some messy csv files and being able to just quote all fields
is very useful,
so that other applications don't misunderstand the file because of some sketchy
characters....
{quote}
When writing there are several quote modes in apache commons csv. (See
https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
This might have to be supported.
However, it looks univocity parser used for writing (it looks currently only
this library is supported) does not support this quote mode. I think we can
drop this backwards compatibility if we are not going to add apache commons csv.
This is a reminder that it will break backwards compatibility for the options,
{{quoteMode}}.
> Support for saving with a quote mode
> ------------------------------------
>
> Key: SPARK-13638
> URL: https://issues.apache.org/jira/browse/SPARK-13638
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Hyukjin Kwon
> Priority: Minor
>
> https://github.com/databricks/spark-csv/pull/254
> tobithiel reported this.
> {quote}
> I'm dealing with some messy csv files and being able to just quote all fields
> is very useful,
> so that other applications don't misunderstand the file because of some
> sketchy characters....
> {quote}
> When writing there are several quote modes in apache commons csv. (See
> https://commons.apache.org/proper/commons-csv/apidocs/org/apache/commons/csv/QuoteMode.html)
> This might have to be supported.
> However, it looks univocity parser used for writing (it looks currently only
> this library is supported) does not support this quote mode. I think we can
> drop this backwards compatibility if we are not going to add apache commons
> csv.
> This is a reminder that it might break backwards compatibility for the
> options, {{quoteMode}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
