[
https://issues.apache.org/jira/browse/SPARK-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15785951#comment-15785951
]
Jacob Wellington commented on SPARK-15493:
------------------------------------------
I'm running into an issue where this doesn't seem to be working for the SQL
interface. I'm connecting to the thrift server using beeline and submitting the
following sql:
{quote}
CREATE TABLE e2
USING csv
OPTIONS (path 'test.csv', quote '"', escapeQuotes 'false',
quoteEscapingEnabled 'false') AS
SELECT
'"G"'
FROM parquet.`test.parquet`;
DROP TABLE e2;
{quote}
When I look at the test.csv output I get this:
{quote}
"\"G\""
{quote}
I'm using spark 2.0.2 with its version of beeline and its hive server. I've
also tried multiple variations of the options.
> Allow setting the quoteEscapingEnabled flag when writing CSV
> ------------------------------------------------------------
>
> Key: SPARK-15493
> URL: https://issues.apache.org/jira/browse/SPARK-15493
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 2.0.0
> Reporter: Jurriaan Pruis
> Assignee: Jurriaan Pruis
> Fix For: 2.0.0
>
>
> See
> https://github.com/uniVocity/univocity-parsers/blob/f3eb2af26374940e60d91d1703bde54619f50c51/src/main/java/com/univocity/parsers/csv/CsvWriterSettings.java#L231-L247
> This kind of functionality is needed to be able to write RFC 4180
> (https://tools.ietf.org/html/rfc4180#section-2) / Amazon Redshift compatible
> CSV files
> (https://docs.aws.amazon.com/redshift/latest/dg/copy-parameters-data-format.html#copy-csv)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
