[ https://issues.apache.org/jira/browse/SPARK-21678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Taran Saini updated SPARK-21678: -------------------------------- Description: Hi, I have the my dataframe cloumn values which can contain commas, double quotes etc. I am transforming the dataframes in order to ensure that all the required values are escaped. However, on doing df.write.format("csv") It again wraps the values in double quotes. How do I disable the same? And even if the double quotes are there to stay why does it do the following : L"\, p' Y a\, C G is written as "L\"\\, p' Y a\\, C G\\, H" i.e double escapes the next already escaped values. and if i myself escape like : L\"\, p' Y a\, C G then that is written as "L\\"\\, p' Y a\\, C G\\, H" How do we just disable this automatic escaping of characters? was: Hi, I have the my dataframe cloumn values which can contain commas, double quotes etc. I am transforming the dataframes in order to ensure that all the required values are escaped. However, on doing df.write.format("csv") It again wraps the values in double quotes. How do I disable the same? And even if the double quotes are there to stay why does it do the following : L"\, p' Y a\, C G is written as "L\"\\, p' Y a\\, C G\\, H" i.e double escapes the next already escaped values. and if i myself escape like : L\"\, p' Y a\, C G then that is written as "L\\"\\, p' Y a\\, C G\\, H" How do we just disable this automatic escaping of characters? > Disabling quotes while writing a dataframe > ------------------------------------------ > > Key: SPARK-21678 > URL: https://issues.apache.org/jira/browse/SPARK-21678 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.2.0 > Reporter: Taran Saini > > Hi, > I have the my dataframe cloumn values which can contain commas, double quotes > etc. > I am transforming the dataframes in order to ensure that all the required > values are escaped. > However, on doing df.write.format("csv") > It again wraps the values in double quotes. How do I disable the same? > And even if the double quotes are there to stay why does it do the following : > L"\, p' Y a\, C G is written as "L\"\\, p' Y a\\, C G\\, H" i.e double > escapes the next already escaped values. > and if i myself escape like : > L\"\, p' Y a\, C G then that is written as "L\\"\\, p' Y a\\, C G\\, H" > How do we just disable this automatic escaping of characters? -- This message was sent by Atlassian JIRA (v6.4.14#64029) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org