I am using spark-core version 2.0.1 with Scala 2.11. I have simple code to read a csv file which has \ escapes.
val myDA = spark.read .option("quote",null) .schema(mySchema) .csv(filePath) As per documentation \ is default escape for csv reader. But it does not work. Spark is reading \ as part of my data. For Ex: City column in csv file is north rocks\,au . I am expecting city column should read in code as northrocks,au. But instead spark reads it as northrocks\ and moves au to next column. I have tried following but did not work: * Explicitly defined escape .option("escape",”\\") * Changed escape to | or : in file and in code * I have tried using spark-csv library Any one facing same issue? Am I missing something? Thanks