Heath Abelson created SPARK-26040:
-------------------------------------
Summary: CSV Row delimiters not consistent between platforms
Key: SPARK-26040
URL: https://issues.apache.org/jira/browse/SPARK-26040
Project: Spark
Issue Type: Bug
Components: Java API
Affects Versions: 2.3.0
Reporter: Heath Abelson
Running a spark job on *nix platforms, only unix style row delimiters (\n) are
recognized. When running the job on windows, only windows style delimiters are
recognized (\r\n).
The result is that, when trying to read a csv generated my MS excel, on spark
running on Linux, extra characters are included in field names and field values
that are last on the line.
Ideally, the CSV parser would be able to handle the 2 different flavors of line
endings regardless of what platform the job is being run on.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
