Weichen Xu created SPARK-15226:
----------------------------------
Summary: CSV file data-line with newline at first line load error
Key: SPARK-15226
URL: https://issues.apache.org/jira/browse/SPARK-15226
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 2.0.0, 2.1.0
Reporter: Weichen Xu
CSV file such as:
---------------
v1,v2,"v
3",v4,v5
a,b,c,d,e
---------------
it contains two row,first row :
v1, v2, v\n3, v4, v5 (in value v\n3 it contains a newline character,it is legal)
second row:
a,b,c,d,e
then in spark-shell run commands like:
val sqlContext = new org.apache.spark.sql.SQLContext(sc);
var reader = sqlContext.read
var df = reader.csv("path/to/csvfile")
df.collect
then we find the load data is wrong,
the load data has only 3 columns, but in fact it has 5 columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
