[ https://issues.apache.org/jira/browse/SPARK-28338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jayadevan M updated SPARK-28338: -------------------------------- Summary: spark.read.format("csv") treat empty string as null if csv file don't quotes in data (was: spark.read.format("csv") treat empty string as null if csv file don't quotes in columns) > spark.read.format("csv") treat empty string as null if csv file don't quotes > in data > ------------------------------------------------------------------------------------ > > Key: SPARK-28338 > URL: https://issues.apache.org/jira/browse/SPARK-28338 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 2.4.3 > Reporter: Jayadevan M > Priority: Major > > The csv input file > cat sample.csv > Name,Lastname,Age > abc,,32 > pqr,xxx,30 > > spark-shell > spark.read.format("csv").option("header", > "true").load("/media/ub_share/projects/*.csv").head(3) > res14: Array[org.apache.spark.sql.Row] = Array([abc,null,32], [pqr,xxx,30]) > > scala> spark.read.format("csv").option("header", "true").option("nullValue", > "?").load("/media/ub_share/projects/*.csv").head(3) > res15: Array[org.apache.spark.sql.Row] = Array([abc,null,32], [pqr,xxx,30]) -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org