Github user MaxGekk commented on a diff in the pull request:
https://github.com/apache/spark/pull/22374#discussion_r216569630
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala
---
@@ -1700,4 +1700,13 @@ class CSVSuite extends QueryTest with
SharedSQLContext with SQLTestUtils with Te
checkCount(2)
countForMalformedCSV(0, Seq(""))
}
+
+ test("SPARK-25387: bad input should not cause NPE") {
+ val schema = StructType(StructField("a", IntegerType) :: Nil)
+ val input = spark.createDataset(Seq("\u0000\u0000\u0001234"))
--- End diff --
The `parseLine` method can return `null` in many cases. See:
https://github.com/uniVocity/univocity-parsers/blob/f616d151b48150bc9cb98943f9b6f8353b704359/src/main/java/com/univocity/parsers/common/AbstractParser.java#L663
https://github.com/uniVocity/univocity-parsers/blob/f616d151b48150bc9cb98943f9b6f8353b704359/src/main/java/com/univocity/parsers/common/AbstractParser.java#L678
It is normal way for the method to indicate about an error.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
