Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/22379#discussion_r224648258
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVDataSource.scala
---
@@ -254,7 +256,7 @@ object TextInputCSVDataSource extends CSVDataSource {
val header = makeSafeHeader(firstRow, caseSensitive, parsedOptions)
val sampled: Dataset[String] = CSVUtils.sample(csv, parsedOptions)
val tokenRDD = sampled.rdd.mapPartitions { iter =>
- val filteredLines = CSVUtils.filterCommentAndEmpty(iter,
parsedOptions)
+ val filteredLines = filterCommentAndEmpty(iter, parsedOptions)
--- End diff --
not a big deal but let's just use `CSVUtils...` usage just for consistency
in this file.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
