Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/20894#discussion_r188553979
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVOptions.scala
---
@@ -153,6 +153,12 @@ class CSVOptions(
val samplingRatio =
parameters.get("samplingRatio").map(_.toDouble).getOrElse(1.0)
+ /**
+ * Forcibly apply the specified or inferred schema to datasource files.
+ * If the option is enabled, headers of CSV files will be ignored.
+ */
+ val enforceSchema = getBool("enforceSchema", true)
--- End diff --
Sorry I missied:
```
+ checkHeaderFlag = !parser.options.enforceSchema, caseSensitive)
```
Can't we simply name the option `checkHeader`?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
