Github user srowen commented on the issue:
https://github.com/apache/spark/pull/23150
Looks like that kind of fallback logic was there for compatibility with
Spark 1.x and 2.0; see some comments about 'backwards compatibility' in for
example
https://github.com/apache/spark/blame/af8250e12490c77f02587275eff9aa225e5dcdba/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala
I think we can remove this fallback and document it as a behavior change.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
