Github user gengliangwang commented on a diff in the pull request:
https://github.com/apache/spark/pull/21878#discussion_r205353836
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSource.scala
---
@@ -635,12 +637,6 @@ object DataSource extends Logging {
"Hive built-in ORC data source must be used with Hive
support enabled. " +
"Please use the native ORC data source by setting
'spark.sql.orc.impl' to " +
"'native'")
- } else if (provider1.toLowerCase(Locale.ROOT) == "avro" ||
- provider1 == "com.databricks.spark.avro") {
- throw new AnalysisException(
--- End diff --
I totally agree with the mapping, we should do it.
The comment here is about when Spark can't find any avro package, we should
show a message for loading the spark-avro jar(org.apache.spark.sql.avro).
Different from CSV, the package spark-avro is not loaded by default within
Spark(at least as I tried spark-shell).
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
