[
https://issues.apache.org/jira/browse/SPARK-26093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16690413#comment-16690413
]
Hyukjin Kwon commented on SPARK-26093:
--------------------------------------
Please follow https://spark.apache.org/docs/latest/sql-data-sources-avro.html
> Read Avro: ClassNotFoundException:
> org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
> -----------------------------------------------------------------------------------------
>
> Key: SPARK-26093
> URL: https://issues.apache.org/jira/browse/SPARK-26093
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.4.0
> Environment: Spark 2.4.0
> Scala 2.11.12
> Java 1.8.0_181
> Reporter: Dagang Wei
> Priority: Major
>
> I downloaded and unpacked spark-2.4.0-bin-hadoop2.7.tgz to my Linux, then I
> followed [Read Avro
> files|https://docs.databricks.com/spark/latest/data-sources/read-avro.html]
> to read a local Avro file in spark-shell:
> $ bin/spark-shell --packages com.databricks:spark-avro_2.11:4.0.0
> ...
> version 2.4.0
> Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_181)
> ...
> scala>
> import com.databricks.spark.avro._
> scala>
> val df = spark.read.avro("file:///.../foo.avro")
> java.lang.ClassNotFoundException: Failed to find data source:
> org.apache.spark.sql.avro.AvroFileFormat. Please find packages at
> http://spark.apache.org/third-party-projects.html
> at
> org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:657)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:194)
> at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178)
> at
> com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
> at
> com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34)
> ... 51 elided
> Caused by: java.lang.ClassNotFoundException:
> org.apache.spark.sql.avro.AvroFileFormat.DefaultSource
> at
> scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634)
> at scala.util.Try$.apply(Try.scala:192)
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
> at
> org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634)
> at scala.util.Try.orElse(Try.scala:84)
> at
> org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:634)
> ... 55 more
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
