[ https://issues.apache.org/jira/browse/SPARK-26093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Hyukjin Kwon resolved SPARK-26093. ---------------------------------- Resolution: Not A Problem > Read Avro: ClassNotFoundException: > org.apache.spark.sql.avro.AvroFileFormat.DefaultSource > ----------------------------------------------------------------------------------------- > > Key: SPARK-26093 > URL: https://issues.apache.org/jira/browse/SPARK-26093 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.4.0 > Environment: Spark 2.4.0 > Scala 2.11.12 > Java 1.8.0_181 > Reporter: Dagang Wei > Priority: Major > > I downloaded and unpacked spark-2.4.0-bin-hadoop2.7.tgz to my Linux, then I > followed [Read Avro > files|https://docs.databricks.com/spark/latest/data-sources/read-avro.html] > to read a local Avro file in spark-shell: > $ bin/spark-shell --packages com.databricks:spark-avro_2.11:4.0.0 > ... > version 2.4.0 > Using Scala version 2.11.12 (OpenJDK 64-Bit Server VM, Java 1.8.0_181) > ... > scala> > import com.databricks.spark.avro._ > scala> > val df = spark.read.avro("file:///.../foo.avro") > java.lang.ClassNotFoundException: Failed to find data source: > org.apache.spark.sql.avro.AvroFileFormat. Please find packages at > http://spark.apache.org/third-party-projects.html > at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:657) > at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:194) > at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:178) > at > com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34) > at > com.databricks.spark.avro.package$AvroDataFrameReader$$anonfun$avro$2.apply(package.scala:34) > ... 51 elided > Caused by: java.lang.ClassNotFoundException: > org.apache.spark.sql.avro.AvroFileFormat.DefaultSource > at > scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > at > org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634) > at > org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20$$anonfun$apply$12.apply(DataSource.scala:634) > at scala.util.Try$.apply(Try.scala:192) > at > org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634) > at > org.apache.spark.sql.execution.datasources.DataSource$$anonfun$20.apply(DataSource.scala:634) > at scala.util.Try.orElse(Try.scala:84) > at > org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:634) > ... 55 more > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org