parisni opened a new issue, #6767:
URL: https://github.com/apache/hudi/issues/6767
The released spark artifacts on maven (eg group: "org.apache.hudi", name:
"hudi-spark3.2.x_2.12", version: "0.12.0") have an bad dependency on scala 2.11.
```
<artifactId>hudi-spark-common_${scala.binary.version}</artifactId>
```
but scala.binary.version is fixed to 2.11
https://github.com/apache/hudi/blob/master/pom.xml#L163
https://github.com/apache/hudi/blob/master/hudi-spark-datasource/hudi-spark3.2.x/pom.xml#L236
this leads to this error
```
org.apache.spark.sql.sources.DataSourceRegister: Provider
org.apache.hudi.Spark3DefaultSource could not be instantiated
java.util.ServiceConfigurationError:
org.apache.spark.sql.sources.DataSourceRegister: Provider
org.apache.hudi.Spark3DefaultSource could not be instantiated
at java.util.ServiceLoader.fail(ServiceLoader.java:232)
at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
at
java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at
scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:46)
at scala.collection.Iterator.foreach(Iterator.scala:943)
at scala.collection.Iterator.foreach$(Iterator.scala:943)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1431)
at scala.collection.IterableLike.foreach(IterableLike.scala:74)
at scala.collection.IterableLike.foreach$(IterableLike.scala:73)
Caused by: java.lang.ClassNotFoundException:
org.apache.hudi.SparkAdapterSupport$class
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 114 more
```
unless you exclude the depency when importing :+1:
```
exclude group: 'org.apache.hudi', module: 'hudi-spark-common_2.11'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
