sadagopan kalyanaraman created SPARK-36548:
----------------------------------------------
Summary: Throwing NoClassDefFoundError for Logging$class
Key: SPARK-36548
URL: https://issues.apache.org/jira/browse/SPARK-36548
Project: Spark
Issue Type: Bug
Components: Kubernetes, Spark Core
Affects Versions: 3.1.1
Reporter: sadagopan kalyanaraman
I'm getting NoClassDefFoundError. is it removed in 3.1.1?
Exception in thread "main" java.util.ServiceConfigurationError:
org.apache.spark.sql.sources.DataSourceRegister: Provider
com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be
instantiatedException in thread "main" java.util.ServiceConfigurationError:
org.apache.spark.sql.sources.DataSourceRegister: Provider
com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be
instantiated at java.base/java.util.ServiceLoader.fail(Unknown Source) at
java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source) at
java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source) at
java.base/java.util.ServiceLoader$3.next(Unknown Source) at
scala.collection.convert.Wrappers$JIteratorWrapper.next(Wrappers.scala:44) at
scala.collection.Iterator.foreach(Iterator.scala:941) at
scala.collection.Iterator.foreach$(Iterator.scala:941) at
scala.collection.AbstractIterator.foreach(Iterator.scala:1429) at
scala.collection.IterableLike.foreach(IterableLike.scala:74) at
scala.collection.IterableLike.foreach$(IterableLike.scala:73) at
scala.collection.AbstractIterable.foreach(Iterable.scala:56) at
scala.collection.TraversableLike.filterImpl(TraversableLike.scala:255) at
scala.collection.TraversableLike.filterImpl$(TraversableLike.scala:249) at
scala.collection.AbstractTraversable.filterImpl(Traversable.scala:108) at
scala.collection.TraversableLike.filter(TraversableLike.scala:347) at
scala.collection.TraversableLike.filter$(TraversableLike.scala:347) at
scala.collection.AbstractTraversable.filter(Traversable.scala:108) at
org.apache.spark.sql.execution.datasources.DataSource$.lookupDataSource(DataSource.scala:659)
at
org.apache.spark.sql.streaming.DataStreamReader.loadInternal(DataStreamReader.scala:209)
at
org.apache.spark.sql.streaming.DataStreamReader.load(DataStreamReader.scala:195)
at
com.xyz.xyz.xyz.AbstractCdcMessageToSqJob.runJob(AbstractCdcMessageToSqJob.java:94)
at
com.xyz.xyz.xyz.CdcMessageToSqlBillerJob.main(CdcMessageToSqlBillerJob.java:30)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown
Source) at java.base/java.lang.reflect.Method.invoke(Unknown Source) at
org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52) at
org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:951)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:180) at
org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203) at
org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:90) at
org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:1030) at
org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1039) at
org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)Caused by:
java.lang.NoClassDefFoundError: org/apache/spark/internal/Logging$class at
com.google.cloud.spark.bigquery.BigQueryUtilScala$.<init>(BigQueryUtil.scala:34)
at
com.google.cloud.spark.bigquery.BigQueryUtilScala$.<clinit>(BigQueryUtil.scala)
at
com.google.cloud.spark.bigquery.BigQueryRelationProvider.<init>(BigQueryRelationProvider.scala:43)
at
com.google.cloud.spark.bigquery.BigQueryRelationProvider.<init>(BigQueryRelationProvider.scala:50)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method) at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(Unknown
Source) at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown
Source) at java.base/java.lang.reflect.Constructor.newInstance(Unknown Source)
... 33 moreCaused by: java.lang.ClassNotFoundException:
org.apache.spark.internal.Logging$class at
java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) at
java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown
Source) at java.base/java.lang.ClassLoader.loadClass(Unknown Source) ... 41 more
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
