Dear Team,
I need your help. I'm trying this job given in the documentation
https://griffin.apache.org/docs/quickstart.html . I have the following
versions installed in my system.
Hadoop:3.2.0
scala: 2.13.1
Spark: 3.0.0
Java: 1.8.0
Hive: 3.1.2
I'm submitting the spark job to run data quality using griffin. I'm getting
below exception can you pls help me in this regard.
2020-04-04 08:37:35,456 INFO cluster.YarnClientSchedulerBackend:
SchedulerBackend is ready for scheduling beginning after waiting
maxRegisteredResourcesWaitingTime: 30000000000(ns)
Exception in thread "main" java.lang.NoSuchMethodError:
scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/
collection/mutable/ArrayOps;
at org.apache.griffin.measure.datasource.connector.batch.
HiveBatchDataConnector.<init>(HiveBatchDataConnector.scala:47)
at org.apache.griffin.measure.datasource.connector.
DataConnectorFactory$$anonfun$getDataConnector$1.apply(
DataConnectorFactory.scala:63)
at org.apache.griffin.measure.datasource.connector.
DataConnectorFactory$$anonfun$getDataConnector$1.apply(
DataConnectorFactory.scala:62)
at scala.util.Try$.apply(Try.scala:213)
at org.apache.griffin.measure.datasource.connector.
DataConnectorFactory$.getDataConnector(DataConnectorFactory.scala:61)
at org.apache.griffin.measure.datasource.DataSourceFactory$$
anonfun$1.apply(DataSourceFactory.scala:58)
at org.apache.griffin.measure.datasource.DataSourceFactory$$
anonfun$1.apply(DataSourceFactory.scala:57)
at scala.collection.TraversableLike.$anonfun$
flatMap$1(TraversableLike.scala:245)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.TraversableLike.flatMap(
TraversableLike.scala:245)
at scala.collection.TraversableLike.flatMap$(
TraversableLike.scala:242)
at scala.collection.immutable.List.flatMap(List.scala:355)
at org.apache.griffin.measure.datasource.DataSourceFactory$.
org$apache$griffin$measure$datasource$DataSourceFactory$$getDataSource(
DataSourceFactory.scala:57)
at org.apache.griffin.measure.datasource.DataSourceFactory$$
anonfun$getDataSources$1.apply(DataSourceFactory.scala:40)
at org.apache.griffin.measure.datasource.DataSourceFactory$$
anonfun$getDataSources$1.apply(DataSourceFactory.scala:38)
at scala.collection.TraversableLike.$anonfun$
flatMap$1(TraversableLike.scala:245)
at scala.collection.immutable.List.foreach(List.scala:392)
at scala.collection.TraversableLike.flatMap(
TraversableLike.scala:245)
at scala.collection.TraversableLike.flatMap$(
TraversableLike.scala:242)
at scala.collection.immutable.List.flatMap(List.scala:355)
at org.apache.griffin.measure.datasource.DataSourceFactory$.
getDataSources(DataSourceFactory.scala:38)
at org.apache.griffin.measure.launch.batch.BatchDQApp$$
anonfun$run$1.apply$mcZ$sp(BatchDQApp.scala:75)
at org.apache.griffin.measure.launch.batch.BatchDQApp$$
anonfun$run$1.apply(BatchDQApp.scala:67)
at org.apache.griffin.measure.launch.batch.BatchDQApp$$
anonfun$run$1.apply(BatchDQApp.scala:67)
at scala.util.Try$.apply(Try.scala:213)
at org.apache.griffin.measure.launch.batch.BatchDQApp.run(
BatchDQApp.scala:67)
at org.apache.griffin.measure.Application$.main(Application.
scala:88)
at org.apache.griffin.measure.Application.main(Application.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(
NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.JavaMainApplication.start(
SparkApplication.scala:52)
at org.apache.spark.deploy.SparkSubmit.org
<http://org.apache.spark.deploy.sparksubmit.org/>$apache$spark$
deploy$SparkSubmit$$runMain(SparkSubmit.scala:928)
at org.apache.spark.deploy.SparkSubmit.doRunMain$1(
SparkSubmit.scala:180)
at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:203)
at org.apache.spark.deploy.SparkSubmit.doSubmit(
SparkSubmit.scala:90)
at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(
SparkSubmit.scala:1007)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:1016)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
