[
https://issues.apache.org/jira/browse/KYLIN-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17445611#comment-17445611
]
xbchao commented on KYLIN-5126:
-------------------------------
thank you for your advice!
I change "log4j.rootLogger=INFO,hdfs" to "log4j.rootLogger=INFO,logFile",and
the $KYLIN_HOME/logs/spark/${job-id}.log is:
2021-11-17 07:49:59,445 ERROR [spark-entry-event-loop] application.JobMonitor :
Job failed the 1 times. java.lang.RuntimeException: Error execute
org.apache.kylin.engine.spark.job.ResourceDetectBeforeCubingJob at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:96)
at org.apache.spark.application.JobWorker$$anon$2.run(JobWorker.scala:55) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748) Caused by:
java.lang.ClassNotFoundException: java.lang.NoClassDefFoundError:
com/amazonaws/AmazonServiceException when creating Hive client using classpath:
file:/opt/apps/kylin/lib/kylin-parquet-job-4.0.0.jar,
file:/opt/apps/apache-kylin-4.0.0-bin-spark2/lib/kylin-parquet-job-4.0.0.jar,
file:/etc/spark/conf.dist/, file:/usr/lib/spark/jars/commons-math3-3.4.1.jar,
file:/usr/lib/spark/jars/HikariCP-java7-2.4.12.jar,
file:/usr/lib/spark/jars/httpclient-4.5.9.jar,
file:/usr/lib/spark/jars/JavaEWAH-0.3.2.jar,
file:/usr/lib/spark/jars/commons-net-3.1.jar,
file:/usr/lib/spark/jars/RoaringBitmap-0.7.45.jar,
file:/usr/lib/spark/jars/jackson-core-asl-1.9.13.jar,
file:/usr/lib/spark/jars/ST4-4.0.4.jar,
file:/usr/lib/spark/jars/commons-pool-1.5.4.jar ...... Please make sure that
jars for your version of hive and hadoop are included in the paths passed to
spark.sql.hive.metastore.jars. at
org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:280)
at
org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:414)
...... at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:304)
at
org.apache.kylin.engine.spark.application.SparkApplication.execute(SparkApplication.java:93)
... 4 more Caused by: java.lang.reflect.InvocationTargetException at
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at
org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:274)
... 90 more Caused by: java.lang.NoClassDefFoundError:
com/amazonaws/AmazonServiceException at
com.amazonaws.glue.catalog.metastore.AWSGlueDataCatalogHiveClientFactory.createMetaStoreClient(AWSGlueDataCatalogHiveClientFactory.java:16)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3113)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3148) at
org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1244) at
org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:183) at
org.apache.hadoop.hive.ql.metadata.Hive.<clinit>(Hive.java:175) at
org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503) at
org.apache.spark.sql.hive.client.HiveClientImpl.newState(HiveClientImpl.scala:185)
... 95 more Caused by: java.lang.ClassNotFoundException:
com.amazonaws.AmazonServiceException at
java.net.URLClassLoader.findClass(URLClassLoader.java:382) at
java.lang.ClassLoader.loadClass(ClassLoader.java:418) ... 104 more 2021-11-17
07:50:00,348 INFO [spark-entry-event-loop]
client.ConfiguredRMFailoverProxyProvider : Failing over to rm2 2021-11-17
07:50:00,349 INFO [spark-entry-event-loop] retry.RetryInvocationHandler :
java.net.ConnectException: Call From ip-local/local to ip-master:8032 failed on
c onnection exception: java.net.ConnectException: Connection refused; For more
details see: [http://wiki.apache.org/hadoop/ConnectionRefused
|http://wiki.apache.org/hadoop/ConnectionRefused], while invoking
ApplicationClientProtocolPBClientImpl.getNewApplication o ver rm2 after 1
failover attempts. Trying to failover after sleeping for 19370ms. ......
2021-11-17 07:50:19,744 INFO [spark-entry-event-loop] resource.ResourceUtils :
Adding resource type - name = vcores, units = , type = COUNTABLE 2021-11-17
07:50:19,748 INFO [spark-entry-event-loop] cluster.YarnInfoFetcher : Cluster
maximum resource allocation ResourceInfo(57344,16) 2021-11-17 07:50:19,749
ERROR [spark-entry-event-loop] application.JobWorkSpace : Job failed
eventually. Reason: Error occurred when generate retry configuration.
java.util.NoSuchElementException: spark.executor.memoryOverhead at
org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246) at
org.apache.spark.SparkConf$$anonfun$get$1.apply(SparkConf.scala:246) EMR
Cluster-id: ID:j-***AA Yarn Application-id:
01efe545-dcbc-4d95-8b76-e4ba86198f54-00_jobId
> Build kylin 4.0, spark has not been able to submit to the yarn cluster
> ----------------------------------------------------------------------
>
> Key: KYLIN-5126
> URL: https://issues.apache.org/jira/browse/KYLIN-5126
> Project: Kylin
> Issue Type: Bug
> Reporter: xbchao
> Priority: Major
>
> When I built kylin 4.0, spark could not be submitted to the yarn cluster. The
> version used was apache-kylin-4.0.0-bin-spark2, which was deployed in the aws
> emr server.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
