[
https://issues.apache.org/jira/browse/KYLIN-3726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16727773#comment-16727773
]
ASF subversion and git services commented on KYLIN-3726:
--------------------------------------------------------
Commit 011ff0887d4a83b3a3373487ad66ce4b652bd7ff in kylin's branch
refs/heads/kylin-on-parquet from Liu Shaohui
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=011ff08 ]
KYLIN-3726: KylinSession should load spark properties from spark-defaults.conf
> KylinSession should load spark properties from spark-defaults.conf
> ------------------------------------------------------------------
>
> Key: KYLIN-3726
> URL: https://issues.apache.org/jira/browse/KYLIN-3726
> Project: Kylin
> Issue Type: Sub-task
> Components: Storage - Parquet
> Reporter: Liu Shaohui
> Assignee: Liu Shaohui
> Priority: Major
>
> When testing parquet storage, the spark session job failed to be submit for
> no JAVA_HOME in executor env. This config is set in the spark default
> property file: spark-defaults.conf.
>
> {code}
> 2018-12-18,15:13:15,466 ERROR org.apache.spark.deploy.yarn.YarnAllocator:
> Failed to launch executor 6 on container
> container_e823_1541646991414_1025309_01_000007
> java.util.NoSuchElementException: key not found: JAVA_HOME at
> scala.collection.MapLike$class.default(MapLike.scala:228) at
> scala.collection.AbstractMap.default(Map.scala:59) at
> scala.collection.mutable.HashMap.apply(HashMap.scala:65) at
> org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:286)
> at
> org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:275)
> at scala.Option.foreach(Option.scala:257) at
> org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:275)
> at
> org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:274)
> at scala.Option.foreach(Option.scala:257) at
> org.apache.spark.deploy.yarn.ExecutorRunnable.prepareEnvironment(ExecutorRunnable.scala:274)
> at
> org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:92)
> at
> org.apache.spark.deploy.yarn.ExecutorRunnable.run(ExecutorRunnable.scala:69)
> at
> org.apache.spark.deploy.yarn.YarnAllocator$$anonfun$runAllocatedContainers$1$$anon$1.run(YarnAllocator.scala:556)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
