lshmouse opened a new pull request #403: Kylin 3726: KylinSession should load spark properties from spark-defaults.conf URL: https://github.com/apache/kylin/pull/403 When testing parquet storage, the spark session job failed to be submit for no JAVA_HOME in executor env. This config is set in the spark default property file: spark-defaults.conf. ``` 2018-12-18,15:13:15,466 ERROR org.apache.spark.deploy.yarn.YarnAllocator: Failed to launch executor 6 on container container_e823_1541646991414_1025309_01_000007 java.util.NoSuchElementException: key not found: JAVA_HOME at scala.collection.MapLike$class.default(MapLike.scala:228) at scala.collection.AbstractMap.default(Map.scala:59) at scala.collection.mutable.HashMap.apply(HashMap.scala:65) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:286) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3$$anonfun$apply$3.apply(ExecutorRunnable.scala:275) at scala.Option.foreach(Option.scala:257) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:275) at org.apache.spark.deploy.yarn.ExecutorRunnable$$anonfun$prepareEnvironment$3.apply(ExecutorRunnable.scala:274) at scala.Option.foreach(Option.scala:257) at org.apache.spark.deploy.yarn.ExecutorRunnable.prepareEnvironment(ExecutorRunnable.scala:274) at org.apache.spark.deploy.yarn.ExecutorRunnable.startContainer(ExecutorRunnable.scala:92) at org.apache.spark.deploy.yarn.ExecutorRunnable.run(ExecutorRunnable.scala:69) at org.apache.spark.deploy.yarn.YarnAllocator$$anonfun$runAllocatedContainers$1$$anon$1.run(YarnAllocator.scala:556) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
