Re: Review Request 30208: HIVE-9449 Push YARN configuration to Spark while deply Spark on YARN[Spark Branch]
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30208/ --- (Updated Jan. 26, 2015, 5:06 a.m.) Review request for hive and Xuefu Zhang. Changes --- Fix unit test failure. Bugs: HIVE-9449 https://issues.apache.org/jira/browse/HIVE-9449 Repository: hive-git Description --- We only push Spark configuration and RSC configuration to Spark while launch Spark cluster now, for Spark on YARN mode, Spark need extra YARN configuration to launch Spark cluster. Besides this, to support dynamically configuration setting for RSC configuration/YARN configuration, we need to recreate SparkSession while RSC configuration/YARN configuration update as well, as they may influence the Spark cluster deployment as well. Diffs (updated) - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d4d98d7 ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java 9dc6c47 Diff: https://reviews.apache.org/r/30208/diff/ Testing --- Thanks, chengxiang li
Review Request 30208: HIVE-9449 Push YARN configuration to Spark while deply Spark on YARN[Spark Branch]
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30208/ --- Review request for hive and Xuefu Zhang. Bugs: HIVE-9449 https://issues.apache.org/jira/browse/HIVE-9449 Repository: hive-git Description --- We only push Spark configuration and RSC configuration to Spark while launch Spark cluster now, for Spark on YARN mode, Spark need extra YARN configuration to launch Spark cluster. Besides this, to support dynamically configuration setting for RSC configuration/YARN configuration, we need to recreate SparkSession while RSC configuration/YARN configuration update as well, as they may influence the Spark cluster deployment as well. Diffs - common/src/java/org/apache/hadoop/hive/conf/HiveConf.java d4d98d7 ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java 9dc6c47 Diff: https://reviews.apache.org/r/30208/diff/ Testing --- Thanks, chengxiang li