-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27987/
-----------------------------------------------------------
(Updated Nov. 14, 2014, 3:34 a.m.)
Review request for hive, Rui Li, Szehon Ho, and Xuefu Zhang.
Changes
-------
1.enable submit spark job through programatic SparkSubmit.
2.make automatic calculating reduce number work at local spark context mode, so
that we don't break qtest.
now this patch should cover all features for this JIRA.
Bugs: HIVE-8833
https://issues.apache.org/jira/browse/HIVE-8833
Repository: hive-git
Description
-------
Hive would support submitting spark job through both local spark client and
remote spark client. we should unify the spark client API, and implement remote
spark client through Remote Spark Context.
Diffs (updated)
-----
ql/pom.xml 06d7f27
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClient.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java
PRE-CREATION
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkClient.java ee16c9e
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 2fea62d
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java e3e6d16
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java
51e0510
ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobRef.java
bf43b6e
ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SetSparkReducerParallelism.java
d4d14a3
spark-client/src/main/java/org/apache/hive/spark/client/SparkClient.java
8346b28
spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java
5af66ee
Diff: https://reviews.apache.org/r/27987/diff/
Testing
-------
Thanks,
chengxiang li
