----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27987/#review61819 -----------------------------------------------------------
Looks mostly good, just some minor nits and basic question as I'm not too familiar. ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java <https://reviews.apache.org/r/27987/#comment103720> Can we correct this typo while we are in this class? ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java <https://reviews.apache.org/r/27987/#comment103725> Should we put this inside the null-check, to avoid NPE? spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java <https://reviews.apache.org/r/27987/#comment103730> Sorry for the basic question, what is 'spark' as spark.master signify? - Szehon Ho On Nov. 17, 2014, 3:47 a.m., chengxiang li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27987/ > ----------------------------------------------------------- > > (Updated Nov. 17, 2014, 3:47 a.m.) > > > Review request for hive, Rui Li, Szehon Ho, and Xuefu Zhang. > > > Bugs: HIVE-8833 > https://issues.apache.org/jira/browse/HIVE-8833 > > > Repository: hive-git > > > Description > ------- > > Hive would support submitting spark job through both local spark client and > remote spark client. we should unify the spark client API, and implement > remote spark client through Remote Spark Context. > > > Diffs > ----- > > ql/pom.xml 06d7f27 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClient.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/HiveSparkClientFactory.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/LocalHiveSparkClient.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkClient.java ee16c9e > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java 2fea62d > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java > e3e6d16 > > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java > 51e0510 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobRef.java > bf43b6e > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SetSparkReducerParallelism.java > d4d14a3 > spark-client/src/main/java/org/apache/hive/spark/client/SparkClient.java > 8346b28 > > spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java > 5af66ee > > Diff: https://reviews.apache.org/r/27987/diff/ > > > Testing > ------- > > > Thanks, > > chengxiang li > >
