> On 一月 23, 2015, 3:02 a.m., chengxiang li wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java, line 371 > > <https://reviews.apache.org/r/30107/diff/4/?file=829688#file829688line371> > > > > #3 this would be executed in akka thread, get extra jar path from > > JobConf, and add to current thread classloader. > > Xuefu Zhang wrote: > what thread is referred as akka thread?
Inside Spark driver, SparkContext submit spark job to DAGSchedule through akka message instead of directly invoke, akka hold a thread pool to handle messages. - chengxiang ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/30107/#review69336 ----------------------------------------------------------- On 一月 22, 2015, 9:23 a.m., chengxiang li wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/30107/ > ----------------------------------------------------------- > > (Updated 一月 22, 2015, 9:23 a.m.) > > > Review request for hive and Xuefu Zhang. > > > Bugs: HIVE-9410 > https://issues.apache.org/jira/browse/HIVE-9410 > > > Repository: hive-git > > > Description > ------- > > The RemoteDriver does not contains added jar in it's classpath, so it would > failed to desrialize SparkWork due to NoClassFoundException. For Hive on MR, > while use add jar through Hive CLI, Hive add jar into CLI classpath(through > thread context classloader) and add it to distributed cache as well. Compare > to Hive on MR, Hive on Spark has an extra RemoteDriver componnet, we should > add added jar into it's classpath as well. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java d7cb111 > ql/src/java/org/apache/hadoop/hive/ql/exec/spark/RemoteHiveSparkClient.java > 30a00a7 > spark-client/src/main/java/org/apache/hive/spark/client/JobContext.java > 00aa4ec > spark-client/src/main/java/org/apache/hive/spark/client/JobContextImpl.java > 1eb3ff2 > > spark-client/src/main/java/org/apache/hive/spark/client/SparkClientImpl.java > 5f9be65 > > spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/30107/diff/ > > > Testing > ------- > > > Thanks, > > chengxiang li > >
