> On Nov. 13, 2014, 12:34 a.m., Chao Sun wrote: > > ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java, > > line 326 > > <https://reviews.apache.org/r/27933/diff/2/?file=760734#file760734line326> > > > > Can we use "SPARKHASHTABLESINK", or something similar? > > Jimmy Xiang wrote: > Does this need match the Operator type? > > Chao Sun wrote: > I think these two are not related. Somebody can correct me if I'm wrong. > One potential issue with using the same name is that RuleRegExp may > become harder to define. > > Jimmy Xiang wrote: > Will change SparkHashtableSinkOperator to extend HashtableSinkOperator. > I prefer not to change the name. In Spark, is it possible a hashtable sink > not a SparkHashtableSinkOperator? > > Suhas Satish wrote: > As Chao correctly pointed out, changing the Operator Name may affect any > RuleRegExp that relies on that name. But I havent come across any specific > rules for HashTableSinks yet. Even I prefer to keep the same name. > > Jimmy, is your question to have a hash table sink without having an > operator ?
I meant that all hastable sink operator should be SparkHashtableSinkOperator with Spark. I checked the suggestion Xuefu mentioned for SparkHashtableSinkOperator to extend HashtableSinkOperator. It has some problem with the OperatorFactory. I am thinking to use HashtableSinkOperator instead, so no new class SparkHashtableSinkOperator. - Jimmy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/27933/#review61150 ----------------------------------------------------------- On Nov. 12, 2014, 11:58 p.m., Jimmy Xiang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/27933/ > ----------------------------------------------------------- > > (Updated Nov. 12, 2014, 11:58 p.m.) > > > Review request for hive, Chao Sun, Szehon Ho, and Xuefu Zhang. > > > Bugs: HIVE-8810 > https://issues.apache.org/jira/browse/HIVE-8810 > > > Repository: hive-git > > > Description > ------- > > Fixed the Spark HashTableSinkOperator > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/HashTableSinkOperator.java > 78d9012 > ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java f1c3564 > ql/src/java/org/apache/hadoop/hive/ql/exec/SparkHashTableSinkOperator.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SparkReduceSinkMapJoinProc.java > a58a6c5 > ql/src/java/org/apache/hadoop/hive/ql/plan/SparkHashTableSinkDesc.java > PRE-CREATION > > Diff: https://reviews.apache.org/r/27933/diff/ > > > Testing > ------- > > > Thanks, > > Jimmy Xiang > >
