> On Feb. 5, 2018, 11:05 p.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java > > Lines 213 (patched) > > <https://reviews.apache.org/r/65479/diff/1/?file=1951925#file1951925line219> > > > > Do we need this? Reducers are always launched when when there is no > > mapper. So, this seems unnecessary. > > Zoltan Haindrich wrote: > I can remove it...but I think IConfigureJobConf can be used as a general > way to make modifications like this - in the current case this setting has > effectively no effect on the reducers...they are being run anyway > > I think that after this patch the code from PlanUtils.configureJobConf > could be moved to FileSinkOperator by using the new interface > I feel that core constructs like ReduceWork should depend less on > explicit operator implementations and more on interfaces if possible. > > Ashutosh Chauhan wrote: > I agree that a general mechanism has some value. However, my concern is > extra CPU we will be wasting doing nothing. In a large plan with 100s of > vertices going through all RSs will be non-zero cost we will be paying. So, > unless we need it we should avoid it.
I see your point, I've removed it. - Zoltan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/65479/#review196840 ----------------------------------------------------------- On Feb. 6, 2018, 10:24 a.m., Zoltan Haindrich wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/65479/ > ----------------------------------------------------------- > > (Updated Feb. 6, 2018, 10:24 a.m.) > > > Review request for hive, Ashutosh Chauhan and Prasanth_J. > > > Bugs: HIVE-18523 > https://issues.apache.org/jira/browse/HIVE-18523 > > > Repository: hive-git > > > Description > ------- > > * ensure that mapper operators are started up - but only if empty grouping is > present > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java 6a0f0de2a5 > ql/src/java/org/apache/hadoop/hive/ql/exec/IConfigureJobConf.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java 941dd58f27 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 976b537033 > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecMapRunner.java > PRE-CREATION > ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecMapper.java 150382a8d5 > ql/src/java/org/apache/hadoop/hive/ql/exec/tez/HiveSplitGenerator.java > 98f4bc01c8 > > ql/src/java/org/apache/hadoop/hive/ql/exec/vector/VectorGroupByOperator.java > 45d809a182 > ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java > e4dfc009d9 > ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java c3b846c4d2 > ql/src/java/org/apache/hadoop/hive/ql/io/NullRowsInputFormat.java > 6a372a3f47 > ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java f2b2fc57a0 > ql/src/java/org/apache/hadoop/hive/ql/plan/ReduceWork.java ecfb118b41 > ql/src/test/queries/clientpositive/groupby_rollup_empty.q 432d8c448a > ql/src/test/results/clientpositive/groupby_rollup_empty.q.out 7359140e29 > ql/src/test/results/clientpositive/llap/groupby_rollup_empty.q.out > d2b57455a3 > > > Diff: https://reviews.apache.org/r/65479/diff/2/ > > > Testing > ------- > > added new testcase for union > > > Thanks, > > Zoltan Haindrich > >
