> On April 27, 2020, 6:02 p.m., Vineet Garg wrote: > > ql/src/test/results/clientpositive/llap/keep_uniform.q.out > > Lines 946 (patched) > > <https://reviews.apache.org/r/72431/diff/2/?file=2227827#file2227827line956> > > > > Why is there an extra join in the plan now?
I run `explain cbo` on master and the branch where this patch is applied with this query. Both plans has 8 HiveJoin operators. Comparing the CBO plans I see that 2 of the joins were reordered: On master: (web_returns + (web_sales + web_sales)) On the branch: (web_sales + web_returns) + web_sales It also turned out that on master the CBO plan contains a HiveProject with all the columns from table `web_returns`. The reason is just the same as in case of the example query mentioned in HIVE-23206. This project has only the necessary columns (wr_order_number only) in the plan created after applying this patch. In the physical plan there are 7 joins on master and 8 when this applied. SharedWorkOptimizer merge two of them i need to investigate further... > On April 27, 2020, 6:02 p.m., Vineet Garg wrote: > > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out > > Lines 240 (patched) > > <https://reviews.apache.org/r/72431/diff/2/?file=2227833#file2227833line243> > > > > This looks like an extra join as compared to earlier (including few > > more in this plan). Any idea why is this? There are 42 HiveJoins both in the old and the new plan. What happened is that `HiveJoinProjectTransposeRule` was able to move out more HiveProjects between joins and than `JoinToMultiJoinRule` was able to merge them and than `LoptOptimizeJoinRule` reordered those ones. - Krisztian ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72431/#review220503 ----------------------------------------------------------- On April 27, 2020, 6:11 a.m., Krisztian Kasa wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72431/ > ----------------------------------------------------------- > > (Updated April 27, 2020, 6:11 a.m.) > > > Review request for hive, Jesús Camacho Rodríguez, Steve Carlin, and Vineet > Garg. > > > Bugs: HIVE-23206 > https://issues.apache.org/jira/browse/HIVE-23206 > > > Repository: hive-git > > > Description > ------- > > Project not defined correctly after reordering a join > > > Diffs > ----- > > itests/src/test/resources/testconfiguration.properties c55f8db61a > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveJoinProjectTransposeRule.java > 492c55e050 > ql/src/test/queries/clientpositive/join_reorder5.q PRE-CREATION > ql/src/test/results/clientpositive/auto_join22.q.out 5a98716fed > ql/src/test/results/clientpositive/correlationoptimizer5.q.out 2e9e6027ae > ql/src/test/results/clientpositive/filter_cond_pushdown.q.out 74a7aa89e7 > ql/src/test/results/clientpositive/join22.q.out ad34bc4310 > ql/src/test/results/clientpositive/llap/correlationoptimizer3.q.out > f063766a1f > ql/src/test/results/clientpositive/llap/join_reorder5.q.out PRE-CREATION > ql/src/test/results/clientpositive/llap/keep_uniform.q.out 54d0b5fab6 > ql/src/test/results/clientpositive/llap/sharedwork.q.out f8d3b4b2f5 > ql/src/test/results/clientpositive/llap/subquery_select.q.out 311cee743d > ql/src/test/results/clientpositive/perf/tez/cbo_query2.q.out 26a98ffcec > ql/src/test/results/clientpositive/perf/tez/cbo_query59.q.out abc5d999b5 > ql/src/test/results/clientpositive/perf/tez/cbo_query95.q.out 218ca7d8b6 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query14.q.out > eaa1defa81 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query2.q.out > 4c90da4476 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query59.q.out > 8d17cc79d1 > ql/src/test/results/clientpositive/perf/tez/constraints/cbo_query95.q.out > ace074316b > ql/src/test/results/clientpositive/perf/tez/constraints/query14.q.out > 8204245245 > ql/src/test/results/clientpositive/perf/tez/constraints/query2.q.out > 66777769e6 > ql/src/test/results/clientpositive/perf/tez/constraints/query59.q.out > f7c7260077 > ql/src/test/results/clientpositive/perf/tez/constraints/query95.q.out > 39d35ec330 > ql/src/test/results/clientpositive/perf/tez/query2.q.out 0e67e97c02 > ql/src/test/results/clientpositive/perf/tez/query59.q.out 1a2ba964f4 > ql/src/test/results/clientpositive/perf/tez/query95.q.out f15afbed4b > ql/src/test/results/clientpositive/runtime_skewjoin_mapjoin_spark.q.out > 9547e4fa7c > ql/src/test/results/clientpositive/smb_mapjoin_25.q.out 8fb82e1659 > > > Diff: https://reviews.apache.org/r/72431/diff/2/ > > > Testing > ------- > > mvn test -Dtest.output.overwrite -DskipSparkTests > -Dtest=TestMiniLlapLocalCliDriver -Dqfile=join_reorder5.q -pl itests/qtest > -Pitests > > > Thanks, > > Krisztian Kasa > >
