> On Jan. 19, 2017, 7:53 a.m., Ashutosh Chauhan wrote: > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java, > > line 234 > > <https://reviews.apache.org/r/55553/diff/2/?file=1608674#file1608674line234> > > > > Any reason for disabling this rule?
For nested scalar subqueries this rule was, for whatever reason, pushing filter on top of LogicalCorrelate to within left input of LogicalCorrelate, which is wrong and causes an exception during decorrelation. I'll add a comment and plan to open a JIRA to investigate why it is doing so. > On Jan. 19, 2017, 7:53 a.m., Ashutosh Chauhan wrote: > > ql/src/test/queries/clientpositive/subquery_in.q, line 63 > > <https://reviews.apache.org/r/55553/diff/2/?file=1608698#file1608698line63> > > > > Instead of removing you may add group by in subquery, unless this query > > pattern is covered elsewhere in tests. This was removed since now we disable corr IN/NOT IN queries with aggregate > On Jan. 19, 2017, 7:53 a.m., Ashutosh Chauhan wrote: > > ql/src/test/queries/clientpositive/subquery_in_having.q, line 43 > > <https://reviews.apache.org/r/55553/diff/2/?file=1608699#file1608699line43> > > > > This satisifies all conditions: correlated and no implied gby. Will > > this still wont work? That criteria works for Scalar queries only for now. For IN/NOT IN we disable corr queries with aggregate for now. > On Jan. 19, 2017, 7:53 a.m., Ashutosh Chauhan wrote: > > ql/src/test/queries/clientpositive/subquery_in_having.q, line 128 > > <https://reviews.apache.org/r/55553/diff/2/?file=1608699#file1608699line128> > > > > Dont we allow correlated predicates here? same reason as above. > On Jan. 19, 2017, 7:53 a.m., Ashutosh Chauhan wrote: > > ql/src/test/queries/clientpositive/subquery_notin.q, line 65 > > <https://reviews.apache.org/r/55553/diff/2/?file=1608700#file1608700line65> > > > > add gby ? this was remove because of same reason as above - Vineet ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/55553/#review162239 ----------------------------------------------------------- On Jan. 19, 2017, 11:58 p.m., Vineet Garg wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/55553/ > ----------------------------------------------------------- > > (Updated Jan. 19, 2017, 11:58 p.m.) > > > Review request for hive, Ashutosh Chauhan and Jesús Camacho Rodríguez. > > > Bugs: HIVE-15544 > https://issues.apache.org/jira/browse/HIVE-15544 > > > Repository: hive-git > > > Description > ------- > > This patch provides support for scalar subqueries in filter clause. Note that > this patch disables the following: > * IN/NOT IN correlated subqueries containing aggregates (HIVE checks for such > queries and throw an exception) > * SCALAR correlated subqueries containing aggregates with non-equi join > predicates on correlated columns (HIVE throws an exception for such queries). > > > Diffs > ----- > > itests/src/test/resources/testconfiguration.properties be5a747 > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java 6f01da0 > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/CalciteSubquerySemanticException.java > PRE-CREATION > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/HivePlannerContext.java > 8beb0dd > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveRelDecorrelator.java > a373cdd > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/rules/HiveSubQueryRemoveRule.java > f1e8ebd > > ql/src/java/org/apache/hadoop/hive/ql/optimizer/calcite/translator/RexNodeConverter.java > 8d2e535 > ql/src/java/org/apache/hadoop/hive/ql/parse/CalcitePlanner.java 9f1b9d5 > ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g cd9adfc > ql/src/java/org/apache/hadoop/hive/ql/parse/QBSubQuery.java 24381b9 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java f275f6a > ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java > 6c30efd > ql/src/java/org/apache/hadoop/hive/ql/plan/ExprNodeSubQueryDesc.java > aec331b > > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSQCountCheck.java > PRE-CREATION > > ql/src/test/org/apache/hadoop/hive/ql/optimizer/calcite/TestCBORuleFiredOnlyOnce.java > 44e157b > ql/src/test/queries/clientnegative/subquery_exists_implicit_gby.q 9013df6 > ql/src/test/queries/clientnegative/subquery_in_implicit_gby.q PRE-CREATION > ql/src/test/queries/clientnegative/subquery_notexists_implicit_gby.q > 852b295 > ql/src/test/queries/clientnegative/subquery_scalar_implicit_gby.q > PRE-CREATION > ql/src/test/queries/clientnegative/subquery_scalar_multi_columns.q > PRE-CREATION > ql/src/test/queries/clientnegative/subquery_scalar_multi_rows.q > PRE-CREATION > ql/src/test/queries/clientnegative/subquery_with_or_cond.q c2c3221 > ql/src/test/queries/clientpositive/perf/query1.q PRE-CREATION > ql/src/test/queries/clientpositive/perf/query23.q e8ebd86 > ql/src/test/queries/clientpositive/perf/query30.q PRE-CREATION > ql/src/test/queries/clientpositive/perf/query6.q PRE-CREATION > ql/src/test/queries/clientpositive/perf/query81.q PRE-CREATION > ql/src/test/queries/clientpositive/subquery_exists.q cac8e1b > ql/src/test/queries/clientpositive/subquery_in.q fe0c9c8 > ql/src/test/queries/clientpositive/subquery_in_having.q 40b7e32 > ql/src/test/queries/clientpositive/subquery_notin.q f9b5405 > ql/src/test/queries/clientpositive/subquery_scalar.q PRE-CREATION > ql/src/test/queries/clientpositive/subquery_unqualcolumnrefs.q bdfa648 > ql/src/test/results/clientnegative/subquery_exists_implicit_gby.q.out > f7251e3 > ql/src/test/results/clientnegative/subquery_in_implicit_gby.q.out > PRE-CREATION > ql/src/test/results/clientnegative/subquery_multiple_cols_in_select.q.out > 7a16bae > ql/src/test/results/clientnegative/subquery_notexists_implicit_gby.q.out > da38f5f > ql/src/test/results/clientnegative/subquery_scalar_implicit_gby.q.out > PRE-CREATION > ql/src/test/results/clientnegative/subquery_scalar_multi_columns.q.out > PRE-CREATION > ql/src/test/results/clientnegative/subquery_scalar_multi_rows.q.out > PRE-CREATION > ql/src/test/results/clientnegative/subquery_windowing_corr.q.out dcd3026 > ql/src/test/results/clientnegative/subquery_with_or_cond.q.out d2d743d > ql/src/test/results/clientpositive/llap/subquery_exists.q.out 35c64e7 > ql/src/test/results/clientpositive/llap/subquery_in.q.out 95c13e6 > ql/src/test/results/clientpositive/llap/subquery_multi.q.out 536bfdb > ql/src/test/results/clientpositive/llap/subquery_notin.q.out 21dea74 > ql/src/test/results/clientpositive/llap/subquery_scalar.q.out PRE-CREATION > ql/src/test/results/clientpositive/perf/query1.q.out PRE-CREATION > ql/src/test/results/clientpositive/perf/query23.q.out 6d4cfca > ql/src/test/results/clientpositive/perf/query30.q.out PRE-CREATION > ql/src/test/results/clientpositive/perf/query6.q.out PRE-CREATION > ql/src/test/results/clientpositive/perf/query81.q.out PRE-CREATION > ql/src/test/results/clientpositive/subquery_exists.q.out 9c02d6a > ql/src/test/results/clientpositive/subquery_in_having.q.out 125187a > ql/src/test/results/clientpositive/subquery_unqualcolumnrefs.q.out 1eb1b93 > > Diff: https://reviews.apache.org/r/55553/diff/ > > > Testing > ------- > > Added q tests > > > Thanks, > > Vineet Garg > >
