Tim Armstrong has posted comments on this change. Change subject: IMPALA-1286: Extract common conjuncts from disjunctions. ......................................................................
Patch Set 2: Code-Review+1 (4 comments) http://gerrit.cloudera.org:8080/#/c/4877/1/fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java File fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java: Line 31: * Extracts common conjuncts from a single disjunctive CompoundPredicate. > Expanded comments on ExprRewriter and modified comment here. I agree we don't need to document the framework here, but I think it's non-obvious that implementing it for a single CompoundPredicate is equally as powerful as implementing it for n CompoundPredicates. If I understand correctly it relies on the bottom-up application of rewrite rules to "bubble up" common predicates. E.g. "This rule extracts common conjuncts from multiple disjunctions when it is applied recursively bottom-up to a tree of CompoundPredicates". Line 49: > Agree that this is problematic. Unfortunately, also non-trivial to fix and Codegen in the BE generally shouldn't be N^2, although the constant factor is much higher. http://gerrit.cloudera.org:8080/#/c/4877/2/fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java File fe/src/main/java/org/apache/impala/rewrite/ExtractCommonConjunctRule.java: Line 51: List<Expr> child0Conjuncts = expr.getChild(0).getConjuncts(); > checkstate that the child conjuncts aren't empty? true or x => true seems like a normalisation rule we should add in a follow-up patch http://gerrit.cloudera.org:8080/#/c/4877/1/fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java File fe/src/test/java/org/apache/impala/analysis/ExprRewriteRulesTest.java: Line 146: "(!(int_col=5 or tinyint_col > 9) and double_col = 7) or " + > Completely agree about having additional rules for normalization. I view th I think we should just be mindful that this change without normalisation added a performance cliff where e.g. flipping x < y to y > x can change the plan. I think prior to this change the two were equivalent. Seems ok as an intermediate state but we shouldn't consider the feature finished until we've made it a little more robust. I think this probably also relates to the constant expr -> literal rewrite that's on my plate. -- To view, visit http://gerrit.cloudera.org:8080/4877 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I3cf9b950afaa3fd753d1b09ba5e540b5258940ad Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Alex Behm <[email protected]> Gerrit-Reviewer: Alex Behm <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Marcel Kornacker <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-HasComments: Yes
