Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/20498 )
Change subject: IMPALA-12018: Consider runtime filter for cardinality reduction ...................................................................... Patch Set 2: (16 comments) http://gerrit.cloudera.org:8080/#/c/20498/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/20498/1//COMMIT_MSG@18 PS1, Line 18: select > Nit: selects Done http://gerrit.cloudera.org:8080/#/c/20498/1//COMMIT_MSG@20 PS1, Line 20: produce > Nit: produces. Done http://gerrit.cloudera.org:8080/#/c/20498/1//COMMIT_MSG@25 PS1, Line 25: option is True) > It's not clear to me how it explains that the cardinality reduction is only Clarified the intention in commit message. I plan to evaluate this in smaller scope first before enabling it in all planning mode. http://gerrit.cloudera.org:8080/#/c/20498/1//COMMIT_MSG@25 PS1, Line 25: mode (CO > Typo: PROCESSING Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/PlanNode.java File fe/src/main/java/org/apache/impala/planner/PlanNode.java: http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/PlanNode.java@128 PS1, Line 128: // invalid: -1 > Mention in the comment that it can also be replaced in replaceCardinality() Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/PlanNode.java@132 PS1, Line 132: d la > Does it need to be a Long? Can't we use -1 as the value indicating an inval I use Long here in case the original value cardinality_ itself is unknown (-1). Therefore, after replaceCardinality() called, (cardinality_=10, originalCardinality_=null) can be differentiated from (cardinality_=10, originalCardinality_=-1). http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/PlanNode.java@386 PS1, Line 386: if (originalCardinality_ != null) { > Shouldn't we add some explanation here about what the value in parentheses Added "changed from". Please check the output in tpcds-processing-cost.test and let me know if this is OK or too verbose. http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java File fe/src/main/java/org/apache/impala/planner/Planner.java: http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@534 PS1, Line 534: refine > Nit: refines. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@547 PS1, Line 547: side > Nit: do we need 'hand' here? Removed. http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@585 PS1, Line 585: ace("refineCardin > Could add a precondition check that 'nodeStack_' is not empty. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@592 PS1, Line 592: Filter filter : scan.ge > Could extract it into a variable. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@592 PS1, Line 592: ator.Runtim > Could use Map::computeIfAbsent() here. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@621 PS1, Line 621: consider > Nit: considers. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/Planner.java@658 PS1, Line 658: require > Nit: requires. Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java File fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java: http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@640 PS1, Line 640: for (RuntimeFilterTarget target : targets_) { > I think it would be cleaner if we inverted the if and returned from there - Done http://gerrit.cloudera.org:8080/#/c/20498/1/fe/src/main/java/org/apache/impala/planner/RuntimeFilterGenerator.java@658 PS1, Line 658: > Can we check if it is a Bloom filter? We could add a Precondition check. I chose to clarify with comment instead. Calling getEstFpp() on other kind of filter is still legal, but will return 0. Please review if it is OK or not. -- To view, visit http://gerrit.cloudera.org:8080/20498 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I033789c9b63a8188484e3afde8e646563918b3e1 Gerrit-Change-Number: 20498 Gerrit-PatchSet: 2 Gerrit-Owner: Riza Suminto <[email protected]> Gerrit-Reviewer: Abhishek Rawat <[email protected]> Gerrit-Reviewer: Csaba Ringhofer <[email protected]> Gerrit-Reviewer: Daniel Becker <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]> Gerrit-Comment-Date: Wed, 18 Oct 2023 00:31:32 +0000 Gerrit-HasComments: Yes
