Vuk Ercegovac has posted comments on this change. ( http://gerrit.cloudera.org:8080/11698 )
Change subject: IMPALA-5004: Switch to sorting node for large TopN queries ...................................................................... Patch Set 3: (3 comments) http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@299 PS3, Line 299: // Add a Sort Node instead of a TopN Node if the estimated size of the materialized : // rows for TopN exceeds the value of TOPN_BYTES_LIMIT : long estimatedTopNMaterializedSize = 0; : f I'd prefer to put this block into SortInfo, called say, estimateMaterializedRecordSize. I see that SortInfo is up for debate, but at least we'll keep such sort-related code in one place. http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@306 PS3, Line 306: estimatedTopNMaterializedSize += sortSlotDesc.getByteSize(); how's this compare with the row size estimate in SortNode.computeNodeResourceProfile? http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@310 PS3, Line 310: *= I doubt we'll run into it in practice, but prefer to use PlanNode.checkedMultiply(..) here to deal with estimate overflows in a uniform way. -- To view, visit http://gerrit.cloudera.org:8080/11698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I34c9db33c9302b55e9978f53f9c7061f2806c8a9 Gerrit-Change-Number: 11698 Gerrit-PatchSet: 3 Gerrit-Owner: Sahil Takiar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Paul Rogers <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Fri, 19 Oct 2018 01:24:45 +0000 Gerrit-HasComments: Yes
