Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/11698 )
Change subject: IMPALA-5004: Switch to sorting node for large TopN queries ...................................................................... Patch Set 7: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: http://gerrit.cloudera.org:8080/#/c/11698/3/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@313 PS3, Line 313: long limit, long offset, boolean hasLimit, boolean disableTopN) > After discussing this a bit more with Tim, it seems that we generally avoid Sahil covered this, but just wanted to confirm that there was a deliberate decision not to pursue, in general, making planner decisions to fit a query into a memory limit. We do give up some opportunities to make more optimal decisions but it's definitely simpler for now. I don't recall seeing a case in practice where Top-N memory consumption caused a query failure but it seems like something that will happen eventually, and this change helps avoid it or at least gives an escape hatch that doesn't require rewriting the query. -- To view, visit http://gerrit.cloudera.org:8080/11698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I34c9db33c9302b55e9978f53f9c7061f2806c8a9 Gerrit-Change-Number: 11698 Gerrit-PatchSet: 7 Gerrit-Owner: Sahil Takiar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Paul Rogers <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Mon, 29 Oct 2018 16:02:27 +0000 Gerrit-HasComments: Yes
