Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/11698 )
Change subject: IMPALA-5004: Switch to sorting node for large TopN queries ...................................................................... Patch Set 5: (4 comments) http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/analysis/SortInfo.java File fe/src/main/java/org/apache/impala/analysis/SortInfo.java: http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/analysis/SortInfo.java@244 PS5, Line 244: getSortTupleDescriptor().getAvgSerializedSize() * (cardinality + offset)); > TODO: can cardinality + offset overflow? Done http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java File fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java: http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@324 PS5, Line 324: // SortNode#computeStats > Can we factor out the formula into a static helper method in SortNode? Done http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java@337 PS5, Line 337: limit + offset > Here and elsewhere: I think limit + offset could overflow. We don't have an Do we have an equivalent to checkedAdd() for C++? I added the calls to checkedAdd() but the queries still fail in the be/. Specifically, exchange-node.h:226 (the call to CopyRows fails on the DCHECK_LE check). I've added the planner tests for this, and they work (since they don't actually run the query, just generate the explain plan). http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/test/java/org/apache/impala/planner/PlannerTest.java File fe/src/test/java/org/apache/impala/planner/PlannerTest.java: http://gerrit.cloudera.org:8080/#/c/11698/5/fe/src/test/java/org/apache/impala/planner/PlannerTest.java@214 PS5, Line 214: public void testTopNBytesLimitDisabled() { > perhaps merge this with the test on L195? settings are the same so unclear deleted this test and merged it with testTopN -- To view, visit http://gerrit.cloudera.org:8080/11698 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I34c9db33c9302b55e9978f53f9c7061f2806c8a9 Gerrit-Change-Number: 11698 Gerrit-PatchSet: 5 Gerrit-Owner: Sahil Takiar <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Paul Rogers <[email protected]> Gerrit-Reviewer: Sahil Takiar <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vuk Ercegovac <[email protected]> Gerrit-Comment-Date: Fri, 26 Oct 2018 14:26:39 +0000 Gerrit-HasComments: Yes
