Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3332: Free local allocations in sorter. ......................................................................
IMPALA-3332: Free local allocations in sorter. Sorter can have runaway memory consumption as it never frees local allocations made in comparator_.Less(). In addition, it doesn't check for errors generated during expression evaluation so it may keep sorting even after failures have occurred. This change fixes the problem by freeing local allocations for every n invocations of comparator_.Less() where n is the row batch size specified in the query options. Various error checks are also added to return early if any error is encountered. Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1 Reviewed-on: http://gerrit.cloudera.org:8080/3116 Reviewed-by: Michael Ho <[email protected]> Tested-by: Internal Jenkins --- M be/src/runtime/sorted-run-merger.cc M be/src/runtime/sorter.cc M be/src/util/tuple-row-compare.h M testdata/workloads/functional-query/queries/QueryTest/spilling.test 4 files changed, 91 insertions(+), 52 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/3116 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I941729b4836e5dbb827d4313a0b45bc5df2fa8e1 Gerrit-PatchSet: 8 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.6.0_5.8.0 Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
