Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-5611: KuduPartitionExpr holds onto memory unnecessarily ......................................................................
IMPALA-5611: KuduPartitionExpr holds onto memory unnecessarily IMPALA-3742 introduced KuduPartitionExpr, which takes a row and passes it to the Kudu client to determine what partitionit belongs to. The DataStreamSender never frees the local allocations for the Kudu partition exprs causing it to hang on to memory longer than it needs to. This patch also fixes two other related issues: - DataStreamSender was dropping the Status from AddRow in the Kudu branch. Adds 'RETURN_IF_ERROR' and 'WARN_UNUSED_RESULT' - Changes the HASH case in DataStreamSender to call FreeLocalAllocations on a per-batch basis, instead of a per-row basis. Testing: - Added an e2e test that runs a large insert with a mem limit that failed with oom previously. Change-Id: Ia661eb8bed114070728a1497ccf7ed6893237e5e Reviewed-on: http://gerrit.cloudera.org:8080/7346 Reviewed-by: Dan Hecht <[email protected]> Reviewed-by: Michael Ho <[email protected]> Tested-by: Impala Public Jenkins --- M be/src/runtime/data-stream-sender.cc M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test 2 files changed, 13 insertions(+), 7 deletions(-) Approvals: Impala Public Jenkins: Verified Michael Ho: Looks good to me, but someone else must approve Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/7346 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ia661eb8bed114070728a1497ccf7ed6893237e5e Gerrit-PatchSet: 6 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
