Thomas Tauber-Marshall has uploaded a new patch set (#2). Change subject: IMPALA-5611: KuduPartitionExpr holds onto memory unnecessarily ......................................................................
IMPALA-5611: KuduPartitionExpr holds onto memory unnecessarily IMPALA-3742 introduced KuduPartitionExpr, which takes a row and passes it to the Kudu client to determine what partitionit belongs to. KuduPartitionExpr never calls ScalarExprEvaluator::FreeLocalAllocations, causing it to hang on to memory longer than it needs it. Since we only need the value of the row for the call into the Kudu client, we can call FreeLocalAllocations after that. This patch also fixes two other related issues: - DataStreamSender was dropping the Status from AddRow in the Kudu branch. Adds 'RETURN_IF_ERROR' and 'WARN_UNUSED_RESULT' - Changes the HASH case in DataStreamSender to call FreeLocalAllocations on a per-batch basis, instead of a per-row basis. Testing: - Manually verified that large Kudu inserts with string columns can pass with much lower mem limits now. Change-Id: Ia661eb8bed114070728a1497ccf7ed6893237e5e --- M be/src/exprs/kudu-partition-expr.cc M be/src/runtime/data-stream-sender.cc M testdata/workloads/functional-query/queries/QueryTest/kudu_insert.test 3 files changed, 14 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/46/7346/2 -- To view, visit http://gerrit.cloudera.org:8080/7346 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia661eb8bed114070728a1497ccf7ed6893237e5e Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
