Tim Armstrong has uploaded a new patch set (#3). Change subject: IMPALA-5497: spilling hash joins that output build rows hit OOM ......................................................................
IMPALA-5497: spilling hash joins that output build rows hit OOM The bug is that the join tried to bring the next spilled partition into memory while still holding onto memory from the current partition. The fix is to return earlier if the output batch is at capacity so that resources are flushed. Also reduce some of the redundancy in the loop that drives the spilling logic. Testing: The failure was originally reproduced by my IMPALA-4703 patch. I was able to cause a query failure with the current code by reducing the memory limit for an existing query. Before it failed with up to 12MB of memory. Now it succeeds with 8MB or less. Ran exhaustive build. Change-Id: I075388d348499c5692d044ac1bc38dd8dd0b10c7 --- M be/src/exec/partitioned-hash-join-node.cc M testdata/workloads/functional-query/queries/QueryTest/spilling.test 2 files changed, 11 insertions(+), 18 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/80/7180/3 -- To view, visit http://gerrit.cloudera.org:8080/7180 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I075388d348499c5692d044ac1bc38dd8dd0b10c7 Gerrit-PatchSet: 3 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]>
