Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15142 )
Change subject: IMPALA-9349: free output_unmatched_batch_ buffers promptly in PHJ ...................................................................... IMPALA-9349: free output_unmatched_batch_ buffers promptly in PHJ This fixes a subtle memory managment issue where freeing of a buffer is delayed longer than it should be. This means that the full buffer pool reservation is not available for repartitioning, which can lead to crashes or hang for very specific queries. The fix is to transfer resources from output_unmatched_batch_ as soon as the last row from the batch is appended to the output batch. This bug would only be triggered by join modes that output unmatched rows from the right side (RIGHT OUTER JOIN, FULL OUTER JOIN, RIGHT ANTI JOIN) *and* have an empty probe side (otherwise unmatched rows are output by iterating over the hash table). Testing: Added DCHECKs to check that all resources are available before repartitioning. Added a regression test that triggered the bug. Change-Id: Ie13b51d4d909afb0fe2e7b7dc00b085c51058fed Reviewed-on: http://gerrit.cloudera.org:8080/15142 Reviewed-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Tested-by: Impala Public Jenkins <impala-public-jenk...@cloudera.com> --- M be/src/exec/partitioned-hash-join-builder.cc M be/src/exec/partitioned-hash-join-node.cc M testdata/workloads/functional-query/queries/QueryTest/spilling-no-debug-action.test 3 files changed, 22 insertions(+), 0 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15142 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ie13b51d4d909afb0fe2e7b7dc00b085c51058fed Gerrit-Change-Number: 15142 Gerrit-PatchSet: 4 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Csaba Ringhofer <csringho...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Zoltan Borok-Nagy <borokna...@cloudera.com>
