Hello Thomas Tauber-Marshall,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/5683
to look at the new patch set (#4).
Change subject: IMPALA-4748: crash in TmpFileMgr when hitting process mem limit
......................................................................
IMPALA-4748: crash in TmpFileMgr when hitting process mem limit
The bug is that FileGroup didn't correctly handle its 'io_ctx_'
being asynchronously cancelled: it left the WriteHandle in an
invalid state. This could happen when the process memory limit
was exceeded.
I fixed this in two ways (either of which would be sufficient
to avoid this exact crash):
* Fix the error handling in TmpFileMgr so that things are left
in a valid state on the error path.
* Stop DiskIoMgr from asynchronously cancelling I/O contexts
with no associated MemTracker. The mem_limit check and error
propagation is necessary when DiskIoMgr will allocate memory
on behalf of the client, but is not necessary when it is not
allocating memory for the client - it just added a redundant
error propagation mechanism.
Testing:
This scenario should no longer be possible for BufferedBlockMgr
since DiskIoMgr won't cancel its I/O context, since it has no
associated MemTracker. However, to test that errors on this path
are correctly handled, I added a simple unit test to TmpFileMgr
that forces cancellation of the I/O context.
Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8
---
M be/src/runtime/disk-io-mgr.cc
M be/src/runtime/tmp-file-mgr-test.cc
M be/src/runtime/tmp-file-mgr.cc
M be/src/runtime/tmp-file-mgr.h
4 files changed, 61 insertions(+), 16 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/5683/4
--
To view, visit http://gerrit.cloudera.org:8080/5683
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8
Gerrit-PatchSet: 4
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-Owner: Tim Armstrong <[email protected]>
Gerrit-Reviewer: Dan Hecht <[email protected]>
Gerrit-Reviewer: Thomas Tauber-Marshall <[email protected]>
Gerrit-Reviewer: Tim Armstrong <[email protected]>
