Tim Armstrong has uploaded a new change for review. http://gerrit.cloudera.org:8080/5683
Change subject: IMPALA-4748: crash in TmpFileMgr when hitting process mem limit ...................................................................... IMPALA-4748: crash in TmpFileMgr when hitting process mem limit The bug is that FileGroup didn't correctly handle its 'io_ctx_' being asynchronously cancelled: it left the WriteHandle in an invalid state. This could happen when the process memory limit was exceeded. I fixed this in two ways (either of which would be sufficient to avoid this exact crash): * Fix the error handling in TmpFileMgr so that things are left in a valid state on the error path. * Stop DiskIoMgr from asynchronously cancelling I/O contexts with no associated MemTracker. The mem_limit check and error propagation is necessary when DiskIoMgr will allocate memory on behalf of the client, but is not necessary when it is not allocating memory for the client - it just added a redundant error propagation mechanism. Testing: This scenario should no longer be possible for BufferedBlockMgr since DiskIoMgr won't cancel its I/O context, since it has no associated MemTracker. However, to test that errors on this path are correctly handled, I added a simple unit test to TmpFileMgr that forces cancellation of the I/O context. Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8 --- M be/src/runtime/disk-io-mgr.cc M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h 4 files changed, 48 insertions(+), 11 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/5683/2 -- To view, visit http://gerrit.cloudera.org:8080/5683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8 Gerrit-PatchSet: 2 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com>