Impala Public Jenkins has submitted this change and it was merged. Change subject: IMPALA-4748: crash in TmpFileMgr when hitting process mem limit ......................................................................
IMPALA-4748: crash in TmpFileMgr when hitting process mem limit The bug is that FileGroup didn't correctly handle its 'io_ctx_' being asynchronously cancelled: it left the WriteHandle in an invalid state. This could happen when the process memory limit was exceeded. I fixed this in two ways (either of which would be sufficient to avoid this exact crash): * Fix the error handling in TmpFileMgr so that things are left in a valid state on the error path. * Stop DiskIoMgr from asynchronously cancelling I/O contexts with no associated MemTracker. The mem_limit check and error propagation is necessary when DiskIoMgr will allocate memory on behalf of the client, but is not necessary when it is not allocating memory for the client - it just added a redundant error propagation mechanism. Testing: This scenario should no longer be possible for BufferedBlockMgr since DiskIoMgr won't cancel its I/O context, since it has no associated MemTracker. However, to test that errors on this path are correctly handled, I added a simple unit test to TmpFileMgr that forces cancellation of the I/O context. Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8 Reviewed-on: http://gerrit.cloudera.org:8080/5683 Reviewed-by: Tim Armstrong <tarmstr...@cloudera.com> Tested-by: Impala Public Jenkins --- M be/src/runtime/disk-io-mgr.cc M be/src/runtime/tmp-file-mgr-test.cc M be/src/runtime/tmp-file-mgr.cc M be/src/runtime/tmp-file-mgr.h 4 files changed, 61 insertions(+), 16 deletions(-) Approvals: Impala Public Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/5683 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: Ib0a624212bc17f7824e6d14ad143c0d5894206f8 Gerrit-PatchSet: 7 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
