Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3936: BufferedBlockMgr fixes for Pin() while write in flight. ......................................................................
IMPALA-3936: BufferedBlockMgr fixes for Pin() while write in flight. Fix multiple bugs that could occur when a block was unpinned then pinned again while the write was in flight. There were two problems: 1. A block's buffer could be transferred while a write is in flight, leaving the block in an invalid state. The fix is to wait for the in-flight write to complete before transferring the buffer. 2. On certain code paths in WriteComplete(), condition variables weren't signalled, leading to threads waiting for write completion not being woken up. The fix is to clarify when condition variables will be signalled and ensure that the appropriate condition variables are always signalled when the write completes. Testing: Added a targeted unit test that exercises these code paths using a debug option that controls timing of writes. Reran the stress test configuration that reproducibly triggered the bug: TPC-H query 18 on a release build with a single impalad. It succeeded. Change-Id: I4be4fad8e6f2303db19ea1e2bd0f13523781ae8e Reviewed-on: http://gerrit.cloudera.org:8080/3832 Reviewed-by: Tim Armstrong <[email protected]> Tested-by: Internal Jenkins --- M be/src/runtime/buffered-block-mgr-test.cc M be/src/runtime/buffered-block-mgr.cc M be/src/runtime/buffered-block-mgr.h 3 files changed, 121 insertions(+), 27 deletions(-) Approvals: Internal Jenkins: Verified Tim Armstrong: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/3832 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I4be4fad8e6f2303db19ea1e2bd0f13523781ae8e Gerrit-PatchSet: 5 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Matthew Jacobs <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
