Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/9680 )
Change subject: IMPALA-6587: free buffers before ScanRange::Cancel() returns ...................................................................... Patch Set 5: (5 comments) http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc File be/src/runtime/io/disk-io-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@714 PS5, Line 714: StartRead > how about GetBufferToReadIntoAndMarkReadStart : this is probably too verbos Not relevant after other changes. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@716 PS5, Line 716: // No buffer available > update comment: No buffer available or cancelled Not relevant after other changes. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@722 PS5, Line 722: > might be a more invasive change, but since all the logic of reading is insi That was a good observation. I also realised that we were holding RequestContext::lock_ for longer than was actually necessary. I ended up refactoring this so to separate the logic that operators on the ScanRange and the logic that updates the RequestContext based on the outcome of the read. I think it is a lot easier to understand now. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-context.cc File be/src/runtime/io/request-context.cc: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-context.cc@108 PS5, Line 108: range->CancelInternal(Status::CANCELLED, false); > Even though more scan ranges will not be started and this problem will not I think it might be more likely to cause bugs than prevent them, since we're holding RequestContext::lock_ and won't be releasing that when waiting on the condition variable. I think it would work after the refactoring I did, but doesn't seem necessary. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-ranges.h File be/src/runtime/io/request-ranges.h: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-ranges.h@491 PS5, Line 491: /// Condition variable for threads in GetNext() that are waiting for the next buffer. : /// Signalled when a buffer is enqueued in 'ready_buffers_' or the scan range is : /// cancelled. > update comment, as wait_on_reader is also using this Done -- To view, visit http://gerrit.cloudera.org:8080/9680 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I87182b6bd51b5fb0b923e7e4c8d08a44e7617db2 Gerrit-Change-Number: 9680 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Mon, 19 Mar 2018 22:05:50 +0000 Gerrit-HasComments: Yes
