Bikramjeet Vig has posted comments on this change. ( http://gerrit.cloudera.org:8080/9680 )
Change subject: IMPALA-6587: free buffers before ScanRange::Cancel() returns ...................................................................... Patch Set 5: (6 comments) http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc File be/src/runtime/io/disk-io-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@714 PS5, Line 714: StartRead how about GetBufferToReadIntoAndMarkReadStart : this is probably too verbose for a method name, but we should come up with something that appropriately signifies what this method does http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@716 PS5, Line 716: // No buffer available update comment: No buffer available or cancelled http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/disk-io-mgr.cc@722 PS5, Line 722: might be a more invasive change, but since all the logic of reading is inside the ScanRange, maybe we should move this logic inside it too and avoid taking out a buffer and moving it around. This way the only method that returns a buffer will be GetNext. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-context.cc File be/src/runtime/io/request-context.cc: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-context.cc@108 PS5, Line 108: range->CancelInternal(Status::CANCELLED, false); Even though more scan ranges will not be started and this problem will not be hit here, but should we still call WaitForInFlightRead here? Step 3 mentioned above would ensure that those disk threads close but I just want to make sure I am not missing anything. http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-ranges.h File be/src/runtime/io/request-ranges.h: http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-ranges.h@473 PS5, Line 473: bool read_in_flight_ = false; > These three boolean flags are all mutually exclusive. I thought about using not strongly in favour of it, might be a bit of an overkill http://gerrit.cloudera.org:8080/#/c/9680/5/be/src/runtime/io/request-ranges.h@491 PS5, Line 491: /// Condition variable for threads in GetNext() that are waiting for the next buffer. : /// Signalled when a buffer is enqueued in 'ready_buffers_' or the scan range is : /// cancelled. update comment, as wait_on_reader is also using this -- To view, visit http://gerrit.cloudera.org:8080/9680 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I87182b6bd51b5fb0b923e7e4c8d08a44e7617db2 Gerrit-Change-Number: 9680 Gerrit-PatchSet: 5 Gerrit-Owner: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Reviewer: Bikramjeet Vig <bikramjeet....@cloudera.com> Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com> Gerrit-Comment-Date: Sat, 17 Mar 2018 01:18:29 +0000 Gerrit-HasComments: Yes
