Michael Ho has posted comments on this change. Change subject: IMPALA-3611: track unused Disk IO buffer memory ......................................................................
Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/3246/1/be/src/runtime/disk-io-mgr.cc File be/src/runtime/disk-io-mgr.cc: Line 700: Consume(buffer_size) Do you think it makes sense to convert this to TryConsume() in the long run or is it too hard to propagate the failure at this point ? Looks like the caller already checks for the status in the buffer descriptor created below so may be we can somehow take advantage of that. Also, it seems a bit convoluted to first count the consumption towards the buffer_mem_tracker_ and then transfer it to the reader's mem-tracker in GetBufferDesc(). May be we should outright count the consumption towards the reader and call Release() on buffer_mem_tracker_ explicitly if we are consuming from the free_buffers_ ? Line 764: SetMemTracker( This is a bit of a misnomer. Do you think it makes sense to call it TransferMemory() or TransferBufferOwnership() or something along that line ? Line 829: process_limit_exceeded = buffer_mem_tracker_->AnyLimitExceeded(); I wonder if this check should be moved to ReturnFreeBuffer() ? Also, this is no longer tracking process memory limit so may be this variable should be renamed if it's kept. -- To view, visit http://gerrit.cloudera.org:8080/3246 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I8777cf76f04d34a46f53d53005412e0f1d63b5b7 Gerrit-PatchSet: 1 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Tim Armstrong <[email protected]> Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-HasComments: Yes
