Hello David Ribeiro Alves, Andrew Wong,
I'd like you to do a code review. Please visit
http://gerrit.cloudera.org:8080/6460
to review the following change.
Change subject: generic_iterators: avoid holding completed iters in merge/union
......................................................................
generic_iterators: avoid holding completed iters in merge/union
Previously, the MergeIterator and UnionIterator would hold on to all of
the iterators they consume until the entire process completed. Each such
iterator may hold on to blocks in the block cache (eg dictionary blocks)
and consume a fair amount of memory.
Without this patch, I noticed that issuing a scan caused the block cache
consumed memory to slowly increase untl the scan finished, at which
point it would quickly drop. With the patch, I no longer see this
behavior.
The one feature lost here is the stringification of MergeIterator which
previously included all of the sub-iterators. We only used this in one
place in a VLOG(1) message anyway, and for typical merges across
thousands of DRS iterators, this wasn't likely to be particularly useful
anyway.
Change-Id: I8e068c256c5855efeff5c6325dbb5fbf770eb900
---
M src/kudu/common/generic_iterators.cc
M src/kudu/common/generic_iterators.h
2 files changed, 48 insertions(+), 51 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/60/6460/1
--
To view, visit http://gerrit.cloudera.org:8080/6460
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I8e068c256c5855efeff5c6325dbb5fbf770eb900
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Todd Lipcon <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: David Ribeiro Alves <[email protected]>
