Michael Ho has uploaded a new patch set (#3). Change subject: IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. ......................................................................
IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. The dtor of a RuntimeState object disconnects instance_mem_tracker_ from its parent before calling the dtor of its child exec nodes (which are kept in the obj_pool of RuntimeState). This ensures that no destroyed MemTracker will be accessed when the MemTracker's hierarchy (e.g. process MemTracker) is traversed. The above only works for those child trackers which are destroyed after the RuntimeState's dtor is called. This is not necessarily the case for DataSink object in PlanFragmentExecutor or the BufferedBlockMgr in RuntimeState itself. To avoid accessing destroyed MemTrackers of a DataSink object, this change explicitly unlinks a DataSink object's MemTracker from its parent at its Close() function. BufferedBlockMgr already does the same thing in its dtor. This change also fixes HdfsSequenceTableWriter::Close() so it will release memory it has consumed and free the memory in its MemPool. Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb --- M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/hbase-table-sink.cc M be/src/exec/hdfs-sequence-table-writer.cc M be/src/exec/hdfs-sequence-table-writer.h M be/src/exec/hdfs-table-sink.cc M be/src/runtime/data-stream-sender.cc M be/src/runtime/plan-fragment-executor.h 8 files changed, 30 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/14/2314/3 -- To view, visit http://gerrit.cloudera.org:8080/2314 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb Gerrit-PatchSet: 3 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.5.0_5.7.0 Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]>
