Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. ......................................................................
IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. The dtor of a RuntimeState object disconnects instance_mem_tracker_ from its parent before calling the dtor of its child exec nodes (which are kept in the obj_pool of RuntimeState). This ensures that no destroyed MemTracker will be accessed when the MemTracker's hierarchy (e.g. process MemTracker) is traversed. The above only works for those child trackers which are destroyed after the RuntimeState's dtor is called. This is not necessarily the case for the DataSink object in PlanFragmentExecutor or the BufferedBlockMgr in RuntimeState itself. To avoid accessing destroyed MemTrackers of a DataSink object, this change explicitly unlinks a DataSink object's MemTracker from its parent at its Close() function. BufferedBlockMgr already does the same thing in its dtor. This change also fixes HdfsSequenceTableWriter::Close() so it will release the memory it has consumed and free MemPool and the memory in it. Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb Reviewed-on: http://gerrit.cloudera.org:8080/2314 Reviewed-by: Michael Ho <[email protected]> Tested-by: Internal Jenkins --- M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/hbase-table-sink.cc M be/src/exec/hdfs-sequence-table-writer.cc M be/src/exec/hdfs-sequence-table-writer.h M be/src/exec/hdfs-table-sink.cc M be/src/runtime/data-stream-sender.cc M be/src/runtime/plan-fragment-executor.h 8 files changed, 32 insertions(+), 7 deletions(-) Approvals: Michael Ho: Looks good to me, approved Internal Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/2314 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb Gerrit-PatchSet: 5 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.5.0_5.7.0 Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Silvius Rus <[email protected]> Gerrit-Reviewer: Skye Wanderman-Milne <[email protected]>
