Huaisi Xu has submitted this change and it was merged. Change subject: IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. ......................................................................
IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions. The dtor of a RuntimeState object disconnects instance_mem_tracker_ from its parent before calling the dtor of its child exec nodes (which are kept in the obj_pool of RuntimeState). This ensures that no destroyed MemTracker will be accessed when the MemTracker's hierarchy (e.g. process MemTracker) is traversed. The above only works for those child trackers which are destroyed after the RuntimeState's dtor is called. This is not necessarily the case for the DataSink object in PlanFragmentExecutor or the BufferedBlockMgr in RuntimeState itself. To avoid accessing destroyed MemTrackers of a DataSink object, this change explicitly unlinks a DataSink object's MemTracker from its parent at its Close() function. BufferedBlockMgr already does the same thing in its dtor. This change also fixes HdfsSequenceTableWriter::Close() so it will release the memory it has consumed and free MemPool and the memory in it. Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb Reviewed-on: http://gerrit.cloudera.org:8080/2314 Reviewed-by: Michael Ho <[email protected]> Tested-by: Internal Jenkins (cherry picked from commit c9544b83759688fa6b124b67f9daea1bd6267e7c) Reviewed-on: http://gerrit.cloudera.org:8080/2424 Reviewed-by: Juan Yu <[email protected]> Tested-by: Huaisi Xu <[email protected]> --- M be/src/exec/data-sink.cc M be/src/exec/data-sink.h M be/src/exec/hbase-table-sink.cc M be/src/exec/hdfs-sequence-table-writer.cc M be/src/exec/hdfs-sequence-table-writer.h M be/src/exec/hdfs-table-sink.cc M be/src/runtime/data-stream-sender.cc M be/src/runtime/plan-fragment-executor.h 8 files changed, 37 insertions(+), 12 deletions(-) Approvals: Huaisi Xu: Verified Juan Yu: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/2424 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb Gerrit-PatchSet: 4 Gerrit-Project: Impala Gerrit-Branch: cdh5-2.2.0_5.4.x Gerrit-Owner: Huaisi Xu <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Juan Yu <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]>
