Hello Michael Ho, Internal Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/2424
to look at the new patch set (#2).
Change subject: IMPALA-3085: Unregister data sinks' MemTrackers at their
Close() functions.
......................................................................
IMPALA-3085: Unregister data sinks' MemTrackers at their Close() functions.
The dtor of a RuntimeState object disconnects instance_mem_tracker_
from its parent before calling the dtor of its child exec nodes
(which are kept in the obj_pool of RuntimeState). This ensures
that no destroyed MemTracker will be accessed when the MemTracker's
hierarchy (e.g. process MemTracker) is traversed. The above
only works for those child trackers which are destroyed after
the RuntimeState's dtor is called. This is not necessarily the
case for the DataSink object in PlanFragmentExecutor or the
BufferedBlockMgr in RuntimeState itself.
To avoid accessing destroyed MemTrackers of a DataSink object,
this change explicitly unlinks a DataSink object's MemTracker
from its parent at its Close() function. BufferedBlockMgr already
does the same thing in its dtor.
This change also fixes HdfsSequenceTableWriter::Close() so it will
release the memory it has consumed and free MemPool and the memory in it.
Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb
Reviewed-on: http://gerrit.cloudera.org:8080/2314
Reviewed-by: Michael Ho <[email protected]>
Tested-by: Internal Jenkins
(cherry picked from commit c9544b83759688fa6b124b67f9daea1bd6267e7c)
---
M be/src/exec/data-sink.cc
M be/src/exec/data-sink.h
M be/src/exec/hbase-table-sink.cc
M be/src/exec/hdfs-sequence-table-writer.cc
M be/src/exec/hdfs-sequence-table-writer.h
M be/src/exec/hdfs-table-sink.cc
M be/src/runtime/data-stream-sender.cc
M be/src/runtime/mem-tracker.cc
M be/src/runtime/plan-fragment-executor.h
9 files changed, 37 insertions(+), 13 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala refs/changes/24/2424/2
--
To view, visit http://gerrit.cloudera.org:8080/2424
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3aec82150a933dc2b261beff41f5f4f022501bfb
Gerrit-PatchSet: 2
Gerrit-Project: Impala
Gerrit-Branch: cdh5-2.2.0_5.4.x
Gerrit-Owner: Huaisi Xu <[email protected]>
Gerrit-Reviewer: Internal Jenkins
Gerrit-Reviewer: Michael Ho <[email protected]>
