Hello Alexey Serbin, Attila Bukor, Kudu Jenkins, Andrew Wong, Mitch Barnett,
Adar Dembo,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11975
to look at the new patch set (#3).
Change subject: [tools] Add tool to dump memtrackers
......................................................................
[tools] Add tool to dump memtrackers
This adds a new tool, `kudu diagnose dump_mem_trackers`, that dumps the
mem-trackers information. It contains information equivalent to the
tracker table on /mem-trackers. There's a new RPC introduced to support
this tool. Output both as JSON and as a table is supported. The table
output type flattens the tracker hierarchy, but it can be
reconstructed from the 'id' and 'parent_id' columns.
Some Kudu community members have reported to me they have started using
the /mem-trackers page to investigate performance and memory issues with
Kudu clusters, and having a way to get the same information in a form
amenable to programmatic searching, sorting, and filtering is
convenient. This tool can also serve as a starting point for more
automated analysis of the tracker information.
The particular advantage of the memtracker dump over, say, the data
generated by heap sampling is that the memtracker shows where memory is
owned. The sampling just shows where it was allocated in a call stack.
So, for example, using the memtracker information it's possible to tell
very quickly exactly which tablets are using a lot of memory in
DeltaMemStores. Heap samples would indicate memory allocated in
update-related code paths, but it would not be possible to tell which
tablet replicas actually held the memory that was allocated.
Here's a small sample of output in csv table format:
tablet-2e02d00d46834b359c6ba0f8d471cbf9,server,-1,2318,2583
txn_tracker,tablet-2e02d00d46834b359c6ba0f8d471cbf9,67108864,0,0
MemRowSet-1,tablet-2e02d00d46834b359c6ba0f8d471cbf9,-1,265,265
DeltaMemStores,tablet-2e02d00d46834b359c6ba0f8d471cbf9,-1,265,265
tablet-2d40450f6556485eab92f64f90756b61,server,-1,2318,2583
txn_tracker,tablet-2d40450f6556485eab92f64f90756b61,67108864,0,0
MemRowSet-1,tablet-2d40450f6556485eab92f64f90756b61,-1,265,265
DeltaMemStores,tablet-2d40450f6556485eab92f64f90756b61,-1,265,265
The columns are id,parent-id,limit,current_consumption,peak_consumption
just like the table on /mem-trackers.
Change-Id: I3e54f809bb6434b8d8e8c95771fe089c9da122d0
---
M src/kudu/server/generic_service.cc
M src/kudu/server/generic_service.h
M src/kudu/server/server_base.proto
M src/kudu/tools/kudu-admin-test.cc
M src/kudu/tools/kudu-tool-test.cc
M src/kudu/tools/tool_action_common.cc
M src/kudu/tools/tool_action_common.h
M src/kudu/tools/tool_action_diagnose.cc
M src/kudu/tools/tool_action_master.cc
M src/kudu/tools/tool_action_tserver.cc
M src/kudu/util/CMakeLists.txt
M src/kudu/util/mem_tracker-test.cc
M src/kudu/util/mem_tracker.cc
M src/kudu/util/mem_tracker.h
A src/kudu/util/mem_tracker.proto
15 files changed, 412 insertions(+), 10 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/11975/3
--
To view, visit http://gerrit.cloudera.org:8080/11975
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I3e54f809bb6434b8d8e8c95771fe089c9da122d0
Gerrit-Change-Number: 11975
Gerrit-PatchSet: 3
Gerrit-Owner: Will Berkeley <[email protected]>
Gerrit-Reviewer: Adar Dembo <[email protected]>
Gerrit-Reviewer: Alexey Serbin <[email protected]>
Gerrit-Reviewer: Andrew Wong <[email protected]>
Gerrit-Reviewer: Attila Bukor <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Mitch Barnett <[email protected]>
Gerrit-Reviewer: Will Berkeley <[email protected]>
