Will Berkeley has uploaded this change for review. ( http://gerrit.cloudera.org:8080/11975
Change subject: [tools] Add tool to dump memtrackers ...................................................................... [tools] Add tool to dump memtrackers This adds a new tool, `kudu diagnose dump_mem_trackers`, that dumps the mem-trackers information. It contains information equivalent to the tracker table on /mem-trackers. There's a new RPC introduced to support this tool. Output both as JSON and as a table is supported. The table output type flattens the tracker hierarchy, but it can be reconstructed from the 'id' and 'parent_id' columns. Some Kudu community members have reported to me they have started using the /mem-trackers page to investigate performance and memory issues with Kudu clusters, and having a way to get the same information in a form amenable to programmatic searching, sorting, and filtering is convenient. This tool can also serve as a starting point for more automated analysis of the tracker information. The particular advantage of the memtracker dump over, say, the data generated by heap sampling is that the memtracker shows where memory is owned. The sampling just shows where it was allocated in a call stack. So, for example, using the memtracker information it's possible to tell very quickly exactly which tablets are using a lot of memory in DeltaMemStores. Heap samples would indicate memory allocated in update-related code paths, but it would not be possible to tell which tablet replicas actually held the memory that was allocated. Here's a small sample of output in csv table format: tablet-2e02d00d46834b359c6ba0f8d471cbf9,server,-1,2318,2583 txn_tracker,tablet-2e02d00d46834b359c6ba0f8d471cbf9,67108864,0,0 MemRowSet-1,tablet-2e02d00d46834b359c6ba0f8d471cbf9,-1,265,265 DeltaMemStores,tablet-2e02d00d46834b359c6ba0f8d471cbf9,-1,265,265 tablet-2d40450f6556485eab92f64f90756b61,server,-1,2318,2583 txn_tracker,tablet-2d40450f6556485eab92f64f90756b61,67108864,0,0 MemRowSet-1,tablet-2d40450f6556485eab92f64f90756b61,-1,265,265 DeltaMemStores,tablet-2d40450f6556485eab92f64f90756b61,-1,265,265 The columns are id,parent-id,limit,current_consumption,peak_consumption just like the table on /mem-trackers. Change-Id: I3e54f809bb6434b8d8e8c95771fe089c9da122d0 --- M src/kudu/server/generic_service.cc M src/kudu/server/generic_service.h M src/kudu/server/server_base.proto M src/kudu/tools/kudu-admin-test.cc M src/kudu/tools/tool_action_diagnose.cc M src/kudu/util/CMakeLists.txt M src/kudu/util/mem_tracker-test.cc M src/kudu/util/mem_tracker.cc M src/kudu/util/mem_tracker.h A src/kudu/util/mem_tracker.proto 10 files changed, 322 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/75/11975/1 -- To view, visit http://gerrit.cloudera.org:8080/11975 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I3e54f809bb6434b8d8e8c95771fe089c9da122d0 Gerrit-Change-Number: 11975 Gerrit-PatchSet: 1 Gerrit-Owner: Will Berkeley <[email protected]>
