Hello David Ribeiro Alves, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/4363
to look at the new patch set (#2).
Change subject: WIP: Delete ancient UNDO delta blocks in the background
......................................................................
WIP: Delete ancient UNDO delta blocks in the background
This patch adds a maintenance manager background task that deletes
"ancient" UNDO delta blocks, which means blocks that correspond to data
that is considered no longer reachable and a candidate for garbage
collection. The task only deletes entire blocks and so does not provoke
write amplification.
The task was written to operate at the server level because the
UpdateStats() part of the task spends a budgeted amount of time opening
(initializing) undo delta blocks in order to read the stats from their
headers. Without reading the delta stats, it is impossible to know
whether a given block is a candidate for GC.
This patch includes the following:
* Adds UNDO delta block GC task at TS / Master level
* Adds a bunch of metrics, both at the tablet and TS level
* Adds flags to enable / disable the GC task as well as flags to
throttle it
* Makes several minor improvements in the maintenance manager
TODO: This patch could use more tests, including the following:
* Rowset-level unit test
* Test coverage utilizing the actual MM tasks (tserver, master)
* Possibly incorporate a randomized test?
Change-Id: I0309bf7acfb6d018860c80f354012c3500da5c68
---
M src/kudu/master/catalog_manager.cc
M src/kudu/master/catalog_manager.h
M src/kudu/master/master.cc
M src/kudu/tablet/delta_tracker.cc
M src/kudu/tablet/delta_tracker.h
M src/kudu/tablet/diskrowset.cc
M src/kudu/tablet/diskrowset.h
M src/kudu/tablet/memrowset.h
M src/kudu/tablet/mock-rowsets.h
M src/kudu/tablet/rowset.h
M src/kudu/tablet/rowset_metadata.cc
M src/kudu/tablet/rowset_metadata.h
M src/kudu/tablet/tablet.cc
M src/kudu/tablet/tablet.h
M src/kudu/tablet/tablet_history_gc-test.cc
M src/kudu/tablet/tablet_metrics.cc
M src/kudu/tablet/tablet_metrics.h
M src/kudu/tserver/CMakeLists.txt
M src/kudu/tserver/tablet_peer_lookup.h
M src/kudu/tserver/tablet_server.cc
M src/kudu/tserver/ts_tablet_manager.cc
M src/kudu/tserver/ts_tablet_manager.h
A src/kudu/tserver/tserver_mm_ops.cc
A src/kudu/tserver/tserver_mm_ops.h
M src/kudu/util/maintenance_manager-test.cc
M src/kudu/util/maintenance_manager.cc
M src/kudu/util/maintenance_manager.h
27 files changed, 947 insertions(+), 34 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/63/4363/2
--
To view, visit http://gerrit.cloudera.org:8080/4363
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I0309bf7acfb6d018860c80f354012c3500da5c68
Gerrit-PatchSet: 2
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Mike Percy <[email protected]>
Gerrit-Reviewer: David Ribeiro Alves <[email protected]>
Gerrit-Reviewer: Kudu Jenkins
