Todd Lipcon has submitted this change and it was merged. Change subject: KUDU-236 (part 1). Implement tablet history GC ......................................................................
KUDU-236 (part 1). Implement tablet history GC This patch defines the concept of an "ancient history mark" (AHM) and a background job that removes history from disk that represents changes occurring prior to the ancient history mark. There is also a mechanism to reject requests for scans if they attempt to open a snapshot scan at a timestamp prior to the AHM. Included tests: * Test for Redo GC via major delta compaction * Test for history GC via tablet flush * Test for Undo GC via merge compaction * Test for entire-row GC via merge compaction * Test for ghost rows in multiple RS reinsert case * Test for reupdating missed deltas * Test for partial rowset history GC using alternating rows * Test for major delta compaction against a subset of columns * Test for reinsert after delete that is prior to AHM * Test for opening a scanner at TS < AHM and rejecting that scanner at the RPC level Tests coming in a follow-up commit: * Randomized test Missing features coming in follow-up commits: * Background task that schedules and performs undo GC * GC history on delta flush Change-Id: If9833a863f118eb82be80ea56204d0d9141611c2 Reviewed-on: http://gerrit.cloudera.org:8080/3076 Tested-by: Kudu Jenkins Reviewed-by: Todd Lipcon <t...@apache.org> --- M src/kudu/common/timestamp.h M src/kudu/integration-tests/CMakeLists.txt A src/kudu/integration-tests/tablet_history_gc-itest.cc M src/kudu/server/clock.h M src/kudu/server/hybrid_clock.cc M src/kudu/server/hybrid_clock.h M src/kudu/tablet/CMakeLists.txt M src/kudu/tablet/compaction-test.cc M src/kudu/tablet/compaction.cc M src/kudu/tablet/compaction.h M src/kudu/tablet/delta_compaction.cc M src/kudu/tablet/delta_compaction.h M src/kudu/tablet/diskrowset.cc M src/kudu/tablet/diskrowset.h M src/kudu/tablet/mutation.h M src/kudu/tablet/rowset_metadata.h M src/kudu/tablet/tablet-harness.h M src/kudu/tablet/tablet-test-base.h M src/kudu/tablet/tablet-test-util.h M src/kudu/tablet/tablet-test.cc M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet.h A src/kudu/tablet/tablet_history_gc-test.cc M src/kudu/tserver/tablet_service.cc 24 files changed, 1,049 insertions(+), 170 deletions(-) Approvals: Todd Lipcon: Looks good to me, approved Kudu Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/3076 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: If9833a863f118eb82be80ea56204d0d9141611c2 Gerrit-PatchSet: 23 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Mike Percy <mpe...@apache.org> Gerrit-Reviewer: Adar Dembo <a...@cloudera.com> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Mike Percy <mpe...@apache.org> Gerrit-Reviewer: Todd Lipcon <t...@apache.org>
