Yingchun Lai has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19670 )
Change subject: [tablet] GC ancient, fully deleted rowsets without live row count stats ...................................................................... [tablet] GC ancient, fully deleted rowsets without live row count stats We added a background op to GC ancient, fully deleted rowsets for KUDU-1625 base on live row count. That patch is very useful, but does not work for older versions(earlier than 1.10) that do not support live row count stats. And during the upgrade process from a lower version to a higher version, live row count feature cannot be enabled for already existing data. To resolve this issue on a lower version of the kudu cluster, I submitted this patch. The main reason is to replace the use of live row count. However, due to the lack of a more accurate counting method, this patch may only release part of the storage space for ancient, fully deleted rows. Therefore, this feature can alleviate the storage space tension of older versions to a certain extent. If you need to enable this feature, enable the flag --enable_gc_deleted_rowsets_without_live_row_count and restart tservers. There's still room for improvement in this implementation in that, currently, we ignored the delete operation in DMS. I will resolve this issue in a follow-up patch. I ran this on a real cluster, the storage space of deleted rowsets that was not previously freed can be GCed as expected. And I also add unit test case to ensure it make sense. Change-Id: Iacdff107b8b07cbd56f47f296a93f4bcfbf56b41 Reviewed-on: http://gerrit.cloudera.org:8080/19670 Tested-by: Kudu Jenkins Reviewed-by: Yingchun Lai <[email protected]> Reviewed-by: Yuqi Du <[email protected]> --- M src/kudu/tablet/delta_tracker.cc M src/kudu/tablet/delta_tracker.h M src/kudu/tablet/diskrowset-test-base.h M src/kudu/tablet/diskrowset-test.cc M src/kudu/tablet/diskrowset.cc M src/kudu/tablet/diskrowset.h M src/kudu/tablet/tablet.cc M src/kudu/tablet/tablet_history_gc-test.cc 8 files changed, 167 insertions(+), 12 deletions(-) Approvals: Kudu Jenkins: Verified Yingchun Lai: Looks good to me, approved Yuqi Du: Looks good to me, but someone else must approve -- To view, visit http://gerrit.cloudera.org:8080/19670 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iacdff107b8b07cbd56f47f296a93f4bcfbf56b41 Gerrit-Change-Number: 19670 Gerrit-PatchSet: 20 Gerrit-Owner: KeDeng <[email protected]> Gerrit-Reviewer: Abhishek Chennaka <[email protected]> Gerrit-Reviewer: Alexey Serbin <[email protected]> Gerrit-Reviewer: KeDeng <[email protected]> Gerrit-Reviewer: Kudu Jenkins (120) Gerrit-Reviewer: Tidy Bot (241) Gerrit-Reviewer: Yingchun Lai <[email protected]> Gerrit-Reviewer: Yuqi Du <[email protected]>
