Yifan Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18152
Change subject: KUDU-3344: catalog manager clean up metadata for deleted tables/tablets ...................................................................... KUDU-3344: catalog manager clean up metadata for deleted tables/tablets Kudu masters now retain metadata for deleted tables and tablets forever, and the leader master loads all of them into memory when starts. If we have a lot of tables and tablets in a cluster, memory usage of the master will be large and master could also take a long time to start. Consider that in many cases we drop tables and partitions, we could clean up useless data in a backgroud task. But it's hard to decide when we should clean them up, because the deletion of tablet replicas is asynchronous, if the metadata has been deleted before the tablet data was successfully deleted, the unknown tablet reported by tablet server would not be processed by catalog manager and we must delete it manually. This patch add a new flag 'deleted_table_and_tablet_reserved_secs', its default value is the same as 'unresponsive_ts_rpc_timeout_ms', we could roughly assume that after this amount of time the tablet data will be actually deleted so we could delete the metadata. Change-Id: Idefa2ee2f5108ba913fe0057a4061c3c28351547 --- M src/kudu/client/client.cc M src/kudu/integration-tests/master-stress-test.cc M src/kudu/integration-tests/ts_tablet_manager-itest.cc M src/kudu/master/auto_rebalancer.cc M src/kudu/master/catalog_manager.cc M src/kudu/master/catalog_manager.h M src/kudu/master/master-test.cc M src/kudu/master/master.proto 8 files changed, 261 insertions(+), 24 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/52/18152/1 -- To view, visit http://gerrit.cloudera.org:8080/18152 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Idefa2ee2f5108ba913fe0057a4061c3c28351547 Gerrit-Change-Number: 18152 Gerrit-PatchSet: 1 Gerrit-Owner: Yifan Zhang <[email protected]>
