Wang Xixu has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19327
Change subject: [Tool] Compact metadata file for old version of Kudu ...................................................................... [Tool] Compact metadata file for old version of Kudu Som old versions of Kudu do not support compacting metadata file in runtime. A log time later, metadata file becomes very big, for example, more than 100G. When restarting Kudu tablet server, it will read all metadata files to check whick blocks is still alive and compact metadata files into new small metadata files which only record alive blocks. As the metadata files are very big, compacting metadata files will cost a lot of time, maybe 2-3 hours. That will cause the service unavailabe for a long time, which is unacceptable. Runtime metadata file compacting has already been supported above version of 1.16.0, see jira: #KUDU-3318. Upgrading old version to above Kudu 1.16.0 still needs to restart Kudu. This tool helps to compact metdata file for old versions of Kudu. It supports 4 operations: --print: Statistic the total blocks and alive blocs for every container. --compact: Compact metadata files when Kudu is still running --merge: Stop Kudu and write incremental records into compacted metdata files. Then backup old metadata file. --clear: Delete metadata file backup file. Restarting old version of Kudu maybe cost 2-3 hours, but using compact tool, service is stopped only last 3 minutes. Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4 --- M src/kudu/fs/log_block_manager-test.cc M src/kudu/fs/log_block_manager.cc M src/kudu/fs/log_block_manager.h M src/kudu/tools/tool_action_fs.cc M src/kudu/util/pb_util.cc M src/kudu/util/pb_util.h 6 files changed, 601 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/27/19327/1 -- To view, visit http://gerrit.cloudera.org:8080/19327 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id61cb6c88d7a093a8448e2497639f5c5c094efa4 Gerrit-Change-Number: 19327 Gerrit-PatchSet: 1 Gerrit-Owner: Wang Xixu <[email protected]>
