Andrew Wong has uploaded a new change for review. http://gerrit.cloudera.org:8080/7617
Change subject: WIP KUDU-1489: move tablet metadata ...................................................................... WIP KUDU-1489: move tablet metadata WIP: a few areas still need cleaning (e.g. data_dirs.cc) Some terminology: - Root: a top-level directory specified by the user via fs_data_dirs. They are canonicalized by the dir manager. - Dir: a suffix used by subdirectories of the roots. Generally should not be used alone, but rather as a suffix (e.g. DataDir). - DataDir: a */data subdirectory of a root - each DataDir has a path instance file - TabletMetadataDir: a */tablet-metadata subdirectory of a root - ConsensusMetadataDir: a */consensus-metadata subdirectory of a root Changes to the tablet lifecycle with striping: - During DataDirManager::Open(), scan through all the directories and map all the tablets' current (c)metadata directories - Additionally, during calls to ListTabletIds(), scan through all directories and update the tablets' current directories - If there are duplicates (not an expected case, but will likely happen), try to delete those that aren't in the disk group - When writing the metas (TabletMetadata::ReplaceSuperBlockUnlocked()), check to see whether the metadata dir is appropriate, if not, write it to a different dir. The crux of this patch is aimed at moving metadata when initializing the TSTabletManager. Rather than always getting metadata from the first data directory, all known directories (both WAL and data dirs) are scanned for metadata instances. Any metadata found in a directory it shouldn't be in is moved to one appropriate for it. An appropriate directory for a tablet's metadata is: 1. any directory in the tablet's directory group, or 2. the WAL dir (if tablet metadata striping is disabled) If striping is enabled, the metadata directory for each tablet will be decided when the tablet is created (or copied). Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a --- M src/kudu/consensus/consensus_meta-test.cc M src/kudu/consensus/consensus_meta.cc M src/kudu/fs/data_dirs.cc M src/kudu/fs/data_dirs.h M src/kudu/fs/fs_manager-test.cc M src/kudu/fs/fs_manager.cc M src/kudu/fs/fs_manager.h M src/kudu/integration-tests/external_mini_cluster_fs_inspector.cc M src/kudu/tablet/tablet_metadata.cc M src/kudu/tablet/tablet_metadata.h M src/kudu/tserver/tablet_copy_client.cc M src/kudu/tserver/ts_tablet_manager.cc M src/kudu/tserver/ts_tablet_manager.h M src/kudu/util/path_util.h 14 files changed, 557 insertions(+), 136 deletions(-) git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/17/7617/1 -- To view, visit http://gerrit.cloudera.org:8080/7617 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a Gerrit-PatchSet: 1 Gerrit-Project: kudu Gerrit-Branch: master Gerrit-Owner: Andrew Wong <aw...@cloudera.com>
