Andrew Wong has uploaded a new change for review.

Change subject: WIP KUDU-1489: move tablet metadata

WIP KUDU-1489: move tablet metadata

WIP: a few areas still need cleaning (e.g.

Some terminology:
- Root: a top-level directory specified by the user via fs_data_dirs.
  They are canonicalized by the dir manager.
- Dir: a suffix used by subdirectories of the roots. Generally should
  not be used alone, but rather as a suffix (e.g. DataDir).
- DataDir: a */data subdirectory of a root
  - each DataDir has a path instance file
- TabletMetadataDir: a */tablet-metadata subdirectory of a root
- ConsensusMetadataDir: a */consensus-metadata subdirectory of a root

Changes to the tablet lifecycle with striping:
- During DataDirManager::Open(), scan through all the directories and
  map all the tablets' current (c)metadata directories
- Additionally, during calls to ListTabletIds(), scan through all
  directories and update the tablets' current directories
- If there are duplicates (not an expected case, but will likely
  happen), try to delete those that aren't in the disk group
- When writing the metas (TabletMetadata::ReplaceSuperBlockUnlocked()),
  check to see whether the metadata dir is appropriate, if not, write it
  to a different dir.

The crux of this patch is aimed at moving metadata when initializing the
TSTabletManager. Rather than always getting metadata from the first data
directory, all known directories (both WAL and data dirs) are scanned
for metadata instances. Any metadata found in a directory it shouldn't be
in is moved to one appropriate for it.

An appropriate directory for a tablet's metadata is:
1. any directory in the tablet's directory group, or
2. the WAL dir (if tablet metadata striping is disabled)

If striping is enabled, the metadata directory for each tablet will be
decided when the tablet is created (or copied).

Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a
M src/kudu/consensus/
M src/kudu/consensus/
M src/kudu/fs/
M src/kudu/fs/data_dirs.h
M src/kudu/fs/
M src/kudu/fs/
M src/kudu/fs/fs_manager.h
M src/kudu/integration-tests/
M src/kudu/tablet/
M src/kudu/tablet/tablet_metadata.h
M src/kudu/tserver/
M src/kudu/tserver/
M src/kudu/tserver/ts_tablet_manager.h
M src/kudu/util/path_util.h
14 files changed, 557 insertions(+), 136 deletions(-)

  git pull ssh:// refs/changes/17/7617/1
To view, visit
To unsubscribe, visit

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia5308f8d4b4b738f353f511e2f1a2634b675a60a
Gerrit-PatchSet: 1
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-Owner: Andrew Wong <>

Reply via email to