Hello Tidy Bot, Yingchun Lai, Kudu Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20309
to look at the new patch set (#5).
Change subject: KUDU-3413 [multi-tenancy] Support for tenant storage path
isolation
......................................................................
KUDU-3413 [multi-tenancy] Support for tenant storage path isolation
This patch belongs to the multi-tenancy feature. In this patch,
I have set up different directories for different tenants to
facilitate the development and maintenance of subsequent features.
When the multi-tenancy feature is disabled, the data directory tree
will be as follows:
`
/rnddata00/kudu/
├── tserver_data
│ ├── data
│ │ ├── b83fe60fc9f541d9a09398b77d2a8fdb.data
│ │ ├── b83fe60fc9f541d9a09398b77d2a8fdb.metadata
│ │ ├── block_manager_instance
│ │ ├── c3aad78253af427dad33641e6e092ec8.data
│ │ ├── c3aad78253af427dad33641e6e092ec8.metadata
│ │ ├── d1aa8feaccd9448a846ff9bcbeea0143.data
│ │ ├── d1aa8feaccd9448a846ff9bcbeea0143.metadata
│ │ ├── d39d42f6c25947f5a062a4e4003a33a3.data
│ │ ├── d39d42f6c25947f5a062a4e4003a33a3.metadata
│ │ ├── d549fa825f6641c898f2b5bcf92e43bb.data
│ │ ├── d549fa825f6641c898f2b5bcf92e43bb.metadata
│ │ ├── ed7e1ff546b9435c9cd793de0ef589cf.data
│ │ ├── ed7e1ff546b9435c9cd793de0ef589cf.metadata
│ │ ├── fff0b5cabbe8444db30a7719490d0214.data
│ │ └── fff0b5cabbe8444db30a7719490d0214.metadata
│ └── instance
└── tserver_wal
├── consensus-meta
│ ├── 0e904e919ba94dcf9f64f8ccca155c71
│ ├── 276b446676f14f589c8e8434876e3ba1
│ ├── 45b3416b328741ceb64c73b4ea43aab0
├── instance
├── tablet-meta
│ ├── 0e904e919ba94dcf9f64f8ccca155c71
│ ├── 276b446676f14f589c8e8434876e3ba1
│ ├── 45b3416b328741ceb64c73b4ea43aab0
└── wals
├── 0e904e919ba94dcf9f64f8ccca155c71
│ ├── index.000000000
│ └── wal-000000001
├── 276b446676f14f589c8e8434876e3ba1
│ ├── index.000000000
│ └── wal-000000001
└── 45b3416b328741ceb64c73b4ea43aab0
├── index.000000000
└── wal-000000001
`
When the multi-tenancy feature is enabled, the data directory tree
will change to the following form:
`
/rnddata00/kudu/
├── tserver_data
│ ├── data
│ │ ├── b83fe60fc9f541d9a09398b77d2a8fdb.data
│ │ ├── b83fe60fc9f541d9a09398b77d2a8fdb.metadata
│ │ └── block_manager_instance
│ ├── instance
│ └── tenants
│ ├── 00000000000000000000000000000001
│ │ ├── data
│ │ │ ├── c3aad78253af427dad33641e6e092ec8.data
│ │ │ ├── c3aad78253af427dad33641e6e092ec8.metadata
│ │ │ └── block_manager_instance
│ │ └── instance
│ └── 00000000000000000000000000000002
│ ├── data
│ │ ├── fff0b5cabbe8444db30a7719490d0214.data
│ │ ├── fff0b5cabbe8444db30a7719490d0214.metadata
│ │ └── block_manager_instance
│ └── instance
└── tserver_wal
├── consensus-meta
│ └── 0e904e919ba94dcf9f64f8ccca155c71
├── instance
├── tablet-meta
│ └── 0e904e919ba94dcf9f64f8ccca155c71
├── wals
│ └── 0e904e919ba94dcf9f64f8ccca155c71
│ ├── index.000000000
│ └── wal-000000001
└── tenants
├── 00000000000000000000000000000001
│ ├── consensus-meta
│ │ └── 276b446676f14f589c8e8434876e3ba1
│ ├── instance
│ ├── tablet-meta
│ │ └── 276b446676f14f589c8e8434876e3ba1
│ └── wals
│ └── 276b446676f14f589c8e8434876e3ba1
│ ├── index.000000000
│ └── wal-000000001
└── 00000000000000000000000000000002
├── consensus-meta
│ └── 45b3416b328741ceb64c73b4ea43aab0
├── instance
├── tablet-meta
│ └── 45b3416b328741ceb64c73b4ea43aab0
└── wals
└── 45b3416b328741ceb64c73b4ea43aab0
├── index.000000000
└── wal-000000001
`
As it involves tenant directory management, I have added some logic
while ensuring that existing logic is not affected, and have also
added some tests to ensure that this logic is effective.
Change-Id: Icb3f3d47e00f93db6f1ce66596de6449fb1dbb73
---
M src/kudu/fs/fs_manager-test.cc
M src/kudu/fs/fs_manager.cc
M src/kudu/fs/fs_manager.h
M src/kudu/fs/log_block_manager.cc
M src/kudu/tablet/delta_compaction.cc
M src/kudu/tools/tool_action_fs.cc
6 files changed, 792 insertions(+), 275 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/kudu refs/changes/09/20309/5
--
To view, visit http://gerrit.cloudera.org:8080/20309
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: kudu
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Icb3f3d47e00f93db6f1ce66596de6449fb1dbb73
Gerrit-Change-Number: 20309
Gerrit-PatchSet: 5
Gerrit-Owner: KeDeng <[email protected]>
Gerrit-Reviewer: Kudu Jenkins (120)
Gerrit-Reviewer: Tidy Bot (241)
Gerrit-Reviewer: Yingchun Lai <[email protected]>
