Ivan Andika created HDDS-14244:
----------------------------------
Summary: Support OM metadata-only bucket migration
Key: HDDS-14244
URL: https://issues.apache.org/jira/browse/HDDS-14244
Project: Apache Ozone
Issue Type: New Feature
Reporter: Ivan Andika
One of our cluster contains multiple OM services which point to a single HDDS
layer (e.g. SCM and DN). However, currently when we want to migrate bucket
between these OM services (same bucket name), we are using a simple sync tool
that simply download the key from the source OM service to the client and the
upload it to the new OM service although technically only OM DB keyTable
metadata and bucketTable metadata needs to be migrated since the block
locations remain the same.
We can support migrating only the OM metadata without any actual block data
transfer.
There are a few approach
# We can ingest the only the SST files from the old OM DB to the new OM DB.
This should be the most performant solution. However, I don't think SST files
between keys in different buckets are isolated, so we might need to so
something about it.
# Introduce a new API for metadata only migration. This might cause a lot of
network overheads.
This might also contribute to the general OM sharding solution in the future.
We can also take a look at other distributed key value store solutions for
ideas (e.g. TiKV, CockroachDB, Yugabyte, FoundationDB, etc).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
