[
https://issues.apache.org/jira/browse/HDDS-9245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
GuoHao updated HDDS-9245:
-------------------------
Description:
Upgrade the tool:
0. Check
0.1 Datanode service is shut down, check process name
0.2 Check the layout version of DN, whether Schema V3 is supported.
0.3 Check if volume rocksdb already exists, if yes, prompt the user whether
to use this rocksdb directly as the storage of metadata after migrate, and
prompt the user that the current rocksdb will be automatically backed up (print
the backup file name and path), if no, end and exit.
0.4 If volume rocksdb does not exist, prompt the user to perform finalizae DN
operation.
0.5 Check if there is a migrate completed file, if so, prompt the user that
the volume has been migrated before, end exit.
0.6 Check whether the DN state is in the containcen state (DN state in the
VERSION file), if not, prompt the user to set the DN to the maintenance state
on the SCM side, end and exit.
1. Migrate data
Support two ways, one to migrate one volume at a time, and one to migrate all
the volumes (multi-threaded, one thread for each volume).
Generate a LOCK file under each volume to be migrated this time, and exit the
end command if the creation fails.
Iterate through each container under a volume (containers with all states)
1.1 If Schema V1, skip.
1.2 If it is Schema V2
a. Open container rocksdb and read all the key values.
b. Convert the key format to schema v3 format.
c. Write the new key and value to the volume rocksdb.
d. Backup the container yaml file
e. Modify the container yaml file with v3 in the version field, modify the
metadata path, and recalculate the file checksume.
If the yaml file is missing, skip the container.
If checksdb is missing, modify the container yaml file directly.
Disk full, disk IO error, the end of this thread (if a single disk operation,
the entire tool ends) - Documentation: We recommend that the user first solve
the disk problem (disk full), prompted to delete the LOCK file under the
volume, for the volume, re-run the command again
2. Choose a container ID at random, get the metadata of V2 and V3 respectively,
and compare whether the metadata is consistent. If the metadata is consistent,
the command ends successfully. If not, an error occurs, exit, and keep the site.
3. Delete the LOCK files under all designed volumes.
4. Generate an identifier file that identifies the completion of the migrate
under the current volume current, and record the timestamp.
> Add container layout v2 upgrade to v3 tool
> ------------------------------------------
>
> Key: HDDS-9245
> URL: https://issues.apache.org/jira/browse/HDDS-9245
> Project: Apache Ozone
> Issue Type: New Feature
> Reporter: GuoHao
> Assignee: GuoHao
> Priority: Major
>
> Upgrade the tool:
> 0. Check
> 0.1 Datanode service is shut down, check process name
> 0.2 Check the layout version of DN, whether Schema V3 is supported.
> 0.3 Check if volume rocksdb already exists, if yes, prompt the user whether
> to use this rocksdb directly as the storage of metadata after migrate, and
> prompt the user that the current rocksdb will be automatically backed up
> (print the backup file name and path), if no, end and exit.
> 0.4 If volume rocksdb does not exist, prompt the user to perform finalizae
> DN operation.
> 0.5 Check if there is a migrate completed file, if so, prompt the user that
> the volume has been migrated before, end exit.
> 0.6 Check whether the DN state is in the containcen state (DN state in the
> VERSION file), if not, prompt the user to set the DN to the maintenance state
> on the SCM side, end and exit.
> 1. Migrate data
> Support two ways, one to migrate one volume at a time, and one to migrate all
> the volumes (multi-threaded, one thread for each volume).
> Generate a LOCK file under each volume to be migrated this time, and exit the
> end command if the creation fails.
> Iterate through each container under a volume (containers with all states)
> 1.1 If Schema V1, skip.
> 1.2 If it is Schema V2
> a. Open container rocksdb and read all the key values.
> b. Convert the key format to schema v3 format.
> c. Write the new key and value to the volume rocksdb.
> d. Backup the container yaml file
> e. Modify the container yaml file with v3 in the version field, modify
> the metadata path, and recalculate the file checksume.
> If the yaml file is missing, skip the container.
> If checksdb is missing, modify the container yaml file directly.
> Disk full, disk IO error, the end of this thread (if a single disk operation,
> the entire tool ends) - Documentation: We recommend that the user first solve
> the disk problem (disk full), prompted to delete the LOCK file under the
> volume, for the volume, re-run the command again
> 2. Choose a container ID at random, get the metadata of V2 and V3
> respectively, and compare whether the metadata is consistent. If the metadata
> is consistent, the command ends successfully. If not, an error occurs, exit,
> and keep the site.
> 3. Delete the LOCK files under all designed volumes.
> 4. Generate an identifier file that identifies the completion of the migrate
> under the current volume current, and record the timestamp.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
