errose28 opened a new pull request, #3465:
URL: https://github.com/apache/ozone/pull/3465
## What changes were proposed in this pull request?
**Please read the design document attached to HDDS-5141 for context when
reviewing**
Add the foundation for finalizing an SCM HA cluster. Non-HA SCM finalization
will also use this code from now on. Other remaining items under the parent
Jira HDDS-5141 which are out of scope for this task are:
- HDDS-6762
- Async finalization from client perspective (return to the client after
the finalizing mark has been written to the DB).
- Ability for the client to query the status of ongoing SCM finalization.
- HDDS-6761
- Resume finalization on the new SCM leader on leader change, and cease
the finalization thread on the old leader.
- Allow followers to finalize from a snapshot.
- This is the purpose of storing the layout version in the DB as
well as the version file.
Portions of the common upgrade framework were refactored under this PR as
well to make it more flexible for supporting SCM HA finalization. For example,
finalization in the `BasicUpgradeFinalizer` was split from one method
finalizing all layout features to one method invoked to finalize each layout
feature.
Some more invasive changes are marked as TODO to reduce the size of this PR:
- Refactor the old `SCMNodeManager` constructor which has been left due
to its high usage in tests.
- Refactor `TestSCMNodeManager` to use junit jupiter for test specific
parameterization.
## What is the link to the Apache JIRA
HDDS-6760
## How was this patch tested?
- Existing unit tests modified.
- New unit tests added.
- `TestHDDSUpgrade#testFinalizationFromInitialVersionToLatestVersion` was
run with 3 SCMs and passed consistently, although this test is excluded from
the regular CI workflow.
- Existing upgrade acceptance tests passed. These tests currently do not use
SCM HA but are still running the code in this PR.
## Outstanding Issues
- This PR is being left as a draft until the following issues are resolved:
- Failure in `TestDatanodeUpgradeToScmHA`.
- Failure in `TestScmFinalization#testResumeFinalizationFromCheckpoint`
- See the `TODO` comment in `BasicUpgradeFinalizer#finalize` for
what needs to be done to fix this test.
- `ScmUpgradeFinalizer` is not receiving injected
`UpgradeFinalizationExecutor`s after refactoring.
- This is an easy fix that will be corrected soon.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
