fapifta commented on code in PR #4456:
URL: https://github.com/apache/ozone/pull/4456#discussion_r1147034102
##########
hadoop-ozone/dist/src/main/compose/upgrade/README.md:
##########
@@ -18,39 +18,60 @@ This directory contains cluster definitions and scripts for
testing upgrades fro
previous version, or to the local build of the code. It is designed to catch
backwards incompatible changes made between
an older release of Ozone and a later release (which may be the local build).
-## IMPORTANT NOTES
+## Quick Guide For Release Managers
-1. Backwards Incompatibility
- - These tests will not catch backwards incompatible changes against
commits in between releases.
- - Example:
- 1. After 1.0.0, a change *c1* is made that is backwards compatible
with 1.0.0.
- 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+- The whole test matrix of upgrading and downgrading to/from previous releases
to the current code is too time consuming to do on every CI run. Instead we
recommend release managers manually test the full matrix before each release,
and let only the tests from the previous release to the current code be run in
CI.
- - This test suite will not raise an error for *c2*, because it
only tests against the last release
- (1.0.0), and not the last commit (*c1*).
+1. Before the release, test the whole matrix of upgrades from the previous
version to the version you are releasing.
+ - This is important manual verification that the release does not break
backwards compatibility, and the results can be included in the release vote
mailing thread.
Review Comment:
```suggestion
- This is an important manual verification that the release does not
break backwards compatibility, and the results can be included in the release
vote mailing thread.
```
##########
hadoop-ozone/dist/src/main/compose/upgrade/README.md:
##########
@@ -18,39 +18,60 @@ This directory contains cluster definitions and scripts for
testing upgrades fro
previous version, or to the local build of the code. It is designed to catch
backwards incompatible changes made between
an older release of Ozone and a later release (which may be the local build).
-## IMPORTANT NOTES
+## Quick Guide For Release Managers
-1. Backwards Incompatibility
- - These tests will not catch backwards incompatible changes against
commits in between releases.
- - Example:
- 1. After 1.0.0, a change *c1* is made that is backwards compatible
with 1.0.0.
- 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+- The whole test matrix of upgrading and downgrading to/from previous releases
to the current code is too time consuming to do on every CI run. Instead we
recommend release managers manually test the full matrix before each release,
and let only the tests from the previous release to the current code be run in
CI.
- - This test suite will not raise an error for *c2*, because it
only tests against the last release
- (1.0.0), and not the last commit (*c1*).
+1. Before the release, test the whole matrix of upgrades from the previous
version to the version you are releasing.
+ - This is important manual verification that the release does not break
backwards compatibility, and the results can be included in the release vote
mailing thread.
+ - To do this, uncomment all lines that contain `run_test` in the *test.sh*
file, and execute *test.sh* either locally or on GitHub actions.
+
+2. After the release is finished and its docker image is published, add the
new version to the test matrix.
+ 1. Change the `OZONE_CURRENT_VERSION` variable to
`OZONE_CURRENT_VERSION=<newly-released-version>`.
+ 2. Comment out all `run_test` lines in *test.sh*.
+ 3. Add a new line: `run_test ha non-rolling-upgrade
<newly-released-version> "$OZONE_CURRENT_VERSION"` before the commented out
lines.
+
+## Important Notes on Test Scope
+
+- These tests will not catch backwards incompatible changes against commits in
between releases.
+ - Example:
+ 1. After 1.2.0, a change *c1* is made that is backwards compatible
with 1.2.0.
+ 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
Review Comment:
```suggestion
2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.2.0 but backwards *incompatible* with *c1*.
```
##########
hadoop-ozone/dist/src/main/compose/upgrade/compose/ha/load.sh:
##########
Review Comment:
I am wondering if we should have a load.sh at all. All three folders are
having this file, and it is almost identical, there are only two differences,
the compose file that is being exported, and the data directories that are
created.
I think it does not make any harm to create all the data directories that
might be used, and the compose file is something we can determine the location
of in the testlib.sh's run_test method where these files are being sourced.
So what if we move this logic out from the load.sh files to the run_test
method or to a separate method in the testlib.sh? It would also be visible that
it is unnecessary to source the testlib.sh, and we don't need the TEST_DIR env
var check.
##########
hadoop-ozone/dist/src/main/compose/upgrade/README.md:
##########
@@ -18,39 +18,60 @@ This directory contains cluster definitions and scripts for
testing upgrades fro
previous version, or to the local build of the code. It is designed to catch
backwards incompatible changes made between
an older release of Ozone and a later release (which may be the local build).
-## IMPORTANT NOTES
+## Quick Guide For Release Managers
-1. Backwards Incompatibility
- - These tests will not catch backwards incompatible changes against
commits in between releases.
- - Example:
- 1. After 1.0.0, a change *c1* is made that is backwards compatible
with 1.0.0.
- 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+- The whole test matrix of upgrading and downgrading to/from previous releases
to the current code is too time consuming to do on every CI run. Instead we
recommend release managers manually test the full matrix before each release,
and let only the tests from the previous release to the current code be run in
CI.
- - This test suite will not raise an error for *c2*, because it
only tests against the last release
- (1.0.0), and not the last commit (*c1*).
+1. Before the release, test the whole matrix of upgrades from the previous
version to the version you are releasing.
+ - This is important manual verification that the release does not break
backwards compatibility, and the results can be included in the release vote
mailing thread.
+ - To do this, uncomment all lines that contain `run_test` in the *test.sh*
file, and execute *test.sh* either locally or on GitHub actions.
+
+2. After the release is finished and its docker image is published, add the
new version to the test matrix.
+ 1. Change the `OZONE_CURRENT_VERSION` variable to
`OZONE_CURRENT_VERSION=<newly-released-version>`.
+ 2. Comment out all `run_test` lines in *test.sh*.
+ 3. Add a new line: `run_test ha non-rolling-upgrade
<newly-released-version> "$OZONE_CURRENT_VERSION"` before the commented out
lines.
+
+## Important Notes on Test Scope
+
+- These tests will not catch backwards incompatible changes against commits in
between releases.
+ - Example:
+ 1. After 1.2.0, a change *c1* is made that is backwards compatible
with 1.2.0.
+ 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+
+ - This test suite will not raise an error for *c2*, because it only
tests against the last release
+ (1.2.0), and not the last commit (*c1*).
+
+## Supported Versions
+
+Non-rolling upgrades and downgrades are supported from 1.1.0 to any later
version. Note that 1.1.0 did not have the non-rolling upgrade framework, so
things like preparing the OMs for upgrade and checking finalization status are
not present in that version. Manual upgrade is the only supported upgrade
option from 1.0.0 to 1.1.0.
Review Comment:
```suggestion
Non-rolling upgrades and downgrades are supported from 1.1.0 to any later
version. Note that 1.0.0 did not have the non-rolling upgrade framework, so
things like preparing the OMs for upgrade and checking finalization status are
not present in that version. Manual upgrade is the only supported upgrade
option from 1.0.0 to 1.1.0.
```
##########
hadoop-ozone/dist/src/main/compose/upgrade/README.md:
##########
@@ -18,39 +18,60 @@ This directory contains cluster definitions and scripts for
testing upgrades fro
previous version, or to the local build of the code. It is designed to catch
backwards incompatible changes made between
an older release of Ozone and a later release (which may be the local build).
-## IMPORTANT NOTES
+## Quick Guide For Release Managers
-1. Backwards Incompatibility
- - These tests will not catch backwards incompatible changes against
commits in between releases.
- - Example:
- 1. After 1.0.0, a change *c1* is made that is backwards compatible
with 1.0.0.
- 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+- The whole test matrix of upgrading and downgrading to/from previous releases
to the current code is too time consuming to do on every CI run. Instead we
recommend release managers manually test the full matrix before each release,
and let only the tests from the previous release to the current code be run in
CI.
- - This test suite will not raise an error for *c2*, because it
only tests against the last release
- (1.0.0), and not the last commit (*c1*).
+1. Before the release, test the whole matrix of upgrades from the previous
version to the version you are releasing.
+ - This is important manual verification that the release does not break
backwards compatibility, and the results can be included in the release vote
mailing thread.
+ - To do this, uncomment all lines that contain `run_test` in the *test.sh*
file, and execute *test.sh* either locally or on GitHub actions.
+
+2. After the release is finished and its docker image is published, add the
new version to the test matrix.
+ 1. Change the `OZONE_CURRENT_VERSION` variable to
`OZONE_CURRENT_VERSION=<newly-released-version>`.
+ 2. Comment out all `run_test` lines in *test.sh*.
+ 3. Add a new line: `run_test ha non-rolling-upgrade
<newly-released-version> "$OZONE_CURRENT_VERSION"` before the commented out
lines.
+
+## Important Notes on Test Scope
+
+- These tests will not catch backwards incompatible changes against commits in
between releases.
+ - Example:
+ 1. After 1.2.0, a change *c1* is made that is backwards compatible
with 1.2.0.
+ 2. After *c1*, a new change *c2* is made that is also backwards
compatible with 1.0.0 but backwards *incompatible* with *c1*.
+
+ - This test suite will not raise an error for *c2*, because it only
tests against the last release
+ (1.2.0), and not the last commit (*c1*).
+
+## Supported Versions
+
+Non-rolling upgrades and downgrades are supported from 1.1.0 to any later
version. Note that 1.1.0 did not have the non-rolling upgrade framework, so
things like preparing the OMs for upgrade and checking finalization status are
not present in that version. Manual upgrade is the only supported upgrade
option from 1.0.0 to 1.1.0.
## Directory Layout
### upgrades
-Each type of upgrade has a subdirectory under the *upgrades* directory. Each
upgrade's steps are controlled by a *driver.sh* script in its
*upgrades/\<upgrade-type>* directory. Callbacks to execute throughout the
upgrade are called by this script and should be placed in a file called
*callback.sh* in the *upgrades/\<upgrade-type>/\<upgrade-from>-\<upgrade-to>*
directory. After the test is run, results and docker volume data for the
upgrade for these versions will also be placed in this directory. The results
of all upgrades run as part of the tests will be placed in a *results* folder
in the top level upgrade directory.
+Each type of upgrade has a subdirectory under the *upgrades* directory.
+
+- Each upgrade's steps are controlled by a *driver.sh* script in its
*upgrades/\<upgrade-type>* directory. Callbacks to execute throughout the
upgrade are called by this script and should be placed in a file called
*callback.sh* in the *upgrades/\<upgrade-type>/\<upgrade-to>* directory.
+
+- As the test is run, result logs and docker volume data for the upgrade for
these versions will be placed in
*upgrades/\<upgrade-type>/execution/\<upgrade-from>-\<upgrade-to>*. This allows
a suite of upgrades to be run without conflicting directory names.
Review Comment:
```suggestion
- As the test is running, result logs and docker volume data for the upgrade
for these versions will be placed in
*upgrades/\<upgrade-type>/execution/\<upgrade-from>-\<upgrade-to>*. This allows
a suite of upgrades to be run without conflicting directory names.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
