This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ozone.git
The following commit(s) were added to refs/heads/master by this push:
new 69206e9681 HDDS-11902. Remove upgrade tests for non-ha and om-ha
(#7610)
69206e9681 is described below
commit 69206e9681cb4df07f886395ce8ec1d5e5ba68bf
Author: Chia-Chuan Yu <[email protected]>
AuthorDate: Fri Jan 3 18:24:14 2025 +0800
HDDS-11902. Remove upgrade tests for non-ha and om-ha (#7610)
---
.../dist/src/main/compose/upgrade/README.md | 18 +--
.../src/main/compose/upgrade/compose/non-ha/.env | 22 ---
.../upgrade/compose/non-ha/docker-compose.yaml | 155 ------------------
.../compose/upgrade/compose/non-ha/docker-config | 49 ------
.../main/compose/upgrade/compose/non-ha/load.sh | 29 ----
.../src/main/compose/upgrade/compose/om-ha/.env | 27 ----
.../upgrade/compose/om-ha/docker-compose.yaml | 178 ---------------------
.../compose/upgrade/compose/om-ha/docker-config | 65 --------
.../src/main/compose/upgrade/compose/om-ha/load.sh | 30 ----
.../upgrade/upgrades/manual-upgrade/README.md | 19 ---
.../upgrade/upgrades/manual-upgrade/driver.sh | 54 -------
11 files changed, 2 insertions(+), 644 deletions(-)
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/README.md
b/hadoop-ozone/dist/src/main/compose/upgrade/README.md
index 49c453800e..345b91d7bd 100644
--- a/hadoop-ozone/dist/src/main/compose/upgrade/README.md
+++ b/hadoop-ozone/dist/src/main/compose/upgrade/README.md
@@ -43,7 +43,7 @@ an older release of Ozone and a later release (which may be
the local build).
## Supported Versions
-Non-rolling upgrades and downgrades are supported from 1.1.0 to any later
version. Note that 1.1.0 did not have the non-rolling upgrade framework, so
things like preparing the OMs for upgrade and checking finalization status are
not present in that version. Manual upgrade is the only supported upgrade
option from 1.0.0 to 1.1.0.
+Non-rolling upgrades and downgrades are supported from 1.1.0 to any later
version. Note that 1.1.0 did not have the non-rolling upgrade framework, so
things like preparing the OMs for upgrade and checking finalization status are
not present in that version.
## Directory Layout
@@ -72,23 +72,9 @@ Each type of upgrade has a subdirectory under the *upgrades*
directory.
4. `with_this_version_finalized`: Run after ozone is stopped in the old
version after donwgrade, started again in the new version pre-finalized, and
then finalized.
- The upgrade is complete when this callback runs.
-#### manual-upgrade
-
-- This is a legacy option that was used before the upgrade framework was
introduced in 1.2.0. This option is left as an example in case it needs to be
used for some reason in the future.
-
-- Any necessary conversion of on disk structures from the old version to the
new version must be done explicitly.
-
-- This is primarily for testing upgrades from versions before the non-rolling
upgrade framework was introduced.
-
-- Supported Callbacks:
- 1. `setup_old_version`: Run before ozone is started in the old version.
- 3. `with_old_version`: Run while ozone is running in the old version.
- 3. `setup_this_version`: Run after ozone is stopped in the old version,
but before it is restarted in the new version.
- 4. `with_this_version`: Run while ozone is running in the new version.
-
### compose
-Docker compose cluster definitions to be used in upgrade testing are defined
in the *compose* directory. A compose cluster can be selected by specifying the
name of its subdirectory as the first argument to `run_test`. `run_test` will
then source the `load.sh` script in the cluster's directory so it is used
during the test. For manual testing, docker compose can be used normally from
the compose cluster directory. Note that some clusters may not work with older
versions. Ozone 1.1.0, for [...]
+Docker compose cluster definitions to be used in upgrade testing are defined
in the *compose* directory. A compose cluster can be selected by specifying the
name of its subdirectory as the first argument to `run_test`. `run_test` will
then source the `load.sh` script in the cluster's directory so it is used
during the test.
## Persisting Data
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
deleted file mode 100644
index babe87a492..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/.env
+++ /dev/null
@@ -1,22 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-HDDS_VERSION=${hdds.version}
-OZONE_RUNNER_VERSION=${docker.ozone-runner.version}
-OZONE_RUNNER_IMAGE=apache/ozone-runner
-OZONE_TEST_IMAGE=apache/ozone-runner:${docker.ozone-runner.version}
-OZONE_DIR=/opt/hadoop
-OZONE_VOLUME=./data
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
deleted file mode 100644
index 7aea9af378..0000000000
---
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-compose.yaml
+++ /dev/null
@@ -1,155 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# reusable fragments (see
https://docs.docker.com/compose/compose-file/#extension-fields)
-x-common-config:
- &common-config
- env_file:
- - docker-config
- image: ${OZONE_TEST_IMAGE}
-
-x-environment:
- &environment
- OZONE-SITE.XML_ozone.server.default.replication:
${OZONE_REPLICATION_FACTOR:-1}
- OZONE_UPGRADE_TO: ${OZONE_UPGRADE_TO:-0}
- OZONE_UPGRADE_FROM: ${OZONE_UPGRADE_FROM:-0}
-
-x-datanode:
- &datanode
- command: ["ozone","datanode"]
- <<: *common-config
- environment:
- <<: *environment
- ports:
- - 19864
- - 9882
-
-x-volumes:
- - &ozone-dir ${TEST_DIR}/../..:${OZONE_DIR}
- - &transformation
${TEST_DIR}/../../libexec/transformation.py:/opt/hadoop/libexec/transformation.py
-
-services:
- om:
- command: ["ozone","om"]
- <<: *common-config
- environment:
- ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
- <<: *environment
- ports:
- - 9862
- - 9874
- networks:
- net:
- ipv4_address: 10.9.0.11
- volumes:
- - ${OZONE_VOLUME}/om:/data
- - *ozone-dir
- - *transformation
- scm:
- command: ["ozone","scm"]
- <<: *common-config
- environment:
- ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION
- OZONE-SITE.XML_hdds.scm.safemode.min.datanode:
${OZONE_SAFEMODE_MIN_DATANODES:-1}
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.12
- ports:
- - 9876:9876
- volumes:
- - ${OZONE_VOLUME}/scm:/data
- - *ozone-dir
- - *transformation
- dn1:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.13
- volumes:
- - ${OZONE_VOLUME}/dn1:/data
- - *ozone-dir
- - *transformation
- dn2:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.14
- volumes:
- - ${OZONE_VOLUME}/dn2:/data
- - *ozone-dir
- - *transformation
- dn3:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.15
- volumes:
- - ${OZONE_VOLUME}/dn3:/data
- - *ozone-dir
- - *transformation
- dn4:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.16
- volumes:
- - ${OZONE_VOLUME}/dn4:/data
- - *ozone-dir
- - *transformation
- dn5:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.17
- volumes:
- - ${OZONE_VOLUME}/dn5:/data
- - *ozone-dir
- - *transformation
- recon:
- command: ["ozone","recon"]
- <<: *common-config
- environment:
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.18
- ports:
- - 9888:9888
- volumes:
- - ${OZONE_VOLUME}/recon:/data
- - *ozone-dir
- - *transformation
- s3g:
- command: ["ozone","s3g"]
- <<: *common-config
- environment:
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.19
- ports:
- - 9878:9878
- volumes:
- - ${OZONE_VOLUME}/s3g:/data
- - *ozone-dir
- - *transformation
-networks:
- net:
- driver: bridge
- ipam:
- config:
- - subnet: 10.9.0.0/16
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
deleted file mode 100644
index ce4a8807e5..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/docker-config
+++ /dev/null
@@ -1,49 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
-
-OZONE-SITE.XML_ozone.client.failover.max.attempts=6
-
-OZONE-SITE.XML_ozone.om.address=om
-OZONE-SITE.XML_ozone.om.http-address=om:9874
-OZONE-SITE.XML_ozone.scm.container.size=1GB
-OZONE-SITE.XML_ozone.scm.datanode.ratis.volume.free-space.min=10MB
-OZONE-SITE.XML_ozone.scm.pipeline.creation.interval=30s
-OZONE-SITE.XML_ozone.scm.pipeline.owner.container.count=1
-OZONE-SITE.XML_ozone.scm.names=scm
-OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data/metadata
-OZONE-SITE.XML_ozone.scm.block.client.address=scm
-OZONE-SITE.XML_ozone.scm.container.size=1GB
-OZONE-SITE.XML_ozone.scm.client.address=scm
-
-OZONE-SITE.XML_hdds.datanode.dir=/data/hdds
-OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB
-
-OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon
-OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m
-OZONE-SITE.XML_hdds.scmclient.max.retry.timeout=30s
-OZONE-SITE.XML_ozone.http.basedir=/tmp/ozone_http
-OZONE-SITE.XML_ozone.fs.hsync.enabled=true
-
-OZONE_CONF_DIR=/etc/hadoop
-OZONE_LOG_DIR=/var/log/hadoop
-
-no_proxy=om,scm,s3g,kdc,localhost,127.0.0.1
-
-# Explicitly enable filesystem snapshot feature for this Docker compose cluster
-# Does not take effect on Ozone versions < 1.4.0
-OZONE-SITE.XML_ozone.filesystem.snapshot.enabled=true
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
deleted file mode 100755
index c1bd0d3724..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/non-ha/load.sh
+++ /dev/null
@@ -1,29 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Fail if required variables are not set.
-set -u
-: "${OZONE_VOLUME}"
-: "${TEST_DIR}"
-set +u
-
-source "$TEST_DIR/testlib.sh"
-
-export COMPOSE_FILE="$TEST_DIR/compose/non-ha/docker-compose.yaml"
-create_data_dirs "${OZONE_VOLUME}"/{om,dn1,dn2,dn3,dn4,dn5,recon,s3g,scm}
-
-echo "Using docker cluster defined in $COMPOSE_FILE"
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env
deleted file mode 100644
index 85c422b5ad..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/.env
+++ /dev/null
@@ -1,27 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-HDDS_VERSION=${hdds.version}
-OZONE_RUNNER_VERSION=${docker.ozone-runner.version}
-OZONE_RUNNER_IMAGE=apache/ozone-runner
-OZONE_TEST_IMAGE=apache/ozone-runner:${docker.ozone-runner.version}
-OZONE_DIR=/opt/hadoop
-OZONE_VOLUME=./data
-OM_SERVICE_ID=omservice
-# Indicates no arguments to the OM.
-# This variable must be set to some non-empty value, or docker compose will
-# expand it to an empty string and pass that to the OM as an argument.
-OM_HA_ARGS=--
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml
deleted file mode 100644
index 880b36ff2b..0000000000
---
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-compose.yaml
+++ /dev/null
@@ -1,178 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# reusable fragments (see
https://docs.docker.com/compose/compose-file/#extension-fields)
-x-common-config:
- &common-config
- env_file:
- - docker-config
- image: ${OZONE_TEST_IMAGE}
-
-x-environment:
- &environment
- OZONE-SITE.XML_ozone.server.default.replication:
${OZONE_REPLICATION_FACTOR:-3}
- OZONE_UPGRADE_TO: ${OZONE_UPGRADE_TO:-0}
- OZONE_UPGRADE_FROM: ${OZONE_UPGRADE_FROM:-0}
-
-x-datanode:
- &datanode
- command: ["ozone","datanode"]
- <<: *common-config
- environment:
- <<: *environment
- ports:
- - 19864
- - 9882
-
-x-om:
- &om
- command: ["ozone","om","${OM_HA_ARGS}"]
- <<: *common-config
- environment:
- ENSURE_OM_INITIALIZED: /data/metadata/om/current/VERSION
- <<: *environment
- ports:
- - 9862
- - 9872
-
-x-volumes:
- - &ozone-dir ../../../..:${OZONE_DIR}
- - &transformation
../../../../libexec/transformation.py:/opt/hadoop/libexec/transformation.py
-
-services:
- om1:
- <<: *om
- networks:
- net:
- ipv4_address: 10.9.0.11
- volumes:
- - ${OZONE_VOLUME}/om1:/data
- - *ozone-dir
- - *transformation
- om2:
- <<: *om
- networks:
- net:
- ipv4_address: 10.9.0.12
- volumes:
- - ${OZONE_VOLUME}/om2:/data
- - *ozone-dir
- - *transformation
- om3:
- <<: *om
- networks:
- net:
- ipv4_address: 10.9.0.13
- volumes:
- - ${OZONE_VOLUME}/om3:/data
- - *ozone-dir
- - *transformation
-
- scm:
- command: ["ozone","scm"]
- <<: *common-config
- environment:
- ENSURE_SCM_INITIALIZED: /data/metadata/scm/current/VERSION
- OZONE-SITE.XML_hdds.scm.safemode.min.datanode:
${OZONE_SAFEMODE_MIN_DATANODES:-1}
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.14
- ports:
- - 9876:9876
- volumes:
- - ${OZONE_VOLUME}/scm:/data
- - *ozone-dir
- - *transformation
- dn1:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.15
- volumes:
- - ${OZONE_VOLUME}/dn1:/data
- - *ozone-dir
- - *transformation
- dn2:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.16
- volumes:
- - ${OZONE_VOLUME}/dn2:/data
- - *ozone-dir
- - *transformation
- dn3:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.17
- volumes:
- - ${OZONE_VOLUME}/dn3:/data
- - *ozone-dir
- - *transformation
- dn4:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.18
- volumes:
- - ${OZONE_VOLUME}/dn4:/data
- - *ozone-dir
- - *transformation
- dn5:
- <<: *datanode
- networks:
- net:
- ipv4_address: 10.9.0.19
- volumes:
- - ${OZONE_VOLUME}/dn5:/data
- - *ozone-dir
- - *transformation
- recon:
- command: ["ozone","recon"]
- <<: *common-config
- environment:
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.20
- ports:
- - 9888:9888
- volumes:
- - ${OZONE_VOLUME}/recon:/data
- - *ozone-dir
- - *transformation
- s3g:
- command: ["ozone","s3g"]
- <<: *common-config
- environment:
- <<: *environment
- networks:
- net:
- ipv4_address: 10.9.0.21
- ports:
- - 9878:9878
- volumes:
- - ${OZONE_VOLUME}/s3g:/data
- - *ozone-dir
- - *transformation
-networks:
- net:
- driver: bridge
- ipam:
- config:
- - subnet: 10.9.0.0/16
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config
deleted file mode 100644
index a049ba5f01..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/docker-config
+++ /dev/null
@@ -1,65 +0,0 @@
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-OZONE-SITE.XML_ozone.metadata.dirs=/data/metadata
-
-OZONE-SITE.XML_ozone.client.failover.max.attempts=6
-
-OZONE-SITE.XML_ozone.om.service.ids=omservice
-OZONE-SITE.XML_ozone.om.nodes.omservice=om1,om2,om3
-OZONE-SITE.XML_ozone.om.address.omservice.om1=om1
-OZONE-SITE.XML_ozone.om.address.omservice.om2=om2
-OZONE-SITE.XML_ozone.om.address.omservice.om3=om3
-OZONE-SITE.XML_ozone.om.ratis.enable=true
-OZONE-SITE.XML_ozone.scm.pipeline.creation.interval=30s
-OZONE-SITE.XML_ozone.scm.pipeline.owner.container.count=1
-OZONE-SITE.XML_ozone.scm.names=scm
-OZONE-SITE.XML_ozone.scm.datanode.id.dir=/data/metadata
-OZONE-SITE.XML_ozone.scm.block.client.address=scm
-OZONE-SITE.XML_ozone.scm.container.size=1GB
-OZONE-SITE.XML_ozone.scm.client.address=scm
-OZONE-SITE.XML_ozone.http.basedir=/tmp/ozone_http
-OZONE-SITE.XML_ozone.fs.hsync.enabled=true
-
-OZONE-SITE.XML_hdds.datanode.dir=/data/hdds
-OZONE-SITE.XML_hdds.datanode.volume.min.free.space=100MB
-
-# If SCM sends container close commands as part of upgrade finalization while
-# datanodes are doing a leader election, all 3 replicas may end up in the
-# CLOSING state. The replication manager must be running to later move them to
-# a CLOSED state so the datanodes can progress with finalization.
-#
-# This config sets the amount of time SCM will wait after safemode exit to
-# start the replication manager and pipeline scrubber. The default of 5 minutes
-# is fine in real clusters to prevent unnecessary over-replication,
-# but it is too long for this test.
-OZONE-SITE.XML_hdds.scm.wait.time.after.safemode.exit=5s
-# If datanodes take too long to close pipelines during finalization, let the
-# scrubber force close them to move the test forward.
-OZONE-SITE.XML_ozone.scm.pipeline.scrub.interval=1m
-OZONE-SITE.XML_ozone.scm.pipeline.allocated.timeout=2m
-
-OZONE-SITE.XML_ozone.recon.db.dir=/data/metadata/recon
-OZONE-SITE.XML_ozone.recon.om.snapshot.task.interval.delay=1m
-OZONE-SITE.XML_ozone.recon.address=recon:9891
-
-no_proxy=om1,om2,om3,scm,s3g,kdc,localhost,127.0.0.1
-
-OM_SERVICE_ID=omservice
-
-# Explicitly enable filesystem snapshot feature for this Docker compose cluster
-# Does not take effect on Ozone versions < 1.4.0
-OZONE-SITE.XML_ozone.filesystem.snapshot.enabled=true
diff --git a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh
b/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh
deleted file mode 100755
index a2f2603414..0000000000
--- a/hadoop-ozone/dist/src/main/compose/upgrade/compose/om-ha/load.sh
+++ /dev/null
@@ -1,30 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# Fail if required variables are not set.
-set -u
-: "${OZONE_VOLUME}"
-: "${TEST_DIR}"
-set +u
-
-source "$TEST_DIR/testlib.sh"
-
-export COMPOSE_FILE="$TEST_DIR/compose/om-ha/docker-compose.yaml"
-export OM_SERVICE_ID=omservice
-create_data_dirs
"${OZONE_VOLUME}"/{om1,om2,om3,dn1,dn2,dn3,dn4,dn5,recon,s3g,scm}
-
-echo "Using docker cluster defined in $COMPOSE_FILE"
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md
b/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md
deleted file mode 100644
index 20c1c3dfb2..0000000000
---
a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/README.md
+++ /dev/null
@@ -1,19 +0,0 @@
-<!---
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License. See accompanying LICENSE file.
--->
-
-# Manual Upgrade Testing
-
-- Since the release of Ozone's non-rolling upgrade framework in 1.2.0, no
manual reformatting steps need to be run on nodes between upgrades.
-
-- The driver for testing this type of upgrade is left here as an example in
case it needs to be used for some reason in the future.
diff --git
a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
b/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
deleted file mode 100755
index c77528684b..0000000000
---
a/hadoop-ozone/dist/src/main/compose/upgrade/upgrades/manual-upgrade/driver.sh
+++ /dev/null
@@ -1,54 +0,0 @@
-#!/usr/bin/env bash
-# Licensed to the Apache Software Foundation (ASF) under one
-# or more contributor license agreements. See the NOTICE file
-# distributed with this work for additional information
-# regarding copyright ownership. The ASF licenses this file
-# to you under the Apache License, Version 2.0 (the
-# "License"); you may not use this file except in compliance
-# with the License. You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# This script tests upgrade from one release to a later one. Docker
-# image with Ozone binaries are required for both versions.
-
-set -e -o pipefail
-
-# Fail if required vars are not set.
-set -u
-: "${OZONE_UPGRADE_FROM}"
-: "${OZONE_UPGRADE_TO}"
-: "${TEST_DIR}"
-: "${OZONE_UPGRADE_CALLBACK}"
-set +u
-
-source "$TEST_DIR"/compose/non-ha/load.sh
-source "$TEST_DIR"/testlib.sh
-[[ -f "$OZONE_UPGRADE_CALLBACK" ]] && source "$OZONE_UPGRADE_CALLBACK"
-
-echo "--- RUNNING MANUAL UPGRADE TEST FROM $OZONE_UPGRADE_FROM TO
$OZONE_UPGRADE_TO ---"
-
-echo "--- SETTING UP OLD VERSION $OZONE_UPGRADE_FROM ---"
-OUTPUT_NAME="$OZONE_UPGRADE_FROM"
-prepare_for_image "$OZONE_UPGRADE_FROM"
-callback setup_old_version
-
-echo "--- RUNNING WITH OLD VERSION $OZONE_UPGRADE_FROM ---"
-start_docker_env
-callback with_old_version
-stop_docker_env
-
-echo "--- SETTING UP NEW VERSION $OZONE_UPGRADE_TO ---"
-OUTPUT_NAME="$OZONE_UPGRADE_TO"
-prepare_for_image "$OZONE_UPGRADE_TO"
-callback setup_this_version
-
-echo "--- RUNNING WITH NEW VERSION $OZONE_UPGRADE_TO ---"
-OZONE_KEEP_RESULTS=true start_docker_env
-callback with_this_version
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
