This is an automated email from the ASF dual-hosted git repository.
adoroszlai pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ozone.git
The following commit(s) were added to refs/heads/master by this push:
new af1f351756 HDDS-8726. Let run.sh start more than 3 datanodes (#4802)
af1f351756 is described below
commit af1f351756191860f8574e78b853524b0d919946
Author: Doroszlai, Attila <[email protected]>
AuthorDate: Sat Jun 3 08:50:34 2023 +0200
HDDS-8726. Let run.sh start more than 3 datanodes (#4802)
---
hadoop-ozone/dist/src/main/compose/ozone/README.md | 9 ++---
hadoop-ozone/dist/src/main/compose/ozone/run.sh | 42 ++++++++++++++++++++--
2 files changed, 44 insertions(+), 7 deletions(-)
diff --git a/hadoop-ozone/dist/src/main/compose/ozone/README.md
b/hadoop-ozone/dist/src/main/compose/ozone/README.md
index 357020d5ef..9de3637979 100644
--- a/hadoop-ozone/dist/src/main/compose/ozone/README.md
+++ b/hadoop-ozone/dist/src/main/compose/ozone/README.md
@@ -29,10 +29,11 @@ TL;DR:
```
./run.sh -d
```
-2. three datanodes for replication:
+2. multiple datanodes for replication:
```
- export OZONE_REPLICATION_FACTOR=3
- ./run.sh -d
+ OZONE_DATANODES=3 ./run.sh -d
+ # or
+ OZONE_DATANODES=5 ./run.sh -d
```
### Basics
@@ -43,7 +44,7 @@ You can change the number of datanodes to start using the
`--scale` option. Eg.
The cluster's replication factor (1 or 3) can be controlled by setting the
`OZONE_REPLICATION_FACTOR` environment variable. It defaults to 1 to match the
number of datanodes started by default, without the `--scale` option.
-For convenience the `run.sh` script can be used to make sure the replication
factor and the number of datanodes match. It also passes any additional
arguments provided on the command-line (eg. `-d`) to `docker-compose`.
+For convenience the `run.sh` script can be used to start multiple datanodes
(by setting the `OZONE_DATANODES` variable), while making sure the replication
factor and the number of datanodes are compatible. It also passes any
additional arguments provided on the command-line (eg. `-d`) to
`docker-compose`.
### Add-ons
diff --git a/hadoop-ozone/dist/src/main/compose/ozone/run.sh
b/hadoop-ozone/dist/src/main/compose/ozone/run.sh
index f18aa78ac4..5bdb2996cb 100755
--- a/hadoop-ozone/dist/src/main/compose/ozone/run.sh
+++ b/hadoop-ozone/dist/src/main/compose/ozone/run.sh
@@ -15,9 +15,45 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-declare -ix OZONE_REPLICATION_FACTOR OZONE_SAFEMODE_MIN_DATANODES
+declare -i OZONE_DATANODES OZONE_REPLICATION_FACTOR
OZONE_SAFEMODE_MIN_DATANODES
+
+ORIG_DATANODES="${OZONE_DATANODES:-}"
+ORIG_REPLICATION_FACTOR="${OZONE_REPLICATION_FACTOR:-}"
+
+# only support replication factor of 1 or 3
+if [[ -v OZONE_REPLICATION_FACTOR ]] && [[ ${OZONE_REPLICATION_FACTOR} -ne 1
]] && [[ ${OZONE_REPLICATION_FACTOR} -ne 3 ]]; then
+ # assume invalid replication factor was intended as "number of datanodes"
+ if [[ -z ${ORIG_DATANODES} ]]; then
+ OZONE_DATANODES=${OZONE_REPLICATION_FACTOR}
+ fi
+ unset OZONE_REPLICATION_FACTOR
+fi
+
+# at least 1 datanode
+if [[ -v OZONE_DATANODES ]] && [[ ${OZONE_DATANODES} -lt 1 ]]; then
+ unset OZONE_DATANODES
+fi
+
+if [[ -v OZONE_DATANODES ]] && [[ -v OZONE_REPLICATION_FACTOR ]]; then
+ # ensure enough datanodes for replication factor
+ if [[ ${OZONE_DATANODES} -lt ${OZONE_REPLICATION_FACTOR} ]]; then
+ OZONE_DATANODES=${OZONE_REPLICATION_FACTOR}
+ fi
+elif [[ -v OZONE_DATANODES ]]; then
+ if [[ ${OZONE_DATANODES} -ge 3 ]]; then
+ OZONE_REPLICATION_FACTOR=3
+ else
+ OZONE_REPLICATION_FACTOR=1
+ fi
+elif [[ -v OZONE_REPLICATION_FACTOR ]]; then
+ OZONE_DATANODES=${OZONE_REPLICATION_FACTOR}
+else
+ OZONE_DATANODES=1
+ OZONE_REPLICATION_FACTOR=1
+fi
-: ${OZONE_REPLICATION_FACTOR:=1}
: ${OZONE_SAFEMODE_MIN_DATANODES:=${OZONE_REPLICATION_FACTOR}}
-docker-compose up --scale datanode=${OZONE_REPLICATION_FACTOR} --no-recreate
"$@"
+export OZONE_DATANODES OZONE_REPLICATION_FACTOR OZONE_SAFEMODE_MIN_DATANODES
+
+docker-compose up --scale datanode=${OZONE_DATANODES} --no-recreate "$@"
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
