hudi-agent commented on code in PR #19489:
URL: https://github.com/apache/hudi/pull/19489#discussion_r3705824202
##########
docker/README.md:
##########
@@ -63,32 +64,39 @@ To build the Docker demo images with `docker` directly,
rather than through the
`build_local_docker_images.sh` above, run the script from under
`<HUDI_REPO_DIR>/docker`:
```shell
-# With no flags, builds Hadoop 2.8.4 / Spark 3.5.3 / Hive 2.3.10, matching
-# docker-compose_hadoop284_hive2310_spark353_{amd64,arm64}.yml
-./build_docker_images.sh
+# Builds Hadoop 3.3.4 / Spark 3.5.3 / Hive 3.1.3, matching
+# docker-compose_hadoop334_hive313_spark353_{amd64,arm64}.yml, which is the
set setup_demo.sh runs
+./build_and_publish_docker_images.sh
```
-You can override the Hadoop, Spark, and Hive versions from the command line.
If you plan to use `setup_demo.sh`,
-build the image set matching the default compose files first. For other flows,
use one of the supported version
-combinations under `docker/compose`.
+To build one of the other version combinations under `docker/compose`, edit
the version block near the top of the
+script before running it:
```shell
-# Matches setup_demo.sh and
-# docker-compose_hadoop334_hive313_spark353_{amd64,arm64}.yml
-./build_docker_images.sh --hadoop-version 3.3.4 --spark-version 3.5.3
--hive-version 3.1.3
-
-# Another supported combination is
-# docker-compose_hadoop340_hive313_spark401_{amd64,arm64}.yml
-./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1
--hive-version 3.1.3
+HADOOP_VERSION="3.3.4"
+SPARK_VERSION="3.5.3"
+HIVE_VERSION="3.1.3"
```
+The base image follows from `SPARK_VERSION`, so setting it to a 4.x release
also switches the set to the Java 17
+base. If you plan to use `setup_demo.sh`, build the image set matching its
compose file first.
+
`setup_demo.sh` currently defaults to
`docker-compose_hadoop334_hive313_spark353_{amd64,arm64}.yml`. If you build a
different image set for the demo flow, update `COMPOSE_FILE_NAME` in
`setup_demo.sh` to point to the matching compose
file before running the script. Run `./setup_demo.sh dev` to use your locally
built images; a plain run pulls the
Docker Hub images over them.
-By default, the script builds images for the current machine architecture and
derives the version tag from the root
-`pom.xml`. Use `--version-tag` to set an explicit tag if needed.
+The script builds images for the current machine architecture and tags each
one `:latest` plus the Hudi version
+taken from the root `pom.xml`. Export `VERSION_TAG` to use a different second
tag:
Review Comment:
🤖 This says to override the second tag by exporting `VERSION_TAG`
(`VERSION_TAG=my-test ./build_and_publish_docker_images.sh`), but the PR
description and the previous docs describe a `--version-tag <tag>` flag for the
same purpose. It would help to state definitively whether the tag override is
an environment variable, a `--version-tag` flag, or both — right now the two
forms coexist in the same doc and readers won't know which one the script
honors.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
docker/README.md:
##########
@@ -63,32 +64,39 @@ To build the Docker demo images with `docker` directly,
rather than through the
`build_local_docker_images.sh` above, run the script from under
`<HUDI_REPO_DIR>/docker`:
```shell
-# With no flags, builds Hadoop 2.8.4 / Spark 3.5.3 / Hive 2.3.10, matching
-# docker-compose_hadoop284_hive2310_spark353_{amd64,arm64}.yml
-./build_docker_images.sh
+# Builds Hadoop 3.3.4 / Spark 3.5.3 / Hive 3.1.3, matching
+# docker-compose_hadoop334_hive313_spark353_{amd64,arm64}.yml, which is the
set setup_demo.sh runs
+./build_and_publish_docker_images.sh
```
-You can override the Hadoop, Spark, and Hive versions from the command line.
If you plan to use `setup_demo.sh`,
-build the image set matching the default compose files first. For other flows,
use one of the supported version
-combinations under `docker/compose`.
+To build one of the other version combinations under `docker/compose`, edit
the version block near the top of the
Review Comment:
🤖 This section tells users to override versions by editing the
`HADOOP_VERSION`/`SPARK_VERSION`/`HIVE_VERSION` block "near the top of the
script," but the Multi-Arch example below (and the PR description) overrides
the same values with `--hadoop-version 3.4.0 --spark-version 4.0.1
--hive-version 3.1.3` flags. These two instructions contradict each other.
Since the script appears to accept
`--hadoop-version`/`--spark-version`/`--hive-version` flags, it would be
clearer (and less error-prone than hand-editing the script) to document the
flags here as well, rather than telling users to modify the source.
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
