This is an automated email from the ASF dual-hosted git repository.
ssulav pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/ozone-installer.git
The following commit(s) were added to refs/heads/master by this push:
new a09df51 HDDS-14720. Set OZONE_HOME and Path only for the service user
bash, include all.yml for OZONE_OPTS (#8)
a09df51 is described below
commit a09df511375bbe3b759a83dddad6c1165e288ccc
Author: Soumitra Sulav <[email protected]>
AuthorDate: Mon Mar 2 21:39:45 2026 +0530
HDDS-14720. Set OZONE_HOME and Path only for the service user bash, include
all.yml for OZONE_OPTS (#8)
---
README.md | 84 ++++++++++++++--------------
ansible.cfg | 1 +
inventories/dev/group_vars/all.yml | 1 -
ozone_installer.py | 35 +++++++++++-
playbooks/cluster.yml | 4 +-
requirements.txt | 1 +
roles/java/tasks/main.yml | 10 ++--
roles/ozone_cleanup/tasks/main.yml | 11 ++++
roles/ozone_config/defaults/main.yml | 1 +
roles/ozone_config/templates/ozone-env.sh.j2 | 5 +-
roles/ozone_layout/tasks/main.yml | 8 +--
11 files changed, 102 insertions(+), 59 deletions(-)
diff --git a/README.md b/README.md
index 64ecb0c..75fc6fc 100644
--- a/README.md
+++ b/README.md
@@ -49,47 +49,6 @@ Ports and service behavior follow Ozone defaults; consult
the official documenta
- SSH server enabled
- Sudo access (if using `--use-sudo`)
-**⚠️ Known Issue: CentOS 8 / RHEL 8 with Python 3.6**
-
-On CentOS 8/RHEL 8, the system's `dnf` package manager may use Python 3.6
(`/usr/libexec/platform-python`), and the DNF Python module (`python3-dnf`) is
only available for Python 3.6, not for Python 3.9+.
-
-The installer works around this by using direct shell commands (e.g.,
`/usr/libexec/platform-python /usr/bin/dnf install`) for package installation
rather than Ansible's package module.
-
-#### Python Version Requirements by OS
-
-| Operating System | Default Python | Available Versions | Installation
Command |
-|-----------------|----------------|-------------------|---------------------|
-| RHEL 9+ / Rocky 9+ | Python 3.9+ ✅ | 3.11, 3.9 | `sudo yum install -y
python3.11` |
-| RHEL 8 / Rocky 8 / CentOS 8 | Python 3.6 ❌ | 3.9, 3.8 (python39, python38) |
`sudo yum install -y python39` |
-| CentOS 7 | Python 3.6 ❌ | 3.6 only | Must use EPEL or SCL for newer versions
|
-| Ubuntu 20.04+ | Python 3.8+ ✅ | 3.11, 3.10, 3.9, 3.8 | `sudo apt-get install
-y python3.11` |
-| Debian 11+ | Python 3.9+ ✅ | 3.11, 3.9 | `sudo apt-get install -y
python3.11` |
-
-**Important**: If your managed nodes have Python 3.6 or older, you must
upgrade:
-
-```bash
-# CentOS 8 / RHEL 8 / Rocky 8 (most common)
-sudo yum install -y python39
-# Verify: /usr/bin/python3.9 --version
-
-# RHEL 9+ / Rocky 9+
-sudo yum install -y python3.11
-# Verify: /usr/bin/python3.11 --version
-
-# Ubuntu / Debian
-sudo apt-get update && sudo apt-get install -y python3.9
-# Verify: /usr/bin/python3.9 --version
-```
-
-**Then specify the Python interpreter when running the installer:**
-```bash
-# For CentOS 8 / RHEL 8
-python3 ozone_installer.py -H hosts -v 2.0.0 --python-interpreter
/usr/bin/python3.9
-
-# For RHEL 9+
-python3 ozone_installer.py -H hosts -v 2.0.0 --python-interpreter
/usr/bin/python3.11
-```
-
### Network and access requirements
- Controller must be on the same network as the target hosts
- Controller requires SSH access (key or password) to all target hosts
@@ -260,7 +219,7 @@ ANSIBLE_CONFIG=ansible.cfg ansible-playbook -i
inventories/dev/hosts.ini playboo
## Notes
- Idempotent where possible; runtime `ozone` init/start guarded with
`creates:`.
-- JAVA_HOME and PATH are persisted for resume; runtime settings are exported
via `ozone-env.sh`.
+- JAVA_HOME, OZONE_HOME, and PATH are set only for the service user (in
`~/.bashrc`); runtime settings are exported via `ozone-env.sh`.
- Local snapshot mode archives from the controller and uploads/extracts on
targets using `unarchive`.
- Logs are written to a per-run file under `logs/` named:
- `ansible-<timestamp>-<hosts_raw_sanitized>.log`
@@ -290,3 +249,44 @@ ANSIBLE_CONFIG=ansible.cfg ansible-playbook -i
inventories/dev/hosts.ini playboo
- The smoke role can optionally install `awscli` on the first S3G host,
configure dummy credentials, and create/list a test bucket against
`http://localhost:9878` (for simple functional verification).
+
+**⚠️ Known Issue: CentOS 8 / RHEL 8 with Python 3.6**
+
+On CentOS 8/RHEL 8, the system's `dnf` package manager may use Python 3.6
(`/usr/libexec/platform-python`), and the DNF Python module (`python3-dnf`) is
only available for Python 3.6, not for Python 3.9+.
+
+The installer works around this by using direct shell commands (e.g.,
`/usr/libexec/platform-python /usr/bin/dnf install`) for package installation
rather than Ansible's package module.
+
+#### Python Version Requirements by OS
+
+| Operating System | Default Python | Available Versions | Installation
Command |
+|-----------------|----------------|-------------------|---------------------|
+| RHEL 9+ / Rocky 9+ | Python 3.9+ ✅ | 3.11, 3.9 | `sudo yum install -y
python3.11` |
+| RHEL 8 / Rocky 8 / CentOS 8 | Python 3.6 ❌ | 3.9, 3.8 (python39, python38) |
`sudo yum install -y python39` |
+| CentOS 7 | Python 3.6 ❌ | 3.6 only | Must use EPEL or SCL for newer versions
|
+| Ubuntu 20.04+ | Python 3.8+ ✅ | 3.11, 3.10, 3.9, 3.8 | `sudo apt-get install
-y python3.11` |
+| Debian 11+ | Python 3.9+ ✅ | 3.11, 3.9 | `sudo apt-get install -y
python3.11` |
+
+**Important**: If your managed nodes have Python 3.6 or older, you must
upgrade:
+
+```bash
+# CentOS 8 / RHEL 8 / Rocky 8 (most common)
+sudo yum install -y python39
+# Verify: /usr/bin/python3.9 --version
+
+# RHEL 9+ / Rocky 9+
+sudo yum install -y python3.11
+# Verify: /usr/bin/python3.11 --version
+
+# Ubuntu / Debian
+sudo apt-get update && sudo apt-get install -y python3.9
+# Verify: /usr/bin/python3.9 --version
+```
+
+**Then specify the Python interpreter when running the installer:**
+```bash
+# For CentOS 8 / RHEL 8
+python3 ozone_installer.py -H hosts -v 2.0.0 --python-interpreter
/usr/bin/python3.9
+
+# For RHEL 9+
+python3 ozone_installer.py -H hosts -v 2.0.0 --python-interpreter
/usr/bin/python3.11
+```
\ No newline at end of file
diff --git a/ansible.cfg b/ansible.cfg
index 4378dc5..4eeccb1 100644
--- a/ansible.cfg
+++ b/ansible.cfg
@@ -30,6 +30,7 @@ callbacks_enabled = timer, profile_tasks, last_failed ; for
execution time profi
deprecation_warnings = False
host_key_checking = False
remote_tmp = /tmp/.ansible-${USER}
+allow_world_readable_tmpfiles = True
[privilege_escalation]
become = True
diff --git a/inventories/dev/group_vars/all.yml
b/inventories/dev/group_vars/all.yml
index e8deab0..89516b1 100644
--- a/inventories/dev/group_vars/all.yml
+++ b/inventories/dev/group_vars/all.yml
@@ -41,7 +41,6 @@ service_group: "ozone"
use_sudo: true
ozone_opts: "-Xmx1024m -XX:ParallelGCThreads=8"
service_command_timeout: 300 # seconds for service init/start commands
-ansible_remote_tmp: "/tmp/.ansible-{{ ansible_user_id }}"
# SSH bootstrap
allow_cluster_ssh_key_deploy: false
diff --git a/ozone_installer.py b/ozone_installer.py
index 920e214..b4c046a 100755
--- a/ozone_installer.py
+++ b/ozone_installer.py
@@ -26,7 +26,12 @@ import tempfile
import logging
from datetime import datetime
from pathlib import Path
-from typing import List, Optional, Tuple
+from typing import Any, Dict, List, Optional, Tuple
+
+try:
+ import yaml
+except ImportError:
+ yaml = None # type: ignore
# Optional nicer interactive prompts (fallback to built-in prompts if
unavailable)
try:
@@ -285,6 +290,32 @@ def expand_braces(expr: str) -> List[str]:
pre, a, b, post = m.group(1), int(m.group(2)), int(m.group(3)), m.group(4)
return [f"{pre}{i}{post}" for i in range(a, b + 1)]
+def _load_group_vars() -> Dict[str, Any]:
+ """Load inventories/dev/group_vars/all.yml if it exists. Returns {} on
failure."""
+ if not yaml:
+ return {}
+ path = ANSIBLE_ROOT / "inventories" / "dev" / "group_vars" / "all.yml"
+ if not path.exists():
+ return {}
+ try:
+ with open(path, encoding="utf-8") as f:
+ data = yaml.safe_load(f)
+ return dict(data) if isinstance(data, dict) else {}
+ except Exception:
+ return {}
+
+
+def _merge_extra_vars(extra_vars: Dict[str, Any]) -> Dict[str, Any]:
+ """Merge group_vars/all.yml into extra_vars; extra_vars override
group_vars.
+
+ Empty-string values from group_vars are excluded so they don't shadow
+ set_fact auto-detection (extra_vars have highest Ansible precedence).
+ """
+ base = {k: v for k, v in _load_group_vars().items() if v != ""}
+ base.update(extra_vars)
+ return base
+
+
def parse_data_dirs(data_raw: Optional[str]) -> str:
"""
Accepts comma-separated data dirs; each may contain brace expansion (e.g.
/data/ozone{1..3}).
@@ -776,6 +807,7 @@ def main(argv: List[str]) -> int:
ask_pass = auth_method == "password" and not password
if stop_only or stop_and_clean:
+ extra_vars = _merge_extra_vars(extra_vars)
with tempfile.NamedTemporaryFile(mode="w", suffix=".ini",
delete=False) as inv_f:
inv_f.write(inventory_text or "")
inv_path = Path(inv_f.name)
@@ -797,6 +829,7 @@ def main(argv: List[str]) -> int:
pass
# Full install: persist config and run cluster playbook
+ extra_vars = _merge_extra_vars(extra_vars)
with tempfile.TemporaryDirectory() as tdir:
inv_path = Path(tdir) / "hosts.ini"
ev_path = Path(tdir) / "vars.json"
diff --git a/playbooks/cluster.yml b/playbooks/cluster.yml
index f800e2d..4444243 100644
--- a/playbooks/cluster.yml
+++ b/playbooks/cluster.yml
@@ -44,10 +44,10 @@
- name: "Pre-install: Ensure Ansible remote tmp exists"
file:
- path: "{{ (ansible_env.TMPDIR | default('/tmp')) ~ '/.ansible-' ~
ansible_user_id }}"
+ path: "{{ (ansible_env.TMPDIR | default('/tmp')) ~ '/.ansible-' ~
(ansible_user_id | default(ansible_user) | default('ansible')) }}"
state: directory
mode: "0700"
- owner: "{{ ansible_user_id }}"
+ owner: "{{ ansible_user_id | default(ansible_user) |
default('ansible') }}"
roles:
- role: ozone_stop
diff --git a/requirements.txt b/requirements.txt
index 615def9..6e6ccd3 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -15,3 +15,4 @@
ansible-core==2.17.*
click==8.*
+pyyaml>=6.0
diff --git a/roles/java/tasks/main.yml b/roles/java/tasks/main.yml
index c05aecd..de0c019 100644
--- a/roles/java/tasks/main.yml
+++ b/roles/java/tasks/main.yml
@@ -39,7 +39,7 @@
path: "{{ item }}"
loop: "{{ java_home_candidates }}"
register: java_candidates
- become: false
+ become: true
- name: "Set ozone_java_home from first existing candidate"
set_fact:
@@ -76,12 +76,12 @@
content: "{{ merged_all | to_nice_json }}"
mode: "0644"
-- name: "Export JAVA_HOME and update PATH in profile.d/ozone.sh"
+- name: "Export JAVA_HOME and update PATH in service user shell config"
blockinfile:
- path: "/etc/profile.d/ozone.sh"
+ path: "{{ (service_user == 'root') | ternary('/root', '/home/' +
service_user) }}/.bashrc"
create: true
- owner: root
- group: root
+ owner: "{{ service_user }}"
+ group: "{{ service_group }}"
mode: "0644"
marker: "# {mark} {{ JAVA_MARKER }}"
block: |
diff --git a/roles/ozone_cleanup/tasks/main.yml
b/roles/ozone_cleanup/tasks/main.yml
index c190ac7..60d6e25 100644
--- a/roles/ozone_cleanup/tasks/main.yml
+++ b/roles/ozone_cleanup/tasks/main.yml
@@ -33,6 +33,17 @@
- name: "Remove install and data directories when install_base exists"
when: install_present
block:
+ - name: "Remove Ozone env blocks from service user .bashrc"
+ blockinfile:
+ path: "{{ (service_user == 'root') | ternary('/root', '/home/' +
service_user) }}/.bashrc"
+ marker: "# {mark} {{ item }}"
+ state: absent
+ create: false
+ loop:
+ - "{{ JAVA_MARKER | default('Apache Ozone Installer Java Home') }}"
+ - "{{ ENV_MARKER | default('Apache Ozone Installer Env') }}"
+ become: true
+
- name: "Remove install directory"
file:
path: "{{ install_base }}"
diff --git a/roles/ozone_config/defaults/main.yml
b/roles/ozone_config/defaults/main.yml
index a527b6b..3ea392b 100644
--- a/roles/ozone_config/defaults/main.yml
+++ b/roles/ozone_config/defaults/main.yml
@@ -17,6 +17,7 @@
install_base: "/opt/ozone"
# data_base: single path or comma-separated paths (e.g. /data/ozone or
/data/ozone1,/data/ozone2)
data_base: "/data/ozone"
+ozone_opts: "-Xmx1024m -XX:ParallelGCThreads=8"
CONFIG_DIR: "" # if provided, can be used to feed additional properties via
vars
diff --git a/roles/ozone_config/templates/ozone-env.sh.j2
b/roles/ozone_config/templates/ozone-env.sh.j2
index dc5c255..d2e09d0 100644
--- a/roles/ozone_config/templates/ozone-env.sh.j2
+++ b/roles/ozone_config/templates/ozone-env.sh.j2
@@ -31,10 +31,7 @@ export HADOOP_CONF_DIR="{{ install_base
}}/current/etc/hadoop"
# Relaxed module access for Java 17/21 (needed by Ozone and dependencies)
export JAVA_TOOL_OPTIONS="${JAVA_TOOL_OPTIONS:+$JAVA_TOOL_OPTIONS}
--add-opens=java.base/jdk.internal.misc=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED
--add-opens=java.base/java.nio=ALL-UNNAMED
--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.util=ALL-UNNAMED"
-{% if ozone_opts | default('-XX:ParallelGCThreads=8') | length > 0 %}
-# Extra JVM options for all Ozone components
-export OZONE_OPTS="{{ ozone_opts | default('-XX:ParallelGCThreads=8') }}"
-{% endif %}
+export OZONE_OPTS="{{ ozone_opts | default('-Xmx1024m
-XX:ParallelGCThreads=8') }}"
export OZONE_OM_USER="{{ service_user }}"
diff --git a/roles/ozone_layout/tasks/main.yml
b/roles/ozone_layout/tasks/main.yml
index 6487765..8f1aa66 100644
--- a/roles/ozone_layout/tasks/main.yml
+++ b/roles/ozone_layout/tasks/main.yml
@@ -29,12 +29,12 @@
}}
become: true
-- name: "Ensure OZONE_HOME and PATH are in profile.d/ozone.sh"
+- name: "Ensure OZONE_HOME and PATH in service user shell config"
blockinfile:
- path: "/etc/profile.d/ozone.sh"
+ path: "{{ (service_user == 'root') | ternary('/root', '/home/' +
service_user) }}/.bashrc"
create: true
- owner: root
- group: root
+ owner: "{{ service_user }}"
+ group: "{{ service_group }}"
mode: "0644"
marker: "# {mark} {{ ENV_MARKER }}"
block: |
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
