asf-tooling commented on issue #245:
URL:
https://github.com/apache/tooling-trusted-releases/issues/245#issuecomment-4410414530
<!-- gofannon-issue-triage-bot v2 -->
**Automated triage** — analyzed at `main@2da7807a`
**Type:** `discussion` • **Classification:** `no_action` •
**Confidence:** `medium`
**Application domain(s):** `shared_infrastructure`, `web_api_infrastructure`
### Summary
This issue requests integrating ATR secrets with Puppet's eyaml mechanism
instead of manually managing secrets.ini. The discussion spans configuration
format (INI vs YAML), mounting strategy (file vs env vars), and hot-reload
capability. @dave2wave's most recent comment (2026-02-16) assumes writing eyaml
configuration through to `secrets/curated/secrets.ini` and asks about a
dedicated ATR SVN ID. Key open questions remain: whether to switch to YAML
format, what constitutes a 'secret' vs. a configuration value, and how to
implement hot-reload of secrets at runtime.
### Where this lives in the code today
#### `atr/config.py` — `_config_secrets` (lines 29-40)
_needs modification_
This is the core secrets-loading function that reads from secrets.ini and
falls back to env vars; it would need modification for YAML support or
hot-reload.
```python
def _config_secrets(key: str, state_dir: str, default: str | None = None,
cast: type = str) -> str | None:
secrets_path = os.path.join(state_dir, "secrets", "curated",
"secrets.ini")
# This code is deprecated and will be removed
# TODO: Remove this once migrations are no longer likely2026-
deprecated_path = os.path.join(state_dir, "secrets.ini")
if os.path.exists(deprecated_path):
if os.path.exists(secrets_path):
raise RuntimeError(f"Conflicting secrets files exist:
{deprecated_path} and {secrets_path}")
return _config_secrets_get(deprecated_path, key, default, cast,
allow_not_found=False)
return _config_secrets_get(secrets_path, key, default, cast)
```
#### `atr/config.py` — `_config_secrets_get` (lines 43-68)
_needs modification_
This function uses decouple.RepositoryIni to parse INI format; would need to
be replaced or augmented with YAML parsing if format changes.
```python
def _config_secrets_get(
secrets_path: str, key: str, default: str | None = None, cast: type =
str, allow_not_found: bool = True
) -> str | None:
try:
repo_ini = decouple.RepositoryIni(secrets_path)
except FileNotFoundError:
if allow_not_found is False:
raise
else:
try:
value = repo_ini[key]
return cast(value)
except KeyError:
pass
# There is no secrets file, or it does not contain the key
# Try getting the value from environment variables
sentinel = object()
# We do not use the cast keyword argument here
# If we did, it would also be applied to the default sentinel value
value = decouple.config(key, default=sentinel)
if value is sentinel:
return default
if not isinstance(value, str):
raise ValueError(f"Secret value for {key} is not a string")
return cast(value)
```
#### `atr/config.py` — `AppConfig` (lines 71-90)
_needs modification_
Secrets are resolved at class definition time (module import), making
hot-reload impossible without restructuring to lazy evaluation or periodic
refresh.
```python
class AppConfig:
ACCOUNT_CHECK_INTERVAL = decouple.config("ACCOUNT_CHECK_INTERVAL",
default=300, cast=int)
ATR_STATUS = decouple.config("ATR_STATUS", default="ALPHA", cast=str)
DISABLE_CHECK_CACHE = decouple.config("DISABLE_CHECK_CACHE",
default=False, cast=bool)
APP_HOST = decouple.config("APP_HOST", default="127.0.0.1")
SSH_HOST = decouple.config("SSH_HOST", default="0.0.0.0")
SSH_PORT = decouple.config("SSH_PORT", default=2222, cast=int)
PROJECT_ROOT =
os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
STATE_DIR = decouple.config("STATE_DIR",
default=os.path.join(PROJECT_ROOT, "state"))
LDAP_BIND_DN = _config_secrets("LDAP_BIND_DN", STATE_DIR, default=None,
cast=str)
LDAP_BIND_PASSWORD = _config_secrets("LDAP_BIND_PASSWORD", STATE_DIR,
default=None, cast=str)
LOG_LEVEL = decouple.config("LOG_LEVEL", default="INFO", cast=lambda x:
x.upper())
LOG_JSON = decouple.config("LOG_JSON", default=False, cast=bool)
LOG_PUBLIC_KEY = _config_secrets("LOG_PUBLIC_KEY", STATE_DIR,
default=None, cast=str)
MAX_SESSION_AGE = decouple.config("MAX_SESSION_AGE", default=60 * 60 *
72, cast=int)
PUBSUB_URL = _config_secrets("PUBSUB_URL", STATE_DIR, default=None,
cast=str)
PUBSUB_USER = _config_secrets("PUBSUB_USER", STATE_DIR, default=None,
cast=str)
PUBSUB_PASSWORD = _config_secrets("PUBSUB_PASSWORD", STATE_DIR,
default=None, cast=str)
SVN_TOKEN = _config_secrets("SVN_TOKEN", STATE_DIR, default=None,
cast=str)
GITHUB_TOKEN = _config_secrets("GITHUB_TOKEN", STATE_DIR, default=None,
cast=str)
```
#### `atr/server.py` — `_app_dirs_setup` (lines 168-186)
_currently does this_
Creates the secrets/curated directory at startup, which is where Puppet
would mount or write the secrets file.
```python
def _app_dirs_setup(state_dir_str: str, hot_reload: bool) -> None:
"""Setup application directories."""
if not os.path.isdir(state_dir_str):
raise RuntimeError(f"State directory not found: {state_dir_str}")
os.chdir(state_dir_str)
if hot_reload is False:
print(f"Working directory changed to: {os.getcwd()}")
# Note that the hypercorn directories are not managed by ATR
directories_to_ensure = [
pathlib.Path(state_dir_str) / "audit",
pathlib.Path(state_dir_str) / "cache",
pathlib.Path(state_dir_str) / "database",
pathlib.Path(state_dir_str) / "hypercorn" / "logs",
pathlib.Path(state_dir_str) / "hypercorn" / "secrets",
pathlib.Path(state_dir_str) / "logs",
pathlib.Path(state_dir_str) / "runtime",
pathlib.Path(state_dir_str) / "secrets" / "curated",
pathlib.Path(state_dir_str) / "secrets" / "generated",
```
### Where new code would go
- `atr/config.py` — after symbol _config_secrets_get
A YAML-based secrets reader function would be added here if the team
decides to switch from INI to YAML format, or a file-watching/reload mechanism
would be added.
### Proposed approach
Based on the discussion, the current direction is for Puppet's eyaml
mechanism to decrypt secrets and write them to `secrets/curated/secrets.ini`
(or a YAML equivalent) which is then mounted into the container. The main code
changes needed are: (1) Potentially switching from INI to YAML parsing in
`_config_secrets_get` using pyyaml (already a transitive dependency), (2)
Implementing hot-reload of secrets so changes to the mounted file take effect
without container restart, as @dave2wave requested. The hot-reload is the
harder part since `AppConfig` class attributes are evaluated at import time.
However, several design questions remain unresolved: whether to keep INI
format (Puppet writes decrypted values to INI) or switch to YAML, what the
policy boundary is for what constitutes a 'secret', whether there's a dedicated
ASF SVN ID for ATR, and the exact Puppet integration pattern to follow
(agenda_server was suggested as a model). No diff is proposed because the
discussion is still converging on a design.
### Open questions
- Should the secrets file format change from INI to YAML, or should Puppet
decrypt eyaml and write INI format to secrets/curated/secrets.ini?
- How should hot-reload of secrets be implemented given that AppConfig class
attributes are evaluated at module import time? (Periodic polling, file
watcher, signal-based?)
- What is the definitive list of values that belong in secrets vs. regular
configuration? (@sbp raised this distinction in the Nov 4 comment)
- Does ATR have a dedicated SVN service account ID? (@dave2wave asked this
in the most recent comment with no answer yet)
- Should the agenda_server Puppet pattern (referenced by @dave2wave) be
followed exactly, or adapted?
_The agent reviewed this issue and is not proposing patches in this run.
Review the existing-code citations and open questions above before deciding
next steps._
### Files examined
- `atr/config.py`
- `atr/server.py`
- `atr/db/__init__.py`
- `atr/storage/__init__.py`
- `atr/models/sql.py`
- `atr/util.py`
- `atr/principal.py`
- `atr/noisy.py`
---
*Draft from a triage agent. A human reviewer should validate before merging
any change. The agent did not run tests or verify diffs apply.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
