bitflicker64 commented on code in PR #3449:
URL: https://github.com/apache/hugegraph/pull/3449#discussion_r4458407777
##########
helm/hugegraph/templates/hubble-deployment.yaml:
##########
@@ -286,6 +296,8 @@ spec:
value: {{ if $vermeerEnabled }}{{ printf "http://%s:%v" (include
"hugegraph.vermeer.masterName" .) .Values.vermeer.master.httpPort | quote }}{{
else }}""{{ end }}
- name: HG_HUBBLE_VERMEER_TOKEN_FILE
value: {{ if $vermeerEnabled
}}"/var/run/hugegraph/vermeer/token"{{ else }}""{{ end }}
+ - name: HG_HUBBLE_TEXT2GQL_MASTER_KEY_FILE
+ value: "/hubble-data/text2gql-master.key"
Review Comment:
**`fsGroup` can make the relocated key unreadable after Pod replacement**
Moving the key onto the `/hubble-data` volume puts it under kubelet's
volume-ownership handling. `values.yaml` tells non-root deployments with
persistence to set `hubble.podSecurityContext.fsGroup`. With the default
`fsGroupChangePolicy: Always`, kubelet recursively adds group read/write
(`0660`) to files on supported volumes each time the volume is mounted.
Hubble creates the key as `0600`, and
`Text2GqlSecretStore.secureAttributes()` rejects any group bit (`allowed =
{OWNER_READ, OWNER_WRITE}`). So the first save works. After the next Pod
replacement the key becomes `rw-rw----`, and both decrypt and encrypt return
503 `secret-unavailable`, even though the PVC and the UID were kept. Before
this PR the key lived in the container filesystem, so `fsGroup` never touched
it.
Suggested fix: restore owner-only mode in the startup wrapper before Hubble
starts, for example `[[ -f "${HG_HUBBLE_TEXT2GQL_MASTER_KEY_FILE}" ]] && chmod
600 "${HG_HUBBLE_TEXT2GQL_MASTER_KEY_FILE}"`. Alternatively, document
`fsGroupChangePolicy: OnRootMismatch`. Please add a test or a manual check for
the `fsGroup` case.
Note: this comes from reading the source and the documented kubelet
behaviour. It was not run on a real cluster, and CSI drivers that manage
permissions themselves may behave differently.
##########
helm/hugegraph/templates/hubble-deployment.yaml:
##########
@@ -267,6 +272,11 @@ spec:
if [[ "${FOUND_SERVER_HOST}" == false ]]; then
printf 'server.host=0.0.0.0\n' >>"${TMP}"
fi
+ # The image default (data/text2gql-master.key, relative to
/hubble)
+ # is outside the data volume; keep the key next to the H2
database.
Review Comment:
**Document the one-time API key re-entry on upgrade**
For an existing release, the ciphertext in H2 was encrypted with a key in
the old container filesystem, at `/hubble/data/text2gql-master.key`. That key
cannot be migrated and is gone once the upgrade replaces the Pod. Hubble does
not create a key at startup or on decrypt (`masterKey(false)` throws
`unavailable()`). So the UI keeps showing a saved key, but generation fails
with 503. Saving other Text2GQL settings without re-entering the key also
fails, because the save path decrypts the stored value.
Losing the key on Pod replacement was already the behaviour before this PR.
However, this PR changes deployment behaviour, and the repository's `AGENTS.md`
asks for matching docs in the same PR. The PR description also says nothing
operational changes. Please add a short note to the Hubble section or
`docs/operations.md`: after upgrading, re-enter the Text2GQL API key once, and
from then on it survives Pod replacement while persistence is enabled.
Optionally, also mention that the key does not survive Pod replacement with
`persistence.enabled=false`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]