miki3421 opened a new issue, #277: URL: https://github.com/apache/openserverless/issues/277
## Problem When optional monitoring is enabled in OPS 0.9.0, ordinary tenant action pods can reach shared Prometheus, kube-state-metrics and Alertmanager listeners. Prometheus additionally uses a fixed NodePort (30299) with its administrative and lifecycle APIs enabled. kube-state-metrics collects Secret metadata and has cluster-wide Secret list/watch privileges. These are shared administration and observability services. Access by one tenant can affect visibility or availability for other tenants. ## Attacker position and risk | Actor / position | Preconditions | Risk | | --- | --- | --- | | Anonymous Internet client | Only the public application HTTP ingress is exposed | No anonymous monitoring access demonstrated through that ingress | | Client able to reach the node's monitoring NodePort | Network access to port 30299 | Unauthenticated Prometheus queries, administrative operations and lifecycle requests | | Ordinary OPS tenant/developer | Can deploy and invoke an action, without Kubernetes credentials | Reachable internal monitoring APIs: cluster metadata disclosure; Prometheus data/availability impact; Alertmanager alert injection and silence manipulation affecting shared notifications | | Host or Kubernetes administrator | Already controls the cluster | Outside this tenant-isolation boundary | The Alertmanager status output in the pinned 0.25 release redacts password and SecretURL fields. Status output alone is not evidence of plaintext Gmail/Slack credential leakage. ## Expected behavior and proposed correction - Remove the Prometheus NodePort and destructive admin API. Preserve its loopback config-reload path and Kubernetes-authorized administrative port-forward access. - Restrict monitoring ingress with NetworkPolicies. Only same-namespace Prometheus may scrape kube-state-metrics on 8080 or deliver alerts on 9093; Alertmanager gossip is restricted to its peers. Permitted selectors exclude every pod carrying the invoker-controlled `user-action-pod` label. - Remove Secret collection and Secret list/watch from kube-state-metrics. - Apply isolation before workloads, roll legacy pods once, wait for deployment readiness and reconcile on operator resume/configuration update. - Keep policies until protected pods terminate during explicit component deletion. - Apply reduced cluster-scoped monitoring RBAC from the administrator-owned task before updating the operator, without granting new operator escalation privileges. This needs coordinated operator and task PRs targeting `0.9.0`. Component versions, receiver configuration and application code are preserved. ## Validation Existing synthetic K3s baseline, with the local patched operator: - 45/45 live boundary and functional checks passed: tenant Service/pod-IP denial, forged trusted labels, foreign namespace denial, trusted scraping and alert delivery, loopback reload, disabled admin API, removed NodePort, reduced Secret RBAC and administrative port-forward. - Repeated operator resume retained monitored pod identities and readiness. - Alpha/Beta: 15/15 application checks; Redis Lua: 84/84; Streamer: 24/24. Application source unchanged; scheduled heartbeats continued. - Existing Kubernetes resource identities and application source hashes were preserved. - The PR candidate rebased onto the current Apache 0.9.0 base passes 88/88 operator unit tests. - Focused Daybreak diff reviews completed without reportable findings in the reviewed patch. ## Deployment requirements and known limits An enforcing NetworkPolicy provider is required. Standard Kind networking without policy enforcement does not establish this boundary. Administrator-created broad allow policies and compromised permitted control-plane pods are outside the ordinary tenant model. Install matching task/operator changes together. An administrator performing an image-only upgrade must first apply the updated monitoring role. This iteration tested an existing-cluster upgrade and real operator resume, not a new full-cluster reinstall or the exact public update command with a released image. Whole-Whisk owner garbage collection still has no guaranteed policy-last deletion ordering; the existing accepted teardown limitation remains documented. Alertmanager receiver credentials stored in a ConfigMap remain a separate hardening concern for principals already authorized to read cluster configuration. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
