This is an automated email from the ASF dual-hosted git repository.
villebro pushed a commit to branch main
in repository
https://gitbox.apache.org/repos/asf/superset-kubernetes-operator.git
The following commit(s) were added to refs/heads/main by this push:
new 7989084 refactor(config): mount superset_config.py at /app/pythonpath
(#29)
7989084 is described below
commit 7989084f5effe63201889079f13948fc348ae0cf
Author: Ville Brofeldt <[email protected]>
AuthorDate: Fri May 8 10:14:49 2026 -0700
refactor(config): mount superset_config.py at /app/pythonpath (#29)
* refactor(config): mount superset_config.py at /app/pythonpath
* fix docs references
---
AGENTS.md | 4 ++--
docs/architecture.md | 2 +-
docs/internals.md | 2 +-
internal/common/names.go | 5 +----
internal/controller/config_test.go | 5 +----
internal/controller/reconcile_test.go | 16 ++--------------
internal/controller/superset_controller.go | 5 -----
internal/resolution/resolver_test.go | 14 +++++++-------
8 files changed, 15 insertions(+), 38 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
index 2a49870..9faabde 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -105,7 +105,7 @@ The operator uses a **two-tier CRD architecture** where the
parent `Superset` re
- **Celery worker configuration**: `spec.celeryWorker.celery` controls Celery
worker command args. Presets set concurrency and pool. Operator constructs the
`celery worker` command from resolved fields. `disabled` preset uses the
hardcoded fallback command.
- **SQLAlchemy engine options**: `spec.sqlaEngineOptions` sets the baseline;
per-component `sqlaEngineOptions` on webServer, celeryWorker, celeryBeat,
mcpServer, lifecycle tasks replaces the top-level entirely (override
semantics). Presets: `disabled` (no rendering), `conservative` (NullPool),
`balanced` (pool_size=1, max_overflow=-1), `performance` (pool_size=workers),
`aggressive` (pool_size=workers×threads). CeleryBeat and lifecycle tasks always
default to NullPool. Pool sizing is com [...]
- **Environment modes**: `environment: dev` allows inline `secretKey`,
`metastore.uri`, `metastore.password`, `valkey.password`,
`lifecycle.adminUser`, and `lifecycle.loadExamples`. `environment: prod`
(default) rejects these via CRD validation; use `secretKeyFrom`,
`metastore.uriFrom`, `metastore.passwordFrom`, or `valkey.passwordFrom` to
reference Kubernetes Secrets (operator injects `valueFrom.secretKeyRef` env
vars).
-- **Env var tiers**: Operator-internal transport vars
(`SUPERSET_OPERATOR__SECRET_KEY`, `SUPERSET_OPERATOR__DB_URI`,
`SUPERSET_OPERATOR__DB_HOST`, `SUPERSET_OPERATOR__VALKEY_HOST`,
`SUPERSET_OPERATOR__FORCE_RELOAD`, etc.) and standard env vars (`PYTHONPATH`).
+- **Env var tiers**: Operator-internal transport vars
(`SUPERSET_OPERATOR__SECRET_KEY`, `SUPERSET_OPERATOR__DB_URI`,
`SUPERSET_OPERATOR__DB_HOST`, `SUPERSET_OPERATOR__VALKEY_HOST`,
`SUPERSET_OPERATOR__FORCE_RELOAD`, etc.).
- **SECRET_KEY validation**: CEL requires either `secretKey` (dev mode) or
`secretKeyFrom` (any mode) to be set.
- **Deployment builder**: All child controllers use `buildDeploymentSpec()`
with flat `FlatComponentSpec`. Reads all fields from the merged
`DeploymentTemplate` hierarchy. No parent lookup needed.
- **Generic child reconciler**: 6 child controllers (all except
SupersetLifecycleTask) use a shared `ChildReconciler` with a `ChildCR`
interface. Each child CRD type implements accessor methods (`GetFlatSpec`,
`GetConfig`, `GetService`, etc.).
@@ -120,7 +120,7 @@ The operator uses a **two-tier CRD architecture** where the
parent `Superset` re
- **Lifecycle tasks**: `spec.lifecycle` on the parent CRD (type
`LifecycleSpec`) defines two sequential tasks: "migrate" (`superset db
upgrade`) and "init" (`superset init`). Each produces a `SupersetLifecycleTask`
child CR named `{parentName}-migrate` and `{parentName}-init`. Tasks run as
bare Pods (restartPolicy: Never) with exponential backoff on failure. Each task
has an independent `strategy`: `VersionChange` (default, runs only on image
changes), `Always` (runs every reconcile), or [...]
- **CRD validation**: All validation uses CEL (`x-kubernetes-validations`) on
CRD types — no admission webhooks. Rules cover: environment mode restrictions,
secret mutual exclusivity, metastore/valkey validation, networking constraints,
monitoring constraints. Defaults (repository, pullPolicy, environment) use
kubebuilder default markers.
- **Metrics**: Operator exposes controller-runtime default metrics (reconcile
counts, durations, leader election) on HTTPS :8443 with Kubernetes auth/authz.
No custom metrics — controller-runtime defaults are sufficient. Superset
instance monitoring via optional `spec.monitoring.serviceMonitor` (creates a
Prometheus ServiceMonitor targeting the web-server component using unstructured
objects; gracefully skips if CRD is absent).
-- **Config mount path**: `/app/superset/config` for superset_config.py.
+- **Config mount path**: `/app/pythonpath` for superset_config.py.
- **All Go files must have the Apache 2.0 copyright header** (see
`hack/boilerplate.go.txt`)
## Naming Conventions
diff --git a/docs/architecture.md b/docs/architecture.md
index ff2e5a9..3bef2cf 100644
--- a/docs/architecture.md
+++ b/docs/architecture.md
@@ -276,7 +276,7 @@ never appear in ConfigMaps or CRD status fields.
### Config Mount Structure
-- `/app/superset/config/` — ConfigMap with `superset_config.py`
+- `/app/pythonpath/` — ConfigMap with `superset_config.py`
---
diff --git a/docs/internals.md b/docs/internals.md
index 3b28d7f..eccca5a 100644
--- a/docs/internals.md
+++ b/docs/internals.md
@@ -278,7 +278,7 @@ the standard Kubernetes mechanism for projecting files into
containers:
The rendered config is already stored on the child CR's `spec.config` field, so
the ConfigMap is technically a derived resource. The child controller creates
it
-from the spec and mounts it at `/app/superset/config/`.
+from the spec and mounts it at `/app/pythonpath/`.
### What Each Component Creates
diff --git a/internal/common/names.go b/internal/common/names.go
index c7b814e..cb3697f 100644
--- a/internal/common/names.go
+++ b/internal/common/names.go
@@ -76,7 +76,7 @@ const (
// Config paths.
const (
ConfigVolumeName = "superset-config"
- ConfigMountPath = "/app/superset/config"
+ ConfigMountPath = "/app/pythonpath"
)
// Init task names.
@@ -110,9 +110,6 @@ const (
EnvAdminFirstName = "SUPERSET_OPERATOR__ADMIN_FIRST_NAME"
EnvAdminLastName = "SUPERSET_OPERATOR__ADMIN_LAST_NAME"
EnvAdminEmail = "SUPERSET_OPERATOR__ADMIN_EMAIL"
-
- // Standard env vars.
- EnvPythonPath = "PYTHONPATH"
)
// ChildName constructs a child resource name from parent name and suffix.
diff --git a/internal/controller/config_test.go
b/internal/controller/config_test.go
index fe77faf..8d7543b 100644
--- a/internal/controller/config_test.go
+++ b/internal/controller/config_test.go
@@ -282,9 +282,6 @@ func TestBuildOperatorInjected(t *testing.T) {
}
envMap := envSliceToMap(injected.Env)
- if envMap["PYTHONPATH"] == "" {
- t.Error("expected PYTHONPATH env var")
- }
if envMap["SUPERSET_OPERATOR__SECRET_KEY"] != "test" {
t.Errorf("expected SUPERSET_OPERATOR__SECRET_KEY=test from
configEnvVars, got %s", envMap["SUPERSET_OPERATOR__SECRET_KEY"])
}
@@ -292,7 +289,7 @@ func TestBuildOperatorInjected(t *testing.T) {
t.Errorf("expected SUPERSET_OPERATOR__FORCE_RELOAD=v2, got %s",
envMap["SUPERSET_OPERATOR__FORCE_RELOAD"])
}
- // Empty config: no volumes, no mounts, no PYTHONPATH.
+ // Empty config: no volumes, no mounts.
empty := buildOperatorInjected("", "child", "", nil)
if len(empty.Volumes) != 0 {
t.Errorf("expected 0 volumes for empty config, got %d",
len(empty.Volumes))
diff --git a/internal/controller/reconcile_test.go
b/internal/controller/reconcile_test.go
index 0521f23..06013bb 100644
--- a/internal/controller/reconcile_test.go
+++ b/internal/controller/reconcile_test.go
@@ -444,18 +444,9 @@ func TestReconcile_AllComponents_FullFeatures(t
*testing.T) {
t.Error("celery worker config should contain WORKER_SETTING")
}
- // WebsocketServer: no PYTHONPATH (Node.js).
+ // WebsocketServer: no config volume (Node.js).
wss := &supersetv1alpha1.SupersetWebsocketServer{}
_ = c.Get(ctx, types.NamespacedName{Name: "full", Namespace:
"default"}, wss)
- hasPythonPath := false
- for _, env := range specEnv(wss.Spec.FlatComponentSpec) {
- if env.Name == "PYTHONPATH" {
- hasPythonPath = true
- }
- }
- if hasPythonPath {
- t.Error("websocket server should not have PYTHONPATH env var")
- }
// MCP server: config should contain MCP_SETTING.
ms := &supersetv1alpha1.SupersetMcpServer{}
@@ -464,7 +455,7 @@ func TestReconcile_AllComponents_FullFeatures(t *testing.T)
{
t.Error("mcp server config should contain MCP_SETTING")
}
- // All Python components should have PYTHONPATH and SECRET_KEY env vars.
+ // All Python components should have SECRET_KEY env vars.
cb := &supersetv1alpha1.SupersetCeleryBeat{}
_ = c.Get(ctx, types.NamespacedName{Name: "full", Namespace:
"default"}, cb)
cf := &supersetv1alpha1.SupersetCeleryFlower{}
@@ -486,9 +477,6 @@ func TestReconcile_AllComponents_FullFeatures(t *testing.T)
{
for _, env := range pc.envs {
envMap[env.Name] = env.Value
}
- if envMap["PYTHONPATH"] == "" {
- t.Errorf("%s should have PYTHONPATH env var", pc.name)
- }
if envMap["SUPERSET_OPERATOR__SECRET_KEY"] != "test-secret-key"
{
t.Errorf("%s should have SUPERSET_OPERATOR__SECRET_KEY
env var in dev mode", pc.name)
}
diff --git a/internal/controller/superset_controller.go
b/internal/controller/superset_controller.go
index bc2d259..c48a518 100644
--- a/internal/controller/superset_controller.go
+++ b/internal/controller/superset_controller.go
@@ -1128,11 +1128,6 @@ func buildOperatorInjected(renderedConfig, childName,
forceReload string, config
MountPath: configMountPath,
ReadOnly: true,
})
-
- injected.Env = append(injected.Env, corev1.EnvVar{
- Name: naming.EnvPythonPath,
- Value: configMountPath,
- })
}
// Add config-derived env vars (secret key, metastore fields, etc.).
diff --git a/internal/resolution/resolver_test.go
b/internal/resolution/resolver_test.go
index c2db8b3..635c135 100644
--- a/internal/resolution/resolver_test.go
+++ b/internal/resolution/resolver_test.go
@@ -189,7 +189,7 @@ func TestResolveChildSpec_ComponentMergesWithTopLevel(t
*testing.T) {
"app.kubernetes.io/component": "celery-worker",
}
operator := &OperatorInjected{
- Env: []corev1.EnvVar{{Name: "PYTHONPATH", Value:
"/app/superset/config"}},
+ Env: []corev1.EnvVar{{Name: "SUPERSET_OPERATOR__SECRET_KEY",
Value: "test"}},
}
result := ResolveChildSpec(ComponentCeleryWorker, topLevel, component,
operatorLabels, operator)
@@ -280,8 +280,8 @@ func TestResolveChildSpec_ComponentMergesWithTopLevel(t
*testing.T) {
if envMap["COMP_ONLY"] != "yes" {
t.Error("component COMP_ONLY should be present")
}
- if envMap["PYTHONPATH"] != "/app/superset/config" {
- t.Error("operator PYTHONPATH should be present")
+ if envMap["SUPERSET_OPERATOR__SECRET_KEY"] != "test" {
+ t.Error("operator SUPERSET_OPERATOR__SECRET_KEY should be
present")
}
// Command: from component (no inheritance)
@@ -348,7 +348,7 @@ func TestResolveChildSpec_OperatorInjectedMerged(t
*testing.T) {
},
}},
},
- VolumeMounts: []corev1.VolumeMount{{Name: "config",
MountPath: "/app/superset/config"}},
+ VolumeMounts: []corev1.VolumeMount{{Name: "config",
MountPath: "/app/pythonpath"}},
InitContainers: []corev1.Container{{Name: "op-init", Image:
"init:op"}},
}
@@ -377,7 +377,7 @@ func TestResolveChildSpec_OperatorInjectedMerged(t
*testing.T) {
}
// VolumeMounts: operator injected
- if len(ct.VolumeMounts) != 1 || ct.VolumeMounts[0].MountPath !=
"/app/superset/config" {
+ if len(ct.VolumeMounts) != 1 || ct.VolumeMounts[0].MountPath !=
"/app/pythonpath" {
t.Error("expected operator volume mount")
}
@@ -408,7 +408,7 @@ func TestResolveChildSpec_OperatorVolumesWinOnConflict(t
*testing.T) {
},
}},
},
- VolumeMounts: []corev1.VolumeMount{{Name: "superset-config",
MountPath: "/app/superset/config"}},
+ VolumeMounts: []corev1.VolumeMount{{Name: "superset-config",
MountPath: "/app/pythonpath"}},
}
result := ResolveChildSpec(ComponentWebServer, topLevel, nil, nil,
operator)
@@ -425,7 +425,7 @@ func TestResolveChildSpec_OperatorVolumesWinOnConflict(t
*testing.T) {
if len(ct.VolumeMounts) != 1 {
t.Fatalf("expected 1 volume mount (merged by name), got %d",
len(ct.VolumeMounts))
}
- if ct.VolumeMounts[0].MountPath != "/app/superset/config" {
+ if ct.VolumeMounts[0].MountPath != "/app/pythonpath" {
t.Errorf("expected operator mount path to win, got %s",
ct.VolumeMounts[0].MountPath)
}
}
