HwangDongJun opened a new pull request, #7752:
URL: https://github.com/apache/kyuubi/pull/7752
### Why are the changes needed?
Kyuubi's SparkProcessBuilder.appendPodNameConf() generates a driver pod name
that
preserves the app name when possible, falling back to a short
kyuubi-$engineRefId-driver
name only if the app-name-based name would exceed the 253-char DNS subdomain
limit that
applies to Kubernetes pod names (KubernetesUtils.generateDriverPodName).
However, this generated name is also propagated as the *value* of Kubernetes
labels
(e.g. via spark.kubernetes.driver.label.*, which Spark in turn uses on
executor pods),
and label values are restricted to the stricter 63-char DNS label limit,
independent of
the 253-char pod name limit. A moderately long spark.app.name (well within
the 253-char
budget) can still produce a driver pod name over 63 characters, causing
executor pod
creation to fail with a Kubernetes API 422 error such as:
Pod "..." is invalid: metadata.labels: Invalid value: "...-driver":
must be no more than 63 characters
This adds a DRIVER_POD_NAME_AS_LABEL_MAX_LENGTH (63) check to
generateDriverPodName(),
so it now falls back to the always-safe kyuubi-$engineRefId-driver name
whenever either
the 253-char pod name limit or the 63-char label limit would be violated.
executor pod name prefixes are not affected by this change: they are only
ever used as
a *name prefix* (with the executor hostname already truncated to 63 chars
separately in
Spark's BasicExecutorFeatureStep), not propagated as a label value.
### How was this patch tested?
Added a new test to KubernetesPodNameSuite reproducing a realistic
spark.app.name (derived from Kyuubi's defaultEngineName pattern) that stays
well
within the 253-char pod name budget but previously resolved to an
83-character driver
pod name, confirming it now falls back to the short, always-safe name and
satisfies
the 63-char label length limit.
Ran the full kyuubi-server test suite locally (mvn test -pl kyuubi-server -am
-P spark-3.5) — all 4 tests in KubernetesPodNameSuite pass, including the 3
pre-existing tests (no regression).
### Was this patch assisted by generative AI tooling?
Assisted-by: claude-sonnet-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
