ruanwenjun opened a new pull request, #7757:
URL: https://github.com/apache/kyuubi/pull/7757
### Why are the changes needed?
Closes #7756.
In multi-server Kubernetes deployments, each Kyuubi server currently
receives events for every Kyuubi Spark Driver Pod in the namespace. When Spark
UI URLs use Services, each server also receives all Service events in that
namespace. This duplicates API server watch delivery and client-side informer
processing across replicas.
This patch adds the optional
`kyuubi.kubernetes.application.ownerScopedWatch.enabled` configuration. When
enabled, Kyuubi derives a stable owner label value from the server hostname,
adds it to Spark Driver Pods and Driver Services, and filters both informers by
that owner. The option is disabled by default. Deployments enabling it must
provide hostnames that are unique across servers and stable across restarts,
such as StatefulSet Pod hostnames.
The behavior applies consistently to Spark SQL engines and batch
applications submitted by Kyuubi.
### How was this patch tested?
- Added tests for stable owner label generation.
- Added tests for enabled and disabled owner label injection.
- Added tests for owner-scoped informer selection.
- Added Spark SQL engine and batch command-generation coverage.
- Ran:
```bash
build/mvn test -pl kyuubi-server -am \
-Pflink-provided,spark-provided,hive-provided \
-Dtest=none \
-DwildcardSuites=org.apache.kyuubi.engine.KubernetesApplicationOperationSuite,org.apache.kyuubi.engine.KyuubiApplicationManagerSuite
build/mvn spotless:check \
-pl kyuubi-common,kyuubi-server -am \
-Pflink-provided,hive-provided,spark-provided
KYUUBI_UPDATE=0 dev/gen/gen_all_config_docs.sh
build/mvn -Pfast package -pl kyuubi-server -am -DskipTests
```
The full Spark process-builder suites were compiled but could not be
executed locally because `SPARK_HOME` is not available in the environment.
### Was this patch assisted by generative AI tooling?
Assisted-by: Command Code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
