miaht94 opened a new issue, #7293: URL: https://github.com/apache/kyuubi/issues/7293
### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug Hi Kyuubi Team, I'm running Kyuubi on k8s and setting `kyuubi.kubernetes.spark.cleanupTerminatedDriverPod.kind=ALL` in Kyuubi Server but it seems not work as expected. After reviewing the source code in version 1.10.2, specifically the Kubernetes informer initialization logic [here](https://github.com/apache/kyuubi/blob/v1.10.2/kyuubi-server/src/main/scala/org/apache/kyuubi/engine/KubernetesApplicationOperation.scala#L186) , I noticed that the informer is only initialized when configurations like the following are provided: ``` kyuubi.kubernetes.client.initialize.list=context1:namespace1 kyuubi.kubernetes.authenticate.context1.oauthToken= kyuubi.kubernetes.authenticate.context1.caCertFile= ... ``` According to the documentation, `kyuubi.kubernetes.client.initialize.list` is only documented as available starting from 1.11.0, yet the corresponding initialization code already exists in 1.10.2. Additionally, this setup feels unnecessarily complex for cases where Kyuubi is running inside Kubernetes and uses the ServiceAccount credentials that was attached to Pod. In such scenarios, users would expect Kyuubi to automatically initialize the Kubernetes informer based on the in-cluster configuration. However, without explicitly setting `kyuubi.kubernetes.client.initialize.list`, no informer is created, which prevents features like terminated driver pod cleanup from working. Could you please clarify: - Whether this behavior is intended in 1.10.2 ? - Kyuubi is expected to work without explicitly setting `kyuubi.kubernetes.client.initialize.list` (using CA cert, token from pod filesystem) ? ### Affects Version(s) 1.10.2 ### Kyuubi Server Log Output ```logtalk ``` ### Kyuubi Engine Log Output ```logtalk ``` ### Kyuubi Server Configurations ```yaml kyuubi.server.administrators=admin # Kyuubi Metrics kyuubi.metrics.enabled=true kyuubi.metrics.reporters=PROMETHEUS kyuubi.metrics.prometheus.port=10019 ## User provided Kyuubi configurations kyuubi.kubernetes.namespace=kyuubi kyuubi.kubernetes.spark.cleanupTerminatedDriverPod.kind=ALL kyuubi.kubernetes.terminatedApplicationRetainPeriod=PT1M kyuubi.ha.addresses=zookeeper.zookeeper.svc:2181 kyuubi.ha.namespace=kyuubi kyuubi.ha.zookeeper.auth.digest=admin:password kyuubi.frontend.rest.bind.host=0.0.0.0 kyuubi.frontend.connection.url.use.hostname=false kyuubi.frontend.thrift.binary.bind.port=10009 kyuubi.frontend.thrift.http.bind.port=10010 kyuubi.frontend.rest.bind.port=10099 kyuubi.frontend.mysql.bind.port=3309 kyuubi.frontend.protocols=MYSQL,REST,THRIFT_BINARY ## User provided Kyuubi configurations kyuubi.engine.pool.size=-1 kyuubi.engine.share.level=CONNECTION kyuubi.session.close.on.disconnect=true kyuubi.engine.kubernetes.submit.timeout=PT90S kyuubi.session.engine.spark.main.resource=local:///opt/kyuubi/externals/engines/spark/kyuubi-spark-sql-engine_2.12-1.10.2.jar ``` ### Kyuubi Engine Configurations ```yaml ``` ### Additional context _No response_ ### Are you willing to submit PR? - [x] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. - [ ] No. I cannot submit a PR at this time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
