Ashray Jain created SPARK-35443:
-----------------------------------
Summary: Mark K8s secrets and config maps as immutable
Key: SPARK-35443
URL: https://issues.apache.org/jira/browse/SPARK-35443
Project: Spark
Issue Type: Improvement
Components: Kubernetes
Affects Versions: 3.1.1
Reporter: Ashray Jain
Kubernetes supports marking secrets and config maps as immutable to gain
performance.
[https://kubernetes.io/docs/concepts/configuration/configmap/#configmap-immutable]
[https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable]
For K8s clusters that run many thousands of Spark applications, this can yield
significant reduction in load on the kube-apiserver.
>From the K8s docs:
{quote}For clusters that extensively use Secrets (at least tens of thousands of
unique Secret to Pod mounts), preventing changes to their data has the
following advantages:
* protects you from accidental (or unwanted) updates that could cause
applications outages
* improves performance of your cluster by significantly reducing load on
kube-apiserver, by closing watches for secrets marked as immutable.{quote}
For any secrets and config maps we create in Spark that are immutable, we could
mark them as immutable by including the following when building the
secret/config map
{code:java}
.withImmutable(true)
{code}
This feature has been supported in K8s as beta since K8s 1.19 and as GA since
K8s 1.21
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
