This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git
commit c4b9dbe53aa585fc8184913412414655a3353fa2 Author: Jacob Ferriero <[email protected]> AuthorDate: Tue Sep 15 03:39:39 2020 -0700 Proposal: remove -serviceaccount suffix from KSA names in helm chart (#10892) * [WIP] remove -serviceaccount suffix in helm chart It's quite annoying to have `-serviceaccount` in each service account name as this is a useless 15 characters that provides no additional information. "why is this so frustrating to you Jake?" GCP service accounts have 30 char name limit https://cloud.google.com/iam/docs/creating-managing-service-accounts#creating For manageability / clarity I'd like to keep KSA and GSA names exactly the same when using workload identity which maps KSA<>GSA 1:1 https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity. (cherry picked from commit 23768f694697be1b29ec00b8deeed5777d2538e8) --- chart/README.md | 10 ++++++++-- chart/templates/cleanup/cleanup-cronjob.yaml | 2 +- chart/templates/cleanup/cleanup-serviceaccount.yaml | 2 +- chart/templates/rbac/pod-cleanup-rolebinding.yaml | 2 +- chart/templates/rbac/pod-launcher-rolebinding.yaml | 4 ++-- chart/templates/scheduler/scheduler-deployment.yaml | 4 ++-- chart/templates/scheduler/scheduler-serviceaccount.yaml | 2 +- chart/templates/workers/worker-deployment.yaml | 2 +- chart/templates/workers/worker-serviceaccount.yaml | 2 +- 9 files changed, 18 insertions(+), 12 deletions(-) diff --git a/chart/README.md b/chart/README.md index 1ae5d6c..d56f114 100644 --- a/chart/README.md +++ b/chart/README.md @@ -35,6 +35,7 @@ cluster using the [Helm](https://helm.sh) package manager. ## Installing the Chart To install this repository from source (using helm 3) + ```bash kubectl create namespace airflow helm repo add stable https://charts.helm.sh/stable/ @@ -48,6 +49,7 @@ section lists the parameters that can be configured during installation. > **Tip**: List all releases using `helm list` ## Upgrading the Chart + To upgrade the chart with the release name `airflow`: ```bash @@ -90,6 +92,7 @@ helm upgrade airflow . \ ``` ## Mounting DAGS using Git-Sync side car without Persistence + This option will use an always running Git-Sync side car on every scheduler,webserver and worker pods. The Git-Sync side car containers will sync DAGs from a git repository every configured number of seconds. If you are using the KubernetesExecutor, Git-sync will run as an initContainer on your worker pods. ```bash @@ -102,6 +105,7 @@ helm upgrade airflow . \ ``` ## Mounting DAGS from an externally populated PVC + In this approach, Airflow will read the DAGs from a PVC which has `ReadOnlyMany` or `ReadWriteMany` accessMode. You will have to ensure that the PVC is populated/updated with the required DAGs(this won't be handled by the chart). You can pass in the name of the volume claim to the chart ```bash @@ -213,12 +217,14 @@ The following tables lists the configurable parameters of the Airflow chart and | `webserver.resources.limits.memory` | Memory Limit of webserver | `~` | | `webserver.resources.requests.cpu` | CPU Request of webserver | `~` | | `webserver.resources.requests.memory` | Memory Request of webserver | `~` | +| `webserver.service.annotations` | Annotations to be added to the webserver service | `{}` | | `webserver.defaultUser` | Optional default airflow user information | `{}` | -| `dags.persistence.*` | Dag persistence configuration | Please refer to `values.yaml` | -| `dags.gitSync.*` | Git sync configuration | Please refer to `values.yaml` | +| `dags.persistence.*` | Dag persistence configuration | Please refer to `values.yaml` | +| `dags.gitSync.*` | Git sync configuration | Please refer to `values.yaml` | | `multiNamespaceMode` | Whether the KubernetesExecutor can launch pods in multiple namespaces | `False` | | `serviceAccountAnnottions.*` | Map of annotations for worker, webserver, scheduler kubernetes service accounts | {} | + Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example, ```bash diff --git a/chart/templates/cleanup/cleanup-cronjob.yaml b/chart/templates/cleanup/cleanup-cronjob.yaml index 3ad3d4f..fd253d2 100644 --- a/chart/templates/cleanup/cleanup-cronjob.yaml +++ b/chart/templates/cleanup/cleanup-cronjob.yaml @@ -51,7 +51,7 @@ spec: {{ toYaml .Values.affinity | indent 12 }} tolerations: {{ toYaml .Values.tolerations | indent 12 }} - serviceAccountName: {{ .Release.Name }}-cleanup-serviceaccount + serviceAccountName: {{ .Release.Name }}-cleanup {{- if or .Values.registry.secretName .Values.registry.connection }} imagePullSecrets: - name: {{ template "registry_secret" . }} diff --git a/chart/templates/cleanup/cleanup-serviceaccount.yaml b/chart/templates/cleanup/cleanup-serviceaccount.yaml index 769cbdc..6ef3aec 100644 --- a/chart/templates/cleanup/cleanup-serviceaccount.yaml +++ b/chart/templates/cleanup/cleanup-serviceaccount.yaml @@ -22,7 +22,7 @@ kind: ServiceAccount apiVersion: v1 metadata: - name: {{ .Release.Name }}-cleanup-serviceaccount + name: {{ .Release.Name }}-cleanup labels: tier: airflow release: {{ .Release.Name }} diff --git a/chart/templates/rbac/pod-cleanup-rolebinding.yaml b/chart/templates/rbac/pod-cleanup-rolebinding.yaml index 4c2dd25..0d09b87 100644 --- a/chart/templates/rbac/pod-cleanup-rolebinding.yaml +++ b/chart/templates/rbac/pod-cleanup-rolebinding.yaml @@ -37,6 +37,6 @@ roleRef: name: {{ .Release.Name }}-cleanup-role subjects: - kind: ServiceAccount - name: {{ .Release.Name }}-cleanup-serviceaccount + name: {{ .Release.Name }}-cleanup namespace: {{ .Release.Namespace }} {{- end }} diff --git a/chart/templates/rbac/pod-launcher-rolebinding.yaml b/chart/templates/rbac/pod-launcher-rolebinding.yaml index f258de7..849f32d 100644 --- a/chart/templates/rbac/pod-launcher-rolebinding.yaml +++ b/chart/templates/rbac/pod-launcher-rolebinding.yaml @@ -51,12 +51,12 @@ roleRef: subjects: {{- if $grantScheduler }} - kind: ServiceAccount - name: {{ .Release.Name }}-scheduler-serviceaccount + name: {{ .Release.Name }}-scheduler namespace: {{ .Release.Namespace }} {{- end }} {{- if $grantWorker }} - kind: ServiceAccount - name: {{ .Release.Name }}-worker-serviceaccount + name: {{ .Release.Name }}-worker namespace: {{ .Release.Namespace }} {{- end }} {{- end }} diff --git a/chart/templates/scheduler/scheduler-deployment.yaml b/chart/templates/scheduler/scheduler-deployment.yaml index 771dd1a..619a16c1 100644 --- a/chart/templates/scheduler/scheduler-deployment.yaml +++ b/chart/templates/scheduler/scheduler-deployment.yaml @@ -19,7 +19,7 @@ ## Airflow Scheduler Deployment/StatefulSet ################################# -# Are we using a local/sequenial executor? +# Are we using a local/sequential executor? {{- $local := or (eq .Values.executor "LocalExecutor") (eq .Values.executor "SequentialExecutor") }} # Is persistence enabled on the _workers_? # This is important because in $local mode, the scheduler assumes the role of the worker @@ -81,7 +81,7 @@ spec: {{ toYaml .Values.tolerations | indent 8 }} restartPolicy: Always terminationGracePeriodSeconds: 10 - serviceAccountName: {{ .Release.Name }}-scheduler-serviceaccount + serviceAccountName: {{ .Release.Name }}-scheduler securityContext: runAsUser: {{ .Values.uid }} fsGroup: {{ .Values.gid }} diff --git a/chart/templates/scheduler/scheduler-serviceaccount.yaml b/chart/templates/scheduler/scheduler-serviceaccount.yaml index f12991d..2d61c76 100644 --- a/chart/templates/scheduler/scheduler-serviceaccount.yaml +++ b/chart/templates/scheduler/scheduler-serviceaccount.yaml @@ -22,7 +22,7 @@ kind: ServiceAccount apiVersion: v1 metadata: - name: {{ .Release.Name }}-scheduler-serviceaccount + name: {{ .Release.Name }}-scheduler labels: tier: airflow release: {{ .Release.Name }} diff --git a/chart/templates/workers/worker-deployment.yaml b/chart/templates/workers/worker-deployment.yaml index 53998da..77d5fe2 100644 --- a/chart/templates/workers/worker-deployment.yaml +++ b/chart/templates/workers/worker-deployment.yaml @@ -72,7 +72,7 @@ spec: {{ toYaml .Values.tolerations | indent 8 }} terminationGracePeriodSeconds: {{ .Values.workers.terminationGracePeriodSeconds }} restartPolicy: Always - serviceAccountName: {{ .Release.Name }}-worker-serviceaccount + serviceAccountName: {{ .Release.Name }}-worker securityContext: runAsUser: {{ .Values.uid }} fsGroup: {{ .Values.gid }} diff --git a/chart/templates/workers/worker-serviceaccount.yaml b/chart/templates/workers/worker-serviceaccount.yaml index f87751b..d7ed927 100644 --- a/chart/templates/workers/worker-serviceaccount.yaml +++ b/chart/templates/workers/worker-serviceaccount.yaml @@ -22,7 +22,7 @@ kind: ServiceAccount apiVersion: v1 metadata: - name: {{ .Release.Name }}-worker-serviceaccount + name: {{ .Release.Name }}-worker labels: tier: airflow release: {{ .Release.Name }}
