juansgr opened a new issue, #44688:
URL: https://github.com/apache/airflow/issues/44688
### Official Helm Chart version
1.15.0 (latest released)
### Apache Airflow version
2.7.1
### Kubernetes Version
v1.25.12+26bab08
### Helm Chart configuration
values.yaml:
```
-Home: &-home /opt/- -:
conn_-: '{{ .Values.data.metadataConnection.protocol }}://{{
.Values.data.metadataConnection.user
}}:{{ .Values.data.metadataConnection.pass }}@{{
.Values.data.metadataConnection.host
}}:{{ .Values.data.metadataConnection.port }}/-_db'
conn_xcom: '{{ .Values.data.metadataConnection.protocol }}://{{
.Values.data.metadataConnection.user
}}:{{ .Values.data.metadataConnection.pass }}@{{
.Values.data.metadataConnection.host
}}:{{ .Values.data.metadataConnection.port }}/{{
.Values.data.metadataConnection.db
}}'
developSourcesVolume:
enabled: false
jobs:
enabled: &jobs_enabled true
create-DBJob:
enabled: true
ttlSecondsAfterFinished: ~
command: ~
args: ["bash", "-c", "exec ./scripts/-.sh"]
annotations: {}
jobAnnotations: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
resources: {}
extraContainers: []
extraVolumeMounts:
---
defaultMode: 0777
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
useHelmHooks: false
applyCustomEnv: false
createImporterPoolsJob:
enabled: true
ttlSecondsAfterFinished: ~
command: ~
args:
- bash
- -c
- exec airflow pools import /opt/-/config/importer_pools.json
annotations: {}
jobAnnotations: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
resources: {}
extraContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
useHelmHooks: false
applyCustomEnv: false
createContentPoolsJob:
enabled: true
ttlSecondsAfterFinished: ~
command: ~
args:
- bash
- -c
- exec airflow pools import /opt/-/config/content_importer_pools.json
annotations: {}
jobAnnotations: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
resources: {}
extraContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
useHelmHooks: false
applyCustomEnv: false
fullnameOverride: ""
nameOverride: ""
useStandardNaming: true
revisionHistoryLimit: ~
uid: 10000
gid: 0
securityContext: {}
securityContexts:
pod: {}
containers: {}
containerLifecycleHooks: {}
airflowHome: /opt/airflow
defaultAirflowRepository: <repo>
defaultAirflowTag: "1.0"
defaultAirflowDigest: ~
airflowVersion: "2.8.3"
images:
airflow:
repository: <repo>
tag: ~
digest: ~
pullPolicy: IfNotPresent
useDefaultImageForMigration: false
migrationsWaitTimeout: 60
pod_template:
repository: <repo>
tag: "1.0"
pullPolicy: IfNotPresent
flower:
repository: ~
tag: ~
pullPolicy: IfNotPresent
statsd:
repository: quay.io/prometheus/statsd-exporter
tag: v0.22.8
pullPolicy: IfNotPresent
redis:
repository: dockerhub.hi.inet/library/redis
tag: 7-bullseye
pullPolicy: IfNotPresent
pgbouncer:
repository: dockerhub.hi.inet/apache/airflow
tag: airflow-pgbouncer-2023.02.24-1.16.1
pullPolicy: IfNotPresent
pgbouncerExporter:
repository: dockerhub.hi.inet/apache/airflow
tag: airflow-pgbouncer-exporter-2023.02.21-0.14.0
pullPolicy: IfNotPresent
gitSync:
repository: registry.k8s.io/git-sync/git-sync
tag: v3.6.9
pullPolicy: IfNotPresent
nodeSelector: {}
affinity: {}
tolerations: []
topologySpreadConstraints: []
schedulerName: ~
labels: {}
ingress:
enabled: ~
web:
enabled: true
annotations: {}
path: "/"
pathType: "ImplementationSpecific"
host: ""
hosts: ["host"]
ingressClassName: ""
tls:
enabled: false
secretName: ""
precedingPaths: []
succeedingPaths: []
flower:
enabled: true
annotations: {}
path: "/"
pathType: "ImplementationSpecific"
host: ""
hosts: ["host"]
ingressClassName: ""
tls:
enabled: false
secretName: ""
networkPolicies:
enabled: false
airflowPodAnnotations: {}
airflowConfigAnnotations: {}
airflowLocalSettings: |-
{{- if semverCompare ">=2.2.0" .Values.airflowVersion }}
{{- if not (or .Values.webserverSecretKey
.Values.webserverSecretKeySecretName) }}
from airflow.www.utils import UIAlert
DASHBOARD_UIALERTS = [
UIAlert(
'Usage of a dynamic webserver secret key detected. We recommend a
static webserver secret key instead.'
' See the <a href='
'"https://airflow.apache.org/docs/helm-chart/stable/production-guide.html
'Helm Chart Production Guide</a> for more details.',
category="warning",
roles=["Admin"],
html=True,
)
]
{{- end }}
{{- end }}
rbac:
create: true
createSCCRoleBinding: true
executor: "CeleryExecutor"
allowPodLaunching: false
env:
- name: "AIRFLOW__API__AUTH_BACKEND"
value: "airflow.api.auth.backend.basic_auth"
- name: "AIRFLOW__LOGGING__LOGGING_LEVEL"
value: "DEBUG"
- name: "AIRFLOW__LOGGING__LOGGING_CONFIG_CLASS"
value: "utils.local_logging.log_config.NEW_LOGGING_CONFIG"
- name: "ENV_VAR_LOCAL_LOGGING_LEVEL"
value: "DEBUG"
- name: "ENV_VAR_TASK_LOG_LEVEL"
value: "WARNING"
- name : "ENV_VAR_COMPONENT_NAME"
value: "top.enablers.-.airflow"
- name : "AIRFLOW__LOGGING__CELERY_LOGGING_LEVEL"
value: "ERROR"
- name : "AIRFLOW__CORE__PARALLELISM"
value: "0"
- name : "AIRFLOW__CORE__MAX_ACTIVE_TASKS_PER_DAG"
value: "32"
- name : "AIRFLOW__CORE__MAX_ACTIVE_RUNS_PER_DAG"
value: "60"
volumes: []
volumeMounts: []
secret: []
enableBuiltInSecretEnvVars:
AIRFLOW__CORE__FERNET_KEY: false
AIRFLOW__CORE__SQL_ALCHEMY_CONN: true
AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: true
AIRFLOW_CONN_AIRFLOW_DB: true
AIRFLOW__WEBSERVER__SECRET_KEY: false
AIRFLOW__CELERY__CELERY_RESULT_BACKEND: true
AIRFLOW__CELERY__RESULT_BACKEND: true
AIRFLOW__CELERY__BROKER_URL: true
AIRFLOW__ELASTICSEARCH__HOST: true
AIRFLOW__ELASTICSEARCH__ELASTICSEARCH_HOST: true
priorityClasses: []
extraSecrets: {}
extraConfigMaps: {}
extraEnv: ~
extraEnvFrom: ~
data:
metadataSecretName: ~
resultBackendSecretName: ~
brokerUrlSecretName: ~
metadataConnection:
<data>
resultBackendConnection: ~
brokerUrl: ~
fernetKey: airflow-fernet-secret
fernetKeySecretName: ~
webserverSecretKey: airflow-webserver-secret
webserverSecretKeySecretName: ~
kerberos:
enabled: false
ccacheMountPath: /var/kerberos-ccache
ccacheFileName: cache
configPath: /etc/krb5.conf
keytabBase64Content: ~
keytabPath: /etc/airflow.keytab
principal: [email protected]
reinitFrequency: 3600
config: |
[logging]
default = "FILE:{{ template "airflow_logs_no_quote" .
}}/kerberos_libs.log"
kdc = "FILE:{{ template "airflow_logs_no_quote" . }}/kerberos_kdc.log"
admin_server = "FILE:{{ template "airflow_logs_no_quote" .
}}/kadmind.log"
[libdefaults]
default_realm = FOO.COM
ticket_lifetime = 10h
renew_lifetime = 7d
forwardable = true
[realms]
FOO.COM = {
kdc = kdc-server.foo.com
admin_server = admin_server.foo.com
}
workers:
replicas: 4
revisionHistoryLimit: ~
command: ~
args:
- "bash"
- "-c"
- |-
exec \
airflow {{ semverCompare ">=2.0.0" .Values.airflowVersion | ternary
"celery worker" "worker" }}
livenessProbe:
enabled: false
initialDelaySeconds: 10
timeoutSeconds: 20
failureThreshold: 5
periodSeconds: 60
command: ~
updateStrategy: ~
strategy:
rollingUpdate:
maxSurge: "100%"
maxUnavailable: "50%"
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
keda:
enabled: false
namespaceLabels: {}
pollingInterval: 5
cooldownPeriod: 30
minReplicaCount: 0
maxReplicaCount: 10
advanced: {}
query: >-
SELECT ceil(COUNT(*)::decimal / {{
.Values.config.celery.worker_concurrency }})
FROM task_instance
WHERE (state='running' OR state='queued')
{{- if eq .Values.executor "CeleryKubernetesExecutor" }}
AND queue != '{{
.Values.config.celery_kubernetes_executor.kubernetes_queue }}'
{{- end }}
usePgbouncer: false
persistence:
enabled: true
size: 100Gi
storageClassName:
fixPermissions: false
annotations: {}
securityContexts:
container: {}
containerLifecycleHooks: {}
kerberosSidecar:
enabled: false
resources: {}
securityContexts:
container: {}
containerLifecycleHooks: {}
resources:
limits:
cpu: 1500m
memory: 16Gi
requests:
cpu: 500m
memory: 4Gi
terminationGracePeriodSeconds: 600
safeToEvict: true
extraContainers: []
extraInitContainers: []
extraVolumeMounts:
---
extraVolumes:
---
nodeSelector:
role: apps
runtimeClassName: ~
priorityClassName: ~
affinity: {}
tolerations: []
topologySpreadConstraints: []
hostAliases: []
annotations: {}
podAnnotations: {}
labels: {}
logGroomerSidecar:
enabled: true
command: ~
args: ["bash", "/clean-logs"]
retentionDays: 15
resources: {}
securityContexts:
container: {}
waitForMigrations:
enabled: true
env: []
securityContexts:
container: {}
env:
- name: "AIRFLOW__CELERY__WORKER_CONCURRENCY"
value: "16"
scheduler:
hostAliases: []
livenessProbe:
initialDelaySeconds: 10
timeoutSeconds: 20
failureThreshold: 5
periodSeconds: 60
command: ~
startupProbe:
failureThreshold: 6
periodSeconds: 10
timeoutSeconds: 20
command: ~
replicas: 1
revisionHistoryLimit: ~
command: ~
args: ["bash", "-c", "exec airflow scheduler"]
updateStrategy: ~
strategy: ~
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
podDisruptionBudget:
enabled: false
config:
maxUnavailable: 1
resources:
limits:
cpu: 600m
memory: 8Gi
requests:
cpu: 300m
memory: 4Gi
safeToEvict: true
extraContainers: []
extraInitContainers: []
extraVolumeMounts:
---
extraVolumes:
---
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
annotations: {}
podAnnotations: {}
labels: {}
logGroomerSidecar:
enabled: true
command: ~
args: ["bash", "/clean-logs"]
retentionDays: 15
resources: {}
securityContexts:
container: {}
containerLifecycleHooks: {}
waitForMigrations:
enabled: *jobs_enabled
env: []
securityContexts:
container: {}
env: []
createUserJob:
ttlSecondsAfterFinished: ~
command: ~
args:
- "bash"
- "-c"
- |-
exec \
airflow {{ semverCompare ">=2.0.0" .Values.airflowVersion | ternary
"users create" "create_user" }} "$@"
- --
- "-r"
- "{{ .Values.webserver.defaultUser.role }}"
- "-u"
- "{{ .Values.webserver.defaultUser.username }}"
- "-e"
- "{{ .Values.webserver.defaultUser.email }}"
- "-f"
- "{{ .Values.webserver.defaultUser.firstName }}"
- "-l"
- "{{ .Values.webserver.defaultUser.lastName }}"
- "-p"
- "{{ .Values.webserver.defaultUser.password }}"
annotations: {}
jobAnnotations: {}
labels: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
extraContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
useHelmHooks: false
applyCustomEnv: false
env: []
resources:
limits:
cpu: 300m
memory: 8Gi
requests:
cpu: 100m
memory: 4Gi
migrateDatabaseJob:
enabled: *jobs_enabled
ttlSecondsAfterFinished: ~
command: ~
args:
- "bash"
- "-c"
- >-
exec \
airflow {{ semverCompare ">=2.7.0" .Values.airflowVersion
| ternary "db migrate" (semverCompare ">=2.0.0" .Values.airflowVersion
| ternary "db upgrade" "upgradedb") }}
annotations: {}
jobAnnotations:
"argocd.argoproj.io/hook": Sync
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
resources:
limits:
cpu: 300m
memory: 8Gi
requests:
cpu: 100m
memory: 4Gi
extraContainers: []
extraVolumeMounts:
---
extraVolumes:
---
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
useHelmHooks: false
applyCustomEnv: false
webserver:
configMapAnnotations: {}
hostAliases: []
allowPodLogReading: true
livenessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
failureThreshold: 5
periodSeconds: 10
scheme: HTTP
readinessProbe:
initialDelaySeconds: 15
timeoutSeconds: 5
failureThreshold: 5
periodSeconds: 10
scheme: HTTP
startupProbe:
timeoutSeconds: 20
failureThreshold: 6
periodSeconds: 10
scheme: HTTP
replicas: 1
revisionHistoryLimit: ~
command: ~
args: ["bash", "-c", "exec airflow webserver"]
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
podDisruptionBudget:
enabled: false
config:
maxUnavailable: 1
strategy: ~
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
extraNetworkPolicies: []
networkPolicy:
ingress:
from: []
ports:
- port: "{{ .Values.ports.airflowUI }}"
resources:
limits:
cpu: 1000m
memory: 8Gi
requests:
cpu: 300m
memory: 4Gi
defaultUser:
enabled: *jobs_enabled
role: Admin
username: admin
email: [email protected]
firstName: admin
lastName: user
password: admin
extraContainers: []
extraInitContainers: []-
extraVolumes: []
extraVolumeMounts: []
webserverConfig: ~
webserverConfigConfigMapName: ~
service:
type: ClusterIP
annotations: {}
ports:
- name: airflow-ui
port: "{{ .Values.ports.airflowUI }}"
loadBalancerIP: ~
loadBalancerSourceRanges: []
nodeSelector:
role: apps
priorityClassName: ~
affinity: {}
tolerations: []
topologySpreadConstraints: []
annotations: {}
podAnnotations: {}
labels: {}
waitForMigrations:
enabled: *jobs_enabled
env: []
securityContexts:
container: {}
env: []
triggerer:
enabled: true
replicas: 1
revisionHistoryLimit: ~
command: ~
args: ["bash", "-c", "exec airflow triggerer"]
updateStrategy: ~
strategy:
rollingUpdate:
maxSurge: "100%"
maxUnavailable: "50%"
livenessProbe:
initialDelaySeconds: 10
timeoutSeconds: 20
failureThreshold: 5
periodSeconds: 60
command: ~
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
persistence:
enabled: true
size: 100Gi
storageClassName:
fixPermissions: false
annotations: {}
resources:
limits:
cpu: 300m
memory: 8Gi
requests:
cpu: 100m
memory: 4Gi
terminationGracePeriodSeconds: 60
safeToEvict: true
extraContainers: []
extraInitContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
annotations: {}
podAnnotations: {}
labels: {}
logGroomerSidecar:
enabled: true
command: ~
args: ["bash", "/clean-logs"]
retentionDays: 15
resources: {}
securityContexts:
container: {}
containerLifecycleHooks: {}
waitForMigrations:
enabled: *jobs_enabled
env: []
securityContexts:
container: {}
env: []
keda:
enabled: false
namespaceLabels: {}
pollingInterval: 5
cooldownPeriod: 30
minReplicaCount: 0
maxReplicaCount: 10
advanced: {}
query: >-
SELECT ceil(COUNT(*)::decimal / {{
.Values.config.triggerer.default_capacity }})
FROM trigger
dagProcessor:
enabled: false
replicas: 1
revisionHistoryLimit: ~
command: ~
args: ["bash", "-c", "exec airflow dag-processor"]
strategy:
rollingUpdate:
maxSurge: "100%"
maxUnavailable: "50%"
livenessProbe:
initialDelaySeconds: 10
timeoutSeconds: 20
failureThreshold: 5
periodSeconds: 60
command: ~
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
resources:
limits:
cpu: 300m
memory: 8Gi
requests:
cpu: 100m
memory: 4Gi
terminationGracePeriodSeconds: 60
safeToEvict: true
extraContainers: []
extraInitContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
annotations: {}
podAnnotations: {}
logGroomerSidecar:
enabled: true
command: ~
args: ["bash", "/clean-logs"]
retentionDays: 15
resources: {}
waitForMigrations:
enabled: true
env: []
env: []
flower:
enabled: true
revisionHistoryLimit: ~
command: ~
args:
- "bash"
- "-c"
- |-
exec \
airflow {{ semverCompare ">=2.0.0" .Values.airflowVersion | ternary
"celery flower" "flower" }}
extraNetworkPolicies: []
networkPolicy:
ingress:
from: []
ports:
- port: "{{ .Values.ports.flowerUI }}"
resources: {}
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
secretName: ~
username: ~
password: ~
service:
type: ClusterIP
annotations: {}
ports:
- name: flower-ui
port: "{{ .Values.ports.flowerUI }}"
loadBalancerIP: ~
loadBalancerSourceRanges: []
extraContainers: []
extraVolumes: []
extraVolumeMounts: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
annotations: {}
podAnnotations: {}
labels: {}
env: []
statsd:
configMapAnnotations: {}
enabled: true
revisionHistoryLimit: ~
args: ["--statsd.mapping-config=/etc/statsd-exporter/mappings.yml"]
annotations: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
uid: 65534
securityContext: {}
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
extraNetworkPolicies: []
resources: {}
service:
extraAnnotations: {}
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
extraMappings: []
overrideMappings: []
podAnnotations: {}
env: []
pgbouncer:
enabled: false
replicas: 1
revisionHistoryLimit: ~
command: ["pgbouncer", "-u", "nobody", "/etc/pgbouncer/pgbouncer.ini"]
args: ~
auth_type: md5
auth_file: /etc/pgbouncer/users.txt
annotations: {}
podAnnotations: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
extraNetworkPolicies: []
metadataPoolSize: 10
resultBackendPoolSize: 5
maxClientConn: 100
configSecretName: ~
podDisruptionBudget:
enabled: false
config:
maxUnavailable: 1
resources: {}
service:
extraAnnotations: {}
verbose: 2
logDisconnections: 1
logConnections: 1
sslmode: "prefer"
ciphers: "normal"
ssl:
ca: ~
cert: ~
key: ~
extraIniMetadata: ~
extraIniResultBackend: ~
extraIni: ~
extraVolumes: []
extraVolumeMounts: []
extraContainers: []
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
priorityClassName: ~
uid: 65534
securityContexts:
pod: {}
container: {}
containerLifecycleHooks:
preStop:
exec:
command: ["/bin/sh", "-c", "killall -INT pgbouncer && sleep
120"]
metricsExporterSidecar:
resources: {}
sslmode: "disable"
statsSecretName: ~
statsSecretKey: ~
securityContexts:
container: {}
containerLifecycleHooks: {}
livenessProbe:
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
readinessProbe:
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
env: []
redis:
enabled: true
terminationGracePeriodSeconds: 600
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
persistence:
enabled: true
size: 1Gi
storageClassName:
annotations: {}
resources:
limits:
cpu: 300m
memory: 8Gi
requests:
cpu: 100m
memory: 4Gi
passwordSecretName: ~
password: ~
safeToEvict: true
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
uid: 0
securityContext: {}
securityContexts:
pod:
fsGroup: 0
runAsGroup: 0
runAsUser: 999
container:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
containerLifecycleHooks: {}
podAnnotations: {}
registry:
secretName: crvpprevrm
elasticsearch:
enabled: false
secretName: ~
connection: {}
ports:
flowerUI: 5555
airflowUI: 8080
workerLogs: 8793
triggererLogs: 8794
redisDB: 6379
statsdIngest: 9125
statsdScrape: 9102
pgbouncer: 6543
pgbouncerScrape: 9127
quotas: {}
limits: []
cleanup:
enabled: false
schedule: "*/15 * * * *"
command: ~
args: ["bash", "-c", "exec airflow kubernetes cleanup-pods
--namespace={{ .Release.Namespace }}"]
jobAnnotations: {}
nodeSelector:
role: apps
affinity: {}
tolerations: []
topologySpreadConstraints: []
podAnnotations: {}
labels: {}
resources: {}
serviceAccount:
automountServiceAccountToken: true
create: true
name: ~
annotations: {}
securityContext: {}
env: []
securityContexts:
pod: {}
container: {}
containerLifecycleHooks: {}
failedJobsHistoryLimit: ~
successfulJobsHistoryLimit: ~
postgresql:
enabled: false
image:
tag: "11"
auth:
enablePostgresUser: true
postgresPassword: postgres
username: ""
password: ""
config:
core:
dags_folder: '/opt/-/dags'
dags_are_paused_at_creation: 'False'
load_examples: 'False'
executor: '{{ .Values.executor }}'
colored_console_log: 'False'
remote_logging: '{{- ternary "True" "False"
.Values.elasticsearch.enabled }}'
logging:
remote_logging: '{{- ternary "True" "False"
.Values.elasticsearch.enabled }}'
colored_console_log: 'False'
metrics:
statsd_on: '{{ ternary "True" "False" .Values.statsd.enabled }}'
statsd_port: 9125
statsd_prefix: airflow
statsd_host: '{{ printf "%s-statsd" .Release.Name }}'
webserver:
enable_proxy_fix: 'True'
rbac: 'True'
celery:
flower_url_prefix: '{{ ternary "" .Values.ingress.flower.path (eq
.Values.ingress.flower.path "/") }}'
worker_concurrency: 16
scheduler:
standalone_dag_processor: '{{ ternary "True" "False"
.Values.dagProcessor.enabled }}'
statsd_on: '{{ ternary "True" "False" .Values.statsd.enabled }}'
statsd_port: 9125
statsd_prefix: airflow
statsd_host: '{{ printf "%s-statsd" .Release.Name }}'
run_duration: 41460
elasticsearch:
json_format: 'True'
log_id_template: "{dag_id}_{task_id}_{execution_date}_{try_number}"
elasticsearch_configs:
max_retries: 3
timeout: 30
retry_timeout: 'True'
kerberos:
keytab: '{{ .Values.kerberos.keytabPath }}'
reinit_frequency: '{{ .Values.kerberos.reinitFrequency }}'
principal: '{{ .Values.kerberos.principal }}'
ccache: '{{ .Values.kerberos.ccacheMountPath }}/{{
.Values.kerberos.ccacheFileName }}'
celery_kubernetes_executor:
kubernetes_queue: 'kubernetes'
kubernetes:
namespace: '{{ .Release.Namespace }}'
airflow_configmap: '{{ include "airflow_config" . }}'
airflow_local_settings_configmap: '{{ include "airflow_config" . }}'
pod_template_file: '{{ include "airflow_pod_template_file" .
}}/pod_template_file.yaml'
worker_container_repository: <repo>
multi_namespace_mode: '{{ ternary "True" "False"
.Values.multiNamespaceMode }}'
kubernetes_executor:
namespace: '{{ .Release.Namespace }}'
pod_template_file: '{{ include "airflow_pod_template_file" .
}}/pod_template_file.yaml'
worker_container_repository: <repo>
multi_namespace_mode: '{{ ternary "True" "False"
.Values.multiNamespaceMode }}'
triggerer:
default_capacity: 1000
multiNamespaceMode: false
podTemplate: ~
dags:
persistence:
annotations: {}
enabled: false
size: 1Gi
storageClassName:
accessMode: ReadWriteOnce
existingClaim:
subPath: ~
gitSync:
enabled: false
repo: https://github.com/apache/airflow.git
branch: v2-2-stable
rev: HEAD
depth: 1
maxFailures: 0
subPath: "tests/dags"
wait: 5
containerName: git-sync
uid: 65533
securityContext: {}
securityContexts:
container: {}
containerLifecycleHooks: {}
extraVolumeMounts: []
env: []
resources: {}
logs:
useDevelopPVC: false
persistence:
enabled: true
size: 100Gi
annotations: {}
storageClassName: "-"
existingClaim: airflow-logs
```
### Docker Image customizations
_No response_
### What happened
I'm deploying Apache Airflow Helm Chart on our corporate environment with
ArgoCD and I would love to get some help on something that I'm not being able
to solve. Thanks in advance.
So far it's working pretty good, I've made some customizations to the
deployment, pv's, added some jobs, etc. The problem is, when upgrading with
ArgoCD, some barriers show up.
Firstly, fernet-key didn't allow upgrade on objects claiming error "missing
fernet-key", so I disabled that temporarily for debug purposes. Now, everytime
we commit into the branch that ArgoCD is deploying, the deploy breaks, being
unable to fully sync the app, claiming errors of "Inmutable fields". I think
the app is not trying to create a new updated object when we try to update, and
instead is trying to modify the already existing objects. The troubleshoting
I've found is just manually deleting the app and letting ArgoCD deploy it
entirely again, but this way we are loosing ArgoCD automatization point.
I’m sure I’m not the only one deploying this Helm chart with ArgoCD, so I’d
like to know if anyone can spot any visible errors or missing configurations.
So far, I’ve followed the documentation at
https://airflow.apache.org/docs/helm-chart/1.8.0/ to deploy with ArgoCD, but I
might be overlooking something.
### What you think should happen instead
ArgoCD should be able to update the deployment configuration and images when
we push into the deployed branch.
### How to reproduce
In an OpenShift cluster with ArgoCD, we deployed the official Helm chart via
ArgoCD, using the configurations previously described in the values.yaml.
The deployment is successful, and the application works correctly. ArgoCD is
able to deploy all objects and identify the synchronization status of the app.
We then attempt to make a change to the deployment, either by committing to
the deployed branch or switching branches. The change could involve modifying a
configuration or updating the image version.
Synchronization via ArgoCD fails. The only solution is to delete the
deployment and let ArgoCD recreate the app from scratch.
### Anything else
Thanks in advance!
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
