Miretpl commented on code in PR #61193:
URL: https://github.com/apache/airflow/pull/61193#discussion_r2743002847
##########
chart/Chart.yaml:
##########
@@ -63,61 +63,393 @@ annotations:
- title: Code View
url:
https://airflow.apache.org/docs/apache-airflow/3.1.6/_images/dag_overview_code.png
artifacthub.io/changes: |
- - description: Allow ConfigMap and Secret references in ``apiServer.env``
+ - description: Add ``PodDisruptionBudget`` for Dag Processor
+ kind: added
+ links:
+ - name: '#60294'
+ url: https://github.com/apache/airflow/pull/60294
+ - description: Add ``PodDisruptionBudget`` for Triggerer and Workers
+ kind: added
+ links:
+ - name: '#59068'
+ url: https://github.com/apache/airflow/pull/59068
+ - description: Add ``HorizontalPodAutoscaler`` (HPA) for API Server
+ kind: added
+ links:
+ - name: '#52392'
+ url: https://github.com/apache/airflow/pull/52392
+ - description: Add support for launching jobs with
``KubernetesJobOperator``
+ kind: added
+ links:
+ - name: '#52024'
+ url: https://github.com/apache/airflow/pull/52024
+ - description: Add ``CronJob`` to clean old records in the database
+ kind: added
+ links:
+ - name: '#58155'
+ url: https://github.com/apache/airflow/pull/58155
+ - description: Improve dag_bundle_config_list Configuration
+ kind: changed
+ links:
+ - name: '#60645'
+ url: https://github.com/apache/airflow/pull/60645
+ - description: Add ``workers.celery.kerberosInitContainer`` &
``workers.kubernetes.kerberosInitContainer``
+ kind: changed
+ links:
+ - name: '#60751'
+ url: https://github.com/apache/airflow/pull/60751
+ - description: Add ``workers.celery.securityContexts`` &
``workers.kubernetes.securityContexts``
+ kind: changed
+ links:
+ - name: '#60396'
+ url: https://github.com/apache/airflow/pull/60396
+ - description: Add ``workers.celery.podManagementPolicy`` field
+ kind: changed
+ links:
+ - name: '#60359'
+ url: https://github.com/apache/airflow/pull/60359
+ - description: Add ``workers.celery.strategy`` field
+ kind: changed
+ links:
+ - name: '#60354'
+ url: https://github.com/apache/airflow/pull/60354
+ - description: Add ``workers.celery.updateStrategy`` field
+ kind: changed
+ links:
+ - name: '#60351'
+ url: https://github.com/apache/airflow/pull/60351
+ - description: Add ``workers.celery.persistence`` section
+ kind: changed
+ links:
+ - name: '#60238'
+ url: https://github.com/apache/airflow/pull/60238
+ - description: Add ``workers.celery.livenessProbe`` section
+ kind: changed
+ links:
+ - name: '#60186'
+ url: https://github.com/apache/airflow/pull/60186
+ - description: Add ``workers.celery.args`` field
+ kind: changed
+ links:
+ - name: '#60163'
+ url: https://github.com/apache/airflow/pull/60163
+ - description: Add ``workers.celery.command`` &
``workers.kubernetes.command``
+ kind: changed
+ links:
+ - name: '#60067'
+ url: https://github.com/apache/airflow/pull/60067
+ - description: Allow custom ``volumeClaimTemplates`` when
``logs.persistence.enabled``
+ is true
Review Comment:
```suggestion
- description: Allow custom ``volumeClaimTemplates`` when
``logs.persistence.enabled`` is true
```
Not sure how the whole process works with the release, so just in case I'm
marking this place (there are more places like this in this file)
##########
chart/RELEASE_NOTES.rst:
##########
@@ -23,6 +23,164 @@ Run ``helm repo update`` before upgrading the chart to the
latest version.
.. towncrier release notes start
+Airflow Helm Chart 1.19.0 (2026-02-02)
+--------------------------------------
+
+Significant Changes
+^^^^^^^^^^^^^^^^^^^
+
+StatsD metrics aggregation now supports configurable TTL-enabled LRU cache to
prevent memory growth in long-running daemons (#60933)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+The Helm Chart now includes new configuration options for StatsD aggregation
management:
+
+* ``statsd.cache.type`` - Enable TTL-enabled ``lru`` cache or ``random`` cache
for metrics aggregation (default: ``lru``)
+* ``statsd.cache.size`` - Maximum number of metrics to cache (default: 1000)
+* ``statsd.cache.ttl`` - Time-to-live for cached metrics in seconds (``0s`` is
TTL disabled) (default: ``0s``)
+
+This feature addresses uncontrolled memory growth in StatsD daemons by
automatically cleaning up stale or unused metric entries. When enabled, the
cache uses both LRU (Least Recently Used) eviction and TTL (Time To Live)
expiration to manage memory usage effectively.
+
+To maintain backward compatibility, the default behaviour remains unchanged.
Users experiencing memory growth issues with StatsD can enable this feature by
setting ``statsd.cache.ttl`` to value higher than ``0`` in their Helm values.
+
+Support for Multiple Celery Worker Sets in the Helm Chart (#58547)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+This change introduces support for advanced Celery Workers topologies to
Apache Airflow Helm Chart, enabling more flexible resource allocation and
precise autoscaling configurations.
+
+**Flexible Worker Topologies**: The new ``workers.celery.enableDefault`` flag
allows users to configure a deployment consisting only of specialized worker
sets defined in ``workers.celery.sets`` section.
+
+**Multi-Queue Autoscaling Support**: Updates the KEDA ScaledObject generation
to support comma-separated queue lists. By using the ``SQL IN (...)`` clause,
we ensure that KEDA scales worker sets based on the precise aggregate workload
of all their assigned queues.
+
+**Granular Configuration Overrides**: This change allows for overwrite of any
currently available workers configuration per worker set. For example, a user
can enable KEDA globally, but explicitly disable it for a specific worker set
that requires a static number of replicas.
+
+Added ``workers.celery.enableDefault`` (default: true) to values.yaml.
+
+Manual Service Account Token Volume configuration for pod-launching executors
(#59156)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+Added support for manual Service Account Token Volume configuration when using
pod-launching executors
+(``CeleryExecutor``, ``CeleryKubernetesExecutor``, ``KubernetesExecutor``,
``LocalKubernetesExecutor``).
+This implements defense-in-depth security with both ServiceAccount and
Pod-level controls, providing
+compatibility with security policies like Kyverno and enabling
container-specific privilege assignment
+following the Principle of Least Privilege.
+
+
+Add ``imagePullSecrets`` option (#58094)
+""""""""""""""""""""""""""""""""""""""""
+
+Add ``.Values.imagePullSecrets`` as the new mechanism for configuring registry
credentials,
+deprecating both ``.Values.registry.secretName`` and the automatic creation of
the ``<RELEASE_NAME>-registry`` secret from ``.Values.registry.connection``.
+
+
+Default Airflow image is updated to ``3.1.6`` (#60453)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+The default Airflow image that is used with the Chart is now ``3.1.6``,
previously it was ``3.0.2``.
+
+Default git-sync image is updated to ``4.4.2`` (#54085)
+"""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+The default git-sync image that is used with the Chart is now ``4.4.2``,
previously it was ``4.3.0``.
+
+New Features
+^^^^^^^^^^^^
+
+- Add ``PodDisruptionBudget`` for Dag Processor (#60294)
+- Add ``PodDisruptionBudget`` for Triggerer and Workers (#59068)
+- Add ``HorizontalPodAutoscaler`` (HPA) for API Server (#52392)
+- Add support for launching jobs with ``KubernetesJobOperator`` (#52024)
+- Add ``CronJob`` to clean old records in the database (#58155)
+
+
+Improvements
+^^^^^^^^^^^^
+
+- Improve dag_bundle_config_list Configuration (#60645)
+- Add ``workers.celery.kerberosInitContainer`` &
``workers.kubernetes.kerberosInitContainer`` (#60751, #60427)
+- Add ``workers.celery.securityContexts`` &
``workers.kubernetes.securityContexts`` (#60396)
+- Add ``workers.celery.podManagementPolicy`` field (#60359)
+- Add ``workers.celery.strategy`` field (#60354)
+- Add ``workers.celery.updateStrategy`` field (#60351)
+- Add ``workers.celery.persistence`` section (#60238)
+- Add ``workers.celery.livenessProbe`` section (#60186)
+- Add ``workers.celery.args`` field (#60163)
+- Add ``workers.celery.command`` & ``workers.kubernetes.command`` (#60067)
+- Allow custom ``volumeClaimTemplates`` when ``logs.persistence.enabled`` is
true (#60118)
+- Add checksum for JWT secret in API server and scheduler deployments (#60111)
+- Add ``workers.celery.revisionHistoryLimit`` field (#60056)
+- Add Redis StatefulSet ``persistentVolumeClaimRetentionPolicy`` support
(#59955)
+- Add ``workers.celery.replicas`` field (#59730)
+- Add custom envs to database cleanup (#59804)
+- Extend ``airflow_ti_running`` metrics by scheduled, queued and deferred
(#58819)
+- Create an explicit control for ``createUserJob`` (#56057)
+- Make cleanup cronjob conditional on kubernetes executor (#58695)
+- Add database cleanup options and remove deprecated ``securityContext`` field
(#58663)
+- Add ability to disable API Server (#56493)
+- Add ``registry.secretNames`` and ``registry.connections`` options to Helm
chart (#58094)
+- Allow custom labels in StatsD, redis and Dag Processor (#55832)
+- Allow setting ``restartPolicy`` for batch jobs in chart (#54354)
+- Add readiness and liveliness support for git sync relay sidecars (#50218)
+- Allow overriding ``schedulerName`` on worker/tasks pods (#53983)
+- Allow additional ``PodDisruptionBudget`` config properties (#58864)
+- Add EdgeExecutor to KEDA query (#55560)
+- Allow ``revisionHistoryLimit`` to be set to 0 (#60340)
+- Allow optional ``subPath`` for logs volume mount (#52350)
+- Move triggerer from ``pod-log-reader-role`` to ``pod-launcher-role`` (#56872)
+
+Bug Fixes
+^^^^^^^^^
+
+- Use the ``bitnamilegacy/postgresql`` image (#61156)
+- Fix Compatibility of Celery Worker Sets with Workers Separation (#60420)
+- Fix database cleanup cronjob ImagePullSecrets (#58626)
+- Remove ``workers.celery`` breaking change (#61049)
+- Fix missing templating in API Server ``extraInitContainers`` (#60812)
+- Fix ``securityContext.containers``/``ingress.apiServer`` in
values.schema.json (#60575)
+- Remove unused ``containerLifecycleHooks`` field (#60239)
+- Remove unneeded logic in api-server (#60147)
+- Remove ``defaultUser`` from API Server in values.schema.json (#59762)
+- Isolate ``defaultUser`` handling in ``createUserJob`` (#59767)
+- Fix rendering condition of ``git_sync_ssh_key_volume`` (#59418)
+- Add watch for events to the Pod launcher role (#59080)
+- Ensure that git-sync actually runs when ``dags.gitSync.enabled=true`` and
``dags.persistence.enabled=true`` (#59123)
+- Don't add labels to non-existent configuration options (#59213)
+- Add log volume to init container for scheduler, triggerer and worker (#56418)
+- Correctly derive celery sync_parallelism from scheduler CPU limits (#58733)
+- Fix ingress notes (#59122)
+- Fix Liveness / Readiness / Startup probe path for Airflow 3.x (#58734)
+- Fix flower network policy condition when multiple executors (#58635)
+- Missing SCC Role bindings for redis and api-server (#57985)
+- Ensure graceful Redis shutdown(#58432)
+- Start Redis directly, not via shell (#58790)
+- Add missing ``airflow.fullname`` on kubernetes objects (#52953)
+- StatsD deployment volume mount without subpath for live reloading (#54986)
+- Fix KEDA query for Kubernetes Executor (#55559)
+- Add API Server config in k8s pod template (#53533)
+- Fix helm schema validation for executor value (#54682)
+- Correct watch verb quoting in Airflow Job Launcher Role (#53822)
+- Trim non-alphanumeric characters from the executor label (#53534)
+- Fix KEDA Query to Use executor Field Instead of queue for Multiple Executors
(#52840)
+
+Doc only changes
+^^^^^^^^^^^^^^^^
+
+- Document how to run the API server behind a reverse proxy (#61095)
+- Clarify ingress settings for Airflow 2 vs 3 in values.yaml (#60434)
+- Add database cleanup docs to Helm productions docs (#58707)
+- KEDA best practices + better documentation (#58246)
+- Update chart info about built-in secrets and environment variables (#58317)
+- Fix typo in PgBouncer section of the Production Guide (#56754)
+- Update webserver secret note in NOTES.txt and Production Guide (#55106)
+- Make term Dag consistent in docs v2 (#55099)
+- Add API Server to container resources docs (#54698)
+- Fix YAML block scalar when providing SSH key for git-sync (#56716)
+
+Misc
+^^^^
+
+- Add Python 3.13 support for Airflow (#46891)
+
Review Comment:
```suggestion
```
The Helm Chart itself is rather Python-independent
##########
chart/RELEASE_NOTES.rst:
##########
@@ -23,6 +23,164 @@ Run ``helm repo update`` before upgrading the chart to the
latest version.
.. towncrier release notes start
+Airflow Helm Chart 1.19.0 (2026-02-02)
+--------------------------------------
+
+Significant Changes
+^^^^^^^^^^^^^^^^^^^
+
+StatsD metrics aggregation now supports configurable TTL-enabled LRU cache to
prevent memory growth in long-running daemons (#60933)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+The Helm Chart now includes new configuration options for StatsD aggregation
management:
+
+* ``statsd.cache.type`` - Enable TTL-enabled ``lru`` cache or ``random`` cache
for metrics aggregation (default: ``lru``)
+* ``statsd.cache.size`` - Maximum number of metrics to cache (default: 1000)
+* ``statsd.cache.ttl`` - Time-to-live for cached metrics in seconds (``0s`` is
TTL disabled) (default: ``0s``)
+
+This feature addresses uncontrolled memory growth in StatsD daemons by
automatically cleaning up stale or unused metric entries. When enabled, the
cache uses both LRU (Least Recently Used) eviction and TTL (Time To Live)
expiration to manage memory usage effectively.
+
+To maintain backward compatibility, the default behaviour remains unchanged.
Users experiencing memory growth issues with StatsD can enable this feature by
setting ``statsd.cache.ttl`` to value higher than ``0`` in their Helm values.
+
+Support for Multiple Celery Worker Sets in the Helm Chart (#58547)
+""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""
+
+This change introduces support for advanced Celery Workers topologies to
Apache Airflow Helm Chart, enabling more flexible resource allocation and
precise autoscaling configurations.
+
+**Flexible Worker Topologies**: The new ``workers.celery.enableDefault`` flag
allows users to configure a deployment consisting only of specialized worker
sets defined in ``workers.celery.sets`` section.
+
+**Multi-Queue Autoscaling Support**: Updates the KEDA ScaledObject generation
to support comma-separated queue lists. By using the ``SQL IN (...)`` clause,
we ensure that KEDA scales worker sets based on the precise aggregate workload
of all their assigned queues.
+
+**Granular Configuration Overrides**: This change allows for overwrite of any
currently available workers configuration per worker set. For example, a user
can enable KEDA globally, but explicitly disable it for a specific worker set
that requires a static number of replicas.
+
+Added ``workers.celery.enableDefault`` (default: true) to values.yaml.
+
Review Comment:
```suggestion
```
Not sure if we need this, as it is mentioned a little higher in the section
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
