dstandish opened a new issue #13181:
URL: https://github.com/apache/airflow/issues/13181
In brand new cluster using k3d locally, I first installed keda:
```bash
helm install keda \
--namespace keda kedacore/keda \
--version "v1.5.0"
```
Next, I installed airflow using this config:
```yaml
executor: CeleryExecutor
defaultAirflowTag: 2.0.0-python3.7
airflowVersion: 2.0.0
workers:
keda:
enabled: true
persistence:
enabled: false
pgbouncer:
enabled: true
```
I think this should create a scaled object `airflow-worker`.
But it does not.
As a result of course no autoscaling occurs.
But, when I applied this scaled object definition, the scaledobject was
created and immediately autascaling started working --- the worker pod
terminated.
```yaml
apiVersion: keda.k8s.io/v1alpha1
kind: ScaledObject
metadata:
name: airflow-worker
labels:
tier: airflow
component: worker-horizontalpodautoscaler
release: airflow
chart: "airflow-master"
heritage: Helm
deploymentName: airflow-worker
spec:
scaleTargetRef:
deploymentName: airflow-worker
pollingInterval: 30 # Optional. Default: 30 seconds
cooldownPeriod: 300 # Optional. Default: 300 seconds
maxReplicaCount: 100 # Optional. Default: 100
triggers:
- type: postgresql
metadata:
targetQueryValue: "1"
connection: AIRFLOW_CONN_AIRFLOW_DB
query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE
state='running' OR state='queued'"
```
@turbaszek and @dimberman you may have insight ...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
