dstandish commented on a change in pull request #13571:
URL: https://github.com/apache/airflow/pull/13571#discussion_r571467658
##########
File path: chart/templates/workers/worker-kedaautoscaler.yaml
##########
@@ -36,13 +36,13 @@ metadata:
spec:
scaleTargetRef:
deploymentName: {{ .Release.Name }}-worker
- pollingInterval: {{ .Values.workers.keda.pollingInterval }} # Optional.
Default: 30 seconds
- cooldownPeriod: {{ .Values.workers.keda.cooldownPeriod }} # Optional.
Default: 300 seconds
- maxReplicaCount: {{ .Values.workers.keda.maxReplicaCount }} # Optional.
Default: 100
+ pollingInterval: {{ .Values.workers.keda.pollingInterval }} # Optional.
Default: 30 seconds
+ cooldownPeriod: {{ .Values.workers.keda.cooldownPeriod }} # Optional.
Default: 300 seconds
+ maxReplicaCount: {{ .Values.workers.keda.maxReplicaCount }} # Optional.
Default: 100
triggers:
- type: postgresql
metadata:
targetQueryValue: "1"
connection: AIRFLOW_CONN_AIRFLOW_DB
- query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE
state='running' OR state='queued'"
+ query: "SELECT ceil(COUNT(*)::decimal / {{
.Values.config.celery.worker_concurrency }}) FROM task_instance WHERE
state='running' OR state='queued'"
Review comment:
the chart does not support mysql _when you run the metastore on k8s
along with airflow_. however, wouldn't it work with mysql if you had an
external metastore e.g. rds (which is what is recommended anyway)?
i know postgres is the better choice for airflow and much more widely
embraced but some folks might use other dbs and off top of my head i think it
would probably work since you just need to specify conn uri and disable
postgres and pgbouncer...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
