dstandish commented on a change in pull request #13571:
URL: https://github.com/apache/airflow/pull/13571#discussion_r571467658
##########
File path: chart/templates/workers/worker-kedaautoscaler.yaml
##########
@@ -36,13 +36,13 @@ metadata:
spec:
scaleTargetRef:
deploymentName: {{ .Release.Name }}-worker
- pollingInterval: {{ .Values.workers.keda.pollingInterval }} # Optional.
Default: 30 seconds
- cooldownPeriod: {{ .Values.workers.keda.cooldownPeriod }} # Optional.
Default: 300 seconds
- maxReplicaCount: {{ .Values.workers.keda.maxReplicaCount }} # Optional.
Default: 100
+ pollingInterval: {{ .Values.workers.keda.pollingInterval }} # Optional.
Default: 30 seconds
+ cooldownPeriod: {{ .Values.workers.keda.cooldownPeriod }} # Optional.
Default: 300 seconds
+ maxReplicaCount: {{ .Values.workers.keda.maxReplicaCount }} # Optional.
Default: 100
triggers:
- type: postgresql
metadata:
targetQueryValue: "1"
connection: AIRFLOW_CONN_AIRFLOW_DB
- query: "SELECT ceil(COUNT(*)::decimal / 16) FROM task_instance WHERE
state='running' OR state='queued'"
+ query: "SELECT ceil(COUNT(*)::decimal / {{
.Values.config.celery.worker_concurrency }}) FROM task_instance WHERE
state='running' OR state='queued'"
Review comment:
the chart does not support mysql when you run the metastore on k8s along
with airflow. however, wouldn't it work with mysql if you had an external
metastore e.g. rds (which is what is recommended anyway)?
i know postgres is the better fit for airflow but some folks might use other
dbs and off top of my head i think it would probably work since you just need
to specify conn uri and disable postgres and pgbouncer...
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
