HsiuChuanHsu commented on PR #56589:
URL: https://github.com/apache/airflow/pull/56589#issuecomment-3481181851
Thanks for your review! Summary of changes:
> If KEDA and HPA is not usable in parallel with the worker groups we should
mention it in the docs as (current) clear limitation.
Agree that this limitation must be clearly documented.
- Added a "not supported" note preceding `workers.keda`, `workers.hpa` &
`workers.queueGroups` settings in `values.yaml`
- Add a "not supported" note description of `queueGroups` in
`values.schema.json`
**Next Steps**: Will continue working on enableing support for HPA & KEDA.
> One potentially missing this is what I saw that all worker groups take the
same celery command line definition, so this means if I want to define
different --concurrency per worker type I need to make this via env, correct?
Maybe also worth noting in the docs (e.g. schema in line 2664).
Yes, your understanding is correct.
I honestly overlooked the need for the $\texttt{--concurrency}$ setting, but
I don't currently have a better implementation idea for handling this. Let's
note the current approach for now
- Added a "not supported" note preceding `workers.queueGroups` settings in
`values.yaml`
- Added a clarifying note regarding the configuration of
`AIRFLOW__CELERY__WORKER_CONCURRENCY` within `values.schema.json`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
