[
https://issues.apache.org/jira/browse/AIRFLOW-3126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17168358#comment-17168358
]
ASF GitHub Bot commented on AIRFLOW-3126:
-----------------------------------------
dimberman edited a comment on pull request #8150:
URL: https://github.com/apache/airflow/pull/8150#issuecomment-666890675
@fengsi Thank you for that breakdown, there's a lot of really valuable
feedback there.
One thing I've been pushing has been the CeleryExecutor with the KEDA
autoscaler. With this set-up you can define your celery workers as deployments
and since they all autoscale to 0, you can create as many "queues" as you'd
like.
I'm sad to hear the pod_template_file still leads to confusion. I'd love to
see if we can figure out a cleaner interface for it s.t. it's more transparent.
Ideally for Airflow 2.0 I'd like to make the pod_template_file the ONLY
option, and remove all other configs. We can offer a "default" value and users
can modify as needed.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> kubernetes executor not allowing new PV
> ---------------------------------------
>
> Key: AIRFLOW-3126
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3126
> Project: Apache Airflow
> Issue Type: Bug
> Components: executors
> Affects Versions: 1.10.0
> Reporter: Rahul Singh
> Assignee: Brandon Willard
> Priority: Major
>
> Kubernetes executor allows only two PV one for dag and other for logs , in
> project scenario there are multiple PV needed , like separate PV to store
> data , PV to store common script files etc . Current implementation is very
> static to only two PV.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
