[
https://issues.apache.org/jira/browse/AIRFLOW-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018873#comment-17018873
]
ASF GitHub Bot commented on AIRFLOW-1467:
-----------------------------------------
kaxil commented on pull request #7160: [AIRFLOW-1467] Dynamic pooling via
allowing tasks to use more than one pool slot (depending upon the need)
URL: https://github.com/apache/airflow/pull/7160
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> allow tasks to use more than one pool slot
> ------------------------------------------
>
> Key: AIRFLOW-1467
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1467
> Project: Apache Airflow
> Issue Type: Improvement
> Components: scheduler
> Reporter: Adrian Bridgett
> Assignee: Lokesh Lal
> Priority: Trivial
> Labels: pool
> Fix For: 1.10.8
>
>
> It would be useful to have tasks use more than a single pool slot.
> Our use case is actually to limit how many tasks run on a head node (due to
> memory constraints), currently we have to set a pool limit limiting how many
> tasks.
> Ideally we could set the pool size to e.g amount of memory and then set those
> tasks pool_usage (or whatever the option would be called) to the amount of
> memory we think they'll use. This way the pool would let lots of small tasks
> run or just a few large tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
