[ https://issues.apache.org/jira/browse/AIRFLOW-1467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17064139#comment-17064139 ]
ASF subversion and git services commented on AIRFLOW-1467: ---------------------------------------------------------- Commit 3fc89bd67ff469940d97d182ce09dabef5634516 in airflow's branch refs/heads/v1-10-test from Lokesh Lal [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=3fc89bd ] [AIRFLOW-1467] Allow tasks to use more than one pool slot (#7160) > allow tasks to use more than one pool slot > ------------------------------------------ > > Key: AIRFLOW-1467 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1467 > Project: Apache Airflow > Issue Type: Improvement > Components: scheduler > Reporter: Adrian Bridgett > Assignee: Lokesh Lal > Priority: Trivial > Labels: pool > Fix For: 1.10.10 > > > It would be useful to have tasks use more than a single pool slot. > Our use case is actually to limit how many tasks run on a head node (due to > memory constraints), currently we have to set a pool limit limiting how many > tasks. > Ideally we could set the pool size to e.g amount of memory and then set those > tasks pool_usage (or whatever the option would be called) to the amount of > memory we think they'll use. This way the pool would let lots of small tasks > run or just a few large tasks. -- This message was sent by Atlassian Jira (v8.3.4#803005)