[jira] [Updated] (AIRFLOW-1557) backfill ignores configured number of slots in a pool
[
https://issues.apache.org/jira/browse/AIRFLOW-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Egor Mateshuk updated AIRFLOW-1557:
---
Attachment: (was: photo_2018-03-07_16-31-23.jpg)
> backfill ignores configured number of slots in a pool
> -
>
> Key: AIRFLOW-1557
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1557
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Ash Berlin-Taylor
>Priority: Major
> Attachments: Backfill_bug_1.8.2.jpg, Screen Shot 2017-09-01 at
> 11.39.32.png
>
>
> I ran a backfill command with this tool:
> The backfill process appears to run as many tasks as possible, even when the
> pool it is running in should limit the numbers:
> {noformat}
> airflow backfill \
> -t fetch_dk_unfiltered \
> --pool brand_index_api \
> -s 2017-07-31 -e 2017-08-31 \
> -x \
> brand_index_fetcher
> {noformat}
> (Nothing else than the backfill is currently using this pool. I wasn't able
> to capture a screen shot of the task instances before the jobs completed.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-1557) backfill ignores configured number of slots in a pool
[
https://issues.apache.org/jira/browse/AIRFLOW-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Egor Mateshuk updated AIRFLOW-1557:
---
Attachment: Backfill_bug_1.8.2.jpg
> backfill ignores configured number of slots in a pool
> -
>
> Key: AIRFLOW-1557
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1557
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Ash Berlin-Taylor
>Priority: Major
> Attachments: Backfill_bug_1.8.2.jpg, Screen Shot 2017-09-01 at
> 11.39.32.png
>
>
> I ran a backfill command with this tool:
> The backfill process appears to run as many tasks as possible, even when the
> pool it is running in should limit the numbers:
> {noformat}
> airflow backfill \
> -t fetch_dk_unfiltered \
> --pool brand_index_api \
> -s 2017-07-31 -e 2017-08-31 \
> -x \
> brand_index_fetcher
> {noformat}
> (Nothing else than the backfill is currently using this pool. I wasn't able
> to capture a screen shot of the task instances before the jobs completed.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-1557) backfill ignores configured number of slots in a pool
[
https://issues.apache.org/jira/browse/AIRFLOW-1557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Egor Mateshuk updated AIRFLOW-1557:
---
Attachment: photo_2018-03-07_16-31-23.jpg
> backfill ignores configured number of slots in a pool
> -
>
> Key: AIRFLOW-1557
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1557
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.8.1
>Reporter: Ash Berlin-Taylor
>Priority: Major
> Attachments: Screen Shot 2017-09-01 at 11.39.32.png,
> photo_2018-03-07_16-31-23.jpg
>
>
> I ran a backfill command with this tool:
> The backfill process appears to run as many tasks as possible, even when the
> pool it is running in should limit the numbers:
> {noformat}
> airflow backfill \
> -t fetch_dk_unfiltered \
> --pool brand_index_api \
> -s 2017-07-31 -e 2017-08-31 \
> -x \
> brand_index_fetcher
> {noformat}
> (Nothing else than the backfill is currently using this pool. I wasn't able
> to capture a screen shot of the task instances before the jobs completed.)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
