[jira] [Updated] (AIRFLOW-921) 1.8.0rc Issues

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-921: Description: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke]

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893157#comment-15893157 ] Dan Davydov commented on AIRFLOW-932: - If the goal is to eventually move all the logic to the

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893158#comment-15893158 ] Dan Davydov commented on AIRFLOW-932: - Thanks for fixing! > Backfills delete existing task instances

[jira] [Assigned] (AIRFLOW-921) 1.8.0rc Issues

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin reassigned AIRFLOW-921: -- Assignee: Bolke de Bruin > 1.8.0rc Issues > -- > > Key:

[jira] [Work started] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-932 started by Bolke de Bruin. -- > Backfills delete existing task instances and mark them as removed >

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893151#comment-15893151 ] Bolke de Bruin commented on AIRFLOW-932: It would avoid the issue, but from a dag run integrity

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893135#comment-15893135 ] Dan Davydov commented on AIRFLOW-932: - I think tasks should only be verified/removed in the scheduler

[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893124#comment-15893124 ] Bolke de Bruin edited comment on AIRFLOW-932 at 3/2/17 10:09 PM: - Issue

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893124#comment-15893124 ] Bolke de Bruin commented on AIRFLOW-932: Issue resides in cli.py and will only happen when a

[jira] [Commented] (AIRFLOW-937) task_stats makes extremely large prepared query

2017-03-02 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893114#comment-15893114 ] ASF subversion and git services commented on AIRFLOW-937: - Commit

[jira] [Commented] (AIRFLOW-937) task_stats makes extremely large prepared query

2017-03-02 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893115#comment-15893115 ] ASF subversion and git services commented on AIRFLOW-937: - Commit

incubator-airflow git commit: [AIRFLOW-937] Improve performance of task_stats

2017-03-02 Thread davydov
Repository: incubator-airflow Updated Branches: refs/heads/master 88d9b0dc9 -> ae44dddcb [AIRFLOW-937] Improve performance of task_stats Please accept this PR that addresses the following issues: - https://issues.apache.org/jira/browse/AIRFLOW-937 Testing Done: - Shouldn't change

[jira] [Created] (AIRFLOW-937) task_stats makes extremely large prepared query

2017-03-02 Thread Alex Guziel (JIRA)
Alex Guziel created AIRFLOW-937: --- Summary: task_stats makes extremely large prepared query Key: AIRFLOW-937 URL: https://issues.apache.org/jira/browse/AIRFLOW-937 Project: Apache Airflow Issue

[jira] [Created] (AIRFLOW-936) Ability to clear/mark success all tasks in a dagrun in the UI

2017-03-02 Thread Dan Davydov (JIRA)
Dan Davydov created AIRFLOW-936: --- Summary: Ability to clear/mark success all tasks in a dagrun in the UI Key: AIRFLOW-936 URL: https://issues.apache.org/jira/browse/AIRFLOW-936 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-921) 1.8.0rc Issues

2017-03-02 Thread Alex Guziel (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893094#comment-15893094 ] Alex Guziel commented on AIRFLOW-921: - The high DB load looks pretty periodic on our end (ie comes

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893083#comment-15893083 ] Bolke de Bruin commented on AIRFLOW-932: Ok. Confirmed now - really strange. > Backfills delete

[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892823#comment-15892823 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:21 PM: - Ah I think I

[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892823#comment-15892823 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:17 PM: - Ah I think I

[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892818#comment-15892818 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:14 PM: - This is off of

[jira] [Updated] (AIRFLOW-927) LDAP authtorization: ldap3 doesn't understand anything coming from configuration file

2017-03-02 Thread Marcelo G. Almiron (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo G. Almiron updated AIRFLOW-927: --- Affects Version/s: Airflow 1.8 > LDAP authtorization: ldap3 doesn't understand

[jira] [Updated] (AIRFLOW-927) LDAP authtorization: ldap3 doesn't understand anything coming from configuration file

2017-03-02 Thread Marcelo G. Almiron (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo G. Almiron updated AIRFLOW-927: --- Affects Version/s: (was: Airflow 1.7.1) Airflow 1.7.1.3 >

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Description: Hi, When using with Airflow with CeleryExecutor, both RabbitMQ && Redis I tested, I

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: (was: scheduler_2.log) > Same {task,execution_date} run multiple times in worker

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: worker_2.log > Same {task,execution_date} run multiple times in worker when using >

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: scheduler_2.log > Same {task,execution_date} run multiple times in worker when using >

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: processes.list > Same {task,execution_date} run multiple times in worker when using >

[jira] [Issue Comment Deleted] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Comment: was deleted (was: I added also: 1. rabbitmq.queue - show that after 5 minutes the broker

[jira] [Issue Comment Deleted] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Comment: was deleted (was: Attached also: 1. scheduler.log to show that scheduler decided to send

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Description: Hi, When using with Airflow with CeleryExecutor, both RabbitMQ && Redis I tested, I

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: dag_runs.png rabbitmq.queue I added also: 1. rabbitmq.queue - show that

[jira] [Comment Edited] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890945#comment-15890945 ] Uri Shamay edited comment on AIRFLOW-928 at 3/2/17 10:03 AM: - Attached also:

[jira] [Updated] (AIRFLOW-928) Same {task,execution_date} run multiple times in worker when using CeleryExecutor

2017-03-02 Thread Uri Shamay (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uri Shamay updated AIRFLOW-928: --- Attachment: scheduler.log > Same {task,execution_date} run multiple times in worker when using >

[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed

2017-03-02 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891767#comment-15891767 ] Bolke de Bruin commented on AIRFLOW-932: I just ran this on my AIRFLOW-910 branch and I am not