[jira] [Created] (AIRFLOW-89) Documentation error on Tutorial site

2016-05-09 Thread Dimitri Vasdekis (JIRA)
Dimitri Vasdekis created AIRFLOW-89: --- Summary: Documentation error on Tutorial site Key: AIRFLOW-89 URL: https://issues.apache.org/jira/browse/AIRFLOW-89 Project: Apache Airflow Issue

[jira] [Updated] (AIRFLOW-78) airflow clear leaves dag_runs

2016-05-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-78: -- Affects Version/s: Airflow 1.6.2 > airflow clear leaves dag_runs >

[jira] [Created] (AIRFLOW-79) End-to-end DAG testing

2016-05-09 Thread Thoralf Gutierrez (JIRA)
Thoralf Gutierrez created AIRFLOW-79: Summary: End-to-end DAG testing Key: AIRFLOW-79 URL: https://issues.apache.org/jira/browse/AIRFLOW-79 Project: Apache Airflow Issue Type: New

[jira] [Updated] (AIRFLOW-79) End-to-end DAG testing

2016-05-09 Thread Siddharth Anand (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand updated AIRFLOW-79: --- External issue URL: https://github.com/apache/incubator-airflow/pull/748 > End-to-end DAG

[jira] [Updated] (AIRFLOW-54) Special tags when referring to other tasks

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini updated AIRFLOW-54: --- Assignee: Adam Mustafa > Special tags when referring to other tasks >

[jira] [Commented] (AIRFLOW-52) Release airflow 1.7.1

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276637#comment-15276637 ] Chris Riccomini commented on AIRFLOW-52: [~aoen], are you planning to burn in [~jlowin]'s fix

[jira] [Resolved] (AIRFLOW-76) `celeryd_concurrency` mandatory configuration

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-76?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini resolved AIRFLOW-76. Resolution: Duplicate Dupe of AIRFLOW-25 > `celeryd_concurrency` mandatory configuration >

[jira] [Commented] (AIRFLOW-71) docker_operator - pulling from private repositories

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276716#comment-15276716 ] Chris Riccomini commented on AIRFLOW-71: [~asnir], do you want add config to the DockerOperator,

[jira] [Commented] (AIRFLOW-71) docker_operator - pulling from private repositories

2016-05-09 Thread Amikam Snir (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-71?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276720#comment-15276720 ] Amikam Snir commented on AIRFLOW-71: Before it pulling the image > docker_operator - pulling from

[jira] [Updated] (AIRFLOW-71) docker_operator - pulling from private repositories

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-71?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini updated AIRFLOW-71: --- Component/s: operators docker > docker_operator - pulling from private

[jira] [Commented] (AIRFLOW-14) DagRun Refactor (Scheduler 2.0)

2016-05-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-14?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276785#comment-15276785 ] Bolke de Bruin commented on AIRFLOW-14: --- Now the scheduler is, in case of using the LocalExecutor,

[jira] [Created] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Stephen Cattaneo (JIRA)
Stephen Cattaneo created AIRFLOW-80: --- Summary: `airflow initdb` fails for new environment Key: AIRFLOW-80 URL: https://issues.apache.org/jira/browse/AIRFLOW-80 Project: Apache Airflow

[jira] [Comment Edited] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276787#comment-15276787 ] Bolke de Bruin edited comment on AIRFLOW-80 at 5/9/16 6:35 PM: --- can you use

[jira] [Commented] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276787#comment-15276787 ] Bolke de Bruin commented on AIRFLOW-80: --- can you use pip install[hive]? > `airflow initdb` fails

[jira] [Comment Edited] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Bolke de Bruin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276787#comment-15276787 ] Bolke de Bruin edited comment on AIRFLOW-80 at 5/9/16 6:36 PM: --- can you use

[jira] [Updated] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Siddharth Anand (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Anand updated AIRFLOW-80: --- Description: I just created a new dev env with "setup.py develop " {code}

[jira] [Commented] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Siddharth Anand (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276791#comment-15276791 ] Siddharth Anand commented on AIRFLOW-80: The better workaround is to move this example to

[jira] [Comment Edited] (AIRFLOW-80) `airflow initdb` fails for new environment

2016-05-09 Thread Siddharth Anand (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-80?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276791#comment-15276791 ] Siddharth Anand edited comment on AIRFLOW-80 at 5/9/16 6:41 PM: The better

[jira] [Created] (AIRFLOW-82) Add configuration option in airflow.cfg for Flask APPLICATION_ROOT

2016-05-09 Thread Sean McIntyre (JIRA)
Sean McIntyre created AIRFLOW-82: Summary: Add configuration option in airflow.cfg for Flask APPLICATION_ROOT Key: AIRFLOW-82 URL: https://issues.apache.org/jira/browse/AIRFLOW-82 Project: Apache

[jira] [Commented] (AIRFLOW-56) Airflow's scheduler can "lose" queued tasks

2016-05-09 Thread Kevin Lin (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-56?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15276824#comment-15276824 ] Kevin Lin commented on AIRFLOW-56: -- Wondering if this issue is only present in 1.7.0. We've observed a

[jira] [Comment Edited] (AIRFLOW-85) Create a viewer/editor roles for UI

2016-05-09 Thread Chris Riccomini (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-85?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277214#comment-15277214 ] Chris Riccomini edited comment on AIRFLOW-85 at 5/9/16 10:42 PM: - Initial

[2/3] incubator-airflow git commit: Add logic to lock DB and avoid race condition

2016-05-09 Thread jlowin
Add logic to lock DB and avoid race condition The scheduler can encounter a queued task twice before the task actually starts to run -- this locks the task and avoids that condition. Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[1/5] incubator-airflow git commit: [AIRFLOW-52] Fix bottlenecks when working with many tasks

2016-05-09 Thread davydov
Repository: incubator-airflow Updated Branches: refs/heads/airbnb_rb1.7.1_3 [created] 77c7bc4ac [AIRFLOW-52] Fix bottlenecks when working with many tasks Dag hash function tried (and failed) to hash the list of tasks, then fell back on repr-ing the list, which took forever. Instead, hash

[2/5] incubator-airflow git commit: Fix corner case with joining processes/queues (#1473)

2016-05-09 Thread davydov
Fix corner case with joining processes/queues (#1473) If a process places items in a queue and the process is joined before the queue is emptied, it can lead to a deadlock under some circumstances. Closes AIRFLOW-61. See for example:

[jira] [Commented] (AIRFLOW-52) Release airflow 1.7.1

2016-05-09 Thread ASF subversion and git services (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-52?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277355#comment-15277355 ] ASF subversion and git services commented on AIRFLOW-52: Commit

[jira] [Updated] (AIRFLOW-52) Release airflow 1.7.1

2016-05-09 Thread Dan Davydov (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-52?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-52: --- Description: Release the airflow 1.7.1 tag. Current status: There are three issues blocking this

[jira] [Created] (AIRFLOW-86) Python3: wrap dict.items() in list

2016-05-09 Thread Jeremiah Lowin (JIRA)
Jeremiah Lowin created AIRFLOW-86: - Summary: Python3: wrap dict.items() in list Key: AIRFLOW-86 URL: https://issues.apache.org/jira/browse/AIRFLOW-86 Project: Apache Airflow Issue Type: Bug

[jira] [Created] (AIRFLOW-87) Setup a devolpment environment

2016-05-09 Thread Amikam Snir (JIRA)
Amikam Snir created AIRFLOW-87: -- Summary: Setup a devolpment environment Key: AIRFLOW-87 URL: https://issues.apache.org/jira/browse/AIRFLOW-87 Project: Apache Airflow Issue Type: Task

[jira] [Commented] (AIRFLOW-62) XCom push not working reliably

2016-05-09 Thread Alex Papanicolaou (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277493#comment-15277493 ] Alex Papanicolaou commented on AIRFLOW-62: -- I think you're right that it's celery. Just ran it

[jira] [Commented] (AIRFLOW-62) XCom push not working reliably

2016-05-09 Thread Alex Papanicolaou (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-62?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277508#comment-15277508 ] Alex Papanicolaou commented on AIRFLOW-62: -- But I have no idea what the problem could be...

[jira] [Commented] (AIRFLOW-47) ExternalTaskSensor causes scheduling dead lock

2016-05-09 Thread Hila Visan (JIRA)
[ https://issues.apache.org/jira/browse/AIRFLOW-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15277581#comment-15277581 ] Hila Visan commented on AIRFLOW-47: --- Hi Siddharth, Did you manage to reproduce the problem? Thanks >