[GitHub] [airflow] stale[bot] commented on issue #3229: [AIRFLOW-2325] Add cloudwatch task handler (IN PROGRESS)

2019-12-06 Thread GitBox
stale[bot] commented on issue #3229: [AIRFLOW-2325] Add cloudwatch task handler (IN PROGRESS) URL: https://github.com/apache/airflow/pull/3229#issuecomment-562823893 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further

[jira] [Commented] (AIRFLOW-6056) Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990307#comment-16990307 ] ASF GitHub Bot commented on AIRFLOW-6056: - aviemzur commented on pull request #6655:

[jira] [Commented] (AIRFLOW-6056) Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990308#comment-16990308 ] ASF GitHub Bot commented on AIRFLOW-6056: - aviemzur commented on pull request #6655:

[GitHub] [airflow] aviemzur opened a new pull request #6655: [AIRFLOW-6056] Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id

2019-12-06 Thread GitBox
aviemzur opened a new pull request #6655: [AIRFLOW-6056] Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id URL: https://github.com/apache/airflow/pull/6655 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following

[GitHub] [airflow] aviemzur closed pull request #6655: [AIRFLOW-6056] Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id

2019-12-06 Thread GitBox
aviemzur closed pull request #6655: [AIRFLOW-6056] Allow EmrAddStepsOperator to accept job_flow_name as alternative to job_flow_id URL: https://github.com/apache/airflow/pull/6655 This is an automated message from the

[GitHub] [airflow] codecov-io commented on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug

2019-12-06 Thread GitBox
codecov-io commented on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug URL: https://github.com/apache/airflow/pull/6732#issuecomment-562805777 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6732?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug URL: https://github.com/apache/airflow/pull/6732#issuecomment-562805777 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6732?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6184) Add `email_on_retry` and `email_on_failure` for DAGs

2019-12-06 Thread Gerard Casas Saez (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gerard Casas Saez updated AIRFLOW-6184: --- Description: Currently, there's support for  `email_on_retry` and

[GitHub] [airflow] codecov-io edited a comment on issue #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#issuecomment-560962980 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6715?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5945) Fix limitation with OperatorLinks when using DAG Serialization

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990215#comment-16990215 ] ASF GitHub Bot commented on AIRFLOW-5945: - kaxil commented on pull request #6715:

[jira] [Resolved] (AIRFLOW-5945) Fix limitation with OperatorLinks when using DAG Serialization

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5945. - Fix Version/s: 1.10.7 2.0.0 Resolution: Fixed > Fix limitation with

[jira] [Commented] (AIRFLOW-5945) Fix limitation with OperatorLinks when using DAG Serialization

2019-12-06 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990216#comment-16990216 ] ASF subversion and git services commented on AIRFLOW-5945: -- Commit

[GitHub] [airflow] kaxil merged pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
kaxil merged pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715 This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-559120739 I need to investigate what is happening with task logging, but otherwise this PR is working and the code is

[GitHub] [airflow] ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-559120739 I need to investigate what is happening with task logging, but otherwise this PR is working and the code is

[jira] [Commented] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990179#comment-16990179 ] ASF subversion and git services commented on AIRFLOW-6192: -- Commit

[jira] [Resolved] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-6192. Resolution: Fixed > Stop creating Hook from SFTPSensor.__init__ >

[GitHub] [airflow] ashb merged pull request #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread GitBox
ashb merged pull request #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__ URL: https://github.com/apache/airflow/pull/6748 This is an automated message from the Apache Git Service. To respond to the

[jira] [Commented] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990178#comment-16990178 ] ASF GitHub Bot commented on AIRFLOW-6192: - ashb commented on pull request #6748: [AIRFLOW-6192]

[GitHub] [airflow] codecov-io edited a comment on issue #6660: [AIRFLOW-6065] Add Stackdriver Task Handler

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6660: [AIRFLOW-6065] Add Stackdriver Task Handler URL: https://github.com/apache/airflow/pull/6660#issuecomment-561766454 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6660?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value URL: https://github.com/apache/airflow/pull/6726#issuecomment-561617155 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6726?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6726: [AIRFLOW-6170] BranchPythonOperator does not do XCom push of returned value URL: https://github.com/apache/airflow/pull/6726#issuecomment-561617155 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6726?src=pr=h1) Report > Merging

[GitHub] [airflow] mik-laj commented on issue #6683: [AIRFLOW-6019] Add flushing in execute method for BigQueryCursor

2019-12-06 Thread GitBox
mik-laj commented on issue #6683: [AIRFLOW-6019] Add flushing in execute method for BigQueryCursor URL: https://github.com/apache/airflow/pull/6683#issuecomment-562733759 The best way to learn this is to read the source test codes for operators and hooks for other GCP services. All

[GitHub] [airflow] zachliu commented on issue #6607: [AIRFLOW-5462] Bump FAB to 2.2.0 to fix OAuth login issue

2019-12-06 Thread GitBox
zachliu commented on issue #6607: [AIRFLOW-5462] Bump FAB to 2.2.0 to fix OAuth login issue URL: https://github.com/apache/airflow/pull/6607#issuecomment-562724548 We've been experiencing a similar issue as AIRFLOW-5462. Everything works as expected for a valid login using the company

[jira] [Commented] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990119#comment-16990119 ] Bjorn Olsen commented on AIRFLOW-6190: -- Thanks for the feedback guys. It does seem similar. Could

[jira] [Commented] (AIRFLOW-6167) Escape col name in MysqlToHive operator

2019-12-06 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990104#comment-16990104 ] ASF subversion and git services commented on AIRFLOW-6167: -- Commit

[jira] [Commented] (AIRFLOW-6167) Escape col name in MysqlToHive operator

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990103#comment-16990103 ] ASF GitHub Bot commented on AIRFLOW-6167: - KevinYang21 commented on pull request #6741:

[GitHub] [airflow] KevinYang21 merged pull request #6741: [AIRFLOW-6167] Escape column name in create table in hive

2019-12-06 Thread GitBox
KevinYang21 merged pull request #6741: [AIRFLOW-6167] Escape column name in create table in hive URL: https://github.com/apache/airflow/pull/6741 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] codecov-io edited a comment on issue #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__ URL: https://github.com/apache/airflow/pull/6748#issuecomment-562700280 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6748?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread GitBox
codecov-io commented on issue #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__ URL: https://github.com/apache/airflow/pull/6748#issuecomment-562700280 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6748?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4184) Add an AWS Athena Helper to insert into table

2019-12-06 Thread Cooper Gillan (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990086#comment-16990086 ] Cooper Gillan commented on AIRFLOW-4184: This looks resolved? > Add an AWS Athena Helper to

[jira] [Commented] (AIRFLOW-4177) Check types in tests

2019-12-06 Thread Cooper Gillan (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990085#comment-16990085 ] Cooper Gillan commented on AIRFLOW-4177: This looks resolved? > Check types in tests >

[GitHub] [airflow] codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-558211492 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6627?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-558211492 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6627?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990034#comment-16990034 ] Kaxil Naik commented on AIRFLOW-6190: - Here is the PR: https://github.com/apache/airflow/pull/6732

[jira] [Comment Edited] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990034#comment-16990034 ] Kaxil Naik edited comment on AIRFLOW-6190 at 12/6/19 5:58 PM: -- Here is the

[jira] [Commented] (AIRFLOW-6175) Executor changes queued tasks to scheduled without removing from task queue

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990033#comment-16990033 ] Kaxil Naik commented on AIRFLOW-6175: - PR: https://github.com/apache/airflow/pull/6732 > Executor

[jira] [Resolved] (AIRFLOW-5144) Add confirmation dialog when deleting variable

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5144. - Assignee: Kaxil Naik Resolution: Fixed > Add confirmation dialog when deleting variable

[GitHub] [airflow] kaxil merged pull request #6745: [Airflow-5144] confirmation dialogue on delete

2019-12-06 Thread GitBox
kaxil merged pull request #6745: [Airflow-5144] confirmation dialogue on delete URL: https://github.com/apache/airflow/pull/6745 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] kaxil merged pull request #6747: [AIRFLOW-XXX] Fix broken link in CONTRIBUTING.rst

2019-12-06 Thread GitBox
kaxil merged pull request #6747: [AIRFLOW-XXX] Fix broken link in CONTRIBUTING.rst URL: https://github.com/apache/airflow/pull/6747 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] kaxil commented on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug

2019-12-06 Thread GitBox
kaxil commented on issue #6732: [AIRFLOW-6175] fixes scheduler queue bug URL: https://github.com/apache/airflow/pull/6732#issuecomment-562673417 Following tests are failing: ``` FAILED tests/jobs/test_backfill_job.py::TestBackfillJob::test_backfill_examples_0_example_branch_operator

[GitHub] [airflow] codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task …

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task … URL: https://github.com/apache/airflow/pull/6652#issuecomment-558277657 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6652?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5523) DAGs never get cleaned up from webservers if DAG file is removed from the scheduler

2019-12-06 Thread Chris Palmer (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990015#comment-16990015 ] Chris Palmer commented on AIRFLOW-5523: --- What is the goal here? If it is to clean up the DAGs

[jira] [Updated] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6192: Description: >From Slack

[jira] [Closed] (AIRFLOW-6176) Form submits change url from https to http when redirectoung

2019-12-06 Thread Ronald Fenner (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ronald Fenner closed AIRFLOW-6176. -- Resolution: Resolved > Form submits change url from https to http when redirectoung >

[jira] [Commented] (AIRFLOW-6176) Form submits change url from https to http when redirectoung

2019-12-06 Thread Ronald Fenner (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990013#comment-16990013 ] Ronald Fenner commented on AIRFLOW-6176: Thanks that enable_proxy_fix worked didn't know it's

[GitHub] [airflow] kaxil opened a new pull request #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread GitBox
kaxil opened a new pull request #6748: [AIRFLOW-6192] Stop creating Hook from SFTPSensor.__init__ URL: https://github.com/apache/airflow/pull/6748 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989992#comment-16989992 ] ASF GitHub Bot commented on AIRFLOW-6192: - kaxil commented on pull request #6748:

[jira] [Created] (AIRFLOW-6192) Stop creating Hook from SFTPSensor.__init__

2019-12-06 Thread Kaxil Naik (Jira)
Kaxil Naik created AIRFLOW-6192: --- Summary: Stop creating Hook from SFTPSensor.__init__ Key: AIRFLOW-6192 URL: https://issues.apache.org/jira/browse/AIRFLOW-6192 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-6176) Form submits change url from https to http when redirectoung

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989985#comment-16989985 ] Ash Berlin-Taylor commented on AIRFLOW-6176:

[jira] [Commented] (AIRFLOW-6173) Remote logging handler does not close at the end of run

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989979#comment-16989979 ] Ash Berlin-Taylor commented on AIRFLOW-6173: That's odd - the logging should all get closed

[GitHub] [airflow] ashb commented on a change in pull request #6723: [AIRFLOW-6168] allows proxyfix to be configurable

2019-12-06 Thread GitBox
ashb commented on a change in pull request #6723: [AIRFLOW-6168] allows proxyfix to be configurable URL: https://github.com/apache/airflow/pull/6723#discussion_r354928703 ## File path: docs/howto/run-behind-proxy.rst ## @@ -79,6 +79,12 @@ middleware in your

[jira] [Commented] (AIRFLOW-4847) Update config template to reflect supporting different Celery pool implementation

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989966#comment-16989966 ] Ash Berlin-Taylor commented on AIRFLOW-4847: Sounds good! > Update config template to

[jira] [Commented] (AIRFLOW-6134) Scheduler hanging every 45 minutes, workers hanging on every job

2019-12-06 Thread Brent Driskill (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989963#comment-16989963 ] Brent Driskill commented on AIRFLOW-6134: - Just as an update, we are investigating this as an

[jira] [Assigned] (AIRFLOW-6134) Scheduler hanging every 45 minutes, workers hanging on every job

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor reassigned AIRFLOW-6134: -- Assignee: Ash Berlin-Taylor > Scheduler hanging every 45 minutes, workers

[jira] [Commented] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989936#comment-16989936 ] Ash Berlin-Taylor commented on AIRFLOW-6190: {quote}The scheduler thinks the next task can

[GitHub] [airflow] kaxil commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
kaxil commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354917096 ## File path: airflow/serialization/serialized_objects.py ## @@ -354,6 +370,65

[GitHub] [airflow] zuku1985 commented on issue #6683: [AIRFLOW-6019] Add flushing in execute method for BigQueryCursor

2019-12-06 Thread GitBox
zuku1985 commented on issue #6683: [AIRFLOW-6019] Add flushing in execute method for BigQueryCursor URL: https://github.com/apache/airflow/pull/6683#issuecomment-562640758 I will need some hints on how to do it. This is an

[jira] [Commented] (AIRFLOW-6188) [Web UI] Support add DAG on web ui

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989933#comment-16989933 ] Ash Berlin-Taylor commented on AIRFLOW-6188: [~Kangtian] if your environment supports it

[jira] [Resolved] (AIRFLOW-6188) [Web UI] Support add DAG on web ui

2019-12-06 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-6188. Resolution: Won't Fix > [Web UI] Support add DAG on web ui >

[GitHub] [airflow] codecov-io edited a comment on issue #6745: [Airflow-5144] confirmation dialogue on delete

2019-12-06 Thread GitBox
codecov-io edited a comment on issue #6745: [Airflow-5144] confirmation dialogue on delete URL: https://github.com/apache/airflow/pull/6745#issuecomment-562636952 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6745?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #6745: [Airflow-5144] confirmation dialogue on delete

2019-12-06 Thread GitBox
codecov-io commented on issue #6745: [Airflow-5144] confirmation dialogue on delete URL: https://github.com/apache/airflow/pull/6745#issuecomment-562636952 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6745?src=pr=h1) Report > Merging

[GitHub] [airflow] ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354901371 ## File path: airflow/serialization/serialized_objects.py ## @@ -354,6 +370,65

[GitHub] [airflow] ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354900317 ## File path: airflow/serialization/serialized_objects.py ## @@ -354,6 +370,65

[GitHub] [airflow] ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354900317 ## File path: airflow/serialization/serialized_objects.py ## @@ -354,6 +370,65

[GitHub] [airflow] ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
ashb commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354899097 ## File path: airflow/serialization/serialized_objects.py ## @@ -354,6 +370,65

[GitHub] [airflow] feluelle commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment

2019-12-06 Thread GitBox
feluelle commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment URL: https://github.com/apache/airflow/pull/6746#issuecomment-562590485 > We can increase verbosity for time being until we eliminate all timeouting tests :< Sounds reasonable.

[GitHub] [airflow] feluelle commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment

2019-12-06 Thread GitBox
feluelle commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment URL: https://github.com/apache/airflow/pull/6746#issuecomment-562589413  for `.` in CI This is an

[GitHub] [airflow] kaxil opened a new pull request #6747: [AIRFLOW-XXX] Fix broken link in CONTRIBUTING.rst

2019-12-06 Thread GitBox
kaxil opened a new pull request #6747: [AIRFLOW-XXX] Fix broken link in CONTRIBUTING.rst URL: https://github.com/apache/airflow/pull/6747 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow

[jira] [Commented] (AIRFLOW-6188) [Web UI] Support add DAG on web ui

2019-12-06 Thread Kangtian (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989768#comment-16989768 ] Kangtian commented on AIRFLOW-6188: --- [~kamil.bregula]   But in k8s cluster, dag folder can be cloud

[GitHub] [airflow] ashb commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
ashb commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-562575923 > * [ ] Test Stackdriver Logging Handler: #6660 :grin: Looking at what that does it should be fine --

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-06 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989740#comment-16989740 ] ASF subversion and git services commented on AIRFLOW-6189: -- Commit

[jira] [Commented] (AIRFLOW-6189) Reduce the maximum test duration to 8 minutes

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989739#comment-16989739 ] ASF GitHub Bot commented on AIRFLOW-6189: - mik-laj commented on pull request #6744:

[GitHub] [airflow] mik-laj merged pull request #6744: [AIRFLOW-6189] Reduce the maximum test duration to 8 minutes

2019-12-06 Thread GitBox
mik-laj merged pull request #6744: [AIRFLOW-6189] Reduce the maximum test duration to 8 minutes URL: https://github.com/apache/airflow/pull/6744 This is an automated message from the Apache Git Service. To respond to the

[jira] [Comment Edited] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989735#comment-16989735 ] Bjorn Olsen edited comment on AIRFLOW-6190 at 12/6/19 1:02 PM: --- I changed

[GitHub] [airflow] nuclearpinguin commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment

2019-12-06 Thread GitBox
nuclearpinguin commented on issue #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment URL: https://github.com/apache/airflow/pull/6746#issuecomment-562562162 @mik-laj @potiuk @ashb @feluelle what are your opinions on pytest verbosity? Should we print all tests on CI

[jira] [Comment Edited] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989735#comment-16989735 ] Bjorn Olsen edited comment on AIRFLOW-6190 at 12/6/19 12:59 PM: I

[GitHub] [airflow] mik-laj commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
mik-laj commented on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-562561472 - [ ] Stackdriver Logging Handler: https://github.com/apache/airflow/pull/6660

[GitHub] [airflow] mik-laj edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
mik-laj edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-562561472 - [ ] Test Stackdriver Logging Handler: https://github.com/apache/airflow/pull/6660

[jira] [Commented] (AIRFLOW-6191) Adjust pytest verbosity in CI and local environment

2019-12-06 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989734#comment-16989734 ] ASF GitHub Bot commented on AIRFLOW-6191: - nuclearpinguin commented on pull request #6746:

[jira] [Commented] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16989735#comment-16989735 ] Bjorn Olsen commented on AIRFLOW-6190: -- I changed the func to include a random duration sleep and

[GitHub] [airflow] nuclearpinguin opened a new pull request #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment

2019-12-06 Thread GitBox
nuclearpinguin opened a new pull request #6746: [AIRFLOW-6191] Adjust pytest verbosity in CI and local environment URL: https://github.com/apache/airflow/pull/6746 Make sure you have checked _all_ steps below. ### Jira - [ ] My PR addresses the following [Airflow

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[jira] [Created] (AIRFLOW-6191) Adjust pytest verbosity in CI and local environment

2019-12-06 Thread Tomasz Urbaszek (Jira)
Tomasz Urbaszek created AIRFLOW-6191: Summary: Adjust pytest verbosity in CI and local environment Key: AIRFLOW-6191 URL: https://issues.apache.org/jira/browse/AIRFLOW-6191 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-06 Thread GitBox
nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740#discussion_r354812890 ## File path: airflow/executors/executor_loader.py ## @@ -51,7 +52,7 @@ def get_default_executor(cls) ->

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-06 Thread GitBox
nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740#discussion_r354812698 ## File path: airflow/models/taskinstance.py ## @@ -885,6 +887,12 @@ def _run_raw_task(

[GitHub] [airflow] kaxil commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization

2019-12-06 Thread GitBox
kaxil commented on a change in pull request #6715: [AIRFLOW-5945] Make inbuilt OperatorLinks work when using Serialization URL: https://github.com/apache/airflow/pull/6715#discussion_r354808615 ## File path: tests/serialization/test_dag_serialization.py ## @@ -381,6

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[GitHub] [airflow] l0n3r4n83r commented on issue #6742: [Airflow-5144] fix alert on delete

2019-12-06 Thread GitBox
l0n3r4n83r commented on issue #6742: [Airflow-5144] fix alert on delete URL: https://github.com/apache/airflow/pull/6742#issuecomment-562548297 @KevinYang21 https://github.com/apache/airflow/pull/6745 opened a new PR This is

[jira] [Updated] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bjorn Olsen updated AIRFLOW-6190: - Description: Below dag creates 20 identical simple tasks which depend on each other in

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-06 Thread GitBox
nuclearpinguin commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740#discussion_r354802412 ## File path: airflow/executors/executor_loader.py ## @@ -51,7 +52,7 @@ def get_default_executor(cls) ->

[GitHub] [airflow] l0n3r4n83r opened a new pull request #6745: [Airflow-5144] confirmation dialogue on delete

2019-12-06 Thread GitBox
l0n3r4n83r opened a new pull request #6745: [Airflow-5144] confirmation dialogue on delete URL: https://github.com/apache/airflow/pull/6745 As per https://github.com/flask-admin/flask-admin/issues/1789 the deleted button did not show a confirmation alert. This issue has been fixed in

[jira] [Created] (AIRFLOW-6190) Task instances queued and dequeued before worker is ready, causing intermittently failed tasks

2019-12-06 Thread Bjorn Olsen (Jira)
Bjorn Olsen created AIRFLOW-6190: Summary: Task instances queued and dequeued before worker is ready, causing intermittently failed tasks Key: AIRFLOW-6190 URL: https://issues.apache.org/jira/browse/AIRFLOW-6190

[GitHub] [airflow] ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-559120739 I need to investigate what is happening with task logging, but otherwise this PR is working and the code is

[GitHub] [airflow] ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution.

2019-12-06 Thread GitBox
ashb edited a comment on issue #6627: [AIRFLOW-5931] Use os.fork when appropriate to speed up task execution. URL: https://github.com/apache/airflow/pull/6627#issuecomment-559120739 I need to investigate what is happening with task logging, but otherwise this PR is working and the code is

[GitHub] [airflow] potiuk commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-06 Thread GitBox
potiuk commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740#discussion_r354788194 ## File path: airflow/executors/executor_loader.py ## @@ -51,7 +52,7 @@ def get_default_executor(cls) ->

[GitHub] [airflow] potiuk commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor

2019-12-06 Thread GitBox
potiuk commented on a change in pull request #6740: [AIRFLOW-6181] Add InProcessExecutor URL: https://github.com/apache/airflow/pull/6740#discussion_r354787964 ## File path: airflow/executors/executor_loader.py ## @@ -51,7 +52,7 @@ def get_default_executor(cls) ->

  1   2   >