[GitHub] [airflow] aviemzur commented on issue #6824: [AIRFLOW-6258] add CloudFormation operators to AWS providers

2020-01-07 Thread GitBox
aviemzur commented on issue #6824: [AIRFLOW-6258] add CloudFormation operators to AWS providers URL: https://github.com/apache/airflow/pull/6824#issuecomment-571933340 @nuclearpinguin made the changes you requested: [Changes after CR

[GitHub] [airflow] yuqian90 commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
yuqian90 commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364092453 ## File path: airflow/models/dag.py ## @@ -955,6 +993,60 @@ def clear(

[GitHub] [airflow] codecov-io edited a comment on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811#issuecomment-568228799 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6811?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811#issuecomment-568228799 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6811?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps() -- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps() -- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[GitHub] [airflow] dazza-codes commented on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs

2020-01-07 Thread GitBox
dazza-codes commented on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811#issuecomment-571888722 Did another self-review and most things look OK and test OK. There is a minor revision to how the client polls the AWS Batch

[GitHub] [airflow] dstandish edited a comment on issue #6850: [AIRFLOW-6296] add mssql odbc hook

2020-01-07 Thread GitBox
dstandish edited a comment on issue #6850: [AIRFLOW-6296] add mssql odbc hook URL: https://github.com/apache/airflow/pull/6850#issuecomment-571734050 also @vamega ... re your comment > I'm not sure how well known this technique is, but you can see a demonstration of it here

[GitHub] [airflow] codecov-io edited a comment on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need)

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need) URL: https://github.com/apache/airflow/pull/6975#issuecomment-569916253 #

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps() -- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Summary: Let emoji encoded properly for json.dumps() -- BaseSQLToGoogleCloudStorageOperator

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Updated] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Damon Liao updated AIRFLOW-6505: Description: Emoji can't be encoded properly when ` json.dumps()` and 'UTF-8', problem fixed by

[jira] [Created] (AIRFLOW-6505) Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator

2020-01-07 Thread Damon Liao (Jira)
Damon Liao created AIRFLOW-6505: --- Summary: Let emoji encoded properly for json.dumps()-- BaseSQLToGoogleCloudStorageOperator Key: AIRFLOW-6505 URL: https://issues.apache.org/jira/browse/AIRFLOW-6505

[GitHub] [airflow] albertusk95 edited a comment on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
albertusk95 edited a comment on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571876662 > existing tests for connection added via db/cli needs to work well, I guess the current tests don't support

[GitHub] [airflow] codecov-io edited a comment on issue #7090: [AIRFLOW-6495] Load DAG only once when running a task using StandardTaskRunner

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7090: [AIRFLOW-6495] Load DAG only once when running a task using StandardTaskRunner URL: https://github.com/apache/airflow/pull/7090#issuecomment-571428318 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7090?src=pr=h1) Report > Merging

[GitHub] [airflow] albertusk95 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
albertusk95 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571876662 > existing tests for connection added via db/cli needs to work well, I think the current tests don't support connection

[GitHub] [airflow] codecov-io edited a comment on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting URL: https://github.com/apache/airflow/pull/7097#issuecomment-571853779 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7097?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting URL: https://github.com/apache/airflow/pull/7097#issuecomment-571853779 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7097?src=pr=h1) Report > Merging

[GitHub] [airflow] lokeshlal commented on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need)

2020-01-07 Thread GitBox
lokeshlal commented on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need) URL: https://github.com/apache/airflow/pull/6975#issuecomment-571872342 Hi @dimberman, I have done changes in few more test cases to pass

[jira] [Commented] (AIRFLOW-6497) Scheduler creates DagBag in the same process with outdated info

2020-01-07 Thread Qian Yu (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010303#comment-17010303 ] Qian Yu commented on AIRFLOW-6497: -- [~kamil.bregula] Do you want to create a PR? Or if you can give

[GitHub] [airflow] kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364042758 ## File path: airflow/example_dags/example_external_task_marker.py ## @@ -0,0

[GitHub] [airflow] kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364043684 ## File path: airflow/models/dag.py ## @@ -955,6 +993,60 @@ def clear(

[GitHub] [airflow] kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364043507 ## File path: airflow/models/dag.py ## @@ -955,6 +993,60 @@ def clear(

[GitHub] [airflow] kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364042612 ## File path: airflow/example_dags/example_external_task_marker.py ## @@ -0,0

[GitHub] [airflow] kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker

2020-01-07 Thread GitBox
kaxil commented on a change in pull request #6633: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/6633#discussion_r364042552 ## File path: airflow/example_dags/example_external_task_marker.py ## @@ -0,0

[GitHub] [airflow] codecov-io edited a comment on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571862145 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7098?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
codecov-io commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571862145 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7098?src=pr=h1) Report > Merging

[GitHub] [airflow] TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571861953 Adding a note. I'm trying to think of a workaround if anyone actually was relying on the previous behavior.

[jira] [Commented] (AIRFLOW-2516) Deadlock found when trying to update task_instance table

2020-01-07 Thread Jarek Potiuk (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010259#comment-17010259 ] Jarek Potiuk commented on AIRFLOW-2516: --- Yep. They are the same. That 's what you would expect.

[jira] [Updated] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Joel Croteau (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joel Croteau updated AIRFLOW-4453: -- Attachment: 3_step.png > none_failed trigger rule cascading skipped state to downstream

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Joel Croteau (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010257#comment-17010257 ] Joel Croteau commented on AIRFLOW-4453: --- [~kamil.bregula], added an extra step with a dummy rule

[GitHub] [airflow] mik-laj commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
mik-laj commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571854915 I think that a note in UPDATING.md could also be useful. Some people may have DAGs that relied on this

[GitHub] [airflow] TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
TV4Fun commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571854759 Yes, but not right now. Might be a little while before I have time. On Tue, Jan 7, 2020, 5:45 PM Kamil

[GitHub] [airflow] mik-laj commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
mik-laj commented on issue #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098#issuecomment-571854505 Can you add tests to avoid regression?

[GitHub] [airflow] codecov-io commented on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread GitBox
codecov-io commented on issue #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting URL: https://github.com/apache/airflow/pull/7097#issuecomment-571853779 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7097?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010244#comment-17010244 ] ASF GitHub Bot commented on AIRFLOW-4453: - TV4Fun commented on pull request #7098:

[GitHub] [airflow] TV4Fun opened a new pull request #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation

2020-01-07 Thread GitBox
TV4Fun opened a new pull request #7098: [AIRFLOW-4453] Make behavior of `none_failed` consistent with documentation URL: https://github.com/apache/airflow/pull/7098 The documentation for the `none_failed` trigger rule (https://airflow.apache.org/docs/stable/concepts.html#trigger-rules)

[GitHub] [airflow] mik-laj commented on issue #6993: [AIRFLOW-4502] new cli command - task_states_for_dag_run

2020-01-07 Thread GitBox
mik-laj commented on issue #6993: [AIRFLOW-4502] new cli command - task_states_for_dag_run URL: https://github.com/apache/airflow/pull/6993#issuecomment-571853166 Each test should delete and create data in the database, but unfortunately, we have one database that is shared by all tests,

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010240#comment-17010240 ] Kamil Bregula commented on AIRFLOW-4453: Perfectly. Can you create PR then? > none_failed

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010239#comment-17010239 ] Kamil Bregula commented on AIRFLOW-4453: Can you check what happens if you add one more task at

[jira] [Commented] (AIRFLOW-2516) Deadlock found when trying to update task_instance table

2020-01-07 Thread Zhidong (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010233#comment-17010233 ] Zhidong commented on AIRFLOW-2516: -- The execution plans for *SELECT ... WHERE FOR UPDATE* and *UPDATE

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Joel Croteau (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010232#comment-17010232 ] Joel Croteau commented on AIRFLOW-4453: --- The problem seems to be [this

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010228#comment-17010228 ] Kamil Bregula commented on AIRFLOW-4453: I am so sorry, my bad. I will correct it in a moment

[jira] [Reopened] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula reopened AIRFLOW-4453: > none_failed trigger rule cascading skipped state to downstream tasks >

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io commented on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io commented on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096#issuecomment-571842522 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7096?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010218#comment-17010218 ] ASF GitHub Bot commented on AIRFLOW-5704: - potiuk commented on pull request #6516:

[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010219#comment-17010219 ] ASF GitHub Bot commented on AIRFLOW-5704: - potiuk commented on pull request #6516:

[GitHub] [airflow] potiuk closed pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

2020-01-07 Thread GitBox
potiuk closed pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk opened a new pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing

2020-01-07 Thread GitBox
potiuk opened a new pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516 Note only last commit is important as this depends on #7084 * Fixed problem that Kubernetes tests were testing latest master

[jira] [Commented] (AIRFLOW-6504) KubernetesExecutor: Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010212#comment-17010212 ] ASF GitHub Bot commented on AIRFLOW-6504: - kaxil commented on pull request #7097:

[GitHub] [airflow] kaxil opened a new pull request #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread GitBox
kaxil opened a new pull request #7097: [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting URL: https://github.com/apache/airflow/pull/7097 Currently `airflow.cfg` file can be passed via ConfigMap using `airflow_configmap` under `[kubernetes]` section setting. We

[jira] [Commented] (AIRFLOW-4453) none_failed trigger rule cascading skipped state to downstream tasks

2020-01-07 Thread Joel Croteau (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010208#comment-17010208 ] Joel Croteau commented on AIRFLOW-4453: --- [~kamil.bregula], this ticket is about a bug in the

[jira] [Work started] (AIRFLOW-6451) self._print_stat() in dag_processing.py should be skippable by config option

2020-01-07 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6451 started by t oo. - > self._print_stat() in dag_processing.py should be skippable by config option >

[GitHub] [airflow] tooptoop4 commented on issue #6993: [AIRFLOW-4502] new cli command - task_states_for_dag_run

2020-01-07 Thread GitBox
tooptoop4 commented on issue #6993: [AIRFLOW-4502] new cli command - task_states_for_dag_run URL: https://github.com/apache/airflow/pull/6993#issuecomment-571833089 @mik-laj do u think tests/www/test_views.py test is failing due to commands being run in tests/cli/task_command.py? I

[jira] [Commented] (AIRFLOW-6451) self._print_stat() in dag_processing.py should be skippable by config option

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010188#comment-17010188 ] ASF GitHub Bot commented on AIRFLOW-6451: - tooptoop4 commented on pull request #7096:

[GitHub] [airflow] tooptoop4 opened a new pull request #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable

2020-01-07 Thread GitBox
tooptoop4 opened a new pull request #7096: [AIRFLOW-6451] self._print_stat() in dag_processing.py should be skippable URL: https://github.com/apache/airflow/pull/7096 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [ ] Description

[jira] [Created] (AIRFLOW-6504) KubernetesExecutor: Allow specifying configmap for Airflow Local Setting

2020-01-07 Thread Kaxil Naik (Jira)
Kaxil Naik created AIRFLOW-6504: --- Summary: KubernetesExecutor: Allow specifying configmap for Airflow Local Setting Key: AIRFLOW-6504 URL: https://issues.apache.org/jira/browse/AIRFLOW-6504 Project:

[jira] [Commented] (AIRFLOW-6322) Add pytest marker for slow tests

2020-01-07 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010165#comment-17010165 ] ASF GitHub Bot commented on AIRFLOW-6322: - dazza-codes commented on pull request #6876:

[GitHub] [airflow] dazza-codes closed pull request #6876: [AIRFLOW-6322] Add pytest.mark.slow marker

2020-01-07 Thread GitBox
dazza-codes closed pull request #6876: [AIRFLOW-6322] Add pytest.mark.slow marker URL: https://github.com/apache/airflow/pull/6876 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] dazza-codes commented on issue #6876: [AIRFLOW-6322] Add pytest.mark.slow marker

2020-01-07 Thread GitBox
dazza-codes commented on issue #6876: [AIRFLOW-6322] Add pytest.mark.slow marker URL: https://github.com/apache/airflow/pull/6876#issuecomment-571824125 The marker definitions will be decided and defined outside of this PR - closing this in deference to something else.

[jira] [Created] (AIRFLOW-6503) support more schedule expressions (ie last day of the month)

2020-01-07 Thread t oo (Jira)
t oo created AIRFLOW-6503: - Summary: support more schedule expressions (ie last day of the month) Key: AIRFLOW-6503 URL: https://issues.apache.org/jira/browse/AIRFLOW-6503 Project: Apache Airflow

[GitHub] [airflow] tooptoop4 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run

2020-01-07 Thread GitBox
tooptoop4 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run URL: https://github.com/apache/airflow/pull/4751#discussion_r363999098 ## File path: airflow/models/dagrun.py ## @@ -263,48 +262,34 @@ def update_state(self,

[GitHub] [airflow] codecov-io edited a comment on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator URL: https://github.com/apache/airflow/pull/5970#issuecomment-571816542 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5970?src=pr=h1) Report > Merging

[GitHub] [airflow] tooptoop4 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run

2020-01-07 Thread GitBox
tooptoop4 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run URL: https://github.com/apache/airflow/pull/4751#discussion_r363998318 ## File path: airflow/models/dagrun.py ## @@ -263,48 +262,34 @@ def update_state(self,

[jira] [Created] (AIRFLOW-6502) Support swarm networks in DockerSwarmOperator

2020-01-07 Thread Arseniy (Jira)
Arseniy created AIRFLOW-6502: Summary: Support swarm networks in DockerSwarmOperator Key: AIRFLOW-6502 URL: https://issues.apache.org/jira/browse/AIRFLOW-6502 Project: Apache Airflow Issue Type:

[GitHub] [airflow] codecov-io commented on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator

2020-01-07 Thread GitBox
codecov-io commented on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator URL: https://github.com/apache/airflow/pull/5970#issuecomment-571816542 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5970?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6501) UI - show next_execution date, first exec_date, creation date (ie timestamp when first parsed into dagbag)

2020-01-07 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6501: -- Summary: UI - show next_execution date, first exec_date, creation date (ie timestamp when first parsed into

[GitHub] [airflow] dazza-codes commented on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs

2020-01-07 Thread GitBox
dazza-codes commented on issue #6811: [AIRFLOW-6245] Add custom waiters for AWS batch jobs URL: https://github.com/apache/airflow/pull/6811#issuecomment-571814074 Rebased and pushed again, travis failed one test with a timeout or something, not something specific to this PR.

[jira] [Created] (AIRFLOW-6501) UI - show next_execution date

2020-01-07 Thread t oo (Jira)
t oo created AIRFLOW-6501: - Summary: UI - show next_execution date Key: AIRFLOW-6501 URL: https://issues.apache.org/jira/browse/AIRFLOW-6501 Project: Apache Airflow Issue Type: New Feature

[jira] [Commented] (AIRFLOW-1510) Scheduler: priority_weight sorting not applied across multiple DAGs

2020-01-07 Thread Luke Bodeen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010141#comment-17010141 ] Luke Bodeen commented on AIRFLOW-1510: -- another idea like [~JosephHarris] said was maybe we remove

[jira] [Updated] (AIRFLOW-6500) dependency 'Execution Date' FAILED: Execution date is in the future

2020-01-07 Thread Pavan Maguluri (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavan Maguluri updated AIRFLOW-6500: Priority: Major (was: Minor) > dependency 'Execution Date' FAILED: Execution date is in

[GitHub] [airflow] tooptoop4 commented on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates

2020-01-07 Thread GitBox
tooptoop4 commented on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-571797817 passed This is an automated message from

[GitHub] [airflow] codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-570935774 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=h1) Report > :exclamation:

[GitHub] [airflow] codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-570935774 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-570935774 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=h1) Report > Merging

[GitHub] [airflow] m1racoli commented on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator

2020-01-07 Thread GitBox
m1racoli commented on issue #5970: [AIRFLOW-2923][AIRFLOW-1784] Implement LatestOnlyOperator as BaseBranchOperator URL: https://github.com/apache/airflow/pull/5970#issuecomment-571794871 Documentation updated and pylint issues fixed.

[jira] [Closed] (AIRFLOW-3435) Example KubernetesOperator docs for Volume mount wrong

2020-01-07 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik closed AIRFLOW-3435. --- Resolution: Implemented Resolved in https://github.com/apache/airflow/pull/3875 > Example

[jira] [Commented] (AIRFLOW-915) Allow task context to be modified by pre_execute hook

2020-01-07 Thread Paul Rhodes (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17010107#comment-17010107 ] Paul Rhodes commented on AIRFLOW-915: - This is no longer required since the it can be achieved by

[jira] [Assigned] (AIRFLOW-915) Allow task context to be modified by pre_execute hook

2020-01-07 Thread Paul Rhodes (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Rhodes reassigned AIRFLOW-915: --- Assignee: Paul Rhodes > Allow task context to be modified by pre_execute hook >

[GitHub] [airflow] codecov-io edited a comment on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need)

2020-01-07 Thread GitBox
codecov-io edited a comment on issue #6975: [AIRFLOW-1467] Dynamic pooling via allowing tasks to use more than one pool slot (depending upon the need) URL: https://github.com/apache/airflow/pull/6975#issuecomment-569916253 #

[GitHub] [airflow] tooptoop4 edited a comment on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
tooptoop4 edited a comment on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571765352 existing tests for connection added via db/cli needs to work

[GitHub] [airflow] tooptoop4 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
tooptoop4 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571765352 existing tests for connection added via/cli needs to work This is

[GitHub] [airflow] tooptoop4 commented on a change in pull request #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
tooptoop4 commented on a change in pull request #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#discussion_r363944283 ## File path: airflow/contrib/hooks/spark_submit_hook.py ## @@ -190,17 +189,22 @@ def

[GitHub] [airflow] tooptoop4 commented on a change in pull request #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
tooptoop4 commented on a change in pull request #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#discussion_r363943123 ## File path: airflow/contrib/hooks/spark_submit_hook.py ## @@ -174,8 +174,7 @@ def

[GitHub] [airflow] tooptoop4 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection

2020-01-07 Thread GitBox
tooptoop4 commented on issue #7075: [AIRFLOW-6212] SparkSubmitHook resolve connection URL: https://github.com/apache/airflow/pull/7075#issuecomment-571762855 > > > @tooptoop4 if we don't remove the spark check on line 177, how to use this hook to track driver status deployed on yarn,

  1   2   3   4   >