[jira] [Created] (AIRFLOW-6804) Add the basic test for example DAGs

2020-02-13 Thread Kamil Bregula (Jira)
Kamil Bregula created AIRFLOW-6804: -- Summary: Add the basic test for example DAGs Key: AIRFLOW-6804 URL: https://issues.apache.org/jira/browse/AIRFLOW-6804 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036762#comment-17036762 ] ASF GitHub Bot commented on AIRFLOW-6590: - nuclearpinguin commented on pull request #7370:

[jira] [Commented] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036766#comment-17036766 ] ASF subversion and git services commented on AIRFLOW-6590: -- Commit

[jira] [Commented] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036765#comment-17036765 ] ASF subversion and git services commented on AIRFLOW-6590: -- Commit

[jira] [Commented] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036763#comment-17036763 ] ASF subversion and git services commented on AIRFLOW-6590: -- Commit

[jira] [Resolved] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread Tomasz Urbaszek (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tomasz Urbaszek resolved AIRFLOW-6590. -- Fix Version/s: 1.10.10 Resolution: Done > Use batch db operations in jobs >

[jira] [Commented] (AIRFLOW-6590) Use batch db operations in jobs

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036764#comment-17036764 ] ASF subversion and git services commented on AIRFLOW-6590: -- Commit

[GitHub] [airflow] nuclearpinguin merged pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-13 Thread GitBox
nuclearpinguin merged pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] mik-laj merged pull request #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
mik-laj merged pull request #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] mik-laj commented on issue #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
mik-laj commented on issue #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418#issuecomment-586137010 Thanks! I merge it. This is an automated message from the Apache Git

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
boring-cyborg[bot] commented on issue #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418#issuecomment-586137058 Awesome work, congrats on your first merged pull request!

[GitHub] [airflow] davidshopback commented on issue #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
davidshopback commented on issue #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418#issuecomment-586135776 Thanks for taking a look! Sorry for the surprise, it's an aggressive security filter as the product is country specific.

[GitHub] [airflow] mik-laj commented on issue #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
mik-laj commented on issue #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418#issuecomment-586129084 https://user-images.githubusercontent.com/12058428/74509438-c6f92600-4f01-11ea-9965-837d9415798b.png;> A surprising message. Could I

[GitHub] [airflow] mik-laj commented on issue #7415: [AIRFLOW-6800] Close file object after parsing ssh config

2020-02-13 Thread GitBox
mik-laj commented on issue #7415: [AIRFLOW-6800] Close file object after parsing ssh config URL: https://github.com/apache/airflow/pull/7415#issuecomment-586128226 Travis is sad. I restarted a failed job. This is an

[GitHub] [airflow] davidshopback opened a new pull request #7418: [AIRFLOW-XXXX] Add ShopBack as an Airflow user

2020-02-13 Thread GitBox
davidshopback opened a new pull request #7418: [AIRFLOW-] Add ShopBack as an Airflow user URL: https://github.com/apache/airflow/pull/7418 Adding ShopBack to the Airflow user list in README.md --- Issue link: WILL BE INSERTED BY

[jira] [Closed] (AIRFLOW-6803) Add ShopBack as an Airflow user

2020-02-13 Thread David Chua (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Chua closed AIRFLOW-6803. --- Resolution: Invalid > Add ShopBack as an Airflow user > --- > >

[jira] [Updated] (AIRFLOW-6803) Add ShopBack as an Airflow user

2020-02-13 Thread David Chua (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Chua updated AIRFLOW-6803: Description: -Add ShopBack as an Airflow user- Sorry, I should read through the documentation

[jira] [Created] (AIRFLOW-6803) Add ShopBack as an Airflow user

2020-02-13 Thread David Chua (Jira)
David Chua created AIRFLOW-6803: --- Summary: Add ShopBack as an Airflow user Key: AIRFLOW-6803 URL: https://issues.apache.org/jira/browse/AIRFLOW-6803 Project: Apache Airflow Issue Type: Task

[GitHub] [airflow] davidshopback closed pull request #7417: Add ShopBack as official user of airflow

2020-02-13 Thread GitBox
davidshopback closed pull request #7417: Add ShopBack as official user of airflow URL: https://github.com/apache/airflow/pull/7417 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] codecov-io commented on issue #7416: [AIRFLOW-6802] honor dag.max_active_run in scheduler

2020-02-13 Thread GitBox
codecov-io commented on issue #7416: [AIRFLOW-6802] honor dag.max_active_run in scheduler URL: https://github.com/apache/airflow/pull/7416#issuecomment-586112833 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7416?src=pr=h1) Report > Merging

[GitHub] [airflow] davidshopback opened a new pull request #7417: Add ShopBack as official user of airflow

2020-02-13 Thread GitBox
davidshopback opened a new pull request #7417: Add ShopBack as official user of airflow URL: https://github.com/apache/airflow/pull/7417 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7417: Add ShopBack as official user of airflow

2020-02-13 Thread GitBox
boring-cyborg[bot] commented on issue #7417: Add ShopBack as official user of airflow URL: https://github.com/apache/airflow/pull/7417#issuecomment-586112668 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about

[GitHub] [airflow] codecov-io edited a comment on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo…

2020-02-13 Thread GitBox
codecov-io edited a comment on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo… URL: https://github.com/apache/airflow/pull/7395#issuecomment-585611569 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7395?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo…

2020-02-13 Thread GitBox
codecov-io edited a comment on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo… URL: https://github.com/apache/airflow/pull/7395#issuecomment-585611569 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7395?src=pr=h1) Report > Merging

[jira] [Commented] (AIRFLOW-6802) scheduler not honoring dag.max_active_run config

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036694#comment-17036694 ] ASF GitHub Bot commented on AIRFLOW-6802: - houqp commented on pull request #7416:

[GitHub] [airflow] houqp opened a new pull request #7416: [AIRFLOW-6802] honor dag.max_active_run in scheduler

2020-02-13 Thread GitBox
houqp opened a new pull request #7416: [AIRFLOW-6802] honor dag.max_active_run in scheduler URL: https://github.com/apache/airflow/pull/7416 commit 50efda5c69c1ddfaa869b408540182fb19f1a286 introduced a bug that prevents the scheduler from enforcing max active run config for all DAGs.

[jira] [Created] (AIRFLOW-6802) scheduler not honoring dag.max_active_run config

2020-02-13 Thread QP Hou (Jira)
QP Hou created AIRFLOW-6802: --- Summary: scheduler not honoring dag.max_active_run config Key: AIRFLOW-6802 URL: https://issues.apache.org/jira/browse/AIRFLOW-6802 Project: Apache Airflow Issue

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-586067049 High level feedback since this was a topic earlier: - the extra linting for the Dockerfile, which comes down to fairly

[GitHub] [airflow] kuromt commented on issue #7365: [AIRFLOW-5221] add host_aliases to KubernetesPodOperator

2020-02-13 Thread GitBox
kuromt commented on issue #7365: [AIRFLOW-5221] add host_aliases to KubernetesPodOperator URL: https://github.com/apache/airflow/pull/7365#issuecomment-586053550 ``` No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the

[GitHub] [airflow] yuqian90 opened a new pull request #7415: [AIRFLOW-6800] Close file object after parsing ssh config

2020-02-13 Thread GitBox
yuqian90 opened a new pull request #7415: [AIRFLOW-6800] Close file object after parsing ssh config URL: https://github.com/apache/airflow/pull/7415 This line of code in ssh/hooks.py is opening a file object without closing it in the scope where it is opened. This causes unnecessary

[jira] [Commented] (AIRFLOW-6800) SSHHook: Close file object when reading ssh config

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036617#comment-17036617 ] ASF GitHub Bot commented on AIRFLOW-6800: - yuqian90 commented on pull request #7415:

[jira] [Created] (AIRFLOW-6800) SSHHook: Close file object when reading ssh config

2020-02-13 Thread Qian Yu (Jira)
Qian Yu created AIRFLOW-6800: Summary: SSHHook: Close file object when reading ssh config Key: AIRFLOW-6800 URL: https://issues.apache.org/jira/browse/AIRFLOW-6800 Project: Apache Airflow Issue

[jira] [Created] (AIRFLOW-6801) Make use of timestamp field in ImportError object

2020-02-13 Thread Chris McLennon (Jira)
Chris McLennon created AIRFLOW-6801: --- Summary: Make use of timestamp field in ImportError object Key: AIRFLOW-6801 URL: https://issues.apache.org/jira/browse/AIRFLOW-6801 Project: Apache Airflow

[GitHub] [airflow] codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task …

2020-02-13 Thread GitBox
codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task … URL: https://github.com/apache/airflow/pull/6652#issuecomment-558277657 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6652?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6685) Add ThresholdCheckOperator for Data Quality Checking

2020-02-13 Thread alex l (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alex l updated AIRFLOW-6685: Description: This PR includes a new operator in *{{CheckOperator}}* that allows users to perform a

[jira] [Updated] (AIRFLOW-6685) Add ThresholdCheckOperator for Data Quality Checking

2020-02-13 Thread alex l (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alex l updated AIRFLOW-6685: Summary: Add ThresholdCheckOperator for Data Quality Checking (was: Add Data Quality Operators ) >

[jira] [Updated] (AIRFLOW-6685) Add Data Quality Operators

2020-02-13 Thread alex l (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] alex l updated AIRFLOW-6685: Description: This PR includes a new operator in `CheckOperator` that allows users to perform a

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-586039637 okay I'm going to assume that this "no pragma" is new and just remove the lines from the top of those files. I also used the

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-586036255 What are these new errors about encoding / pragma? I literally didn't change any of these files ```

[GitHub] [airflow] simis2626 commented on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo…

2020-02-13 Thread GitBox
simis2626 commented on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo… URL: https://github.com/apache/airflow/pull/7395#issuecomment-586034102 Thank you @potiuk appreciate the diagnosis!

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-586021498 okay to so to update everyone following (and future me reading back to remember this) we are first trying a simple approach

[jira] [Closed] (AIRFLOW-1947) airflow json file created i /tmp get wrong permission when using run_as_user

2020-02-13 Thread Soeren Laursen (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Soeren Laursen closed AIRFLOW-1947. --- Resolution: Fixed > airflow json file created i /tmp get wrong permission when using

[jira] [Created] (AIRFLOW-6799) webgui cannot display all tasks

2020-02-13 Thread Soeren Laursen (Jira)
Soeren Laursen created AIRFLOW-6799: --- Summary: webgui cannot display all tasks Key: AIRFLOW-6799 URL: https://issues.apache.org/jira/browse/AIRFLOW-6799 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-6789) Concurrency parameter does not work in the latest stable version 1.10.9

2020-02-13 Thread Denis Scheglow (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Scheglow updated AIRFLOW-6789: Component/s: worker > Concurrency parameter does not work in the latest stable version

[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-13 Thread GitBox
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585978655 Hey @ashb @mik-laj @nuclearpinguin -> all green for the system tests ready for backporting. BTW, I upladed the docs and

[GitHub] [airflow] potiuk commented on issue #7401: [AIRFLOW-6778] Add a configurable DAGs volume mount path for Kubernetes

2020-02-13 Thread GitBox
potiuk commented on issue #7401: [AIRFLOW-6778] Add a configurable DAGs volume mount path for Kubernetes URL: https://github.com/apache/airflow/pull/7401#issuecomment-585977885 Yeah. we have timeouts in various places - issue raised to Travis. Hopefully we can get the final step on moving

[GitHub] [airflow] codecov-io edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-13 Thread GitBox
codecov-io edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-583898179 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7389?src=pr=h1) Report > Merging

[GitHub] [airflow] potiuk commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-13 Thread GitBox
potiuk commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#discussion_r379122368 ## File path: BREEZE.rst ## @@ -611,7 +617,13 @@ This is the current syntax for `./breeze

[GitHub] [airflow] potiuk commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-13 Thread GitBox
potiuk commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#discussion_r379122655 ## File path: TESTING.rst ## @@ -31,8 +31,7 @@ Airflow Test Infrastructure * **System

[GitHub] [airflow] brandonwillard commented on issue #7401: [AIRFLOW-6778] Add a configurable DAGs volume mount path for Kubernetes

2020-02-13 Thread GitBox
brandonwillard commented on issue #7401: [AIRFLOW-6778] Add a configurable DAGs volume mount path for Kubernetes URL: https://github.com/apache/airflow/pull/7401#issuecomment-585974726 @potiuk, looks like the `[kubernetes][git]` test timed-out again. These errors arein some senserandom,

[GitHub] [airflow] ashb commented on a change in pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds

2020-02-13 Thread GitBox
ashb commented on a change in pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds URL: https://github.com/apache/airflow/pull/7363#discussion_r379119387 ## File path: tests/providers/google/cloud/operators/test_dataproc.py ## @@ -104,7 +104,7 @@

[jira] [Resolved] (AIRFLOW-2906) DataDog Integration for Airflow

2020-02-13 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-2906. Fix Version/s: 1.10.10 Resolution: Fixed I've marked this as fix for

[jira] [Commented] (AIRFLOW-2906) DataDog Integration for Airflow

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036499#comment-17036499 ] ASF GitHub Bot commented on AIRFLOW-2906: - ashb commented on pull request #7376: [AIRFLOW-2906]

[jira] [Commented] (AIRFLOW-2906) DataDog Integration for Airflow

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036500#comment-17036500 ] ASF subversion and git services commented on AIRFLOW-2906: -- Commit

[GitHub] [airflow] ashb merged pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics

2020-02-13 Thread GitBox
ashb merged pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics URL: https://github.com/apache/airflow/pull/7376 This is an automated message from the Apache Git Service. To respond to

[jira] [Updated] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Matthew Bruce (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bruce updated AIRFLOW-6796: --- Description: With serialization of DAGs enabled, `SerializedDagModel.remove_deleted_dags`

[jira] [Updated] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Matthew Bruce (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bruce updated AIRFLOW-6796: --- Description: With serialization of DAGs enabled, `SerializedDagModel.remove_deleted_dags`

[jira] [Updated] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Matthew Bruce (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bruce updated AIRFLOW-6796: --- Description: With serialization of DAGs enabled, `SerializedDagModel.remove_deleted_dags`

[jira] [Created] (AIRFLOW-6798) Add option for service account values for KubernetesPodOperator

2020-02-13 Thread Parhy (Jira)
Parhy created AIRFLOW-6798: -- Summary: Add option for service account values for KubernetesPodOperator Key: AIRFLOW-6798 URL: https://issues.apache.org/jira/browse/AIRFLOW-6798 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Matthew Bruce (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matthew Bruce updated AIRFLOW-6796: --- Description: With serialization of DAGs enabled, `SerializedDagModel.remove_deleted_dags`

[GitHub] [airflow] ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs

2020-02-13 Thread GitBox
ashb commented on a change in pull request #7370: [AIRFLOW-6590] Use batch db operations in jobs URL: https://github.com/apache/airflow/pull/7370#discussion_r379113503 ## File path: scripts/perf/sql_queries.py ## @@ -0,0 +1,178 @@ +# Licensed to the Apache Software

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585967157 Perfecto! I'll give this a shot. This is an automated

[jira] [Created] (AIRFLOW-6797) Create policy hooks for DAGs

2020-02-13 Thread Matthew Bruce (Jira)
Matthew Bruce created AIRFLOW-6797: -- Summary: Create policy hooks for DAGs Key: AIRFLOW-6797 URL: https://issues.apache.org/jira/browse/AIRFLOW-6797 Project: Apache Airflow Issue Type: New

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585964299 > 1. installing Singularity as a binary inside a container (these bases already exist) > 2. running the container via

[jira] [Updated] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6796?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor updated AIRFLOW-6796: --- Description: With serialization of DAGs enabled,

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585946960 > I assume we can start singularity from the host and be able to forward this connection to inside the airlfow-testing

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585943248 > okay just to clarify - you want a Singularity + Airflow container run via a similar kind cluster? You said something about

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585941481 > okay so I'm tracing the kubernetes (runtime) as an example, and I have a quick question. In

[GitHub] [airflow] brtasavpatel commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error

2020-02-13 Thread GitBox
brtasavpatel commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error URL: https://github.com/apache/airflow/pull/6643#issuecomment-585941357 > Was this resolved? Setting AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS: '{ "_request_timeout": "50" }' did not

[GitHub] [airflow] potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
potiuk commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585939693 Somehow I missed it completely. Sorry. Responding now.

[jira] [Created] (AIRFLOW-6796) Serialized DAGs can be incorrectly deleted

2020-02-13 Thread Matthew Bruce (Jira)
Matthew Bruce created AIRFLOW-6796: -- Summary: Serialized DAGs can be incorrectly deleted Key: AIRFLOW-6796 URL: https://issues.apache.org/jira/browse/AIRFLOW-6796 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-6795) serialized_dag table's data column text type is too small for mysql

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036456#comment-17036456 ] ASF GitHub Bot commented on AIRFLOW-6795: - nritholtz commented on pull request #7414:

[GitHub] [airflow] nritholtz opened a new pull request #7414: [AIRFLOW-6795] Increase text size on data column in serialized_dag for MySQL

2020-02-13 Thread GitBox
nritholtz opened a new pull request #7414: [AIRFLOW-6795] Increase text size on data column in serialized_dag for MySQL URL: https://github.com/apache/airflow/pull/7414 …r MySQL --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg)

[jira] [Updated] (AIRFLOW-6795) serialized_dag table's data column text type is too small for mysql

2020-02-13 Thread Nathaniel Ritholtz (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathaniel Ritholtz updated AIRFLOW-6795: Summary: serialized_dag table's data column text type is too small for mysql

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379058038 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379057502 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the

[GitHub] [airflow] dhegberg commented on issue #3229: [AIRFLOW-2325] Add cloudwatch task handler (IN PROGRESS)

2020-02-13 Thread GitBox
dhegberg commented on issue #3229: [AIRFLOW-2325] Add cloudwatch task handler (IN PROGRESS) URL: https://github.com/apache/airflow/pull/3229#issuecomment-585918439 @ericabertugli Are you still working on this? I started doing some testing and I'm happy to take over. I'd

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056518 ## File path: airflow/providers/salesforce/operators/tableau_refresh_workbook.py ## @@ -0,0 +1,74 @@

[GitHub] [airflow] feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration

2020-02-13 Thread GitBox
feluelle commented on a change in pull request #7410: [AIRFLOW-6790] Add basic Tableau Integration URL: https://github.com/apache/airflow/pull/7410#discussion_r379056816 ## File path: airflow/providers/salesforce/hooks/tableau.py ## @@ -0,0 +1,61 @@ +# Licensed to the

[jira] [Created] (AIRFLOW-6795) serialized_dag table's data type max length is too small

2020-02-13 Thread Nathaniel Ritholtz (Jira)
Nathaniel Ritholtz created AIRFLOW-6795: --- Summary: serialized_dag table's data type max length is too small Key: AIRFLOW-6795 URL: https://issues.apache.org/jira/browse/AIRFLOW-6795 Project:

[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow

2020-02-13 Thread GitBox
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-585869248 ping... This is an automated message from the Apache Git

[GitHub] [airflow] alexzlue commented on issue #7353: [AIRFLOW-6685] Data Quality Check operators

2020-02-13 Thread GitBox
alexzlue commented on issue #7353: [AIRFLOW-6685] Data Quality Check operators URL: https://github.com/apache/airflow/pull/7353#issuecomment-585860338 @eladkal Thanks for bringing this to mind. I do see that there is some functionality that I have that `CheckOperator` does not have. I will

[GitHub] [airflow] msb217 commented on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task …

2020-02-13 Thread GitBox
msb217 commented on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task … URL: https://github.com/apache/airflow/pull/6652#issuecomment-585848563 @ashb any comments or opinions? This is an

[GitHub] [airflow] eladkal commented on issue #7353: [AIRFLOW-6685] Data Quality Check operators

2020-02-13 Thread GitBox
eladkal commented on issue #7353: [AIRFLOW-6685] Data Quality Check operators URL: https://github.com/apache/airflow/pull/7353#issuecomment-585830929 In general the operators in this PR sounds like enhancement of

[jira] [Commented] (AIRFLOW-6344) CI builds with TAG fail with "unknown revision or path not in the working tree"

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036312#comment-17036312 ] ASF subversion and git services commented on AIRFLOW-6344: -- Commit

[GitHub] [airflow] dpmccabe commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R

2020-02-13 Thread GitBox
dpmccabe commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R URL: https://github.com/apache/airflow/pull/3115#issuecomment-585817970 Worth mentioning https://github.com/ropensci/drake/ as an alternative for people who are building data pipelines in R. Not the best option in

[jira] [Commented] (AIRFLOW-6344) CI builds with TAG fail with "unknown revision or path not in the working tree"

2020-02-13 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036308#comment-17036308 ] ASF subversion and git services commented on AIRFLOW-6344: -- Commit

[jira] [Resolved] (AIRFLOW-6344) CI builds with TAG fail with "unknown revision or path not in the working tree"

2020-02-13 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-6344. Fix Version/s: 1.10.10 Resolution: Fixed > CI builds with TAG fail with

[jira] [Commented] (AIRFLOW-6344) CI builds with TAG fail with "unknown revision or path not in the working tree"

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036307#comment-17036307 ] ASF GitHub Bot commented on AIRFLOW-6344: - ashb commented on pull request #7411: [AIRFLOW-6344]

[GitHub] [airflow] ashb merged pull request #7411: [AIRFLOW-6344] Fix travis CI for tag builds

2020-02-13 Thread GitBox
ashb merged pull request #7411: [AIRFLOW-6344] Fix travis CI for tag builds URL: https://github.com/apache/airflow/pull/7411 This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (AIRFLOW-6649) Google storage to Snowflake

2020-02-13 Thread nexoriv (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036305#comment-17036305 ] nexoriv commented on AIRFLOW-6649: -- [~kamil.bregula] that sounds interesting!

[jira] [Updated] (AIRFLOW-6768) Graph view rendering angular edges

2020-02-13 Thread Nathan Hadfield (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Hadfield updated AIRFLOW-6768: - Fix Version/s: 1.10.10 2.0.0 > Graph view rendering angular edges >

[GitHub] [airflow] zhongjiajie commented on a change in pull request #6900: [AIRFLOW-6346] Enhance dag default_view and orientation

2020-02-13 Thread GitBox
zhongjiajie commented on a change in pull request #6900: [AIRFLOW-6346] Enhance dag default_view and orientation URL: https://github.com/apache/airflow/pull/6900#discussion_r378914969 ## File path: airflow/serialization/schema.json ## @@ -93,7 +93,7 @@

[jira] [Assigned] (AIRFLOW-6794) Allow AWS Operator RedshiftToS3Transfer To Run a Custom Query

2020-02-13 Thread Roger Russel Droique Neris (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roger Russel Droique Neris reassigned AIRFLOW-6794: --- Assignee: Roger Russel Droique Neris > Allow AWS Operator

[jira] [Created] (AIRFLOW-6794) Allow AWS Operator RedshiftToS3Transfer To Run a Custom Query

2020-02-13 Thread Roger Russel Droique Neris (Jira)
Roger Russel Droique Neris created AIRFLOW-6794: --- Summary: Allow AWS Operator RedshiftToS3Transfer To Run a Custom Query Key: AIRFLOW-6794 URL: https://issues.apache.org/jira/browse/AIRFLOW-6794

[GitHub] [airflow] mik-laj commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-13 Thread GitBox
mik-laj commented on a change in pull request #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#discussion_r378910666 ## File path: TESTING.rst ## @@ -31,8 +31,7 @@ Airflow Test Infrastructure * **System

[jira] [Commented] (AIRFLOW-6793) airflow config command doesn't respect env variable

2020-02-13 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036281#comment-17036281 ] ASF GitHub Bot commented on AIRFLOW-6793: - mik-laj commented on pull request #7413:

[GitHub] [airflow] mik-laj opened a new pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command

2020-02-13 Thread GitBox
mik-laj opened a new pull request #7413: [AIRFLOW-6793] Respect env variable in airflow config command URL: https://github.com/apache/airflow/pull/7413 `airflow config` command always returns a value from a file. It does not read information from environment variables. This bug was

  1   2   >