[GitHub] [airflow] feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398033455 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the

[GitHub] [airflow] roitvt opened a new pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
roitvt opened a new pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875 add spark_kubernetes system test --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes

[GitHub] [airflow] roitvt commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
roitvt commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398097561 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[jira] [Closed] (AIRFLOW-622) airflow cannot recover itself after mysql connection was once gone away and not trigger an error from email alert

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-622. --- Resolution: Auto Closed > airflow cannot recover itself after mysql connection was once

[jira] [Closed] (AIRFLOW-93) Allow specifying multiple task execution deltas for ExternalTaskSensors

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-93?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-93. -- Resolution: Auto Closed > Allow specifying multiple task execution deltas for

[jira] [Closed] (AIRFLOW-1190) SSH Connection still running, inspite of killing tasks

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-1190. Resolution: Auto Closed > SSH Connection still running, inspite of killing tasks >

[jira] [Commented] (AIRFLOW-2036) "Broken Dag" messages are basically useless for troubleshooting

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067020#comment-17067020 ] Daniel Imberman commented on AIRFLOW-2036: -- [~aizhamal] should I transfer this ticket to GH or

[GitHub] [airflow] potiuk commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
potiuk commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398122902 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] potiuk commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
potiuk commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398121943 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] kaxil commented on a change in pull request #7454: [AIRFLOW-6833] HA for webhdfs connection

2020-03-25 Thread GitBox
kaxil commented on a change in pull request #7454: [AIRFLOW-6833] HA for webhdfs connection URL: https://github.com/apache/airflow/pull/7454#discussion_r397996727 ## File path: airflow/providers/apache/hdfs/hooks/webhdfs.py ## @@ -56,27 +57,35 @@ def __init__(self,

[GitHub] [airflow] turbaszek commented on a change in pull request #7869: [AIRFLOW-5801] Get GCP credentials from file instead of JSON blob

2020-03-25 Thread GitBox
turbaszek commented on a change in pull request #7869: [AIRFLOW-5801] Get GCP credentials from file instead of JSON blob URL: https://github.com/apache/airflow/pull/7869#discussion_r398004042 ## File path: airflow/providers/google/cloud/hooks/cloud_sql.py ## @@ -520,12

[GitHub] [airflow] kaxil opened a new pull request #7874: Fix static check failure: end-of-file-fixer

2020-03-25 Thread GitBox
kaxil opened a new pull request #7874: Fix static check failure: end-of-file-fixer URL: https://github.com/apache/airflow/pull/7874 https://github.com/apache/airflow/pull/7789 introduced this bug by adding a new line at the end of the file ``` Fix End of

[GitHub] [airflow] feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398051173 ## File path: tests/providers/mysql/operators/test_s3_to_mysql.py ## @@ -0,0 +1,82 @@ +# Licensed to the

[jira] [Closed] (AIRFLOW-1583) PostgreSQL error when pushing a value bigger than 1GB

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-1583. Resolution: Fixed > PostgreSQL error when pushing a value bigger than 1GB >

[jira] [Updated] (AIRFLOW-6722) Can't modify a variable with an existing value to have an empty value

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6722: Fix Version/s: (was: 1.10.10) > Can't modify a variable with an existing value to have an

[GitHub] [airflow] zeroam opened a new pull request #7873: Add help message in celery command

2020-03-25 Thread GitBox
zeroam opened a new pull request #7873: Add help message in celery command URL: https://github.com/apache/airflow/pull/7873 Celery command is available just for CeleryExecutor. The others cannot recognize. It needs to show help message for celery command even if the executor is not

[jira] [Commented] (AIRFLOW-6420) ui - show summary of task/dagrun stats across ALL dags

2020-03-25 Thread Roster (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6420?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066857#comment-17066857 ] Roster commented on AIRFLOW-6420: - It would also be cool to have two metrics: 1. Overall time from dag

[jira] [Created] (AIRFLOW-7111) Expose generate_presigned_url of boto3 to S3Hook

2020-03-25 Thread korni (Jira)
korni created AIRFLOW-7111: -- Summary: Expose generate_presigned_url of boto3 to S3Hook Key: AIRFLOW-7111 URL: https://issues.apache.org/jira/browse/AIRFLOW-7111 Project: Apache Airflow Issue Type:

[GitHub] [airflow] kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398036533 ## File path: tests/providers/mysql/operators/test_s3_to_mysql.py ## @@ -0,0 +1,82 @@ +# Licensed to the

[GitHub] [airflow] feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398037514 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the

[GitHub] [airflow] dimberman commented on issue #7365: [AIRFLOW-5221] add host_aliases to KubernetesPodOperator

2020-03-25 Thread GitBox
dimberman commented on issue #7365: [AIRFLOW-5221] add host_aliases to KubernetesPodOperator URL: https://github.com/apache/airflow/pull/7365#issuecomment-603984814 ``` tests/runtime/kubernetes/test_kubernetes_executor.py .F

[jira] [Closed] (AIRFLOW-78) airflow clear leaves dag_runs

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-78?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-78. -- Resolution: Auto Closed > airflow clear leaves dag_runs > - > >

[jira] [Closed] (AIRFLOW-197) Import SubDagOperator not consistent

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-197. --- Resolution: Auto Closed > Import SubDagOperator not consistent >

[jira] [Updated] (AIRFLOW-5231) S3Hook delete fails with over 1000 keys

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-5231: Fix Version/s: (was: 1.10.10) 2.0.0 > S3Hook delete fails with over 1000

[GitHub] [airflow] kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398038266 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the Apache

[GitHub] [airflow] kaxil commented on issue #7872: New dags fail to run

2020-03-25 Thread GitBox
kaxil commented on issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872#issuecomment-603985455 This is the expected behavior. The Scheduler needs to do various validation before it can execute tasks and hence it needs the DAG to be in the database and the

[GitHub] [airflow] kaxil closed issue #7872: New dags fail to run

2020-03-25 Thread GitBox
kaxil closed issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[jira] [Closed] (AIRFLOW-421) code view cannot find source files consistently since 1.7.1

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-421. --- Resolution: Auto Closed > code view cannot find source files consistently since 1.7.1 >

[jira] [Commented] (AIRFLOW-4363) Encounter JSON Decode Error when using docker operator

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067022#comment-17067022 ] ASF subversion and git services commented on AIRFLOW-4363: -- Commit

[GitHub] [airflow] potiuk commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
potiuk commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398121943 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] GuzikJakub commented on a change in pull request #7454: [AIRFLOW-6833] HA for webhdfs connection

2020-03-25 Thread GitBox
GuzikJakub commented on a change in pull request #7454: [AIRFLOW-6833] HA for webhdfs connection URL: https://github.com/apache/airflow/pull/7454#discussion_r397991670 ## File path: airflow/providers/apache/hdfs/hooks/webhdfs.py ## @@ -56,27 +57,35 @@ def __init__(self,

[GitHub] [airflow] feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398031833 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the

[GitHub] [airflow] kaxil merged pull request #7874: Fix static check failure: end-of-file-fixer

2020-03-25 Thread GitBox
kaxil merged pull request #7874: Fix static check failure: end-of-file-fixer URL: https://github.com/apache/airflow/pull/7874 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] mik-laj commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398092115 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] mik-laj commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398100570 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[jira] [Closed] (AIRFLOW-97) "airflow" "DAG" strings in file necessary to import dag

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-97?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-97. -- Resolution: Auto Closed > "airflow" "DAG" strings in file necessary to import dag >

[GitHub] [airflow] dimberman opened a new issue #7876: API endpoint to fetch DAG instance expected start time

2020-03-25 Thread GitBox
dimberman opened a new issue #7876: API endpoint to fetch DAG instance expected start time URL: https://github.com/apache/airflow/issues/7876 **Description** From:

[jira] [Assigned] (AIRFLOW-621) Scheduled DAG: How to fetch DAG instance expected start time

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman reassigned AIRFLOW-621: --- Assignee: Daniel Imberman > Scheduled DAG: How to fetch DAG instance expected

[jira] [Updated] (AIRFLOW-6768) Graph view rendering angular edges

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6768: Fix Version/s: (was: 1.10.10) > Graph view rendering angular edges >

[jira] [Updated] (AIRFLOW-6437) sql filters - remove in (NULL)

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6437: Fix Version/s: (was: 1.10.10) > sql filters - remove in (NULL) >

[GitHub] [airflow] kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398034873 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the Apache

[GitHub] [airflow] kaxil commented on issue #7872: New dags fail to run

2020-03-25 Thread GitBox
kaxil commented on issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872#issuecomment-603998341 > So when adding a new dag to airflow the process is: add dag to repo/directory, reinitialize the db, then run? That wasn't the process before as I said in

[jira] [Commented] (AIRFLOW-4363) Encounter JSON Decode Error when using docker operator

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066952#comment-17066952 ] ASF GitHub Bot commented on AIRFLOW-4363: - dimberman commented on pull request #7628:

[GitHub] [airflow] dimberman merged pull request #7628: [AIRFLOW-4363] Fix JSON encoding error

2020-03-25 Thread GitBox
dimberman merged pull request #7628: [AIRFLOW-4363] Fix JSON encoding error URL: https://github.com/apache/airflow/pull/7628 This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Commented] (AIRFLOW-4363) Encounter JSON Decode Error when using docker operator

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17066953#comment-17066953 ] ASF subversion and git services commented on AIRFLOW-4363: -- Commit

[GitHub] [airflow] mik-laj commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398092115 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] mik-laj commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398092115 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] cccs-cat001 opened a new issue #7872: New dags fail to run

2020-03-25 Thread GitBox
cccs-cat001 opened a new issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872 **Apache Airflow version**: 1.10.9 **Kubernetes version (if you are using kubernetes)** (use `kubectl version`): N/A **Environment**: [JupyterLab docker

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7872: New dags fail to run

2020-03-25 Thread GitBox
boring-cyborg[bot] commented on issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872#issuecomment-603947117 Thanks for opening your first issue here! Be sure to follow the issue template!

[GitHub] [airflow] kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
kaxil commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398035816 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the Apache

[GitHub] [airflow] feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator

2020-03-25 Thread GitBox
feluelle commented on a change in pull request #6578: [AIRFLOW-5907] Add S3 to MySql Operator URL: https://github.com/apache/airflow/pull/6578#discussion_r398035829 ## File path: airflow/providers/mysql/operators/s3_to_mysql.py ## @@ -0,0 +1,84 @@ +# Licensed to the

[GitHub] [airflow] cccs-cat001 commented on issue #7872: New dags fail to run

2020-03-25 Thread GitBox
cccs-cat001 commented on issue #7872: New dags fail to run URL: https://github.com/apache/airflow/issues/7872#issuecomment-603996612 So when adding a new dag to airflow the process is: add dag to repo/directory, reinitialize the db, then run? That wasn't the process before as I said in

[GitHub] [airflow] turbaszek commented on a change in pull request #7864: GCP SparkR operator

2020-03-25 Thread GitBox
turbaszek commented on a change in pull request #7864: GCP SparkR operator URL: https://github.com/apache/airflow/pull/7864#discussion_r398086582 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -1359,6 +1359,128 @@ def execute(self, context):

[GitHub] [airflow] turbaszek commented on a change in pull request #7864: GCP SparkR operator

2020-03-25 Thread GitBox
turbaszek commented on a change in pull request #7864: GCP SparkR operator URL: https://github.com/apache/airflow/pull/7864#discussion_r398088165 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -1359,6 +1359,128 @@ def execute(self, context):

[GitHub] [airflow] turbaszek commented on a change in pull request #7864: GCP SparkR operator

2020-03-25 Thread GitBox
turbaszek commented on a change in pull request #7864: GCP SparkR operator URL: https://github.com/apache/airflow/pull/7864#discussion_r398088165 ## File path: airflow/providers/google/cloud/operators/dataproc.py ## @@ -1359,6 +1359,128 @@ def execute(self, context):

[GitHub] [airflow] mik-laj commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398094493 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[jira] [Resolved] (AIRFLOW-4363) Encounter JSON Decode Error when using docker operator

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-4363. - Fix Version/s: 1.10.10 Resolution: Fixed > Encounter JSON Decode Error when using

[jira] [Closed] (AIRFLOW-797) CLONE - No such transport: sqla when using CeleryExecutor

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-797. --- Resolution: Auto Closed > CLONE - No such transport: sqla when using CeleryExecutor >

[jira] [Updated] (AIRFLOW-6850) Handle not being able to display code more gracefully

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6850: Fix Version/s: (was: 1.10.10) > Handle not being able to display code more gracefully >

[jira] [Commented] (AIRFLOW-7000) Allow passing in env var JSON dict in task_test

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067054#comment-17067054 ] ASF subversion and git services commented on AIRFLOW-7000: -- Commit

[jira] [Closed] (AIRFLOW-2935) CLONE - Tasks never run in externally triggered DAGRun

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-2935. Resolution: Auto Closed > CLONE - Tasks never run in externally triggered DAGRun >

[jira] [Closed] (AIRFLOW-462) Concurrent Scheduler Jobs pushing the same task to queue

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-462. --- Resolution: Auto Closed > Concurrent Scheduler Jobs pushing the same task to queue >

[jira] [Closed] (AIRFLOW-63) Dangling Running Jobs

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-63?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-63. -- Resolution: Auto Closed > Dangling Running Jobs > - > >

[jira] [Closed] (AIRFLOW-210) sqlalchemy warns about task_instance.execution_date invoked with an empty sequence

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-210. --- Resolution: Auto Closed > sqlalchemy warns about task_instance.execution_date invoked with

[jira] [Closed] (AIRFLOW-57) Rename concurrency configuration variables to be more clear

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-57?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-57. -- Resolution: Auto Closed > Rename concurrency configuration variables to be more clear >

[jira] [Closed] (AIRFLOW-621) Scheduled DAG: How to fetch DAG instance expected start time

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Imberman closed AIRFLOW-621. --- Resolution: Auto Closed > Scheduled DAG: How to fetch DAG instance expected start time >

[jira] [Commented] (AIRFLOW-621) Scheduled DAG: How to fetch DAG instance expected start time

2020-03-25 Thread Daniel Imberman (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067030#comment-17067030 ] Daniel Imberman commented on AIRFLOW-621: - moved to

[GitHub] [airflow] mik-laj commented on issue #7873: Add help message in celery command

2020-03-25 Thread GitBox
mik-laj commented on issue #7873: Add help message in celery command URL: https://github.com/apache/airflow/pull/7873#issuecomment-604096154 Hi. We recently had discussions about whether it is a good idea to hide commands in CLI. We decided that we prefer it to be visible, but with the

[GitHub] [airflow] lsimoneau commented on issue #7374: [AIRFLOW-6399] Serialization: DAG access_control field should be deco…

2020-03-25 Thread GitBox
lsimoneau commented on issue #7374: [AIRFLOW-6399] Serialization: DAG access_control field should be deco… URL: https://github.com/apache/airflow/pull/7374#issuecomment-604133178 The test output is hard to read, but I _think_ this is failing because `can_dag_edit` and `can_dag_read` are

[GitHub] [airflow] mik-laj commented on issue #7871: Kubernetes Executor wastefully creates pods for DummyOperator tasks

2020-03-25 Thread GitBox
mik-laj commented on issue #7871: Kubernetes Executor wastefully creates pods for DummyOperator tasks URL: https://github.com/apache/airflow/issues/7871#issuecomment-604158761 Interesting idea. I think you can just mark these tasks automatically in a success state. This made this change

[GitHub] [airflow] mik-laj commented on issue #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
mik-laj commented on issue #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#issuecomment-604063097 @potiuk I am a bit busy working on client stuff. I will try to look at it and run on Monday.

[GitHub] [airflow] xinbinhuang commented on issue #7861: [AIRFLOW-7108] Fix CloudSecretsManagerBackend invalid connections_prefix

2020-03-25 Thread GitBox
xinbinhuang commented on issue #7861: [AIRFLOW-7108] Fix CloudSecretsManagerBackend invalid connections_prefix URL: https://github.com/apache/airflow/pull/7861#issuecomment-604072028 @zhongjiajie @kaxil Just rebased. Going through Travis now. Should be good to go after that

[GitHub] [airflow] mik-laj commented on a change in pull request #7873: Add help message in celery command

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7873: Add help message in celery command URL: https://github.com/apache/airflow/pull/7873#discussion_r398184886 ## File path: airflow/cli/cli_parser.py ## @@ -55,6 +56,28 @@ def command(*args, **kwargs): return command

[GitHub] [airflow] mik-laj opened a new pull request #7878: Load all jobs models at once

2020-03-25 Thread GitBox
mik-laj opened a new pull request #7878: Load all jobs models at once URL: https://github.com/apache/airflow/pull/7878 We use model polymorphism in `airflow.jobs` classes, so we must load all classes each time.

[GitHub] [airflow] mik-laj commented on a change in pull request #7841: Requirements now depend on python version

2020-03-25 Thread GitBox
mik-laj commented on a change in pull request #7841: Requirements now depend on python version URL: https://github.com/apache/airflow/pull/7841#discussion_r398188442 ## File path: Dockerfile.ci ## @@ -40,14 +43,19 @@ ARG DEPENDENCIES_EPOCH_NUMBER="2" ENV

[GitHub] [airflow] roitvt commented on a change in pull request #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
roitvt commented on a change in pull request #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#discussion_r398188191 ## File path: tests/providers/cncf/kubernetes/operators/test_spark_kubernetes_system.py ## @@ -0,0 +1,72 @@ +# +#

[GitHub] [airflow] roitvt commented on issue #7875: add spark_kubernetes system test

2020-03-25 Thread GitBox
roitvt commented on issue #7875: add spark_kubernetes system test URL: https://github.com/apache/airflow/pull/7875#issuecomment-604103872 @potiuk it's working on 1.10 I checked it with breeze against 1.10.9 This is an

[GitHub] [airflow] kaxil merged pull request #7877: Fix grammar in setup.py

2020-03-25 Thread GitBox
kaxil merged pull request #7877: Fix grammar in setup.py URL: https://github.com/apache/airflow/pull/7877 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067109#comment-17067109 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

[jira] [Updated] (AIRFLOW-6530) Allow for custom Statsd client

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6530: Fix Version/s: (was: 2.0.0) 1.10.10 > Allow for custom Statsd client >

[jira] [Updated] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-4057: Fix Version/s: (was: 2.0.0) 1.10.10 > airflow should handle invalid stats

[jira] [Updated] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-03-25 Thread Kaxil Naik (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6711: Fix Version/s: (was: 2.0.0) 1.10.10 > Drop plugin support for

[jira] [Commented] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067112#comment-17067112 ] ASF subversion and git services commented on AIRFLOW-6711: -- Commit

[jira] [Commented] (AIRFLOW-2906) DataDog Integration for Airflow

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067113#comment-17067113 ] ASF subversion and git services commented on AIRFLOW-2906: -- Commit

[jira] [Commented] (AIRFLOW-6530) Allow for custom Statsd client

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067114#comment-17067114 ] ASF subversion and git services commented on AIRFLOW-6530: -- Commit

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067111#comment-17067111 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

[jira] [Assigned] (AIRFLOW-4166) Add CSV file support to CassandraToGoogleCloudStorageOperator

2020-03-25 Thread Ephraim E Anierobi (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ephraim E Anierobi reassigned AIRFLOW-4166: --- Assignee: (was: Ephraim E Anierobi) > Add CSV file support to

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067110#comment-17067110 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

[GitHub] [airflow] zeroam commented on issue #7873: Add help message in celery command

2020-03-25 Thread GitBox
zeroam commented on issue #7873: Add help message in celery command URL: https://github.com/apache/airflow/pull/7873#issuecomment-604145199 > Hi. We recently had discussions about whether it is a good idea to hide commands in CLI. We decided that we prefer it to be visible, but with the

[GitHub] [airflow] mik-laj opened a new pull request #7880: Don't schedule dummy task

2020-03-25 Thread GitBox
mik-laj opened a new pull request #7880: Don't schedule dummy task URL: https://github.com/apache/airflow/pull/7880 https://github.com/apache/airflow/issues/7871 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7879: [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization

2020-03-25 Thread GitBox
boring-cyborg[bot] commented on issue #7879: [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization URL: https://github.com/apache/airflow/pull/7879#issuecomment-604146697 Congratulations on your first Pull Request and welcome to the Apache

[jira] [Commented] (AIRFLOW-6399) Serialization: DAG access_control field should be decorated field in DAG serialization

2020-03-25 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067228#comment-17067228 ] ASF GitHub Bot commented on AIRFLOW-6399: - lsimoneau commented on pull request #7879:

[GitHub] [airflow] lsimoneau opened a new pull request #7879: [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization

2020-03-25 Thread GitBox
lsimoneau opened a new pull request #7879: [AIRFLOW-6399] Serialization: DAG access_control field should be decorated field in DAG serialization URL: https://github.com/apache/airflow/pull/7879 This is a modification of #7374 to fix a failing test caused by inconsistent ordering between

[GitHub] [airflow] zeroam commented on a change in pull request #7873: Add help message in celery command

2020-03-25 Thread GitBox
zeroam commented on a change in pull request #7873: Add help message in celery command URL: https://github.com/apache/airflow/pull/7873#discussion_r398236137 ## File path: airflow/cli/cli_parser.py ## @@ -55,6 +56,28 @@ def command(*args, **kwargs): return command

[GitHub] [airflow] mik-laj commented on issue #7450: [AIRFLOW-6829][WIP] Introduce BaseOperatorMetaClass - auto-apply apply_default

2020-03-25 Thread GitBox
mik-laj commented on issue #7450: [AIRFLOW-6829][WIP] Introduce BaseOperatorMetaClass - auto-apply apply_default URL: https://github.com/apache/airflow/pull/7450#issuecomment-604093947 Let's wait with this PR until we stop relasing the package of backported operators. This PR shows that

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067194#comment-17067194 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067195#comment-17067195 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

[jira] [Commented] (AIRFLOW-6711) Drop plugin support for stat_name_handler

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067196#comment-17067196 ] ASF subversion and git services commented on AIRFLOW-6711: -- Commit

[jira] [Commented] (AIRFLOW-2906) DataDog Integration for Airflow

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067197#comment-17067197 ] ASF subversion and git services commented on AIRFLOW-2906: -- Commit

[jira] [Commented] (AIRFLOW-4057) airflow should handle invalid stats name

2020-03-25 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067193#comment-17067193 ] ASF subversion and git services commented on AIRFLOW-4057: -- Commit

  1   2   >