[GitHub] [airflow] gnatamania commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R

2020-02-12 Thread GitBox
gnatamania commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R URL: https://github.com/apache/airflow/pull/3115#issuecomment-585118703 so... is this the end of my hopes and dreams of seeing the ROperator in Airflow? 

[jira] [Commented] (AIRFLOW-6065) Task Handler allowing us to save logs to Stackdriver Logging

2020-02-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035235#comment-17035235 ] ASF subversion and git services commented on AIRFLOW-6065: -- Commit

[jira] [Commented] (AIRFLOW-6065) Task Handler allowing us to save logs to Stackdriver Logging

2020-02-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035234#comment-17035234 ] ASF GitHub Bot commented on AIRFLOW-6065: - mik-laj commented on pull request #6660:

[GitHub] [airflow] chandulal commented on a change in pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics

2020-02-12 Thread GitBox
chandulal commented on a change in pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics URL: https://github.com/apache/airflow/pull/7376#discussion_r378174065 ## File path: airflow/stats.py ## @@ -135,19 +135,87 @@ def timing(self,

[GitHub] [airflow] mik-laj commented on issue #7399: [AIRFLOW-6505] Let emoji encoded properly for json.dumps()

2020-02-12 Thread GitBox
mik-laj commented on issue #7399: [AIRFLOW-6505] Let emoji encoded properly for json.dumps() URL: https://github.com/apache/airflow/pull/7399#issuecomment-585181548 Travis is sad. Can you fix it? This is an automated message

[GitHub] [airflow] mik-laj merged pull request #6660: [AIRFLOW-6065] Add Stackdriver Task Handler

2020-02-12 Thread GitBox
mik-laj merged pull request #6660: [AIRFLOW-6065] Add Stackdriver Task Handler URL: https://github.com/apache/airflow/pull/6660 This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] ashb commented on a change in pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics URL: https://github.com/apache/airflow/pull/7376#discussion_r378166778 ## File path: airflow/stats.py ## @@ -135,19 +135,87 @@ def timing(self, stat,

[GitHub] [airflow] stale[bot] commented on issue #6210: [AIRFLOW-5567] BaseReschedulePokeOperator

2020-02-12 Thread GitBox
stale[bot] commented on issue #6210: [AIRFLOW-5567] BaseReschedulePokeOperator URL: https://github.com/apache/airflow/pull/6210#issuecomment-585179579 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-12 Thread GitBox
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585195708 Hello @ashb @mik-laj @nuclearpinguin @kaxil @dimberman -> I updated the implementation now and it works really nicely.

[GitHub] [airflow] feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared URL: https://github.com/apache/airflow/pull/7276#discussion_r378236282 ## File path: airflow/models/skipmixin.py ## @@ -98,3 +98,13 @@ def

[GitHub] [airflow] potiuk commented on a change in pull request #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations

2020-02-12 Thread GitBox
potiuk commented on a change in pull request #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations URL: https://github.com/apache/airflow/pull/7404#discussion_r378261236 ## File path: tests/test_project_structure.py ## @@ -131,3 +131,124 @@ def

[GitHub] [airflow] JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378276464 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +#

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[GitHub] [airflow] potiuk commented on a change in pull request #6597: [AIRFLOW-6005] Common classes should always be imported from airflow, Depends on [AIRFLOW-6128]

2020-02-12 Thread GitBox
potiuk commented on a change in pull request #6597: [AIRFLOW-6005] Common classes should always be imported from airflow, Depends on [AIRFLOW-6128] URL: https://github.com/apache/airflow/pull/6597#discussion_r378269370 ## File path: .pre-commit-config.yaml ## @@ -237,6

[jira] [Commented] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035476#comment-17035476 ] Maksym Shalak commented on AIRFLOW-6782: I don't think so: I haven't seen any of those errors

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-12 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r378238926 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] potiuk edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-12 Thread GitBox
potiuk edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585195708 Hello @ashb @mik-laj @nuclearpinguin @kaxil @dimberman -> I updated the implementation now and it works really

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs

2020-02-12 Thread GitBox
nuclearpinguin commented on a change in pull request #7400: [AIRFLOW-6759] Added MLEngine operator/hook to cancel MLEngine jobs URL: https://github.com/apache/airflow/pull/7400#discussion_r378238724 ## File path: airflow/providers/google/cloud/operators/mlengine.py ## @@

[GitHub] [airflow] codecov-io commented on issue #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations

2020-02-12 Thread GitBox
codecov-io commented on issue #7404: [AIRFLOW-6781] Enforce example_dags and guide for Google integrations URL: https://github.com/apache/airflow/pull/7404#issuecomment-585205003 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7404?src=pr=h1) Report > Merging

[GitHub] [airflow] feluelle commented on a change in pull request #6597: [AIRFLOW-6005] Common classes should always be imported from airflow, Depends on [AIRFLOW-6128]

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6597: [AIRFLOW-6005] Common classes should always be imported from airflow, Depends on [AIRFLOW-6128] URL: https://github.com/apache/airflow/pull/6597#discussion_r378268793 ## File path: .pre-commit-config.yaml ## @@ -237,6

[GitHub] [airflow] potiuk commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R

2020-02-12 Thread GitBox
potiuk commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R URL: https://github.com/apache/airflow/pull/3115#issuecomment-585219279 Hmm. @gnatamania Unless you find someone who can pick this up @briandconnelly -> maybe you still want to continue that? Or maybe you can pick

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[jira] [Created] (AIRFLOW-6783) s3_to_redshift operator's usage is too limited

2020-02-12 Thread Peter Ring (Jira)
Peter Ring created AIRFLOW-6783: --- Summary: s3_to_redshift operator's usage is too limited Key: AIRFLOW-6783 URL: https://issues.apache.org/jira/browse/AIRFLOW-6783 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035465#comment-17035465 ] Kamil Bregula commented on AIRFLOW-6782: Is it related to this ticket?

[GitHub] [airflow] feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared URL: https://github.com/apache/airflow/pull/7276#discussion_r378236863 ## File path: airflow/models/skipmixin.py ## @@ -98,3 +98,13 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared URL: https://github.com/apache/airflow/pull/7276#discussion_r378244769 ## File path: tests/operators/test_python.py ## @@ -541,6 +542,62 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #7276: [AIRFLOW-5391] Do not run skipped tasks when they are cleared URL: https://github.com/apache/airflow/pull/7276#discussion_r378244504 ## File path: tests/operators/test_python.py ## @@ -541,6 +542,62 @@ def

[GitHub] [airflow] potiuk edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests

2020-02-12 Thread GitBox
potiuk edited a comment on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-585195708 Hello @ashb @mik-laj @nuclearpinguin @kaxil @dimberman -> I updated the implementation now and it works really

[GitHub] [airflow] feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378265450 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +# Licensed

[GitHub] [airflow] feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378264953 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +# Licensed

[GitHub] [airflow] feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378264658 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +# Licensed

[GitHub] [airflow] JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378274635 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +#

[GitHub] [airflow] feluelle commented on a change in pull request #6075: [AIRFLOW-5266] Allow aws_athena_hook to get all query results

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6075: [AIRFLOW-5266] Allow aws_athena_hook to get all query results URL: https://github.com/apache/airflow/pull/6075#discussion_r378274183 ## File path: tests/providers/amazon/aws/hooks/test_athena.py ## @@ -0,0 +1,120 @@ +#

[GitHub] [airflow] shcherbin commented on issue #7375: [AIRFLOW-5231] Fix S3Hook.delete_objects method

2020-02-12 Thread GitBox
shcherbin commented on issue #7375: [AIRFLOW-5231] Fix S3Hook.delete_objects method URL: https://github.com/apache/airflow/pull/7375#issuecomment-585223066 @mik-laj, would you please have another look? @feluelle approved the PR.

[GitHub] [airflow] feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378281115 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +# Licensed

[GitHub] [airflow] briandconnelly commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R

2020-02-12 Thread GitBox
briandconnelly commented on issue #3115: [AIRFLOW-2193] Add ROperator for using R URL: https://github.com/apache/airflow/pull/3115#issuecomment-585247174 Yeah, I don't plan on pursuing this further. Building and maintaining the test framework for it is complicated, and I just don't have

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Attachment: Screenshot from 2020-02-12 16-19-49.png > airflow-xcom-sidecar container

[jira] [Updated] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksym Shalak updated AIRFLOW-6782: --- Description: {color:#172b4d}I launch tasks using KubernetesPodOperator with

[GitHub] [airflow] mik-laj commented on issue #7406: [AIRFLOW-XXXX] Add architecture section to k8sexec docs

2020-02-12 Thread GitBox
mik-laj commented on issue #7406: [AIRFLOW-] Add architecture section to k8sexec docs URL: https://github.com/apache/airflow/pull/7406#issuecomment-585283178 Can you add some labels to the diagram, please? The diagram uses the Airflow logo to describe many different components. It is

[GitHub] [airflow] feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#discussion_r378270803 ## File path: docs/operators-and-hooks-ref.rst ## @@ -98,6 +98,12 @@ Foundation.

[GitHub] [airflow] feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#discussion_r378270803 ## File path: docs/operators-and-hooks-ref.rst ## @@ -98,6 +98,12 @@ Foundation.

[GitHub] [airflow] hmike96 commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file

2020-02-12 Thread GitBox
hmike96 commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-585236235 This is awesome! thanks for the hard work! I was wondering, will I be able to now create custom Kubernetes

[jira] [Commented] (AIRFLOW-5231) S3Hook delete fails with over 1000 keys

2020-02-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035462#comment-17035462 ] ASF GitHub Bot commented on AIRFLOW-5231: - mik-laj commented on pull request #7375:

[jira] [Commented] (AIRFLOW-5231) S3Hook delete fails with over 1000 keys

2020-02-12 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035463#comment-17035463 ] ASF subversion and git services commented on AIRFLOW-5231: -- Commit

[GitHub] [airflow] mik-laj merged pull request #7375: [AIRFLOW-5231] Fix S3Hook.delete_objects method

2020-02-12 Thread GitBox
mik-laj merged pull request #7375: [AIRFLOW-5231] Fix S3Hook.delete_objects method URL: https://github.com/apache/airflow/pull/7375 This is an automated message from the Apache Git Service. To respond to the message, please

[jira] [Resolved] (AIRFLOW-5231) S3Hook delete fails with over 1000 keys

2020-02-12 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-5231. Resolution: Fixed > S3Hook delete fails with over 1000 keys >

[GitHub] [airflow] dimberman opened a new pull request #7406: Improve k8s executor documentation

2020-02-12 Thread GitBox
dimberman opened a new pull request #7406: Improve k8s executor documentation URL: https://github.com/apache/airflow/pull/7406 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x]

[GitHub] [airflow] feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378266851 ## File path: airflow/operators/mysql_to_s3_operator.py ## @@ -0,0 +1,122 @@ +# -*- coding: utf-8 -*- +# +#

[GitHub] [airflow] potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages

2020-02-12 Thread GitBox
potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-585218154 > I've got the wheel files built by CI and am going to check out they install and uninstall in all the different permutations I am think of

[GitHub] [airflow] feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator

2020-02-12 Thread GitBox
feluelle commented on a change in pull request #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#discussion_r378272407 ## File path: docs/operators-and-hooks-ref.rst ## @@ -98,6 +98,12 @@ Foundation.

[GitHub] [airflow] JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator

2020-02-12 Thread GitBox
JavierLopezT commented on a change in pull request #6670: [AIRFLOW-4816]MySqlToS3Operator URL: https://github.com/apache/airflow/pull/6670#discussion_r378274635 ## File path: tests/providers/amazon/aws/operators/test_mysql_to_s3_operator.py ## @@ -0,0 +1,61 @@ +# +#

[jira] [Created] (AIRFLOW-6782) airflow-xcom-sidecar container prevents pods from completion

2020-02-12 Thread Maksym Shalak (Jira)
Maksym Shalak created AIRFLOW-6782: -- Summary: airflow-xcom-sidecar container prevents pods from completion Key: AIRFLOW-6782 URL: https://issues.apache.org/jira/browse/AIRFLOW-6782 Project: Apache

[GitHub] [airflow] fox91 commented on issue #7030: [AIRFLOW-6440][AIP-29] Add AWS Fargate Executor

2020-02-12 Thread GitBox
fox91 commented on issue #7030: [AIRFLOW-6440][AIP-29] Add AWS Fargate Executor URL: https://github.com/apache/airflow/pull/7030#issuecomment-585297496 Here is my question because I don't know how much Jira is used: > I've a question: why is this executor called "AWS Fargate" and not

[GitHub] [airflow] ashb commented on a change in pull request #7406: [AIRFLOW-XXXX] Add architecture section to k8sexec docs

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7406: [AIRFLOW-] Add architecture section to k8sexec docs URL: https://github.com/apache/airflow/pull/7406#discussion_r378420245 ## File path: docs/executor/kubernetes.rst ## @@ -34,3 +34,71 @@ The volumes are optional and

[jira] [Updated] (AIRFLOW-6785) DagBag tries to run hook inside SubDagOperator

2020-02-12 Thread Darwin Yip (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darwin Yip updated AIRFLOW-6785: Description: The following worked in 1.10.1, but not in 1.10.9. It seems that the DagBag tries

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7407: Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
boring-cyborg[bot] commented on issue #7407: Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#issuecomment-585364531 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or

[GitHub] [airflow] houqp commented on issue #7402: [AIRFLOW-6779] fix scheduler bug related to task concurrency and depends on past

2020-02-12 Thread GitBox
houqp commented on issue #7402: [AIRFLOW-6779] fix scheduler bug related to task concurrency and depends on past URL: https://github.com/apache/airflow/pull/7402#issuecomment-585380922 @ashb updated PR This is an automated

[jira] [Created] (AIRFLOW-6784) Graph view is missing on some DAGs (Google chrome)

2020-02-12 Thread ohad (Jira)
ohad created AIRFLOW-6784: - Summary: Graph view is missing on some DAGs (Google chrome) Key: AIRFLOW-6784 URL: https://issues.apache.org/jira/browse/AIRFLOW-6784 Project: Apache Airflow Issue Type:

[GitHub] [airflow] dimberman commented on issue #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842]

2020-02-12 Thread GitBox
dimberman commented on issue #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842] URL: https://github.com/apache/airflow/pull/6500#issuecomment-585307699 @potiuk is there any way to break this into smaller PRs? Merging 78 files of configuration is

[GitHub] [airflow] ashb commented on a change in pull request #7402: [AIRFLOW-6779] fix scheduler bug related to task concurrency and depends on past

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7402: [AIRFLOW-6779] fix scheduler bug related to task concurrency and depends on past URL: https://github.com/apache/airflow/pull/7402#discussion_r378429677 ## File path: tests/jobs/test_scheduler_job.py ## @@ -469,6 +469,105

[GitHub] [airflow] ashb commented on issue #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics

2020-02-12 Thread GitBox
ashb commented on issue #7376: [AIRFLOW-2906] Add datadog(dogstatsd) support to send airflow metrics URL: https://github.com/apache/airflow/pull/7376#issuecomment-585349655 See also https://issues.apache.org/jira/browse/AIRFLOW-6530 -- there's some overlap in the ask there and this PR.

[jira] [Resolved] (AIRFLOW-5818) Very bad webserver performance when defining many dags with many operators

2020-02-12 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-5818. Fix Version/s: 1.10.7 Resolution: Fixed I think this is fixed now in

[jira] [Resolved] (AIRFLOW-4064) Edit DAG in FAB UI

2020-02-12 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-4064. Resolution: Won't Fix General editing of the DAG model is not supported. Was

[jira] [Resolved] (AIRFLOW-5279) Should use links instead of buttons for Airflow modals

2020-02-12 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-5279. Resolution: Duplicate Fix in master, not backported. > Should use links instead

[GitHub] [airflow] dferguson992 commented on issue #1415: Add KafkaConsumerHook and KafkaSensor

2020-02-12 Thread GitBox
dferguson992 commented on issue #1415: Add KafkaConsumerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/1415#issuecomment-585365130 @ashb I've created #7407 with the co-authoring steps, but I cannot find the original AIRFLOW- issue referenced in this original commit.

[GitHub] [airflow] ashb commented on a change in pull request #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842]

2020-02-12 Thread GitBox
ashb commented on a change in pull request #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842] URL: https://github.com/apache/airflow/pull/6500#discussion_r378382540 ## File path: scripts/ci/utils/_run.sh ## @@ -0,0 +1,150 @@

[GitHub] [airflow] dimberman commented on a change in pull request #7406: [AIRFLOW-XXXX] Add architecture section to k8sexec docs

2020-02-12 Thread GitBox
dimberman commented on a change in pull request #7406: [AIRFLOW-] Add architecture section to k8sexec docs URL: https://github.com/apache/airflow/pull/7406#discussion_r378420805 ## File path: docs/executor/kubernetes.rst ## @@ -34,3 +34,71 @@ The volumes are optional

[jira] [Created] (AIRFLOW-6785) DagBag tries to run hook inside SubDagOperator

2020-02-12 Thread Darwin Yip (Jira)
Darwin Yip created AIRFLOW-6785: --- Summary: DagBag tries to run hook inside SubDagOperator Key: AIRFLOW-6785 URL: https://issues.apache.org/jira/browse/AIRFLOW-6785 Project: Apache Airflow

[jira] [Resolved] (AIRFLOW-4580) The new version 1.10.3 has compatibility problems with custom plugins

2020-02-12 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-4580. Resolution: Cannot Reproduce > The new version 1.10.3 has compatibility

[jira] [Resolved] (AIRFLOW-4538) Add support for FAB API with plugins

2020-02-12 Thread Ash Berlin-Taylor (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-4538. Resolution: Fixed Flask-AppBuilder as been updated to ~2.2.0 now. > Add support

[GitHub] [airflow] ashb commented on issue #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
ashb commented on issue #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#issuecomment-585413257 The first thing I notice is that we now need to rename this: - airflow/contrib/hooks/kafka_consumer_hook.py

[GitHub] [airflow] aKumpan commented on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response

2020-02-12 Thread GitBox
aKumpan commented on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response URL: https://github.com/apache/airflow/pull/6256#issuecomment-585416447 UP @OmerJog , @XD-DENG Conflicts resolved, please check again

[jira] [Updated] (AIRFLOW-6785) DagBag tries to run hook inside SubDagOperator

2020-02-12 Thread Darwin Yip (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darwin Yip updated AIRFLOW-6785: Affects Version/s: 1.10.4 1.10.5 1.10.6

[GitHub] [airflow] codecov-io edited a comment on issue #7353: [AIRFLOW-6685] Data Quality Check operators

2020-02-12 Thread GitBox
codecov-io edited a comment on issue #7353: [AIRFLOW-6685] Data Quality Check operators URL: https://github.com/apache/airflow/pull/7353#issuecomment-582190961 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7353?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-6788) The Goddamn installation instructions for using Postgres as a backend need to be scrapped and rewritten

2020-02-12 Thread DevLocalCA (Jira)
DevLocalCA created AIRFLOW-6788: --- Summary: The Goddamn installation instructions for using Postgres as a backend need to be scrapped and rewritten Key: AIRFLOW-6788 URL:

[GitHub] [airflow] aKumpan commented on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response

2020-02-12 Thread GitBox
aKumpan commented on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response URL: https://github.com/apache/airflow/pull/6256#issuecomment-585384765 @OmerJog, @XD-DENG could you please merge it? This is an

[GitHub] [airflow] codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task …

2020-02-12 Thread GitBox
codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task … URL: https://github.com/apache/airflow/pull/6652#issuecomment-558277657 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6652?src=pr=h1) Report > Merging

[jira] [Created] (AIRFLOW-6786) Adding KafkaConsumerHook, KafkaProducerHook, and KafkaSensor

2020-02-12 Thread Daniel Ferguson (Jira)
Daniel Ferguson created AIRFLOW-6786: Summary: Adding KafkaConsumerHook, KafkaProducerHook, and KafkaSensor Key: AIRFLOW-6786 URL: https://issues.apache.org/jira/browse/AIRFLOW-6786 Project:

[GitHub] [airflow] potiuk commented on issue #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842]

2020-02-12 Thread GitBox
potiuk commented on issue #6500: [AIRFLOW-5828] Vastly simplify build scripts. Depends on [AIRFLOW-5704] [AIRFLOW-5842] URL: https://github.com/apache/airflow/pull/6500#issuecomment-585408099 It does... I am actually already sneaking in some changes from this PR into other PRs one-by-one

[GitHub] [airflow] vasinata commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error

2020-02-12 Thread GitBox
vasinata commented on issue #6643: [AIRFLOW-6040] Fix KubernetesJobWatcher Read time out error URL: https://github.com/apache/airflow/pull/6643#issuecomment-585414414 Was this resolved? Setting AIRFLOW__KUBERNETES__KUBE_CLIENT_REQUEST_ARGS: '{ "_request_timeout": "50" }' did not resolve

[jira] [Commented] (AIRFLOW-3877) Scheduler sending full file path of dags to workers

2020-02-12 Thread Alexander Tronchin-James (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035716#comment-17035716 ] Alexander Tronchin-James commented on AIRFLOW-3877: --- Also experiencing this error

[jira] [Comment Edited] (AIRFLOW-3877) Scheduler sending full file path of dags to workers

2020-02-12 Thread Alexander Tronchin-James (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035716#comment-17035716 ] Alexander Tronchin-James edited comment on AIRFLOW-3877 at 2/12/20 9:32 PM:

[jira] [Updated] (AIRFLOW-6785) DagBag tries to run hook inside SubDagOperator

2020-02-12 Thread Darwin Yip (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darwin Yip updated AIRFLOW-6785: Description: The following worked in 1.10.1, but not in 1.10.9. It seems that the DagBag tries

[GitHub] [airflow] codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task …

2020-02-12 Thread GitBox
codecov-io edited a comment on issue #6652: [AIRFLOW-5548] [AIRFLOW-5550] REST API enhancement - dag info, task … URL: https://github.com/apache/airflow/pull/6652#issuecomment-558277657 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6652?src=pr=h1) Report > Merging

[jira] [Updated] (AIRFLOW-6785) DagBag tries to run hook inside SubDagOperator

2020-02-12 Thread Darwin Yip (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Darwin Yip updated AIRFLOW-6785: Description: The following worked in 1.10.1, but not in 1.10.9. It seems that the DagBag tries

[jira] [Updated] (AIRFLOW-6787) Make ClientRequestToken Optional

2020-02-12 Thread Lindsay Portelli (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lindsay Portelli updated AIRFLOW-6787: -- Description: Boto3 doesn't accept parameters that are None so if you want to use

[jira] [Created] (AIRFLOW-6787) Make ClientRequestToken Optional

2020-02-12 Thread Lindsay Portelli (Jira)
Lindsay Portelli created AIRFLOW-6787: - Summary: Make ClientRequestToken Optional Key: AIRFLOW-6787 URL: https://issues.apache.org/jira/browse/AIRFLOW-6787 Project: Apache Airflow Issue

[jira] [Updated] (AIRFLOW-6787) Make ClientRequestToken Optional

2020-02-12 Thread Lindsay Portelli (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lindsay Portelli updated AIRFLOW-6787: -- Description: Boto3 doesn't accept parameters that are None so if you want to use

[GitHub] [airflow] codecov-io edited a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response

2020-02-12 Thread GitBox
codecov-io edited a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response URL: https://github.com/apache/airflow/pull/6256#issuecomment-538202231 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6256?src=pr=h1) Report > Merging

[GitHub] [airflow] codecov-io edited a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response

2020-02-12 Thread GitBox
codecov-io edited a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response URL: https://github.com/apache/airflow/pull/6256#issuecomment-538202231 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6256?src=pr=h1) Report > Merging

[GitHub] [airflow] aKumpan removed a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response

2020-02-12 Thread GitBox
aKumpan removed a comment on issue #6256: [AIRFLOW-5590] Add run_id to trigger DAG run API response URL: https://github.com/apache/airflow/pull/6256#issuecomment-585384765 @OmerJog, @XD-DENG could you please merge it? This

[jira] [Commented] (AIRFLOW-6787) Make ClientRequestToken Optional

2020-02-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6787?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17035699#comment-17035699 ] ASF GitHub Bot commented on AIRFLOW-6787: - lindsable commented on pull request #7408:

[GitHub] [airflow] lindsable opened a new pull request #7408: [AIRFLOW-6787] make ClientRequestToken optional

2020-02-12 Thread GitBox
lindsable opened a new pull request #7408: [AIRFLOW-6787] make ClientRequestToken optional URL: https://github.com/apache/airflow/pull/7408 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) https://issues.apache.org/jira/browse/AIRFLOW-6787

[GitHub] [airflow] ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r378508326 ## File path: airflow/contrib/hooks/kafka_producer_hook.py ## @@ -0,0 +1,56 @@

[GitHub] [airflow] ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r378507748 ## File path: airflow/contrib/hooks/kafka_consumer_hook.py ## @@ -0,0 +1,69 @@

[GitHub] [airflow] ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r378508501 ## File path: airflow/contrib/hooks/kafka_producer_hook.py ## @@ -0,0 +1,56 @@

[GitHub] [airflow] ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor

2020-02-12 Thread GitBox
ashb commented on a change in pull request #7407: [AIRFLOW-6786] Add KafkaConsumerHook, KafkaProduerHook and KafkaSensor URL: https://github.com/apache/airflow/pull/7407#discussion_r378509192 ## File path: airflow/contrib/sensors/kafka_sensor.py ## @@ -0,0 +1,50 @@ +#

  1   2   >