[GitHub] [airflow] zhongjiajie commented on issue #7579: [AIRFLOW-6952] Use property for dag default_view

2020-02-27 Thread GitBox
zhongjiajie commented on issue #7579: [AIRFLOW-6952] Use property for dag default_view URL: https://github.com/apache/airflow/pull/7579#issuecomment-592391945 @kaxil @potiuk @mik-laj PTAL, a small change here This is an

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385543236 ## File path: airflow/providers/amazon/aws/sensors/sqs.py ## @@ -56,6 +56,7 @@ def __init__(self,

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385543222 ## File path: airflow/providers/amazon/aws/sensors/s3_prefix.py ## @@ -69,12 +70,11 @@ def

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385543208 ## File path: airflow/providers/amazon/aws/sensors/redshift.py ## @@ -43,9 +43,9 @@ def

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542845 ## File path: tests/providers/amazon/aws/operators/test_ecs.py ## @@ -48,12 +48,10 @@ ] }

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542306 ## File path: tests/providers/amazon/aws/operators/test_ecs.py ## @@ -171,7 +166,7 @@ def

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542156 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -47,11 +48,29 @@ class

[GitHub] [airflow] roitvt commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor

2020-02-27 Thread GitBox
roitvt commented on a change in pull request #7163: [AIRFLOW-6542] add spark-on-k8s operator/hook/sensor URL: https://github.com/apache/airflow/pull/7163#discussion_r385542795 ## File path: tests/test_project_structure.py ## @@ -36,6 +36,9 @@

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542845 ## File path: tests/providers/amazon/aws/operators/test_ecs.py ## @@ -48,12 +48,10 @@ ] }

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542185 ## File path: tests/providers/amazon/aws/sensors/test_sqs.py ## @@ -18,8 +18,9 @@ import

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542306 ## File path: tests/providers/amazon/aws/operators/test_ecs.py ## @@ -171,7 +166,7 @@ def

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385542156 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -47,11 +48,29 @@ class

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385541955 ## File path: airflow/providers/amazon/aws/hooks/base_aws.py ## @@ -232,6 +251,38 @@ def

[GitHub] [airflow] saguziel commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option

2020-02-27 Thread GitBox
saguziel commented on issue #7269: [AIRFLOW-6651] Add Redis Heartbeat option URL: https://github.com/apache/airflow/pull/7269#issuecomment-592382747 DB load. One example is from like from 9% to 6% load.. but this is very dependent on the configs used. It would reduce writes by 30-40% on

[jira] [Updated] (AIRFLOW-6952) Use property for dag default_view

2020-02-27 Thread zhongjiajie (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhongjiajie updated AIRFLOW-6952: - Summary: Use property for dag default_view (was: Dag default_view should use property) > Use

[jira] [Commented] (AIRFLOW-6952) Dag default_view should use property

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047252#comment-17047252 ] ASF GitHub Bot commented on AIRFLOW-6952: - zhongjiajie commented on pull request #7579:

[GitHub] [airflow] zhongjiajie opened a new pull request #7579: [AIRFLOW-6952] Use property for dag default_view

2020-02-27 Thread GitBox
zhongjiajie opened a new pull request #7579: [AIRFLOW-6952] Use property for dag default_view URL: https://github.com/apache/airflow/pull/7579 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before

[jira] [Created] (AIRFLOW-6952) Dag default_view should use property

2020-02-27 Thread zhongjiajie (Jira)
zhongjiajie created AIRFLOW-6952: Summary: Dag default_view should use property Key: AIRFLOW-6952 URL: https://issues.apache.org/jira/browse/AIRFLOW-6952 Project: Apache Airflow Issue Type:

[GitHub] [airflow] baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client

2020-02-27 Thread GitBox
baolsen commented on a change in pull request #7541: [AIRFLOW-6822] AWS hooks should cache boto3 client URL: https://github.com/apache/airflow/pull/7541#discussion_r385517283 ## File path: tests/providers/amazon/aws/operators/test_ecs.py ## @@ -48,12 +48,10 @@ ] }

[jira] [Created] (AIRFLOW-6951) sql_alchemy_connect_args missing from Configuration Reference in new website

2020-02-27 Thread Damian Shaw (Jira)
Damian Shaw created AIRFLOW-6951: Summary: sql_alchemy_connect_args missing from Configuration Reference in new website Key: AIRFLOW-6951 URL: https://issues.apache.org/jira/browse/AIRFLOW-6951

[GitHub] [airflow] zhongjiajie commented on issue #7544: [AIRFLOW-6922] Support python3.8

2020-02-27 Thread GitBox
zhongjiajie commented on issue #7544: [AIRFLOW-6922] Support python3.8 URL: https://github.com/apache/airflow/pull/7544#issuecomment-592321041 It seem that 3.8 `typing` have some different to 3.6 and 3.7, So maybe we could not just support it by this way

[GitHub] [airflow] zhongjiajie commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
zhongjiajie commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385510607 ## File path: airflow/settings.py ## @@ -173,8 +173,8 @@ def

[GitHub] [airflow] codecov-io commented on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

2020-02-27 Thread GitBox
codecov-io commented on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files URL: https://github.com/apache/airflow/pull/6788#issuecomment-592316893 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6788?src=pr=h1) Report > Merging

[GitHub] [airflow] kaxil commented on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

2020-02-27 Thread GitBox
kaxil commented on issue #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files URL: https://github.com/apache/airflow/pull/6788#issuecomment-592308818 **ToDo**: - [ ] Add new column in SerializedDagTable to store unrendered template fields - [ ] Add

[GitHub] [airflow] kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files URL: https://github.com/apache/airflow/pull/6788#discussion_r385502185 ## File path: airflow/models/templatedfields.py ## @@ -0,0 +1,97 @@ +# -*-

[GitHub] [airflow] kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files URL: https://github.com/apache/airflow/pull/6788#discussion_r38550 ## File path: airflow/models/templatedfields.py ## @@ -0,0 +1,97 @@ +# -*-

[GitHub] [airflow] kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #6788: WIP: [AIRFLOW-5944] Rendering templated_fields without accessing DAG files URL: https://github.com/apache/airflow/pull/6788#discussion_r385502167 ## File path: airflow/models/dagrun.py ## @@ -419,6 +420,17 @@ def

[GitHub] [airflow] kaxil commented on issue #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db

2020-02-27 Thread GitBox
kaxil commented on issue #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db URL: https://github.com/apache/airflow/pull/7577#issuecomment-592301688 CI failure is unrelated This is an automated

[GitHub] [airflow] kaxil edited a comment on issue #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db

2020-02-27 Thread GitBox
kaxil edited a comment on issue #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db URL: https://github.com/apache/airflow/pull/7577#issuecomment-592301688 CI failure looks unrelated This is an

[GitHub] [airflow] kaxil opened a new pull request #7578: [AIRFLOW-6856] BugFix: Paused Dags still Scheduled

2020-02-27 Thread GitBox
kaxil opened a new pull request #7578: [AIRFLOW-6856] BugFix: Paused Dags still Scheduled URL: https://github.com/apache/airflow/pull/7578 https://github.com/apache/airflow/pull/7476/ introduced a bug due to which Paused Dags where still Scheduled. The Bug is the following query

[jira] [Commented] (AIRFLOW-6856) Bulk fetch paused_dag_ids

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047189#comment-17047189 ] ASF GitHub Bot commented on AIRFLOW-6856: - kaxil commented on pull request #7578:

[jira] [Assigned] (AIRFLOW-6747) UI - Show count of tasks in each dag on the main dags page

2020-02-27 Thread Nityananda Gohain (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nityananda Gohain reassigned AIRFLOW-6747: -- Assignee: Nityananda Gohain > UI - Show count of tasks in each dag on the

[jira] [Comment Edited] (AIRFLOW-4078) Allow filtering by all columns in Browse Logs view

2020-02-27 Thread Ebrima Jallow (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047168#comment-17047168 ] Ebrima Jallow edited comment on AIRFLOW-4078 at 2/28/20 2:57 AM: -

[jira] [Commented] (AIRFLOW-4078) Allow filtering by all columns in Browse Logs view

2020-02-27 Thread Ebrima Jallow (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047168#comment-17047168 ] Ebrima Jallow commented on AIRFLOW-4078: Hello,  My name is Ebrima. I am currently doing my

[GitHub] [airflow] zhongjiajie commented on issue #7148: [AIRFLOW-6472] Correct short option in cli

2020-02-27 Thread GitBox
zhongjiajie commented on issue #7148: [AIRFLOW-6472] Correct short option in cli URL: https://github.com/apache/airflow/pull/7148#issuecomment-592285746 Conflict with UPDATING.md file, will rebase on master after get some feedback from community.

[jira] [Commented] (AIRFLOW-6472) Some of the options we have are long options with single -

2020-02-27 Thread zhongjiajie (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047167#comment-17047167 ] zhongjiajie commented on AIRFLOW-6472: -- Related dev mail list 

[GitHub] [airflow] zhongjiajie commented on a change in pull request #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator

2020-02-27 Thread GitBox
zhongjiajie commented on a change in pull request #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator URL: https://github.com/apache/airflow/pull/7575#discussion_r385480964 ## File path:

[jira] [Commented] (AIRFLOW-6950) Remove refresh_executor_config from refresh_from_db

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047138#comment-17047138 ] ASF GitHub Bot commented on AIRFLOW-6950: - kaxil commented on pull request #7577:

[GitHub] [airflow] kaxil opened a new pull request #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db

2020-02-27 Thread GitBox
kaxil opened a new pull request #7577: [AIRFLOW-6950] Remove refresh_executor_config from ti.refresh_from_db URL: https://github.com/apache/airflow/pull/7577 https://github.com/apache/airflow/pull/5926 added "refresh_executor_config" argument to "refresh_from_db" which is never used. So

[jira] [Created] (AIRFLOW-6950) Remove refresh_executor_config from refresh_from_db

2020-02-27 Thread Kaxil Naik (Jira)
Kaxil Naik created AIRFLOW-6950: --- Summary: Remove refresh_executor_config from refresh_from_db Key: AIRFLOW-6950 URL: https://issues.apache.org/jira/browse/AIRFLOW-6950 Project: Apache Airflow

[jira] [Commented] (AIRFLOW-6730) is_alive uses seconds and not total_seconds

2020-02-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047135#comment-17047135 ] ASF subversion and git services commented on AIRFLOW-6730: -- Commit

[jira] [Commented] (AIRFLOW-6730) is_alive uses seconds and not total_seconds

2020-02-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047134#comment-17047134 ] ASF subversion and git services commented on AIRFLOW-6730: -- Commit

[GitHub] [airflow] saguziel commented on issue #7363: [AIRFLOW-6730] Use total_seconds instead of seconds

2020-02-27 Thread GitBox
saguziel commented on issue #7363: [AIRFLOW-6730] Use total_seconds instead of seconds URL: https://github.com/apache/airflow/pull/7363#issuecomment-592269856 This passed on my travis-ci so i will merge This is an automated

[GitHub] [airflow] saguziel merged pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds

2020-02-27 Thread GitBox
saguziel merged pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds URL: https://github.com/apache/airflow/pull/7363 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Commented] (AIRFLOW-6730) is_alive uses seconds and not total_seconds

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047132#comment-17047132 ] ASF GitHub Bot commented on AIRFLOW-6730: - saguziel commented on pull request #7363:

[GitHub] [airflow] zhongjiajie commented on issue #7573: [AIRFLOW-6719] Add pyupgrade

2020-02-27 Thread GitBox
zhongjiajie commented on issue #7573: [AIRFLOW-6719] Add pyupgrade URL: https://github.com/apache/airflow/pull/7573#issuecomment-592267628 Related PR https://github.com/apache/airflow/pull/7343 This is an automated message

[jira] [Commented] (AIRFLOW-6933) Pass in env vars for all operators

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047055#comment-17047055 ] ASF GitHub Bot commented on AIRFLOW-6933: - saguziel commented on pull request #7576:

[GitHub] [airflow] saguziel opened a new pull request #7576: [AIRFLOW-6933] Pass in env vars for all operators

2020-02-27 Thread GitBox
saguziel opened a new pull request #7576: [AIRFLOW-6933] Pass in env vars for all operators URL: https://github.com/apache/airflow/pull/7576 Pass in the airflow context env vars for all operators by doing it in ti._run_raw. Remove it from PythonOperator since it is unnecessary. Leave it

[GitHub] [airflow] abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385377395 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[GitHub] [airflow] abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385377395 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[GitHub] [airflow] stale[bot] commented on issue #6565: [AIRFLOW-5909] Enable mapping to BYTEs type to sql_to_gcs operator

2020-02-27 Thread GitBox
stale[bot] commented on issue #6565: [AIRFLOW-5909] Enable mapping to BYTEs type to sql_to_gcs operator URL: https://github.com/apache/airflow/pull/6565#issuecomment-592221601 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no

[GitHub] [airflow] ashb merged pull request #7553: [AIRFLOW-XXXX] Update LICENSE versions and remove old licenses

2020-02-27 Thread GitBox
ashb merged pull request #7553: [AIRFLOW-] Update LICENSE versions and remove old licenses URL: https://github.com/apache/airflow/pull/7553 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ashb commented on issue #6075: [AIRFLOW-5266] Allow aws_athena_hook to get all query results

2020-02-27 Thread GitBox
ashb commented on issue #6075: [AIRFLOW-5266] Allow aws_athena_hook to get all query results URL: https://github.com/apache/airflow/pull/6075#issuecomment-592219938 I'll take a look at this tomorrow This is an automated

[GitHub] [airflow] codecov-io commented on issue #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator

2020-02-27 Thread GitBox
codecov-io commented on issue #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator URL: https://github.com/apache/airflow/pull/7575#issuecomment-592215039 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7575?src=pr=h1) Report > Merging

[GitHub] [airflow] saguziel commented on a change in pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds

2020-02-27 Thread GitBox
saguziel commented on a change in pull request #7363: [AIRFLOW-6730] Use total_seconds instead of seconds URL: https://github.com/apache/airflow/pull/7363#discussion_r385388599 ## File path: tests/providers/google/cloud/operators/test_dataproc.py ## @@ -104,7 +104,7 @@

[GitHub] [airflow] ashb commented on issue #7573: [AIRFLOW-6719] Add pyupgrade

2020-02-27 Thread GitBox
ashb commented on issue #7573: [AIRFLOW-6719] Add pyupgrade URL: https://github.com/apache/airflow/pull/7573#issuecomment-592186638 Should we hold off on this PR until we are done with 1.10.x backports entirely, as that PR makes any future backports all but impossible.

[GitHub] [airflow] ashb opened a new pull request #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator

2020-02-27 Thread GitBox
ashb opened a new pull request #7575: [AIRFLOW-6949] Respect explicit `spark.kubernetes.namespace` conf to SparkSubmitOperator URL: https://github.com/apache/airflow/pull/7575 This means the value from the Operator/dag file takes precedence over the connection The previous

[jira] [Commented] (AIRFLOW-6949) SparkSubitOperator ignores explicit spark.kubernetes.namespace config option

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046981#comment-17046981 ] ASF GitHub Bot commented on AIRFLOW-6949: - ashb commented on pull request #7575: [AIRFLOW-6949]

[jira] [Created] (AIRFLOW-6949) SparkSubitOperator ignores explicit spark.kubernetes.namespace config option

2020-02-27 Thread Ash Berlin-Taylor (Jira)
Ash Berlin-Taylor created AIRFLOW-6949: -- Summary: SparkSubitOperator ignores explicit spark.kubernetes.namespace config option Key: AIRFLOW-6949 URL: https://issues.apache.org/jira/browse/AIRFLOW-6949

[GitHub] [airflow] abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385377395 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[GitHub] [airflow] abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
abdulbasitds commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385377395 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[GitHub] [airflow] anitakar commented on a change in pull request #7217: [AIRFLOW-5946] Store source code in db

2020-02-27 Thread GitBox
anitakar commented on a change in pull request #7217: [AIRFLOW-5946] Store source code in db URL: https://github.com/apache/airflow/pull/7217#discussion_r385374472 ## File path: docs/dag-serialization.rst ## @@ -63,6 +63,8 @@ Add the following settings in ``airflow.cfg``:

[GitHub] [airflow] stale[bot] commented on issue #5388: [AIRFLOW-4490] DagRun.conf should return empty dictionary by default

2020-02-27 Thread GitBox
stale[bot] commented on issue #5388: [AIRFLOW-4490] DagRun.conf should return empty dictionary by default URL: https://github.com/apache/airflow/pull/5388#issuecomment-592150338 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if

[jira] [Resolved] (AIRFLOW-6856) Bulk fetch paused_dag_ids

2020-02-27 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6856. Fix Version/s: 2.0.0 Resolution: Fixed > Bulk fetch paused_dag_ids >

[jira] [Commented] (AIRFLOW-6856) Bulk fetch paused_dag_ids

2020-02-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046898#comment-17046898 ] ASF subversion and git services commented on AIRFLOW-6856: -- Commit

[jira] [Commented] (AIRFLOW-6856) Bulk fetch paused_dag_ids

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046897#comment-17046897 ] ASF GitHub Bot commented on AIRFLOW-6856: - mik-laj commented on pull request #7476:

[GitHub] [airflow] mik-laj merged pull request #7476: [AIRFLOW-6856] Bulk fetch paused_dag_ids

2020-02-27 Thread GitBox
mik-laj merged pull request #7476: [AIRFLOW-6856] Bulk fetch paused_dag_ids URL: https://github.com/apache/airflow/pull/7476 This is an automated message from the Apache Git Service. To respond to the message, please log on

[jira] [Assigned] (AIRFLOW-5946) Store & Read code from DB for Code View

2020-02-27 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula reassigned AIRFLOW-5946: -- Assignee: Anita Fronczak (was: Kaxil Naik) > Store & Read code from DB for Code

[jira] [Commented] (AIRFLOW-6931) One migration failed during "airflow initdb" in mssql server 2017

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046882#comment-17046882 ] ASF GitHub Bot commented on AIRFLOW-6931: - BaoshanGu commented on pull request #7574:

[GitHub] [airflow] BaoshanGu opened a new pull request #7574: [AIRFLOW-6931] Fixed migrations to find all dependencies for mssql

2020-02-27 Thread GitBox
BaoshanGu opened a new pull request #7574: [AIRFLOW-6931] Fixed migrations to find all dependencies for mssql URL: https://github.com/apache/airflow/pull/7574 [AIRFLOW-6931] Migration file 74effc47d867_change_datetime_to_datetime2_6_on_mssql_.py failed during "airflow initdb" in mssql

[GitHub] [airflow] boring-cyborg[bot] commented on issue #7574: [AIRFLOW-6931] Fixed migrations to find all dependencies for mssql

2020-02-27 Thread GitBox
boring-cyborg[bot] commented on issue #7574: [AIRFLOW-6931] Fixed migrations to find all dependencies for mssql URL: https://github.com/apache/airflow/pull/7574#issuecomment-592114910 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any

[jira] [Commented] (AIRFLOW-6361) Run LocalTaskJob directly in Celery task

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046846#comment-17046846 ] ASF GitHub Bot commented on AIRFLOW-6361: - mik-laj commented on pull request #6905:

[GitHub] [airflow] mik-laj opened a new pull request #6905: [AIRFLOW-6361] Run LocalTaskJob directly in Celery task

2020-02-27 Thread GitBox
mik-laj opened a new pull request #6905: [AIRFLOW-6361] Run LocalTaskJob directly in Celery task URL: https://github.com/apache/airflow/pull/6905 Hello, The executor runs multiple processes to perform one task. Many processes have a very short life cycle, so the process of starting

[GitHub] [airflow] mik-laj commented on issue #7343: [AIRFLOW-6719] Introduce pyupgrade to enforce latest syntax

2020-02-27 Thread GitBox
mik-laj commented on issue #7343: [AIRFLOW-6719] Introduce pyupgrade to enforce latest syntax URL: https://github.com/apache/airflow/pull/7343#issuecomment-592090216 @mik-laj please create new PR to apache/airflow but reuse JIRA number.

[GitHub] [airflow] zhongjiajie opened a new pull request #7573: Add pyupgrade

2020-02-27 Thread GitBox
zhongjiajie opened a new pull request #7573: Add pyupgrade URL: https://github.com/apache/airflow/pull/7573 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x]

[GitHub] [airflow] kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385258199 ## File path: airflow/cli/commands/version_command.py ## @@ -16,9 +16,8 @@ # under the

[GitHub] [airflow] kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385258199 ## File path: airflow/cli/commands/version_command.py ## @@ -16,9 +16,8 @@ # under the

[GitHub] [airflow] dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-27 Thread GitBox
dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi… URL: https://github.com/apache/airflow/pull/7437#discussion_r385254375 ## File path: docs/howto/write-logs.rst ## @@ -115,6 +115,29 @@ To configure

[GitHub] [airflow] kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
kaxil commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385258385 ## File path: airflow/cli/commands/version_command.py ## @@ -16,9 +16,8 @@ # under the

[GitHub] [airflow] dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-27 Thread GitBox
dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi… URL: https://github.com/apache/airflow/pull/7437#discussion_r385254375 ## File path: docs/howto/write-logs.rst ## @@ -115,6 +115,29 @@ To configure

[GitHub] [airflow] dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi…

2020-02-27 Thread GitBox
dhegberg commented on a change in pull request #7437: [AIRFLOW-2325] Add CloudwatchTaskHandler option for remote task loggi… URL: https://github.com/apache/airflow/pull/7437#discussion_r385254375 ## File path: docs/howto/write-logs.rst ## @@ -115,6 +115,29 @@ To configure

[jira] [Resolved] (AIRFLOW-6862) Do not check the freshness of fresh DAG

2020-02-27 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6862. Fix Version/s: 2.0.0 Resolution: Fixed > Do not check the freshness of fresh DAG

[jira] [Commented] (AIRFLOW-6862) Do not check the freshness of fresh DAG

2020-02-27 Thread ASF subversion and git services (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046817#comment-17046817 ] ASF subversion and git services commented on AIRFLOW-6862: -- Commit

[jira] [Commented] (AIRFLOW-6862) Do not check the freshness of fresh DAG

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046816#comment-17046816 ] ASF GitHub Bot commented on AIRFLOW-6862: - mik-laj commented on pull request #7481:

[GitHub] [airflow] mik-laj merged pull request #7481: [AIRFLOW-6862] Do not check the freshness of fresh DAG

2020-02-27 Thread GitBox
mik-laj merged pull request #7481: [AIRFLOW-6862] Do not check the freshness of fresh DAG URL: https://github.com/apache/airflow/pull/7481 This is an automated message from the Apache Git Service. To respond to the message,

[jira] [Resolved] (AIRFLOW-6650) Google Cloud Platform Connection key json documentation or code is wrong

2020-02-27 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6650?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6650. Fix Version/s: 2.0.0 Resolution: Invalid > Google Cloud Platform Connection key

[jira] [Resolved] (AIRFLOW-6682) Move GCP classes to providers package

2020-02-27 Thread Kamil Bregula (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-6682. Fix Version/s: 2.0.0 Resolution: Fixed > Move GCP classes to providers package >

[GitHub] [airflow] nuclearpinguin commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
nuclearpinguin commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385223627 ## File path: airflow/settings.py ## @@ -173,8 +173,8 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
feluelle commented on a change in pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572#discussion_r385220207 ## File path: airflow/settings.py ## @@ -173,8 +173,8 @@ def

[GitHub] [airflow] feluelle commented on a change in pull request #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-27 Thread GitBox
feluelle commented on a change in pull request #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#discussion_r385217048 ## File path: airflow/providers/mysql/hooks/mysql.py ## @@ -113,8

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-27 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-591519540 The issue is the new version of `mysql-connector-python` 8.0.19, don't know exactly what it

[GitHub] [airflow] feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook

2020-02-27 Thread GitBox
feluelle edited a comment on issue #6576: [AIRFLOW-5922] Add option to specify the mysql client library used in MySqlHook URL: https://github.com/apache/airflow/pull/6576#issuecomment-591519540 The issue is the new version of `mysql-connector-python` 8.0.19, don't know exactly what it

[jira] [Updated] (AIRFLOW-6904) Airflow 1.10.7 suppresses Operator logs

2020-02-27 Thread RAHUL JAIN (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] RAHUL JAIN updated AIRFLOW-6904: Summary: Airflow 1.10.7 suppresses Operator logs (was: Airflow 1.10.9 suppresses Operator logs)

[GitHub] [airflow] feluelle commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
feluelle commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385213897 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[jira] [Commented] (AIRFLOW-6948) Remove ASCII Airflow from version command

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046766#comment-17046766 ] ASF GitHub Bot commented on AIRFLOW-6948: - nuclearpinguin commented on pull request #7572:

[GitHub] [airflow] nuclearpinguin opened a new pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command

2020-02-27 Thread GitBox
nuclearpinguin opened a new pull request #7572: [AIRFLOW-6948] Remove ASCII Airflow from version command URL: https://github.com/apache/airflow/pull/7572 Before: ``` root@3f957e731266:/opt/airflow# airflow version [2020-02-27 16:09:09,092] {settings.py:177} INFO -

[GitHub] [airflow] feluelle commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration

2020-02-27 Thread GitBox
feluelle commented on a change in pull request #6007: [AIRFLOW-2310] Enable AWS Glue Job Integration URL: https://github.com/apache/airflow/pull/6007#discussion_r385209674 ## File path: docs/integration.rst ## @@ -17,7 +17,6 @@ Integration === - Review

[jira] [Commented] (AIRFLOW-6361) Run LocalTaskJob directly in Celery task

2020-02-27 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046757#comment-17046757 ] ASF GitHub Bot commented on AIRFLOW-6361: - stale[bot] commented on pull request #6905:

[GitHub] [airflow] stale[bot] closed pull request #6905: [AIRFLOW-6361] Run LocalTaskJob directly in Celery task

2020-02-27 Thread GitBox
stale[bot] closed pull request #6905: [AIRFLOW-6361] Run LocalTaskJob directly in Celery task URL: https://github.com/apache/airflow/pull/6905 This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] codecov-io commented on issue #7570: [AIRFLOW-6946] [WIP] Switch to MySQL 5.7 in 2.0 as base

2020-02-27 Thread GitBox
codecov-io commented on issue #7570: [AIRFLOW-6946] [WIP] Switch to MySQL 5.7 in 2.0 as base URL: https://github.com/apache/airflow/pull/7570#issuecomment-592035504 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7570?src=pr=h1) Report > Merging

  1   2   3   >