[GitHub] [airflow] mik-laj edited a comment on pull request #10256: fixes http hook using schema field from airflow.models.connection.Con…

2020-08-09 Thread GitBox
mik-laj edited a comment on pull request #10256: URL: https://github.com/apache/airflow/pull/10256#issuecomment-671086076 @kubatyszko Can you give the value you have in the Secret backet? Do you realize that in the backend secret you should store the connection representation as a URI,

[GitHub] [airflow] mik-laj commented on pull request #10256: fixes http hook using schema field from airflow.models.connection.Con…

2020-08-09 Thread GitBox
mik-laj commented on pull request #10256: URL: https://github.com/apache/airflow/pull/10256#issuecomment-671086076 @kubatyszko Can you give the value you have in the Secret backet? Have you tried to create a connection using the Web UI and then view the URI representations of the

[GitHub] [airflow] mik-laj edited a comment on pull request #9907: Added feature to Import connections from a file.

2020-08-09 Thread GitBox
mik-laj edited a comment on pull request #9907: URL: https://github.com/apache/airflow/pull/9907#issuecomment-671062255 We already have the `airflow connections import` command. Your command will be a great complement to it.

[GitHub] [airflow] mik-laj commented on issue #8715: Documentation about loading packages in Python / Airflow

2020-08-09 Thread GitBox
mik-laj commented on issue #8715: URL: https://github.com/apache/airflow/issues/8715#issuecomment-671087782 It's related to: https://github.com/apache/airflow/issues/9498 https://github.com/apache/airflow/issues/9507 So I added this to the Airflow 2.0 milestone

[GitHub] [airflow] mik-laj edited a comment on issue #10007: Sagemaker contrib module - should be independant

2020-08-09 Thread GitBox
mik-laj edited a comment on issue #10007: URL: https://github.com/apache/airflow/issues/10007#issuecomment-671088094 @shlomiken Is there any progress here? This is a ticket that got a lot of thumbs up so would like to know if this ticket is lockable or needs to be processed.

[GitHub] [airflow] mik-laj commented on issue #10007: Sagemaker contrib module - should be independant

2020-08-09 Thread GitBox
mik-laj commented on issue #10007: URL: https://github.com/apache/airflow/issues/10007#issuecomment-671088094 @shlomiken Is there any progress here? This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mik-laj commented on issue #8495: Allow configuration of args vs command to support container entrypoint in kubernetes executor

2020-08-09 Thread GitBox
mik-laj commented on issue #8495: URL: https://github.com/apache/airflow/issues/8495#issuecomment-671088585 @dmayle @matthieu-foucault @humbledude Would you like to contribute this change to the project? This looks like a much needed change so I'd love to help with the review. CC:

[airflow] branch v1-10-test updated: add documentation for k8s fixes

2020-08-09 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 90fe915 add documentation for k8s

[GitHub] [airflow] mik-laj commented on issue #8715: Documentation about loading packages in Python / Airflow

2020-08-09 Thread GitBox
mik-laj commented on issue #8715: URL: https://github.com/apache/airflow/issues/8715#issuecomment-671088716 @rootcss Could you please leave a comment so I can assign you to this ticket? This is an automated message from the

[GitHub] [airflow] mik-laj commented on issue #9181: Make KubernetesExecutor PersistentVolumeClaim mounts more flexible.

2020-08-09 Thread GitBox
mik-laj commented on issue #9181: URL: https://github.com/apache/airflow/issues/9181#issuecomment-671089535 Can the expected result be achieved with this configuration option? https://airflow.readthedocs.io/en/latest/configurations-ref.html#pod-template-file

[GitHub] [airflow-site] potiuk commented on pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
potiuk commented on pull request #279: URL: https://github.com/apache/airflow-site/pull/279#issuecomment-671090633 Fantastic ! Thanks! This is an automated message from the Apache Git Service. To respond to the message,

[airflow-site] branch master updated: Move announcements page from confluence to website (#279)

2020-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/master by this push: new 6d47dfe Move announcements page from

[GitHub] [airflow-site] potiuk merged pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
potiuk merged pull request #279: URL: https://github.com/apache/airflow-site/pull/279 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk closed issue #10196: Move Announcement page to airflow.apache.org

2020-08-09 Thread GitBox
potiuk closed issue #10196: URL: https://github.com/apache/airflow/issues/10196 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] dimberman opened a new pull request #10266: Add reconcile_metadata to reconcile_pods

2020-08-09 Thread GitBox
dimberman opened a new pull request #10266: URL: https://github.com/apache/airflow/pull/10266 metadata objects require a more complex merge strategy then a simple "merge pods" for merging labels and other features --- **^ Add meaningful description above**

[GitHub] [airflow] mik-laj commented on pull request #10192: Deprecate BaseHook.get_connections method (#10135)

2020-08-09 Thread GitBox
mik-laj commented on pull request #10192: URL: https://github.com/apache/airflow/pull/10192#issuecomment-671097020 I added `airflow connection get` command: https://github.com/apache/airflow/pull/10214 This is an automated

[GitHub] [airflow] potiuk closed issue #10155: Airflow 1.10.10 + DAG SERIALIZATION = fails to start manually the DAG's operators

2020-08-09 Thread GitBox
potiuk closed issue #10155: URL: https://github.com/apache/airflow/issues/10155 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk merged pull request #10246: Added DataprepGetJobsForJobGroupOperator

2020-08-09 Thread GitBox
potiuk merged pull request #10246: URL: https://github.com/apache/airflow/pull/10246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on pull request #10246: Added DataprepGetJobsForJobGroupOperator

2020-08-09 Thread GitBox
potiuk commented on pull request #10246: URL: https://github.com/apache/airflow/pull/10246#issuecomment-671099018 Thanks @michalslowikowski00 ! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on issue #10264: Add Sphinx Spell-Checker

2020-08-09 Thread GitBox
potiuk commented on issue #10264: URL: https://github.com/apache/airflow/issues/10264#issuecomment-671098812 BTW. I got a lot better at it with recent changes in IntelliJ where spell-checking and grammar is built in :) This

[GitHub] [airflow] potiuk commented on issue #10264: Add Sphinx Spell-Checker

2020-08-09 Thread GitBox
potiuk commented on issue #10264: URL: https://github.com/apache/airflow/issues/10264#issuecomment-671098732 Love it. I make a lot of spelling mistakes (usually because I want to go fast). So having an automated check would be awesome!

[GitHub] [airflow] shlomiken commented on issue #10007: Sagemaker contrib module - should be independant

2020-08-09 Thread GitBox
shlomiken commented on issue #10007: URL: https://github.com/apache/airflow/issues/10007#issuecomment-671099821 Hi @mik-laj , unfortunately we don't have the capacity now to refactor this. we decided to install sagemaker as it worked till now. this means that the use case of working

[GitHub] [airflow] mik-laj commented on issue #8657: Add a configuration option to enable Airflow to look for DAGs in a specified S3 bucket.

2020-08-09 Thread GitBox
mik-laj commented on issue #8657: URL: https://github.com/apache/airflow/issues/8657#issuecomment-671089031 Such a feature is not planned. It is recommended to set up a separate process (e.g. sidecar) that will be responsible for file synchronization. However, I would be happy if there

[GitHub] [airflow] mik-laj edited a comment on issue #9181: Make KubernetesExecutor PersistentVolumeClaim mounts more flexible.

2020-08-09 Thread GitBox
mik-laj edited a comment on issue #9181: URL: https://github.com/apache/airflow/issues/9181#issuecomment-671089535 Can the expected result be achieved with pod_template_file option? https://airflow.readthedocs.io/en/latest/configurations-ref.html#pod-template-file

[GitHub] [airflow] mik-laj commented on issue #10038: Using >= or =~ in dependency management is a very bad practice

2020-08-09 Thread GitBox
mik-laj commented on issue #10038: URL: https://github.com/apache/airflow/issues/10038#issuecomment-671089649 Is there anything left to do in this ticket? We have docs about it: https://airflow.readthedocs.io/en/latest/installation.html

[airflow-site] branch asf-site updated: Update asf-site to output generated at 6d47dfe

2020-08-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/asf-site by this push: new 3a1233d Update asf-site to

[GitHub] [airflow] potiuk closed issue #10038: Using >= or =~ in dependency management is a very bad practice

2020-08-09 Thread GitBox
potiuk closed issue #10038: URL: https://github.com/apache/airflow/issues/10038 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #8657: Add a configuration option to enable Airflow to look for DAGs in a specified S3 bucket.

2020-08-09 Thread GitBox
potiuk commented on issue #8657: URL: https://github.com/apache/airflow/issues/8657#issuecomment-671098446 Yep. There was an extensive discussion about it at the devlist

[GitHub] [airflow] potiuk closed issue #8657: Add a configuration option to enable Airflow to look for DAGs in a specified S3 bucket.

2020-08-09 Thread GitBox
potiuk closed issue #8657: URL: https://github.com/apache/airflow/issues/8657 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow-site] potiuk commented on pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
potiuk commented on pull request #279: URL: https://github.com/apache/airflow-site/pull/279#issuecomment-671098545 The page is live. Thanks Again :) This is an automated message from the Apache Git Service. To respond to the

[airflow] branch master updated (06a1836 -> ef08831)

2020-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 06a1836 Add Missing Apache Providers to docs/installation.rst (#10265) add ef08831 Added

[GitHub] [airflow] mik-laj commented on issue #10007: Sagemaker contrib module - should be independant

2020-08-09 Thread GitBox
mik-laj commented on issue #10007: URL: https://github.com/apache/airflow/issues/10007#issuecomment-671100253 I close the ticket because the problem is no longer with Airflow, but with the Sagemaker SDK. @shlomiken Thank you for sharing your experience about this operator.

[GitHub] [airflow-site] LeonY1 edited a comment on issue #275: The buttons for the Use Cases feel reversed

2020-08-09 Thread GitBox
LeonY1 edited a comment on issue #275: URL: https://github.com/apache/airflow-site/issues/275#issuecomment-671061272 When we look at the use cases page we can see that there are a lot of different use cases ordered from left to right. So the example I will be talking about will be between

[GitHub] [airflow-site] LeonY1 commented on issue #275: The buttons for the Use Cases feel reversed

2020-08-09 Thread GitBox
LeonY1 commented on issue #275: URL: https://github.com/apache/airflow-site/issues/275#issuecomment-671061272 When we look at the use cases page we can see that there are a lot of different use cases ordered from left to right. So the example I will be talking about will be between Adobe

[GitHub] [airflow] mik-laj commented on pull request #9907: Added feature to Import connections from a file.

2020-08-09 Thread GitBox
mik-laj commented on pull request #9907: URL: https://github.com/apache/airflow/pull/9907#issuecomment-671062255 We already have the `airflow connections import` command. This command will be a great improvement for the CLI. We already have the command `airflow connections import`. Your

[GitHub] [airflow] kaxil merged pull request #10263: Fix various typos in the repo

2020-08-09 Thread GitBox
kaxil merged pull request #10263: URL: https://github.com/apache/airflow/pull/10263 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] kaxil opened a new pull request #10265: Add Missing Apache Proviers to docs/installation.rst

2020-08-09 Thread GitBox
kaxil opened a new pull request #10265: URL: https://github.com/apache/airflow/pull/10265 The following were missing: - kylin - livy - sqoop - pig --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] mik-laj commented on issue #8091: Introduce Pydocstyle

2020-08-09 Thread GitBox
mik-laj commented on issue #8091: URL: https://github.com/apache/airflow/issues/8091#issuecomment-671069863 @kaxil @potiuk What else are we planning to do with this ticket? This is an automated message from the Apache Git

[GitHub] [airflow] kaxil closed issue #8091: Introduce Pydocstyle

2020-08-09 Thread GitBox
kaxil closed issue #8091: URL: https://github.com/apache/airflow/issues/8091 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] kaxil commented on issue #8091: Introduce Pydocstyle

2020-08-09 Thread GitBox
kaxil commented on issue #8091: URL: https://github.com/apache/airflow/issues/8091#issuecomment-671069981 This can be closed, I will create a new ticket on the following steps This is an automated message from the Apache Git

[GitHub] [airflow] kaxil merged pull request #10265: Add Missing Apache Providers to docs/installation.rst

2020-08-09 Thread GitBox
kaxil merged pull request #10265: URL: https://github.com/apache/airflow/pull/10265 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] dimberman commented on a change in pull request #10230: Fix KubernetesPodOperator reattachment

2020-08-09 Thread GitBox
dimberman commented on a change in pull request #10230: URL: https://github.com/apache/airflow/pull/10230#discussion_r467603846 ## File path: airflow/providers/cncf/kubernetes/operators/kubernetes_pod.py ## @@ -304,14 +297,40 @@ def execute(self, context) -> Optional[str]:

[GitHub] [airflow] dimberman commented on pull request #10230: Fix KubernetesPodOperator reattachment

2020-08-09 Thread GitBox
dimberman commented on pull request #10230: URL: https://github.com/apache/airflow/pull/10230#issuecomment-671073461 > LGTM, logic is much clearer, thank you. > > One thing to consider is the comment by @dakov here: [#6377

[airflow] branch master updated: Fix Warning when using a different Sphinx Builder (#10262)

2020-08-09 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 5503a6a Fix Warning when using a different

[GitHub] [airflow] kaxil opened a new pull request #10263: Fix various typos in the repo

2020-08-09 Thread GitBox
kaxil opened a new pull request #10263: URL: https://github.com/apache/airflow/pull/10263 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] kaxil opened a new issue #10264: Add Spell-Checker

2020-08-09 Thread GitBox
kaxil opened a new issue #10264: URL: https://github.com/apache/airflow/issues/10264 We have been fixing various typos in the project but it would be good if we can enable a Spell Checker for our Docsite. So our docs are typo-free. We can use

[GitHub] [airflow] VijayantSoni commented on pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-08-09 Thread GitBox
VijayantSoni commented on pull request #8701: URL: https://github.com/apache/airflow/pull/8701#issuecomment-671067874 Hi @mik-laj , rebased with master and tests ran fine this time. Thanks. Will it be possible for you to review in absence of @ashb ? Thanks !

[GitHub] [airflow] mik-laj commented on pull request #8701: Adding ElastiCache Hook for creating, describing and deleting replication groups

2020-08-09 Thread GitBox
mik-laj commented on pull request #8701: URL: https://github.com/apache/airflow/pull/8701#issuecomment-671068349 I am not an AWS expert. @feluelle can you take a look? This is an automated message from the Apache Git

[airflow] branch master updated (5503a6a -> b43f90a)

2020-08-09 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git. from 5503a6a Fix Warning when using a different Sphinx Builder (#10262) add b43f90a Fix various typos in the

[airflow] branch master updated: Add Missing Apache Providers to docs/installation.rst (#10265)

2020-08-09 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 06a1836 Add Missing Apache Providers to

[GitHub] [airflow-site] yesemsanthoshkumar edited a comment on pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
yesemsanthoshkumar edited a comment on pull request #279: URL: https://github.com/apache/airflow-site/pull/279#issuecomment-671075081 > > 1. Oct 18, 2019 > >Same update as Nov 22, 2019. Should I remove one of them? > > Yep. mistake. Remove the Nov 22nd one. Done > >

[GitHub] [airflow-site] yesemsanthoshkumar commented on pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
yesemsanthoshkumar commented on pull request #279: URL: https://github.com/apache/airflow-site/pull/279#issuecomment-671075081 > > 1. Oct 18, 2019 > >Same update as Nov 22, 2019. Should I remove one of them? > > Yep. mistake. Remove the Nov 22nd one. Done > > 1.

[GitHub] [airflow] rootcss commented on issue #8715: Documentation about loading packages in Python / Airflow

2020-08-09 Thread GitBox
rootcss commented on issue #8715: URL: https://github.com/apache/airflow/issues/8715#issuecomment-671142928 @mik-laj yes, can you assign this issue to me? This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] roitvt commented on pull request #10023: spark-on-k8s sensor - add driver logs

2020-08-09 Thread GitBox
roitvt commented on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-671017812 LGTM works well with spark-pi. Great addition thanks :) @bbenzikry @mik-laj This is an automated

[GitHub] [airflow] roitvt removed a comment on pull request #10023: spark-on-k8s sensor - add driver logs

2020-08-09 Thread GitBox
roitvt removed a comment on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-671018435 > > > @roitvt, sorry to bother you - do you think you'll have time to take a look this week? > > > > > > Hi Beni, I'm really sorry I'll do it this week.

[GitHub] [airflow] roitvt commented on pull request #10023: spark-on-k8s sensor - add driver logs

2020-08-09 Thread GitBox
roitvt commented on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-671018435 > > > @roitvt, sorry to bother you - do you think you'll have time to take a look this week? > > > > > > Hi Beni, I'm really sorry I'll do it this week. thank you

[GitHub] [airflow] roitvt commented on pull request #10023: spark-on-k8s sensor - add driver logs

2020-08-09 Thread GitBox
roitvt commented on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-671018479 > > > @roitvt, sorry to bother you - do you think you'll have time to take a look this week? > > > > > > Hi Beni, I'm really sorry I'll do it this week. thank you

[GitHub] [airflow] JeffryMAC commented on a change in pull request #10241: Create "Managing variable" in howto directory

2020-08-09 Thread GitBox
JeffryMAC commented on a change in pull request #10241: URL: https://github.com/apache/airflow/pull/10241#discussion_r467544808 ## File path: docs/howto/variable.rst ## @@ -0,0 +1,72 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

[GitHub] [airflow] mik-laj commented on a change in pull request #10241: Create "Managing variable" in howto directory

2020-08-09 Thread GitBox
mik-laj commented on a change in pull request #10241: URL: https://github.com/apache/airflow/pull/10241#discussion_r467550086 ## File path: docs/howto/variable.rst ## @@ -0,0 +1,72 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

[GitHub] [airflow] mik-laj commented on a change in pull request #10241: Create "Managing variable" in howto directory

2020-08-09 Thread GitBox
mik-laj commented on a change in pull request #10241: URL: https://github.com/apache/airflow/pull/10241#discussion_r467550010 ## File path: docs/howto/variable.rst ## @@ -0,0 +1,72 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more contributor

[GitHub] [airflow] bbenzikry commented on pull request #10023: spark-on-k8s sensor - add driver logs

2020-08-09 Thread GitBox
bbenzikry commented on pull request #10023: URL: https://github.com/apache/airflow/pull/10023#issuecomment-671020059 > > > > @roitvt, sorry to bother you - do you think you'll have time to take a look this week? > > > > > > > > > Hi Beni, I'm really sorry I'll do it this week.

[GitHub] [airflow] potiuk commented on pull request #10220: Increase number of runs for quarantined tests

2020-08-09 Thread GitBox
potiuk commented on pull request #10220: URL: https://github.com/apache/airflow/pull/10220#issuecomment-671020470 :D Shall we :) This is an automated message from the Apache Git Service. To respond to the message, please log

[airflow] branch v1-10-test updated: Fix init containers

2020-08-09 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a commit to branch v1-10-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v1-10-test by this push: new 9e2557a Fix init containers

[GitHub] [airflow] potiuk merged pull request #10236: Update example on docs/howto/connection/index.rst

2020-08-09 Thread GitBox
potiuk merged pull request #10236: URL: https://github.com/apache/airflow/pull/10236 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk merged pull request #10257: Improve guide about Google Cloud Secret Manager Backend

2020-08-09 Thread GitBox
potiuk merged pull request #10257: URL: https://github.com/apache/airflow/pull/10257 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[airflow] branch master updated: Improve guide about Google Cloud Secret Manager Backend (#10257)

2020-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 55021b7 Improve guide about Google Cloud

[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #10246: Added DataprepGetJobsForJobGroupOperator

2020-08-09 Thread GitBox
michalslowikowski00 commented on a change in pull request #10246: URL: https://github.com/apache/airflow/pull/10246#discussion_r467566854 ## File path: tests/providers/google/cloud/hooks/test_dataprep.py ## @@ -0,0 +1,97 @@ +# +# Licensed to the Apache Software Foundation

[GitHub] [airflow] kaxil commented on a change in pull request #10227: Use Hash of Serialized DAG to determine DAG is changed or not

2020-08-09 Thread GitBox
kaxil commented on a change in pull request #10227: URL: https://github.com/apache/airflow/pull/10227#discussion_r467569173 ## File path: airflow/models/serialized_dag.py ## @@ -76,6 +82,7 @@ def __init__(self, dag: DAG): self.fileloc_hash =

[GitHub] [airflow] potiuk commented on pull request #10256: fixes http hook using schema field from airflow.models.connection.Con…

2020-08-09 Thread GitBox
potiuk commented on pull request #10256: URL: https://github.com/apache/airflow/pull/10256#issuecomment-671040183 Happy to help with the System Tests as well if needed -> just ping us in "#system-tests" channel in Airflow's slack. The plans are that we fully automate system tests

[GitHub] [airflow] Minyus edited a comment on issue #10037: KubernetesPodOperator namespace argument conflict when using pod_template_file

2020-08-09 Thread GitBox
Minyus edited a comment on issue #10037: URL: https://github.com/apache/airflow/issues/10037#issuecomment-671041151 I'm trying to use `pod_template_file` with `in_cluster=True` using Airflow 1.10.11 (Docker image at

[GitHub] [airflow] Minyus commented on issue #10037: KubernetesPodOperator namespace argument conflict when using pod_template_file

2020-08-09 Thread GitBox
Minyus commented on issue #10037: URL: https://github.com/apache/airflow/issues/10037#issuecomment-671041151 I'm trying to use `pod_template_file` with `in_cluster=True` using Airflow 1.10.11 (Docker image from

[GitHub] [airflow] Minyus edited a comment on issue #10037: KubernetesPodOperator namespace argument conflict when using pod_template_file

2020-08-09 Thread GitBox
Minyus edited a comment on issue #10037: URL: https://github.com/apache/airflow/issues/10037#issuecomment-671041151 I'm trying to use `pod_template_file` with `in_cluster=True` using Airflow 1.10.11 (Docker image at

[GitHub] [airflow] ljb7977 edited a comment on pull request #9022: Fix AwsGlueJobSensor

2020-08-09 Thread GitBox
ljb7977 edited a comment on pull request #9022: URL: https://github.com/apache/airflow/pull/9022#issuecomment-669986197 > @ljb7977 how is this going? Could you please also do a rebase together with the changes? :) @feluelle Sorry for late replying. I made changes that you suggested

[GitHub] [airflow] kubatyszko commented on pull request #10256: fixes http hook using schema field from airflow.models.connection.Con…

2020-08-09 Thread GitBox
kubatyszko commented on pull request #10256: URL: https://github.com/apache/airflow/pull/10256#issuecomment-671083935 I'm reconsider my solution to the issue. I think the conn_type always needs to be "http" - meaning protocol, while the scheme is supposed to reflect http or https.

[GitHub] [airflow] kaxil merged pull request #10247: Create separate section for Cron Presets

2020-08-09 Thread GitBox
kaxil merged pull request #10247: URL: https://github.com/apache/airflow/pull/10247 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated: Create separate section for Cron Presets (#10247)

2020-08-09 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 637a2c1 Create separate section for Cron

[airflow] branch constraints-master updated: Updating constraints. GH run id:201193219

2020-08-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-master by this push: new 64e06a3 Updating

[GitHub] [airflow] mik-laj commented on a change in pull request #10257: Improve guide about Google Cloud Secret Manager Backend

2020-08-09 Thread GitBox
mik-laj commented on a change in pull request #10257: URL: https://github.com/apache/airflow/pull/10257#discussion_r467563553 ## File path: docs/howto/secrets-backend/google-cloud-secret-manager-backend.rst ## @@ -26,7 +26,15 @@ a secret backend and how to manage secrets.

[GitHub] [airflow] mik-laj commented on a change in pull request #10257: Improve guide about Google Cloud Secret Manager Backend

2020-08-09 Thread GitBox
mik-laj commented on a change in pull request #10257: URL: https://github.com/apache/airflow/pull/10257#discussion_r467563372 ## File path: docs/howto/secrets-backend/google-cloud-secret-manager-backend.rst ## @@ -26,7 +26,15 @@ a secret backend and how to manage secrets.

[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #10246: Added DataprepGetJobsForJobGroupOperator

2020-08-09 Thread GitBox
michalslowikowski00 commented on a change in pull request #10246: URL: https://github.com/apache/airflow/pull/10246#discussion_r467565793 ## File path: airflow/providers/google/cloud/hooks/dataprep.py ## @@ -0,0 +1,74 @@ +# +# Licensed to the Apache Software Foundation (ASF)

[GitHub] [airflow] michalslowikowski00 commented on a change in pull request #10246: Added DataprepGetJobsForJobGroupOperator

2020-08-09 Thread GitBox
michalslowikowski00 commented on a change in pull request #10246: URL: https://github.com/apache/airflow/pull/10246#discussion_r467565824 ## File path: tests/providers/google/cloud/hooks/test_dataprep.py ## @@ -0,0 +1,97 @@ +# +# Licensed to the Apache Software Foundation

[GitHub] [airflow] kaxil opened a new pull request #10258: Add Syntax Highlights to code-blocks in docs/best-practices.rst

2020-08-09 Thread GitBox
kaxil opened a new pull request #10258: URL: https://github.com/apache/airflow/pull/10258 **Before**: ![image](https://user-images.githubusercontent.com/8811558/89730390-037f8780-da36-11ea-9006-f1c505dbc657.png) **After**:

[GitHub] [airflow] kaxil commented on a change in pull request #10227: Use Hash of Serialized DAG to determine DAG is changed or not

2020-08-09 Thread GitBox
kaxil commented on a change in pull request #10227: URL: https://github.com/apache/airflow/pull/10227#discussion_r467569001 ## File path: airflow/models/serialized_dag.py ## @@ -65,6 +66,11 @@ class SerializedDagModel(Base): fileloc_hash = Column(BigInteger,

[GitHub] [airflow] potiuk commented on a change in pull request #10227: Use Hash of Serialized DAG to determine DAG is changed or not

2020-08-09 Thread GitBox
potiuk commented on a change in pull request #10227: URL: https://github.com/apache/airflow/pull/10227#discussion_r467570749 ## File path: airflow/models/serialized_dag.py ## @@ -76,6 +82,7 @@ def __init__(self, dag: DAG): self.fileloc_hash =

[GitHub] [airflow] tooptoop4 opened a new pull request #7637: [AIRFLOW-6994] SparkSubmitOperator re-launches spark driver even when original driver still running

2020-08-09 Thread GitBox
tooptoop4 opened a new pull request #7637: URL: https://github.com/apache/airflow/pull/7637 --- Issue link: [AIRFLOW-6994](https://issues.apache.org/jira/browse/AIRFLOW-6994) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context

[jira] [Commented] (AIRFLOW-6994) SparkSubmitOperator re launches spark driver even when original driver still running

2020-08-09 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17173812#comment-17173812 ] ASF GitHub Bot commented on AIRFLOW-6994: - tooptoop4 opened a new pull request #7637: URL:

[GitHub] [airflow-site] yesemsanthoshkumar opened a new pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
yesemsanthoshkumar opened a new pull request #279: URL: https://github.com/apache/airflow-site/pull/279 Resolves https://github.com/apache/airflow/issues/10196 This is an automated message from the Apache Git Service. To

[GitHub] [airflow] kaxil commented on a change in pull request #5499: [AIRFLOW-3964][AIP-17] Build smart sensor

2020-08-09 Thread GitBox
kaxil commented on a change in pull request #5499: URL: https://github.com/apache/airflow/pull/5499#discussion_r467573812 ## File path: airflow/utils/log/file_task_handler.py ## @@ -71,15 +71,25 @@ def close(self): def _render_filename(self, ti, try_number): if

[GitHub] [airflow] mik-laj opened a new pull request #10259: Fix redirects URLs

2020-08-09 Thread GitBox
mik-laj opened a new pull request #10259: URL: https://github.com/apache/airflow/pull/10259 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] kaxil commented on issue #10155: Airflow 1.10.10 + DAG SERIALIZATION = fails to start manually the DAG's operators

2020-08-09 Thread GitBox
kaxil commented on issue #10155: URL: https://github.com/apache/airflow/issues/10155#issuecomment-671041668 This has been fixed in 1.10.11 - https://github.com/apache/airflow/pull/8775 This is an automated message from the

[GitHub] [airflow] mik-laj opened a new issue #10260: Automatic reference to configuration reference docs

2020-08-09 Thread GitBox
mik-laj opened a new issue #10260: URL: https://github.com/apache/airflow/issues/10260 Hello, I would like it to be possible to easily refer to the reference list with configuration options. Currently, most of the guides that require a description of configuration options are

[GitHub] [airflow] mik-laj commented on issue #9414: Environment variables reference

2020-08-09 Thread GitBox
mik-laj commented on issue #9414: URL: https://github.com/apache/airflow/issues/9414#issuecomment-671044613 @ghost Do you have any problem? Do you have any questions? I will be very happy to help you if I can This is an

[GitHub] [airflow] mik-laj commented on a change in pull request #10259: Fix redirects URLs

2020-08-09 Thread GitBox
mik-laj commented on a change in pull request #10259: URL: https://github.com/apache/airflow/pull/10259#discussion_r467577727 ## File path: docs/redirects.txt ## @@ -68,5 +68,5 @@ howto/operator/google/firebase/index.rst howto/operator/google/index.rst # Other redirects

[GitHub] [airflow] aranjanthakur commented on issue #10166: Add capability to specify gunicorn access log format for airflow webserver

2020-08-09 Thread GitBox
aranjanthakur commented on issue #10166: URL: https://github.com/apache/airflow/issues/10166#issuecomment-671048749 Hi @mik-laj , I have made the required changes. Could you please guide how I can test it? I have run integration tests in breeze env. I want to build it locally and test

[GitHub] [airflow-site] potiuk commented on pull request #279: Move announcements page from confluence to website

2020-08-09 Thread GitBox
potiuk commented on pull request #279: URL: https://github.com/apache/airflow-site/pull/279#issuecomment-671048559 Looks great! > Resolves [apache/airflow#10196](https://github.com/apache/airflow/issues/10196) > > @potiuk I have the following questions. > > 1. Nov 21,

[GitHub] [airflow-site] mik-laj commented on issue #275: The buttons for the Use Cases feel reversed

2020-08-09 Thread GitBox
mik-laj commented on issue #275: URL: https://github.com/apache/airflow-site/issues/275#issuecomment-671050715 @LeonY1 Can you take a screenshot and highlight which page item you are talking about? I can't find the problem either.

[airflow] branch master updated: Create "Managing variable" in howto directory (#10241)

2020-08-09 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 183cb8d Create "Managing variable" in

[GitHub] [airflow] mik-laj merged pull request #10241: Create "Managing variable" in howto directory

2020-08-09 Thread GitBox
mik-laj merged pull request #10241: URL: https://github.com/apache/airflow/pull/10241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] kaxil commented on pull request #10227: Use Hash of Serialized DAG to determine DAG is changed or not

2020-08-09 Thread GitBox
kaxil commented on pull request #10227: URL: https://github.com/apache/airflow/pull/10227#issuecomment-671029234 Ping @potiuk This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] potiuk commented on a change in pull request #10227: Use Hash of Serialized DAG to determine DAG is changed or not

2020-08-09 Thread GitBox
potiuk commented on a change in pull request #10227: URL: https://github.com/apache/airflow/pull/10227#discussion_r467566683 ## File path: airflow/models/serialized_dag.py ## @@ -76,6 +82,7 @@ def __init__(self, dag: DAG): self.fileloc_hash =

  1   2   >