[GitHub] [airflow] potiuk commented on pull request #9084: Remove TaskInstance.log_filepath attribute

2020-05-31 Thread GitBox
potiuk commented on pull request #9084: URL: https://github.com/apache/airflow/pull/9084#issuecomment-636650043 Whil I agree with the idea, there should be an UPDATING.md entry for that one. This is an automated message from

[GitHub] [airflow] turbaszek commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
turbaszek commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r433071133 ## File path: tests/providers/google/cloud/operators/test_bigquery.py ## @@ -788,3 +789,65 @@ def test_execute(self, mock_hook): projec

[GitHub] [airflow] shawnngtq commented on issue #8752: BUG: Unable to view or browse dags in web UI after migrated airflow 1.9.0 to 1.10.10

2020-05-31 Thread GitBox
shawnngtq commented on issue #8752: URL: https://github.com/apache/airflow/issues/8752#issuecomment-636644468 I face the same issue. Commenting here for record This is an automated message from the Apache Git Service. To resp

[GitHub] [airflow] boring-cyborg[bot] commented on issue #9086: Task Instance Slots Available' FAILED

2020-05-31 Thread GitBox
boring-cyborg[bot] commented on issue #9086: URL: https://github.com/apache/airflow/issues/9086#issuecomment-636635049 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated

[GitHub] [airflow] zhengyscn opened a new issue #9086: Task Instance Slots Available' FAILED

2020-05-31 Thread GitBox
zhengyscn opened a new issue #9086: URL: https://github.com/apache/airflow/issues/9086 dependency 'Task Instance Slots Available' FAILED: The maximum number of running tasks (16) for this task's DAG 'create_instance_test' has been reached. FIXME: Rescheduling due to concurrency limits re

[GitHub] [airflow] potiuk commented on a change in pull request #8942: #8525 Add SQL Branch Operator

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8942: URL: https://github.com/apache/airflow/pull/8942#discussion_r433044218 ## File path: airflow/operators/sql_branch_operator.py ## @@ -0,0 +1,174 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contri

[GitHub] [airflow] potiuk commented on a change in pull request #8942: #8525 Add SQL Branch Operator

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8942: URL: https://github.com/apache/airflow/pull/8942#discussion_r433044093 ## File path: airflow/operators/sql_branch_operator.py ## @@ -0,0 +1,174 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contri

[GitHub] [airflow] potiuk closed issue #9016: Get query for methods in DBApi hook from files

2020-05-31 Thread GitBox
potiuk closed issue #9016: URL: https://github.com/apache/airflow/issues/9016 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[GitHub] [airflow] potiuk commented on issue #9016: Get query for methods in DBApi hook from files

2020-05-31 Thread GitBox
potiuk commented on issue #9016: URL: https://github.com/apache/airflow/issues/9016#issuecomment-636616110 Agree with @eladkal - using ext and custom operator is the right ting to do. template_ext is the mechanism foreseen to do do similar thing and it's better to utilise it. ---

[GitHub] [airflow] turbaszek commented on pull request #8296: Make execute and on_kill as abstractmethod in baseoperator

2020-05-31 Thread GitBox
turbaszek commented on pull request #8296: URL: https://github.com/apache/airflow/pull/8296#issuecomment-636605404 From docs https://docs.python.org/3/library/abc.html : >Using this decorator requires that the class’s metaclass is ABCMeta or is derived from it. A class that has a metacla

[jira] [Commented] (AIRFLOW-6808) Add Google Cloud Kubernetes Engine Operators

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120742#comment-17120742 ] ASF GitHub Bot commented on AIRFLOW-6808: - stale[bot] closed pull request #7431

[GitHub] [airflow] stale[bot] closed pull request #7431: [AIRFLOW-6808]: Add GCP GKE Operators and Hooks

2020-05-31 Thread GitBox
stale[bot] closed pull request #7431: URL: https://github.com/apache/airflow/pull/7431 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[jira] [Commented] (AIRFLOW-7033) Change dag and task state meanwhile

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120734#comment-17120734 ] ASF GitHub Bot commented on AIRFLOW-7033: - zhongjiajie commented on pull reques

[GitHub] [airflow] zhongjiajie commented on pull request #7683: [AIRFLOW-7033] Change dag and task state meanwhile

2020-05-31 Thread GitBox
zhongjiajie commented on pull request #7683: URL: https://github.com/apache/airflow/pull/7683#issuecomment-636591463 No stale, sorry about that, I got some health problem recently, will continue after recover from illness. T

[GitHub] [airflow] zhongjiajie commented on pull request #8296: Make execute and on_kill as abstractmethod in baseoperator

2020-05-31 Thread GitBox
zhongjiajie commented on pull request #8296: URL: https://github.com/apache/airflow/pull/8296#issuecomment-636591384 No stale, sorry about that, I got some health problem recently, will continue after recover from illness. T

[GitHub] [airflow] gwind edited a comment on issue #8564: Can we add configable debug settings for delay pod delete when there is a `Error` state of pods ?

2020-05-31 Thread GitBox
gwind edited a comment on issue #8564: URL: https://github.com/apache/airflow/issues/8564#issuecomment-636585115 > Hi guys! > > How did you solve the problem ? > > > > ModuleNotFoundError: No module named 'argcomplete' > > is there any setting etc to fix it??? Thi

[GitHub] [airflow] gwind commented on issue #8564: Can we add configable debug settings for delay pod delete when there is a `Error` state of pods ?

2020-05-31 Thread GitBox
gwind commented on issue #8564: URL: https://github.com/apache/airflow/issues/8564#issuecomment-636585115 > Hi guys! > > How did you solve the problem ? > > > > ModuleNotFoundError: No module named 'argcomplete' > > is there any setting etc to fix it??? This bug c

[GitHub] [airflow] gwind edited a comment on issue #8564: Can we add configable debug settings for delay pod delete when there is a `Error` state of pods ?

2020-05-31 Thread GitBox
gwind edited a comment on issue #8564: URL: https://github.com/apache/airflow/issues/8564#issuecomment-636585115 > Hi guys! > > How did you solve the problem ? > > > > ModuleNotFoundError: No module named 'argcomplete' > > is there any setting etc to fix it??? Thi

[GitHub] [airflow] mik-laj opened a new pull request #9085: Improve TestCliConfig in local env

2020-05-31 Thread GitBox
mik-laj opened a new pull request #9085: URL: https://github.com/apache/airflow/pull/9085 --- Make sure to mark the boxes below before creating PR: [x] - [X] Description above provides context of the change - [X] Unit tests coverage for changes (not needed for documentation chan

[GitHub] [airflow] ousatov-ua commented on issue #8564: Can we add configable debug settings for delay pod delete when there is a `Error` state of pods ?

2020-05-31 Thread GitBox
ousatov-ua commented on issue #8564: URL: https://github.com/apache/airflow/issues/8564#issuecomment-636551393 Hi guys! How did you solve the problem ? >> ModuleNotFoundError: No module named 'argcomplete' is there any setting etc to fix it??? -

[GitHub] [airflow] xinbinhuang commented on a change in pull request #8888: Add AWS ECS system test

2020-05-31 Thread GitBox
xinbinhuang commented on a change in pull request #: URL: https://github.com/apache/airflow/pull/#discussion_r432993143 ## File path: tests/providers/amazon/aws/operators/test_ecs_system.py ## @@ -0,0 +1,99 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] mustafagok commented on a change in pull request #8888: Add AWS ECS system test

2020-05-31 Thread GitBox
mustafagok commented on a change in pull request #: URL: https://github.com/apache/airflow/pull/#discussion_r432989031 ## File path: tests/providers/amazon/aws/operators/test_ecs_system.py ## @@ -0,0 +1,99 @@ +# +# Licensed to the Apache Software Foundation (ASF) under

[GitHub] [airflow] mik-laj commented on pull request #9051: Solution proposal to fix mounting a volume problem with dockerswarmop…

2020-05-31 Thread GitBox
mik-laj commented on pull request #9051: URL: https://github.com/apache/airflow/pull/9051#issuecomment-636534731 Unit tests are sad. Can you update them? This is an automated message from the Apache Git Service. To respond t

[GitHub] [airflow] mik-laj opened a new pull request #9084: Remove TaskInstance.log_filepath attribute

2020-05-31 Thread GitBox
mik-laj opened a new pull request #9084: URL: https://github.com/apache/airflow/pull/9084 This method returns incorrect values. We have a different way of determining this path that uses the template. But more important to me is that you can't write a method that works in every case, becau

[GitHub] [airflow] mik-laj opened a new issue #9083: Links to Stackdriver in WebUI

2020-05-31 Thread GitBox
mik-laj opened a new issue #9083: URL: https://github.com/apache/airflow/issues/9083 **Description** We currently have integration between web UI and Elasticsearch. It was added by this PR: https://github.com/apache/airflow/pull/5164 ![](https://user-images.githubusercontent

[GitHub] [airflow] Acehaidrey commented on pull request #8598: make hive macros return string type vs bytes

2020-05-31 Thread GitBox
Acehaidrey commented on pull request #8598: URL: https://github.com/apache/airflow/pull/8598#issuecomment-636525454 @turbaszek I just rebased instead of git pull etc. @ashb mind please taking a look one last time to close this out once and for all?

[GitHub] [airflow] mik-laj commented on issue #9036: Task instance log_filepath doesn't include try_number

2020-05-31 Thread GitBox
mik-laj commented on issue #9036: URL: https://github.com/apache/airflow/issues/9036#issuecomment-636520376 It seems that this property is no longer supported by Airflow and we should delete it. To generate a log path you must use FileProcessorHandler -

[GitHub] [airflow] mik-laj commented on issue #9060: Connection support for repeated extra fields

2020-05-31 Thread GitBox
mik-laj commented on issue #9060: URL: https://github.com/apache/airflow/issues/9060#issuecomment-636518696 https://github.com/apache/airflow/blob/ffbbbfc1a7c2266a19293e0c06b80d46e394fe57/airflow/models/connection.py#L211 If we want arrays to be created via the URL, we need to improve th

[GitHub] [airflow] mik-laj merged pull request #9072: Add num-executions to the "dags next_execution" cli command

2020-05-31 Thread GitBox
mik-laj merged pull request #9072: URL: https://github.com/apache/airflow/pull/9072 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[airflow] branch master updated: Add displaying multiple dates in airflow next_execution command (#9072)

2020-05-31 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new c002b25 Add displaying multiple dates in

[jira] [Commented] (AIRFLOW-578) BaseJob does not check return code of a process

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120604#comment-17120604 ] ASF GitHub Bot commented on AIRFLOW-578: lucafuji commented on pull request #687

[GitHub] [airflow] lucafuji commented on pull request #6870: [AIRFLOW-0578] Check return code

2020-05-31 Thread GitBox
lucafuji commented on pull request #6870: URL: https://github.com/apache/airflow/pull/6870#issuecomment-636504108 My I know why it's stale? If you take a look at this PR, the latest comment is mine and it seems it's the assignee has no time to review my PR. If this is the case, maybe we ca

[airflow] branch master updated: Test queries when number of active DAG Run is not zero (#9082)

2020-05-31 Thread kamilbregula
This is an automated email from the ASF dual-hosted git repository. kamilbregula pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 93b8f3e Test queries when number of acti

[GitHub] [airflow] mik-laj merged pull request #9082: Count queries when number of active DAG Run is not zero

2020-05-31 Thread GitBox
mik-laj merged pull request #9082: URL: https://github.com/apache/airflow/pull/9082 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[jira] [Commented] (AIRFLOW-3607) Decreasing scheduler delay between tasks

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120594#comment-17120594 ] ASF GitHub Bot commented on AIRFLOW-3607: - houqp commented on pull request #750

[GitHub] [airflow] houqp commented on pull request #7503: [AIRFLOW-3607] fixed the bug with picking just special cases while maintaining the p…

2020-05-31 Thread GitBox
houqp commented on pull request #7503: URL: https://github.com/apache/airflow/pull/7503#issuecomment-636498570 @ashb @mik-laj gentle ping for a final review :) This is an automated message from the Apache Git Service. To resp

[GitHub] [airflow] szha edited a comment on issue #9009: Use Terraform for System Testing

2020-05-31 Thread GitBox
szha edited a comment on issue #9009: URL: https://github.com/apache/airflow/issues/9009#issuecomment-636414596 HashiCorp recently added a clause to their [term of evaluation](https://www.hashicorp.com/terms-of-evaluation) that forbids the usage of their enterprise software in China. While

[GitHub] [airflow] mik-laj opened a new pull request #9082: Test queries when number of active DAG Run is not zero

2020-05-31 Thread GitBox
mik-laj opened a new pull request #9082: URL: https://github.com/apache/airflow/pull/9082 --- Make sure to mark the boxes below before creating PR: [x] - [X] Description above provides context of the change - [X] Unit tests coverage for changes (not needed for documentation chan

[GitHub] [airflow] ywan2017 commented on pull request #8964: Mark sure airflow get the right job status after log stream interrupt when schedule spark job on k8s

2020-05-31 Thread GitBox
ywan2017 commented on pull request #8964: URL: https://github.com/apache/airflow/pull/8964#issuecomment-636490368 This pr is closed ,and instead by a new pull request #9081 This is an automated message from the Apache Git Se

[GitHub] [airflow] ywan2017 closed pull request #8964: Mark sure airflow get the right job status after log stream interrupt when schedule spark job on k8s

2020-05-31 Thread GitBox
ywan2017 closed pull request #8964: URL: https://github.com/apache/airflow/pull/8964 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] ywan2017 opened a new pull request #9081: Get spark driver pod status if log stream interrupted accidentally

2020-05-31 Thread GitBox
ywan2017 opened a new pull request #9081: URL: https://github.com/apache/airflow/pull/9081 #8963 ## Description I am using airflow SparkSubmitOperator to schedule my spark jobs on kubernetes cluster. But for some reason, kubernetes often throw 'too old resource versio

[GitHub] [airflow] stale[bot] commented on pull request #8304: [Airflow-5565] Add extra_args param to s3hook

2020-05-31 Thread GitBox
stale[bot] commented on pull request #8304: URL: https://github.com/apache/airflow/pull/8304#issuecomment-636489821 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] turbaszek edited a comment on pull request #8805: Resolve upstream tasks when template field is XComArg

2020-05-31 Thread GitBox
turbaszek edited a comment on pull request #8805: URL: https://github.com/apache/airflow/pull/8805#issuecomment-636334773 I did a simple test: ```python from datetime import datetime from airflow import DAG from airflow.operators.dummy_operator import DummyOperator from ai

[GitHub] [airflow] mik-laj commented on a change in pull request #8651: [AIP-31] [AIRFLOW-8058] Retrieve current execution context without altering function's signature

2020-05-31 Thread GitBox
mik-laj commented on a change in pull request #8651: URL: https://github.com/apache/airflow/pull/8651#discussion_r432954063 ## File path: docs/howto/use-additional-execute-contextmanager.rst ## @@ -0,0 +1,68 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +

[jira] [Commented] (AIRFLOW-6290) Create guide for GKE service operator

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120556#comment-17120556 ] ASF GitHub Bot commented on AIRFLOW-6290: - mik-laj commented on pull request #8

[GitHub] [airflow] mik-laj commented on pull request #8883: [AIRFLOW-6290] Create guide for GKE operators

2020-05-31 Thread GitBox
mik-laj commented on pull request #8883: URL: https://github.com/apache/airflow/pull/8883#issuecomment-636479785 @tanjinP It looks good. Should I merge it? This is an automated message from the Apache Git Service. To respond

[jira] [Commented] (AIRFLOW-3607) Decreasing scheduler delay between tasks

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-3607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120544#comment-17120544 ] ASF GitHub Bot commented on AIRFLOW-3607: - stale[bot] commented on pull request

[GitHub] [airflow] stale[bot] commented on pull request #7503: [AIRFLOW-3607] fixed the bug with picking just special cases while maintaining the p…

2020-05-31 Thread GitBox
stale[bot] commented on pull request #7503: URL: https://github.com/apache/airflow/pull/7503#issuecomment-636474711 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[GitHub] [airflow] mik-laj commented on issue #8286: Cloud Memorystore for Memcached operators

2020-05-31 Thread GitBox
mik-laj commented on issue #8286: URL: https://github.com/apache/airflow/issues/8286#issuecomment-636473688 1. I would prefer that you still have one file, but you can create multiple classes in one module. 2. Yes. I think it's a good idea to create a new hook in this case. 3. This is

[jira] [Commented] (AIRFLOW-6294) Create guide for Dataflow service operator

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120542#comment-17120542 ] ASF GitHub Bot commented on AIRFLOW-6294: - mik-laj commented on pull request #8

[GitHub] [airflow] mik-laj commented on pull request #8809: [AIRFLOW-6294] Create guide for Dataflow operators

2020-05-31 Thread GitBox
mik-laj commented on pull request #8809: URL: https://github.com/apache/airflow/pull/8809#issuecomment-636473033 > I added a new example, would this break anything? I know for sure it will not run as some of the arguments were made up. I know the CI would catch this if anything is wrong, b

[GitHub] [airflow] mik-laj closed issue #8205: Create guide for Cloud Key Management Service (KMS) operators

2020-05-31 Thread GitBox
mik-laj closed issue #8205: URL: https://github.com/apache/airflow/issues/8205 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] mik-laj commented on issue #8205: Create guide for Cloud Key Management Service (KMS) operators

2020-05-31 Thread GitBox
mik-laj commented on issue #8205: URL: https://github.com/apache/airflow/issues/8205#issuecomment-636472447 I checked again and we actually don't have operators. A list of operators and hooks is available.: https://airflow.readthedocs.io/en/latest/operators-and-hooks-ref.html#id15 -

[GitHub] [airflow] mik-laj commented on pull request #9079: WIP: [8970] Improve KubernetesPodOperator guide

2020-05-31 Thread GitBox
mik-laj commented on pull request #9079: URL: https://github.com/apache/airflow/pull/9079#issuecomment-636472245 Very good direction. We just need it. This is an automated message from the Apache Git Service. To respond to th

[GitHub] [airflow] turbaszek edited a comment on pull request #8805: Resolve upstream tasks when template field is XComArg

2020-05-31 Thread GitBox
turbaszek edited a comment on pull request #8805: URL: https://github.com/apache/airflow/pull/8805#issuecomment-636334773 I did a simple test: ```python from datetime import datetime from airflow import DAG from airflow.operators.dummy_operator import DummyOperator from ai

[GitHub] [airflow] mik-laj commented on a change in pull request #9079: WIP: [8970] Improve KubernetesPodOperator guide

2020-05-31 Thread GitBox
mik-laj commented on a change in pull request #9079: URL: https://github.com/apache/airflow/pull/9079#discussion_r432947632 ## File path: docs/howto/operator/kubernetes.rst ## @@ -22,7 +22,46 @@ KubernetesPodOperator = -The :class:`~airflow.providers.cn

[airflow] branch master updated: Create guide for Dataproc Operators (#9037)

2020-05-31 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new 29eb68b Create guide for Dataproc Operators

[GitHub] [airflow] turbaszek commented on pull request #9037: Create guide for Dataproc Operators

2020-05-31 Thread GitBox
turbaszek commented on pull request #9037: URL: https://github.com/apache/airflow/pull/9037#issuecomment-636471714 Thanks @joppevos ! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] turbaszek merged pull request #9037: Create guide for Dataproc Operators

2020-05-31 Thread GitBox
turbaszek merged pull request #9037: URL: https://github.com/apache/airflow/pull/9037 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] mik-laj closed issue #8142: API Endpoint - XCOM/Variables lookup

2020-05-31 Thread GitBox
mik-laj closed issue #8142: URL: https://github.com/apache/airflow/issues/8142 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the s

[GitHub] [airflow] mik-laj commented on issue #8142: API Endpoint - XCOM/Variables lookup

2020-05-31 Thread GitBox
mik-laj commented on issue #8142: URL: https://github.com/apache/airflow/issues/8142#issuecomment-636465350 We removed these endpoints. Now they are part of ticket https://github.com/apache/airflow/issues/8134 and https://github.com/apache/airflow/issues/8133

[GitHub] [airflow] vbmade2000 commented on issue #8136: API Endpoint - Config

2020-05-31 Thread GitBox
vbmade2000 commented on issue #8136: URL: https://github.com/apache/airflow/issues/8136#issuecomment-636465426 > We haven't started implementation yet. We're working on a specification, but next week everything will start. Are you already on our Slack? We have #sig-api channel. To join, pl

[GitHub] [airflow] mik-laj commented on issue #8142: API Endpoint - XCOM/Variables lookup

2020-05-31 Thread GitBox
mik-laj commented on issue #8142: URL: https://github.com/apache/airflow/issues/8142#issuecomment-636465180 Yes. We want to enable the integration of various systems with Airflow. This is an automated message from the Apache

[GitHub] [airflow] mik-laj commented on a change in pull request #9077: Add metavar to CLI arguments

2020-05-31 Thread GitBox
mik-laj commented on a change in pull request #9077: URL: https://github.com/apache/airflow/pull/9077#discussion_r432941898 ## File path: airflow/cli/cli_parser.py ## @@ -1347,7 +1355,7 @@ def _add_action_command(sub: ActionCommand, sub_proc: argparse.ArgumentParser) - def

[GitHub] [airflow] mik-laj commented on a change in pull request #9077: Add metavar to CLI arguments

2020-05-31 Thread GitBox
mik-laj commented on a change in pull request #9077: URL: https://github.com/apache/airflow/pull/9077#discussion_r432941957 ## File path: airflow/cli/cli_parser.py ## @@ -1347,7 +1355,7 @@ def _add_action_command(sub: ActionCommand, sub_proc: argparse.ArgumentParser) - def

[GitHub] [airflow] mik-laj commented on issue #8136: API Endpoint - Config

2020-05-31 Thread GitBox
mik-laj commented on issue #8136: URL: https://github.com/apache/airflow/issues/8136#issuecomment-636464383 We haven't started implementation yet. We're working on a specification, but next week everything will start. Are you already on our Slack? We have #sig-api channel. To join, please

[GitHub] [airflow] vbmade2000 commented on issue #8136: API Endpoint - Config

2020-05-31 Thread GitBox
vbmade2000 commented on issue #8136: URL: https://github.com/apache/airflow/issues/8136#issuecomment-636463979 This looks good first one. This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [airflow] mdediana commented on a change in pull request #9077: Add metavar to CLI arguments

2020-05-31 Thread GitBox
mdediana commented on a change in pull request #9077: URL: https://github.com/apache/airflow/pull/9077#discussion_r432941231 ## File path: airflow/cli/cli_parser.py ## @@ -1347,7 +1355,7 @@ def _add_action_command(sub: ActionCommand, sub_proc: argparse.ArgumentParser) - def

[GitHub] [airflow] mdediana commented on pull request #9072: Add num-executions to the "dags next_execution" cli command

2020-05-31 Thread GitBox
mdediana commented on pull request #9072: URL: https://github.com/apache/airflow/pull/9072#issuecomment-636463270 @mik-laj Looking better now, thanks. For the invalid positive int message, I used similar wording as the existing argparse checks. Invalid int: ``` airflow webs

[GitHub] [airflow] mik-laj commented on issue #8138: API Endpoint - Dags structure/Task

2020-05-31 Thread GitBox
mik-laj commented on issue #8138: URL: https://github.com/apache/airflow/issues/8138#issuecomment-636462441 @vbmade2000 This task looks simple but requires integration with DAG Serialization and many test cases. What do you think about [API Endpoint - Config](https://github.com/apache/airf

[GitHub] [airflow] vbmade2000 edited a comment on issue #8138: API Endpoint - Dags structure/Task

2020-05-31 Thread GitBox
vbmade2000 edited a comment on issue #8138: URL: https://github.com/apache/airflow/issues/8138#issuecomment-636462130 @mik-laj I want to take this task. If I succeed then want go for more API endpoint implementation in Airflow. -

[GitHub] [airflow] vbmade2000 commented on issue #8138: API Endpoint - Dags structure/Task

2020-05-31 Thread GitBox
vbmade2000 commented on issue #8138: URL: https://github.com/apache/airflow/issues/8138#issuecomment-636462130 @mik-laj I want to take take this task. This is an automated message from the Apache Git Service. To respond to t

[GitHub] [airflow] potiuk commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432937816 ## File path: tests/providers/google/cloud/operators/test_bigquery.py ## @@ -788,3 +789,65 @@ def test_execute(self, mock_hook): project_i

[GitHub] [airflow] potiuk commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432937764 ## File path: tests/providers/google/cloud/operators/test_bigquery.py ## @@ -788,3 +789,65 @@ def test_execute(self, mock_hook): project_i

[GitHub] [airflow] potiuk commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432937559 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -1570,3 +1576,76 @@ def execute(self, context): table_resource=self.t

[GitHub] [airflow] turbaszek commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
turbaszek commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432936866 ## File path: airflow/providers/google/cloud/operators/bigquery.py ## @@ -1570,3 +1576,76 @@ def execute(self, context): table_resource=sel

[GitHub] [airflow] turbaszek commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
turbaszek commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432936839 ## File path: docs/howto/operator/gcp/bigquery.rst ## @@ -255,32 +255,18 @@ Let's say you would like to execute the following query. :end-before: [

[GitHub] [airflow] turbaszek commented on a change in pull request #8868: Add BigQueryInsertJobOperator

2020-05-31 Thread GitBox
turbaszek commented on a change in pull request #8868: URL: https://github.com/apache/airflow/pull/8868#discussion_r432936461 ## File path: tests/providers/google/cloud/operators/test_bigquery.py ## @@ -788,3 +789,65 @@ def test_execute(self, mock_hook): projec

[GitHub] [airflow] mik-laj commented on a change in pull request #9072: Add num-executions to the "dags next_execution" cli command

2020-05-31 Thread GitBox
mik-laj commented on a change in pull request #9072: URL: https://github.com/apache/airflow/pull/9072#discussion_r432935842 ## File path: airflow/cli/commands/dag_command.py ## @@ -277,17 +277,26 @@ def dag_next_execution(args): if dag.get_is_paused(): print("[INF

[jira] [Commented] (AIRFLOW-7115) display the dag owner in the dag view

2020-05-31 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120485#comment-17120485 ] ASF GitHub Bot commented on AIRFLOW-7115: - stale[bot] commented on pull request

[GitHub] [airflow] stale[bot] commented on pull request #8397: [AIRFLOW-7115] Display the dag owner in the dag view

2020-05-31 Thread GitBox
stale[bot] commented on pull request #8397: URL: https://github.com/apache/airflow/pull/8397#issuecomment-636454690 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[airflow] branch master updated: Add metric for job start/end task run (#8680)

2020-05-31 Thread turbaszek
This is an automated email from the ASF dual-hosted git repository. turbaszek pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/master by this push: new a3fc8be Add metric for job start/end task r

[GitHub] [airflow] turbaszek merged pull request #8680: Add metric for start/end task run

2020-05-31 Thread GitBox
turbaszek merged pull request #8680: URL: https://github.com/apache/airflow/pull/8680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [airflow] turbaszek commented on pull request #9037: Create guide for Dataproc Operators

2020-05-31 Thread GitBox
turbaszek commented on pull request #9037: URL: https://github.com/apache/airflow/pull/9037#issuecomment-636454231 @joppevos can you adjust tests? We've got test that checks that guides are not missing... ``` tests/test_project_structure.py:218: AssertionError ``` --

[GitHub] [airflow] potiuk opened a new issue #9080: Build and publish API clients automatically

2020-05-31 Thread GitBox
potiuk opened a new issue #9080: URL: https://github.com/apache/airflow/issues/9080 **Description** The OpenAPI allows to generate automatically clients for various languages. There are however some unresolved problems with some of the constructs in the Open API - for example oneOf

[GitHub] [airflow] potiuk commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r432931719 ## File path: openapi.yaml ## @@ -0,0 +1,2412 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] potiuk commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r432931164 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] potiuk commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r432931164 ## File path: openapi.yaml ## @@ -0,0 +1,2411 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] potiuk commented on a change in pull request #8721: Add OpenAPI specification (II)

2020-05-31 Thread GitBox
potiuk commented on a change in pull request #8721: URL: https://github.com/apache/airflow/pull/8721#discussion_r432930815 ## File path: openapi.yaml ## @@ -0,0 +1,2412 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] joppevos commented on a change in pull request #9037: Create guide for Dataproc Operators

2020-05-31 Thread GitBox
joppevos commented on a change in pull request #9037: URL: https://github.com/apache/airflow/pull/9037#discussion_r432930675 ## File path: docs/howto/operator/gcp/dataproc.rst ## @@ -0,0 +1,188 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more cont