[GitHub] [airflow] github-actions[bot] commented on pull request #17524: Decreases cassandra memory use during Integration tests

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17524: URL: https://github.com/apache/airflow/pull/17524#issuecomment-895762858 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] lwyszomi commented on pull request #17137: [Airflow 13779] use provided parameters in the wait_for_pipeline_state hook

2021-08-10 Thread GitBox
lwyszomi commented on pull request #17137: URL: https://github.com/apache/airflow/pull/17137#issuecomment-895770385 @potiuk Thank you for your response. I know that feeling when in big project we have some parts which randomly failing. I will try to what I can to help community to improve

[GitHub] [airflow] uranusjr commented on a change in pull request #15330: Add a Docker Taskflow decorator

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #15330: URL: https://github.com/apache/airflow/pull/15330#discussion_r685734272 ## File path: airflow/decorators/__init__.py ## @@ -41,103 +45,12 @@ def __call__( """ return

[GitHub] [airflow] potiuk commented on pull request #17137: [Airflow 13779] use provided parameters in the wait_for_pipeline_state hook

2021-08-10 Thread GitBox
potiuk commented on pull request #17137: URL: https://github.com/apache/airflow/pull/17137#issuecomment-895776838 @lwyszomi - actually your comment yesterday made me take a closer look to some of the failing cases (I am no vacations officialy but well, it's fun :) ) and I might have found

[GitHub] [airflow] Sonins commented on a change in pull request #17428: Fix elasticsearch-secret template port default function

2021-08-10 Thread GitBox
Sonins commented on a change in pull request #17428: URL: https://github.com/apache/airflow/pull/17428#discussion_r685708706 ## File path: chart/tests/test_elasticsearch_secret.py ## @@ -96,3 +96,24 @@ def test_should_correctly_handle_password_with_special_characters(self):

[GitHub] [airflow] github-actions[bot] commented on pull request #16012: Fixing Glue hooks/operators

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #16012: URL: https://github.com/apache/airflow/pull/16012#issuecomment-895766066 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] huozhanfeng commented on issue #17437: It's too slow to recognize new dag file when there are a log of dags files

2021-08-10 Thread GitBox
huozhanfeng commented on issue #17437: URL: https://github.com/apache/airflow/issues/17437#issuecomment-895755619 > I've put this against the 2.2 milestone -- no promise if we (the core team) will get in in for 2.2, but PRs greatly accepted and we're happy to help you get your POC PR

[GitHub] [airflow] uranusjr commented on a change in pull request #17526: remove redundant commit

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17526: URL: https://github.com/apache/airflow/pull/17526#discussion_r685723456 ## File path: airflow/jobs/scheduler_job.py ## @@ -766,7 +766,17 @@ def _clean_tis_without_dagrun(self, session):

[GitHub] [airflow] uranusjr commented on a change in pull request #17526: remove redundant commit

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17526: URL: https://github.com/apache/airflow/pull/17526#discussion_r685723456 ## File path: airflow/jobs/scheduler_job.py ## @@ -766,7 +766,17 @@ def _clean_tis_without_dagrun(self, session):

[GitHub] [airflow] potiuk merged pull request #17525: Remove hard-coded container name for trino

2021-08-10 Thread GitBox
potiuk merged pull request #17525: URL: https://github.com/apache/airflow/pull/17525 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Remove hard-coded container name for trino (#17525)

2021-08-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new bd0a3d5 Remove hard-coded container name for

[airflow] branch main updated: Decreases cassandra memory requirement during Integration tests (#17524)

2021-08-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 19777f1 Decreases cassandra memory requirement

[GitHub] [airflow] stijndehaes commented on issue #16625: Task is not retried when worker pod fails to start

2021-08-10 Thread GitBox
stijndehaes commented on issue #16625: URL: https://github.com/apache/airflow/issues/16625#issuecomment-895776139 We noticed a similar issue with Airflow 2.1.2. Job went from queued to failed without retry, looking at the code I am not sure how to fix it. It is clear that in

[GitHub] [airflow] potiuk merged pull request #17524: Decreases cassandra memory use during Integration tests

2021-08-10 Thread GitBox
potiuk merged pull request #17524: URL: https://github.com/apache/airflow/pull/17524 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eladkal commented on a change in pull request #17519: Doc: Add FAQ to speed up parsing with tons of dag files

2021-08-10 Thread GitBox
eladkal commented on a change in pull request #17519: URL: https://github.com/apache/airflow/pull/17519#discussion_r685730346 ## File path: docs/apache-airflow/faq.rst ## @@ -119,6 +119,18 @@ How do I trigger tasks based on another task's failure? You can achieve this with

[GitHub] [airflow] stijndehaes edited a comment on issue #16625: Task is not retried when worker pod fails to start

2021-08-10 Thread GitBox
stijndehaes edited a comment on issue #16625: URL: https://github.com/apache/airflow/issues/16625#issuecomment-895776139 We noticed this issue with Airflow 2.1.2. Job went from queued to failed without retry, looking at the code I am not sure how to fix it. It is clear that in

[GitHub] [airflow] github-actions[bot] commented on pull request #17525: Remove hard-coded container name for trino

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17525: URL: https://github.com/apache/airflow/pull/17525#issuecomment-895762468 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] JavierLopezT commented on pull request #17515: Initializes RedshiftSqlHook by extending the PostgresHook

2021-08-10 Thread GitBox
JavierLopezT commented on pull request #17515: URL: https://github.com/apache/airflow/pull/17515#issuecomment-895784949 I would call it just `RedshiftHook` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] stijndehaes edited a comment on issue #16625: Task is not retried when worker pod fails to start

2021-08-10 Thread GitBox
stijndehaes edited a comment on issue #16625: URL: https://github.com/apache/airflow/issues/16625#issuecomment-895776139 We noticed this issue with Airflow 2.1.2. Job went from queued to failed without retry, looking at the code I am not sure how to fix it. It is clear that in

[GitHub] [airflow] uranusjr merged pull request #17405: Add XCom.clear so it's hookable in custom XCom backend

2021-08-10 Thread GitBox
uranusjr merged pull request #17405: URL: https://github.com/apache/airflow/pull/17405 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (19777f1 -> 82229b3)

2021-08-10 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 19777f1 Decreases cassandra memory requirement during Integration tests (#17524) add 82229b3 Add XCom.clear

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16475: Add the ability to rerun success state task_instance in the Backfill API(#16452)

2021-08-10 Thread GitBox
ephraimbuddy commented on a change in pull request #16475: URL: https://github.com/apache/airflow/pull/16475#discussion_r685764959 ## File path: airflow/jobs/backfill_job.py ## @@ -471,6 +476,14 @@ def _per_task_process(key, ti, session=None):

[GitHub] [airflow] enima2684 commented on a change in pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
enima2684 commented on a change in pull request #17474: URL: https://github.com/apache/airflow/pull/17474#discussion_r685786839 ## File path: airflow/providers/docker/operators/docker_swarm.py ## @@ -121,12 +153,16 @@ def _run_service(self) -> None:

[GitHub] [airflow] gmsantos commented on issue #16833: Chart: Add docs on using custom pod-template

2021-08-10 Thread GitBox
gmsantos commented on issue #16833: URL: https://github.com/apache/airflow/issues/16833#issuecomment-895818962 Does make sense to support helm templates in `podTemplate`? As of today, there is no easy way to customize the`files/pod-template-file.kubernetes-helm-yaml` content. -- This

[GitHub] [airflow] kaxil commented on issue #17437: It's too slow to recognize new dag file when there are a log of dags files

2021-08-10 Thread GitBox
kaxil commented on issue #17437: URL: https://github.com/apache/airflow/issues/17437#issuecomment-895830829 @huozhanfeng the idea is that since you have a higher (maybe even 30 mins) min_file_process_interval the file path queue would be empty after the first pass, and then it will only

[GitHub] [airflow] uranusjr commented on pull request #17501: Simplify 404 page

2021-08-10 Thread GitBox
uranusjr commented on pull request #17501: URL: https://github.com/apache/airflow/pull/17501#issuecomment-895845363 Runs are being starved  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] huozhanfeng commented on issue #17437: It's too slow to recognize new dag file when there are a log of dags files

2021-08-10 Thread GitBox
huozhanfeng commented on issue #17437: URL: https://github.com/apache/airflow/issues/17437#issuecomment-895863014 > @huozhanfeng the idea is that since you have a higher (maybe even 30 mins) min_file_process_interval the file path queue would be empty after the first pass, and then it

[GitHub] [airflow] huozhanfeng removed a comment on issue #17437: It's too slow to recognize new dag file when there are a log of dags files

2021-08-10 Thread GitBox
huozhanfeng removed a comment on issue #17437: URL: https://github.com/apache/airflow/issues/17437#issuecomment-895755619 > I've put this against the 2.2 milestone -- no promise if we (the core team) will get in in for 2.2, but PRs greatly accepted and we're happy to help you get your POC

[GitHub] [airflow] LionelZhao28 commented on pull request #17502: localize the dag_run

2021-08-10 Thread GitBox
LionelZhao28 commented on pull request #17502: URL: https://github.com/apache/airflow/pull/17502#issuecomment-895880253 I think `pendulum` works fine when dealing with the DST time. I've changed the codes and add the unit test for the local run_id -- This is an automated message from

[GitHub] [airflow] github-actions[bot] commented on pull request #17519: Doc: Add FAQ to speed up parsing with tons of dag files

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17519: URL: https://github.com/apache/airflow/pull/17519#issuecomment-895891375 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[GitHub] [airflow] JavierLopezT edited a comment on pull request #17515: Initializes RedshiftSqlHook by extending the PostgresHook

2021-08-10 Thread GitBox
JavierLopezT edited a comment on pull request #17515: URL: https://github.com/apache/airflow/pull/17515#issuecomment-895784949 What do you think about calling it just `RedshiftHook`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] aa1371 commented on pull request #16829: Queue support for DaskExecutor using Dask Worker Resources

2021-08-10 Thread GitBox
aa1371 commented on pull request #16829: URL: https://github.com/apache/airflow/pull/16829#issuecomment-895785831 @potiuk - fixed static check errors -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [airflow] LionelZhao28 edited a comment on pull request #17502: localize the dag_run

2021-08-10 Thread GitBox
LionelZhao28 edited a comment on pull request #17502: URL: https://github.com/apache/airflow/pull/17502#issuecomment-895793877 > I think the timezone part will change. For example, if you’re in England, 2am without DST (Greenwich Mean Time) would be 02:00:00+00:00, and 2am with DST

[GitHub] [airflow] LionelZhao28 commented on pull request #17502: localize the dag_run

2021-08-10 Thread GitBox
LionelZhao28 commented on pull request #17502: URL: https://github.com/apache/airflow/pull/17502#issuecomment-895793877 > `:00:00+01:00`. If that, I think it will work fine if we can make sure the DST offset is in the `run_id`. Or, we can add another filed to show whether the

[GitHub] [airflow] lwyszomi commented on pull request #17137: [Airflow 13779] use provided parameters in the wait_for_pipeline_state hook

2021-08-10 Thread GitBox
lwyszomi commented on pull request #17137: URL: https://github.com/apache/airflow/pull/17137#issuecomment-895802566 @potiuk I rebased PR, I hope this help :) sorry that I ping you when you are on the vacation. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] uranusjr commented on a change in pull request #17528: Ignores exception raised during closing SSH connection

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17528: URL: https://github.com/apache/airflow/pull/17528#discussion_r685779664 ## File path: airflow/providers/ssh/operators/ssh.py ## @@ -149,7 +149,13 @@ def execute(self, context) -> Union[bytes, str, bool]:

[GitHub] [airflow] stijndehaes edited a comment on issue #16625: Task is not retried when worker pod fails to start

2021-08-10 Thread GitBox
stijndehaes edited a comment on issue #16625: URL: https://github.com/apache/airflow/issues/16625#issuecomment-895776139 We noticed this issue with Airflow 2.1.2. Job went from queued to failed without retry, looking at the code I am not sure how to fix it. It is clear that in

[GitHub] [airflow] uranusjr commented on a change in pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17474: URL: https://github.com/apache/airflow/pull/17474#discussion_r685170203 ## File path: airflow/providers/docker/operators/docker_swarm.py ## @@ -121,12 +153,16 @@ def _run_service(self) -> None:

[GitHub] [airflow] github-actions[bot] commented on pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17474: URL: https://github.com/apache/airflow/pull/17474#issuecomment-895850274 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] potiuk commented on issue #11309: FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'

2021-08-10 Thread GitBox
potiuk commented on issue #11309: URL: https://github.com/apache/airflow/issues/11309#issuecomment-895850195 @john-james-sf - this issue is closed and I think your have a different environment/error. Please open a new discussion (best) rather than issue and describe your circumstances,

[GitHub] [airflow] stijndehaes commented on issue #17370: Task set to be failed by scheduler directly, while all up streams success

2021-08-10 Thread GitBox
stijndehaes commented on issue #17370: URL: https://github.com/apache/airflow/issues/17370#issuecomment-895877622 > > May be related with: #16625 > > If so, then it's resolved in #16301 which will be released in 2.1.3 @ephraimbuddy I don't think that #16301 solves #16625, the

[GitHub] [airflow] enima2684 commented on issue #9310: Delay until Docker Swarm recognizes finished service

2021-08-10 Thread GitBox
enima2684 commented on issue #9310: URL: https://github.com/apache/airflow/issues/9310#issuecomment-895893308 The issue https://github.com/docker/docker-py/issues/931 is still not yet fixed and the DockerSwarmOperator still have the 60s delay before finishing a completed service.

[GitHub] [airflow] uranusjr commented on a change in pull request #17137: [Airflow 13779] use provided parameters in the wait_for_pipeline_state hook

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17137: URL: https://github.com/apache/airflow/pull/17137#discussion_r685778883 ## File path: tests/providers/google/cloud/hooks/test_datafusion.py ## @@ -197,13 +196,6 @@ def test_start_pipeline(self,

[GitHub] [airflow] potiuk opened a new pull request #17528: Ignores exception raised during closing SSH connection

2021-08-10 Thread GitBox
potiuk opened a new pull request #17528: URL: https://github.com/apache/airflow/pull/17528 Sometimes, when the connection gets closed after shutdown, a race condition causes "bad socket" error to be thrown. Actually there is nothing wrong that can happen when closing a closed

[GitHub] [airflow] uranusjr commented on issue #17527: Broken json_client

2021-08-10 Thread GitBox
uranusjr commented on issue #17527: URL: https://github.com/apache/airflow/issues/17527#issuecomment-895815549 This was probably missed when we migrated from requests to httpx. Would you be able to submit a pull request for this? -- This is an automated message from the Apache Git

[GitHub] [airflow] uranusjr commented on a change in pull request #17414: Allow custom timetable as a DAG argument

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17414: URL: https://github.com/apache/airflow/pull/17414#discussion_r685803516 ## File path: airflow/www/views.py ## @@ -2601,7 +2601,7 @@ def landing_times(self, session=None): x_points[task_id] = []

[GitHub] [airflow] github-actions[bot] commented on pull request #17414: Allow custom timetable as a DAG argument

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17414: URL: https://github.com/apache/airflow/pull/17414#issuecomment-895834636 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] uranusjr commented on pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
uranusjr commented on pull request #17474: URL: https://github.com/apache/airflow/pull/17474#issuecomment-895841252 We now have * `from docker import types` * `from docker.types import ...` * `from docker.types.services import ...` Around the same proximation. They

[GitHub] [airflow] uranusjr edited a comment on pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
uranusjr edited a comment on pull request #17474: URL: https://github.com/apache/airflow/pull/17474#issuecomment-895841252 We now have * `from docker import types` * `from docker.types import ...` * `from docker.types.services import ...` Around the same proximity. They

[GitHub] [airflow] enima2684 commented on pull request #17474: Add support for configs, secrets, networks and replicas for DockerSwarmOperator

2021-08-10 Thread GitBox
enima2684 commented on pull request #17474: URL: https://github.com/apache/airflow/pull/17474#issuecomment-895847448 Fixed the imports as asked :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk edited a comment on pull request #17033: Add autoscaling subcluster support and remove defaults

2021-08-10 Thread GitBox
potiuk edited a comment on pull request #17033: URL: https://github.com/apache/airflow/pull/17033#issuecomment-895854216 We release all providers in waves in ~ 1 month cadence. Once this is merged it will be included. The next wave is going to happen in ~ 1.5 - 2 weeks. -- This is an

[GitHub] [airflow] potiuk commented on pull request #17033: Add autoscaling subcluster support and remove defaults

2021-08-10 Thread GitBox
potiuk commented on pull request #17033: URL: https://github.com/apache/airflow/pull/17033#issuecomment-895854216 We release all providers in waves in ~ 1 month cadence. Once this is merged it will be included. The next wave is going to happen ~ 1.5 - 2 weeks. -- This is an automated

[GitHub] [airflow] ashb commented on a change in pull request #17100: WIP: Advanced Params using json-schema

2021-08-10 Thread GitBox
ashb commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r681628602 ## File path: airflow/models/dag.py ## @@ -2312,6 +2327,19 @@ def set_edge_info(self, upstream_task_id: str, downstream_task_id: str, info: Ed

[GitHub] [airflow] uranusjr commented on pull request #17502: localize the dag_run

2021-08-10 Thread GitBox
uranusjr commented on pull request #17502: URL: https://github.com/apache/airflow/pull/17502#issuecomment-895789203 I think the timezone part will change. For example, if you’re in England, 2am without DST (Greenwich Mean Time) would be `02:00:00+00:00`, and 2am with DST (British Summer

[GitHub] [airflow] uranusjr commented on pull request #17502: localize the dag_run

2021-08-10 Thread GitBox
uranusjr commented on pull request #17502: URL: https://github.com/apache/airflow/pull/17502#issuecomment-895796041 > whether the `run_id` is generated in DST This would be a bit difficult to determine unless the timezone implementation correctly implements

[GitHub] [airflow] sw1010 opened a new issue #17527: Broken json_client

2021-08-10 Thread GitBox
sw1010 opened a new issue #17527: URL: https://github.com/apache/airflow/issues/17527 **Apache Airflow version**: 2.1.2 **Apache Airflow Provider versions** (please include all providers that are relevant to your bug): apache-airflow-providers-ftp==2.0.0

[GitHub] [airflow] boring-cyborg[bot] commented on issue #17527: Broken json_client

2021-08-10 Thread GitBox
boring-cyborg[bot] commented on issue #17527: URL: https://github.com/apache/airflow/issues/17527#issuecomment-895811251 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] stijndehaes edited a comment on issue #16625: Task is not retried when worker pod fails to start

2021-08-10 Thread GitBox
stijndehaes edited a comment on issue #16625: URL: https://github.com/apache/airflow/issues/16625#issuecomment-895776139 We noticed this issue with Airflow 2.1.2. Job went from queued to failed without retry, looking at the code I am not sure how to fix it. It is clear that in

[GitHub] [airflow] github-actions[bot] commented on pull request #17528: Ignores exception raised during closing SSH connection

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17528: URL: https://github.com/apache/airflow/pull/17528#issuecomment-895816244 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] amiriry commented on issue #17443: No link for kibana when using frontend configuration

2021-08-10 Thread GitBox
amiriry commented on issue #17443: URL: https://github.com/apache/airflow/issues/17443#issuecomment-895825612 I did the command: `docker exec $(docker ps | awk '/worker/ {print $NF}') airflow providers list | grep elasticsearc` The output: `apache-airflow-providers-elasticsearch

[GitHub] [airflow] yyhecust commented on a change in pull request #16475: Add the ability to rerun success state task_instance in the Backfill API(#16452)

2021-08-10 Thread GitBox
yyhecust commented on a change in pull request #16475: URL: https://github.com/apache/airflow/pull/16475#discussion_r685802843 ## File path: tests/jobs/test_backfill_job.py ## @@ -929,8 +960,11 @@ def run_backfill(cond): dag_maker.create_dagrun(

[GitHub] [airflow] peter-volkov commented on pull request #17033: Add autoscaling subcluster support and remove defaults

2021-08-10 Thread GitBox
peter-volkov commented on pull request #17033: URL: https://github.com/apache/airflow/pull/17033#issuecomment-895848814 @potiuk how do we make a release of https://pypi.org/project/apache-airflow-providers-yandex/ ? -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] uranusjr commented on a change in pull request #17100: WIP: Advanced Params using json-schema

2021-08-10 Thread GitBox
uranusjr commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r685850326 ## File path: airflow/models/dag.py ## @@ -2312,6 +2327,19 @@ def set_edge_info(self, upstream_task_id: str, downstream_task_id: str, info: Ed

[GitHub] [airflow] sw1010 opened a new pull request #17529: Fixed the broken json_client

2021-08-10 Thread GitBox
sw1010 opened a new pull request #17529: URL: https://github.com/apache/airflow/pull/17529 The json_client depends on httpx and thus the response doesn't contain the attribute 'ok'. To fix the problem, the missing attribute has been changed to is_error. closes: #17527 --

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #17529: Fixed the broken json_client

2021-08-10 Thread GitBox
boring-cyborg[bot] commented on pull request #17529: URL: https://github.com/apache/airflow/pull/17529#issuecomment-895885569 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] ephraimbuddy commented on issue #17370: Task set to be failed by scheduler directly, while all up streams success

2021-08-10 Thread GitBox
ephraimbuddy commented on issue #17370: URL: https://github.com/apache/airflow/issues/17370#issuecomment-895885489 > > > May be related with: #16625 > > > > > > If so, then it's resolved in #16301 which will be released in 2.1.3 > > @ephraimbuddy I don't think that #16301

[GitHub] [airflow] kaxil commented on a change in pull request #17519: Doc: Add FAQ to speed up parsing with tons of dag files

2021-08-10 Thread GitBox
kaxil commented on a change in pull request #17519: URL: https://github.com/apache/airflow/pull/17519#discussion_r685864333 ## File path: docs/apache-airflow/faq.rst ## @@ -119,6 +119,18 @@ How do I trigger tasks based on another task's failure? You can achieve this with

[GitHub] [airflow] kaxil commented on pull request #17519: Doc: Add FAQ to speed up parsing with tons of dag files

2021-08-10 Thread GitBox
kaxil commented on pull request #17519: URL: https://github.com/apache/airflow/pull/17519#issuecomment-895888005 > Might be worth mentioning this only helps with "simple" DAGs, e.g. if the main DAG file itself actually changes. Not sure exactly how to phrase it in this context, but this

[GitHub] [airflow] kaxil opened a new pull request #17530: Update pre-commit hooks

2021-08-10 Thread GitBox
kaxil opened a new pull request #17530: URL: https://github.com/apache/airflow/pull/17530 black 21.6b0 -> 21.7b0. pyupgrade v2.21.0 -> v2.23.3. yamllint v1.26.1 -> v1.26.2. isort: 5.9.2 -> 5.9.3. --- **^ Add meaningful description above** Read the **[Pull

[GitHub] [airflow] kaxil closed issue #17437: It's too slow to recognize new dag file when there are a log of dags files

2021-08-10 Thread GitBox
kaxil closed issue #17437: URL: https://github.com/apache/airflow/issues/17437 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil merged pull request #17519: Doc: Add FAQ to speed up parsing with tons of dag files

2021-08-10 Thread GitBox
kaxil merged pull request #17519: URL: https://github.com/apache/airflow/pull/17519 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (82229b3 -> 7dfc520)

2021-08-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 82229b3 Add XCom.clear so it's hookable in custom XCom backend (#17405) add 7dfc520 Doc: Add FAQ to speed

[GitHub] [airflow] github-actions[bot] commented on pull request #17530: Update pre-commit hooks

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17530: URL: https://github.com/apache/airflow/pull/17530#issuecomment-895905694 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[GitHub] [airflow] uranusjr closed pull request #17414: Allow custom timetable as a DAG argument

2021-08-10 Thread GitBox
uranusjr closed pull request #17414: URL: https://github.com/apache/airflow/pull/17414 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] github-actions[bot] commented on pull request #17529: Fixed the broken json_client

2021-08-10 Thread GitBox
github-actions[bot] commented on pull request #17529: URL: https://github.com/apache/airflow/pull/17529#issuecomment-895923724 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] thaysa closed issue #17481: Execution date is in future

2021-08-10 Thread GitBox
thaysa closed issue #17481: URL: https://github.com/apache/airflow/issues/17481 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil closed issue #17527: Broken json_client

2021-08-10 Thread GitBox
kaxil closed issue #17527: URL: https://github.com/apache/airflow/issues/17527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #17529: Fixed the broken json_client

2021-08-10 Thread GitBox
boring-cyborg[bot] commented on pull request #17529: URL: https://github.com/apache/airflow/pull/17529#issuecomment-895950473 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] kaxil merged pull request #17529: Fixed the broken json_client

2021-08-10 Thread GitBox
kaxil merged pull request #17529: URL: https://github.com/apache/airflow/pull/17529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (663f173 -> fbddefe)

2021-08-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 663f173 Update pre-commit hooks (#17530) add fbddefe Fixed broken json_client (#17529) No new revisions

[GitHub] [airflow] kaxil commented on a change in pull request #15385: Added table to view providers in airflow ui under admin tab

2021-08-10 Thread GitBox
kaxil commented on a change in pull request #15385: URL: https://github.com/apache/airflow/pull/15385#discussion_r685937377 ## File path: airflow/security/permissions.py ## @@ -41,6 +41,7 @@ RESOURCE_PLUGIN = "Plugins" RESOURCE_PROVIDER = "Providers" RESOURCE_ROLE = "Roles"

[GitHub] [airflow] frankcash commented on pull request #17515: Initializes RedshiftSqlHook by extending the PostgresHook

2021-08-10 Thread GitBox
frankcash commented on pull request #17515: URL: https://github.com/apache/airflow/pull/17515#issuecomment-895961108 I have no problem with that, in-fact I thought about it. I just feel it could be confusing as there already is a `RedshiftHook`

[GitHub] [airflow] uranusjr commented on issue #17534: Dags are not triggered at right time when start date is set

2021-08-10 Thread GitBox
uranusjr commented on issue #17534: URL: https://github.com/apache/airflow/issues/17534#issuecomment-895987034 This is correct. Aifrlow has an implicit notion of *intervals*. For a DAG started at 2021-07-29 and runs every day, it needs one initial day before executing, so the initial run

[GitHub] [airflow] uranusjr closed issue #17534: Dags are not triggered at right time when start date is set

2021-08-10 Thread GitBox
uranusjr closed issue #17534: URL: https://github.com/apache/airflow/issues/17534 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #16475: Add the ability to rerun success state task_instance in the Backfill API(#16452)

2021-08-10 Thread GitBox
ephraimbuddy commented on a change in pull request #16475: URL: https://github.com/apache/airflow/pull/16475#discussion_r685879899 ## File path: airflow/jobs/backfill_job.py ## @@ -471,6 +476,13 @@ def _per_task_process(key, ti, session=None):

[GitHub] [airflow] uranusjr commented on pull request #17414: Allow custom timetable as a DAG argument

2021-08-10 Thread GitBox
uranusjr commented on pull request #17414: URL: https://github.com/apache/airflow/pull/17414#issuecomment-895919125 OK I think this is good now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] uranusjr merged pull request #17530: Update pre-commit hooks

2021-08-10 Thread GitBox
uranusjr merged pull request #17530: URL: https://github.com/apache/airflow/pull/17530 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (7dfc520 -> 663f173)

2021-08-10 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7dfc520 Doc: Add FAQ to speed up parsing with tons of dag files (#17519) add 663f173 Update pre-commit

[GitHub] [airflow] massamany opened a new pull request #17531: fix port overriding in cli for postgres

2021-08-10 Thread GitBox
massamany opened a new pull request #17531: URL: https://github.com/apache/airflow/pull/17531 When configuring the sql alchemy url like this : ``` sql_alchemy_conn = postgresql://user:password@localhost:/airflowdb ``` Then the Airflow shell can't be used, because the port is

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #17531: fix port overriding in cli for postgres

2021-08-10 Thread GitBox
boring-cyborg[bot] commented on pull request #17531: URL: https://github.com/apache/airflow/pull/17531#issuecomment-895919947 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] yyhecust commented on a change in pull request #16475: Add the ability to rerun success state task_instance in the Backfill API(#16452)

2021-08-10 Thread GitBox
yyhecust commented on a change in pull request #16475: URL: https://github.com/apache/airflow/pull/16475#discussion_r685929982 ## File path: airflow/jobs/backfill_job.py ## @@ -471,6 +476,13 @@ def _per_task_process(key, ti, session=None):

[GitHub] [airflow] kaxil merged pull request #17528: Ignores exception raised during closing SSH connection

2021-08-10 Thread GitBox
kaxil merged pull request #17528: URL: https://github.com/apache/airflow/pull/17528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (fbddefe -> f424780)

2021-08-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fbddefe Fixed broken json_client (#17529) add f424780 Ignores exception raised during closing SSH

[GitHub] [airflow] kaxil opened a new pull request #17532: Minor README fixes

2021-08-10 Thread GitBox
kaxil opened a new pull request #17532: URL: https://github.com/apache/airflow/pull/17532 - Markdown was not formatted correctly in the in-line code-block - Just adds single quotes around pip install as the square bracket errors out on some terminals --- **^ Add

[GitHub] [airflow] kaxil merged pull request #17532: Minor README fixes

2021-08-10 Thread GitBox
kaxil merged pull request #17532: URL: https://github.com/apache/airflow/pull/17532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (f424780 -> dec1e9d)

2021-08-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from f424780 Ignores exception raised during closing SSH connection (#17528) add dec1e9d Minor README fixes

[GitHub] [airflow] yyhecust commented on a change in pull request #16475: Add the ability to rerun success state task_instance in the Backfill API(#16452)

2021-08-10 Thread GitBox
yyhecust commented on a change in pull request #16475: URL: https://github.com/apache/airflow/pull/16475#discussion_r685942656 ## File path: airflow/jobs/backfill_job.py ## @@ -471,6 +476,13 @@ def _per_task_process(key, ti, session=None):

[GitHub] [airflow] kaxil commented on a change in pull request #17506: Use `dag_maker` fixture in test_processor.py

2021-08-10 Thread GitBox
kaxil commented on a change in pull request #17506: URL: https://github.com/apache/airflow/pull/17506#discussion_r685951621 ## File path: tests/conftest.py ## @@ -484,3 +489,41 @@ def __call__(self, dag_id='test_dag', session=None, **kwargs): return self

[GitHub] [airflow] massamany commented on pull request #17531: fix port overriding in cli for postgres

2021-08-10 Thread GitBox
massamany commented on pull request #17531: URL: https://github.com/apache/airflow/pull/17531#issuecomment-895977218 > This PR is against v1-10-stable which has reached EOL currently. https://github.com/apache/airflow#version-life-cycle > > Please check the "main" branch and PR

[GitHub] [airflow] kaxil commented on pull request #17531: fix port overriding in cli for postgres

2021-08-10 Thread GitBox
kaxil commented on pull request #17531: URL: https://github.com/apache/airflow/pull/17531#issuecomment-895984895 > > This PR is against v1-10-stable which has reached EOL currently. https://github.com/apache/airflow#version-life-cycle > > Please check the "main" branch and PR against it

  1   2   3   >