[GitHub] [airflow] ephraimbuddy commented on issue #18041: Tasks intermittently gets terminated with SIGTERM on kubernetes executor

2021-09-10 Thread GitBox
ephraimbuddy commented on issue #18041: URL: https://github.com/apache/airflow/issues/18041#issuecomment-916691668 @laserpedro It will be very helpful if you can provide a simple dag to reproduce this behaviour. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] potiuk commented on issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
potiuk commented on issue #18133: URL: https://github.com/apache/airflow/issues/18133#issuecomment-916787541 Just started to answer when @uranusjr did :) ,. so to repeat - as a solution for now you need to set the encoding properly via LANG* variables in your environment (for all

[GitHub] [airflow] potiuk commented on issue #18136: Allow airflow standard images to run in openshift utilising the official helm chart

2021-09-10 Thread GitBox
potiuk commented on issue #18136: URL: https://github.com/apache/airflow/issues/18136#issuecomment-916799112 Good Idea. While the Airflow Official Image supports Open-Shift arbitrary user, the Charts might indeed need some modifications to be able to support it out-of-the-box cc:

[GitHub] [airflow] potiuk commented on issue #18136: Allow airflow standard images to run in openshift utilising the official helm chart

2021-09-10 Thread GitBox
potiuk commented on issue #18136: URL: https://github.com/apache/airflow/issues/18136#issuecomment-916799362 Happy to help with review @nwalens ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] lvjg commented on issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
lvjg commented on issue #18133: URL: https://github.com/apache/airflow/issues/18133#issuecomment-916845288 thanks!!! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] ashb commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-10 Thread GitBox
ashb commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r706136935 ## File path: airflow/cli/commands/dag_command.py ## @@ -106,21 +106,25 @@ def dag_backfill(args, dag=None): dag_run_state=State.NONE,

[GitHub] [airflow] ashb commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-10 Thread GitBox
ashb commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r706137492 ## File path: airflow/serialization/serialized_objects.py ## @@ -878,6 +920,34 @@ def from_dict(cls, serialized_obj: dict) -> 'SerializedDAG':

[GitHub] [airflow] eladkal commented on issue #18031: Alert if a task misses deadline

2021-09-10 Thread GitBox
eladkal commented on issue #18031: URL: https://github.com/apache/airflow/issues/18031#issuecomment-916727949 I find no value with the current SLA feature. I don't know many people who are using it due to it's limitations. I would say that a better approch is to deprecate the current

[GitHub] [airflow] Taragolis commented on issue #18125: Create pytest plugin for prepare test environment

2021-09-10 Thread GitBox
Taragolis commented on issue #18125: URL: https://github.com/apache/airflow/issues/18125#issuecomment-916741336 Most probably my explanation not clear. I shouldn't have created it in midnight, I even wasn't use my main account :-) Let me explain wit some sample, probably after

[GitHub] [airflow] cablespaghetti commented on pull request #17211: Chart: Use stable API versions where available

2021-09-10 Thread GitBox
cablespaghetti commented on pull request #17211: URL: https://github.com/apache/airflow/pull/17211#issuecomment-916773094 Ok it is quite rightly erroring since I changed the default for Ingress path from "" to "/". I'll change that in the schema and update the UPDATING.rst file. The other

[GitHub] [airflow] uranusjr commented on pull request #18042: Fixing ses email backend

2021-09-10 Thread GitBox
uranusjr commented on pull request #18042: URL: https://github.com/apache/airflow/pull/18042#issuecomment-916724995 It seems like the CI failed since it ran out of resource. I think you can disregard those failures. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [airflow] eladkal commented on issue #17336: Celery worker fails with RecursionError when asctime is in log_format but is not in json_fields

2021-09-10 Thread GitBox
eladkal commented on issue #17336: URL: https://github.com/apache/airflow/issues/17336#issuecomment-916768827 Correct me if i'm wrong but if `json_format=True` and `asctime` isn't in `json_fields` then you can't reference `asctime` in `log_format` -- This is an automated message

[airflow] branch v2-1-test updated: Sane detection of the host/port in entrypoint prod (#17847)

2021-09-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-1-test by this push: new c11d368 Sane detection of the host/port

[GitHub] [airflow] laserpedro edited a comment on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-10 Thread GitBox
laserpedro edited a comment on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-915407555 cf https://github.com/apache/airflow/issues/18041#issuecomment-915383493 For me for the moment the pb is on backfill mode. tonight my processes will run on this

[GitHub] [airflow] uranusjr commented on issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
uranusjr commented on issue #18133: URL: https://github.com/apache/airflow/issues/18133#issuecomment-916786534 You probably need this: https://stackoverflow.com/a/27931669/1376863 Although explicitly using UTF-8 is probably a correct thing to do anyway. IIRC the source code read

[GitHub] [airflow] raefezzeldin opened a new issue #18135: Dags persistence on MultiNode Kubernetes Cluster

2021-09-10 Thread GitBox
raefezzeldin opened a new issue #18135: URL: https://github.com/apache/airflow/issues/18135 ### Apache Airflow version 2.1.3 (latest released) ### Operating System Official image os ### Versions of Apache Airflow Providers _No response_ ###

[GitHub] [airflow] potiuk commented on pull request #17576: Add pre/post execution hooks

2021-09-10 Thread GitBox
potiuk commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-916792442 > I'd like to get 2.2 release train started, and in order to not delay that unnecessarily how about we mark this feature as experimental -- that way if we decide we _don't_

[GitHub] [airflow] nathadfield commented on a change in pull request #18064: Fixing Vault AppRole authentication with CONN_URI

2021-09-10 Thread GitBox
nathadfield commented on a change in pull request #18064: URL: https://github.com/apache/airflow/pull/18064#discussion_r706113088 ## File path: airflow/providers/hashicorp/hooks/vault.py ## @@ -148,7 +149,21 @@ def __init__( except ValueError:

[GitHub] [airflow] nathadfield commented on a change in pull request #18064: Fixing Vault AppRole authentication with CONN_URI

2021-09-10 Thread GitBox
nathadfield commented on a change in pull request #18064: URL: https://github.com/apache/airflow/pull/18064#discussion_r706118393 ## File path: airflow/providers/hashicorp/hooks/vault.py ## @@ -148,7 +149,21 @@ def __init__( except ValueError:

[GitHub] [airflow] github-actions[bot] commented on pull request #18141: Fix quarentine tests affected by AIP-39

2021-09-10 Thread GitBox
github-actions[bot] commented on pull request #18141: URL: https://github.com/apache/airflow/pull/18141#issuecomment-916860597 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916868864 Fix here: https://github.com/apache/airflow/pull/18142 (there were quite a few missing ``__init__.py` files. Stil do not what triggered the error though (and did not trigger

[GitHub] [airflow] github-actions[bot] commented on pull request #18139: Add a note about no back-compat guarantees for experimental features

2021-09-10 Thread GitBox
github-actions[bot] commented on pull request #18139: URL: https://github.com/apache/airflow/pull/18139#issuecomment-916833483 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might decide

[airflow] branch main updated (e6cb2f7 -> b8926ee)

2021-09-10 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from e6cb2f7 ECSOperator returns last logs when ECS task fails (#17209) add b8926ee Add a note about no back-compat

[GitHub] [airflow] ashb merged pull request #18139: Add a note about no back-compat guarantees for experimental features

2021-09-10 Thread GitBox
ashb merged pull request #18139: URL: https://github.com/apache/airflow/pull/18139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] lvjg commented on issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
lvjg commented on issue #18133: URL: https://github.com/apache/airflow/issues/18133#issuecomment-916845161 thanks~ > You probably need this: https://stackoverflow.com/a/27931669/1376863 > > Although explicitly using UTF-8 is probably a correct thing to do anyway. IIRC the

[GitHub] [airflow] ephraimbuddy edited a comment on issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
ephraimbuddy edited a comment on issue #18131: URL: https://github.com/apache/airflow/issues/18131#issuecomment-916862728 > I believe this is same as #17786 and fix will come in 2.1.4 (shortly). @ephraimbuddy - can you confirm? The fix is https://github.com/apache/airflow/pull/17945

[GitHub] [airflow] ashb merged pull request #18140: Mark passing pre/post execute callbacks to operators as experimental.

2021-09-10 Thread GitBox
ashb merged pull request #18140: URL: https://github.com/apache/airflow/pull/18140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Mark passing pre/post execute callbacks to operators as experimental. (#18140)

2021-09-10 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0e3b06b Mark passing pre/post execute callbacks to

[GitHub] [airflow] ashb commented on issue #18069: Move auto-refresh interval to config variable

2021-09-10 Thread GitBox
ashb commented on issue #18069: URL: https://github.com/apache/airflow/issues/18069#issuecomment-916864057 Yeah, absolutely no problem with your PR, I was just a bit surprised by the effects! -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916869739 Sorry for that @subkanthi ... That was somewhat unexpected :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] laserpedro edited a comment on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-10 Thread GitBox
laserpedro edited a comment on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-915407555 cf https://github.com/apache/airflow/issues/18041#issuecomment-915383493 For me for the moment the pb is on backfill mode. tonight my processes will run on this

[GitHub] [airflow] potiuk commented on issue #18031: Alert if a task misses deadline

2021-09-10 Thread GitBox
potiuk commented on issue #18031: URL: https://github.com/apache/airflow/issues/18031#issuecomment-916796799 Yeah. I think we should have a more comprehensive/better/complete/usable SLA implementation replacing the current one. -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916849947 Found the problem. Fix is coming. I am not sure how it was triggered now but was not triggered before. Tableau test package misses ``_init_.py` simply ... I will add it

[GitHub] [airflow] uranusjr commented on a change in pull request #18141: Fix quarentine tests affected by AIP-39

2021-09-10 Thread GitBox
uranusjr commented on a change in pull request #18141: URL: https://github.com/apache/airflow/pull/18141#discussion_r706131668 ## File path: airflow/jobs/scheduler_job.py ## @@ -954,7 +954,7 @@ def _schedule_dag_run( unfinished_task_instances = (

[GitHub] [airflow] mobuchowski commented on issue #17984: Add possibility to LineageBackend to be notified of task instance execution start and failure

2021-09-10 Thread GitBox
mobuchowski commented on issue #17984: URL: https://github.com/apache/airflow/issues/17984#issuecomment-916862802 Hey @potiuk, maybe you had time to look at this :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] ephraimbuddy commented on issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
ephraimbuddy commented on issue #18131: URL: https://github.com/apache/airflow/issues/18131#issuecomment-916862728 > I believe this is same as #17786 and fix will come in 2.1.4 (shortly). @ephraimbuddy - can you confirm? The fix is https://github.com/apache/airflow/pull/18061,

[airflow] branch v2-1-test updated (c11d368 -> 6866a7c)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from c11d368 Sane detection of the host/port in entrypoint prod (#17847) new 59bf5b5 Do not delete running

[airflow] 01/03: Do not delete running DAG from the UI (#17630)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 59bf5b5167192ebc60e2c9235d0dc1e999bf228a Author: Ephraim Anierobi AuthorDate: Mon Aug 16 09:20:35 2021 +0100

[airflow] 03/03: Fix wrong query on running tis (#17631)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 6866a7c2e131b5e8fcbef5473abd4835cad1c972 Author: Ephraim Anierobi AuthorDate: Tue Aug 17 09:37:56 2021 +0100

[airflow] 02/03: Add root to tree refresh url (#17633)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 34fbe0ddebefa35cd46498c21cdeb1377334f68a Author: Brent Bovenzi AuthorDate: Mon Aug 16 18:00:02 2021 +0200

[GitHub] [airflow] potiuk opened a new pull request #18142: Add missing __init__.py files for some test packages

2021-09-10 Thread GitBox
potiuk opened a new pull request #18142: URL: https://github.com/apache/airflow/pull/18142 Lacl of the __init__.py caused failures in some specific cases - especially when new providers have been added. This PR adds missing ``__init__.py`` files and modifies pre-commit check

[GitHub] [airflow] malthe commented on pull request #17576: Add pre/post execution hooks

2021-09-10 Thread GitBox
malthe commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-916790800 @ashb yes that sounds very reasonable – to me, `pre_execute` is really just a workaround. In a perfect world, the scheduling system is able to support these use-cases out of

[GitHub] [airflow] potiuk closed issue #18135: Dags persistence on MultiNode Kubernetes Cluster

2021-09-10 Thread GitBox
potiuk closed issue #18135: URL: https://github.com/apache/airflow/issues/18135 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on issue #18135: Dags persistence on MultiNode Kubernetes Cluster

2021-09-10 Thread GitBox
potiuk commented on issue #18135: URL: https://github.com/apache/airflow/issues/18135#issuecomment-916790317 You need to share the disks between the node. Airflow does not support separate volumes per schedule. Your persistent volume claim has to be one of the distributed fllesystems that

[GitHub] [airflow] potiuk edited a comment on issue #18135: Dags persistence on MultiNode Kubernetes Cluster

2021-09-10 Thread GitBox
potiuk edited a comment on issue #18135: URL: https://github.com/apache/airflow/issues/18135#issuecomment-916790317 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] ashb opened a new pull request #18140: Mark passing pre/post execute callbacks to operators as experimental.

2021-09-10 Thread GitBox
ashb opened a new pull request #18140: URL: https://github.com/apache/airflow/pull/18140 My primary concern here is that by being able to arbitrarily "change" what an operator does will greatly increase the "accidental complexity" of both Airflow (for us as developers) and of the DAG

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916840245 There is indeed something fishy going on here with test collection. I am taking a look, BTW (for the future @eladkal @subkanthi ) - if you look at the logs, it is

[GitHub] [airflow] ashb commented on pull request #18140: Mark passing pre/post execute callbacks to operators as experimental.

2021-09-10 Thread GitBox
ashb commented on pull request #18140: URL: https://github.com/apache/airflow/pull/18140#issuecomment-916840480 /cc @malthe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] eladkal commented on issue #18089: KeyError when ECS failed to start image

2021-09-10 Thread GitBox
eladkal commented on issue #18089: URL: https://github.com/apache/airflow/issues/18089#issuecomment-916758390 @theagilehacker feel free to submit PR with a fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] potiuk commented on pull request #17576: Add pre/post execution hooks

2021-09-10 Thread GitBox
potiuk commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-916793392 We can even print warning when it is used, that it is experimental. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on a change in pull request #18064: Fixing Vault AppRole authentication with CONN_URI

2021-09-10 Thread GitBox
potiuk commented on a change in pull request #18064: URL: https://github.com/apache/airflow/pull/18064#discussion_r706089404 ## File path: airflow/providers/hashicorp/hooks/vault.py ## @@ -148,7 +149,21 @@ def __init__( except ValueError: raise

[GitHub] [airflow] ashb opened a new pull request #18139: Add a note about no back-compat guarantees for experimental features

2021-09-10 Thread GitBox
ashb opened a new pull request #18139: URL: https://github.com/apache/airflow/pull/18139 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] potiuk commented on issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
potiuk commented on issue #18131: URL: https://github.com/apache/airflow/issues/18131#issuecomment-916837455 I believe this is duplicate of #17786 and fix will come in 2.1.4 (shortly). @ephraimbuddy - can you confirm? -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916841729 also @uranusjr -> for the future ^^ - for similar investigations, you might not be aware of it but we have full reproducibility of the images used during tests locally (the

[GitHub] [airflow] potiuk edited a comment on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk edited a comment on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-916849947 Found the problem. Fix is coming. I am not sure how it was triggered now but was not triggered before. Tableau test package misses `_init_.py` simply ... I will add it

[GitHub] [airflow] github-actions[bot] commented on pull request #18140: Mark passing pre/post execute callbacks to operators as experimental.

2021-09-10 Thread GitBox
github-actions[bot] commented on pull request #18140: URL: https://github.com/apache/airflow/pull/18140#issuecomment-916860998 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] lvjg opened a new issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
lvjg opened a new issue #18133: URL: https://github.com/apache/airflow/issues/18133 ### Apache Airflow version main (development) ### Operating System centos7 ### Versions of Apache Airflow Providers _No response_ ### Deployment Other

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18133: bulk_sync_to_db got UnicodeDecodeError when Chinese characters in dag code

2021-09-10 Thread GitBox
boring-cyborg[bot] commented on issue #18133: URL: https://github.com/apache/airflow/issues/18133#issuecomment-916761145 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ashb commented on pull request #17576: Add pre/post execution hooks

2021-09-10 Thread GitBox
ashb commented on pull request #17576: URL: https://github.com/apache/airflow/pull/17576#issuecomment-916788361 I'd like to get 2.2 release train started, and in order to not delay that unnecessarily how about we mark this feature as experimental -- that way if we decide we _don't_ want

[GitHub] [airflow] potiuk edited a comment on issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
potiuk edited a comment on issue #18131: URL: https://github.com/apache/airflow/issues/18131#issuecomment-916837455 I believe this is same as #17786 and fix will come in 2.1.4 (shortly). @ephraimbuddy - can you confirm? -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] ephraimbuddy commented on pull request #18071: Change from dynamic date to fixed date in examples

2021-09-10 Thread GitBox
ephraimbuddy commented on pull request #18071: URL: https://github.com/apache/airflow/pull/18071#issuecomment-916860013 I have tried to experiment with what you said @collinmcnulty but unfortunately, I can't wrap my head around it. For example, this dag when parsed today: ```

[GitHub] [airflow] potiuk commented on issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
potiuk commented on issue #18131: URL: https://github.com/apache/airflow/issues/18131#issuecomment-916871892 Closing then and marking as 2.1.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk closed issue #18131: Airflow scheduler stuck with large DAG and concurrency limit

2021-09-10 Thread GitBox
potiuk closed issue #18131: URL: https://github.com/apache/airflow/issues/18131 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] nwalens opened a new issue #18136: Allow airflow standard images to run in openshift utilising the official helm chart

2021-09-10 Thread GitBox
nwalens opened a new issue #18136: URL: https://github.com/apache/airflow/issues/18136 ### Description Airflow helm chart is very powerful and configurable, however in order to run it in a on-premises openshift 4 environment, one must manually create security context constraints or

[GitHub] [airflow] boring-cyborg[bot] commented on issue #18136: Allow airflow standard images to run in openshift utilising the official helm chart

2021-09-10 Thread GitBox
boring-cyborg[bot] commented on issue #18136: URL: https://github.com/apache/airflow/issues/18136#issuecomment-916797458 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] uranusjr opened a new pull request #18141: Fix quarentine tests affected by AIP-39

2021-09-10 Thread GitBox
uranusjr opened a new pull request #18141: URL: https://github.com/apache/airflow/pull/18141 When we did `TaskInstance`’s migration from `execution_date` to `run_id`, we forgot to check the quarentined tests. This fixes them. Note that this does _not_ make the quarentined tests to

[GitHub] [airflow] kaxil commented on a change in pull request #17100: Advanced Params using json-schema

2021-09-10 Thread GitBox
kaxil commented on a change in pull request #17100: URL: https://github.com/apache/airflow/pull/17100#discussion_r706138480 ## File path: airflow/serialization/serialized_objects.py ## @@ -878,6 +920,34 @@ def from_dict(cls, serialized_obj: dict) -> 'SerializedDAG':

[GitHub] [airflow] potiuk commented on issue #17984: Add possibility to LineageBackend to be notified of task instance execution start and failure

2021-09-10 Thread GitBox
potiuk commented on issue #17984: URL: https://github.com/apache/airflow/issues/17984#issuecomment-916871219 Not yet. Life took over. But I will -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] mik-laj edited a comment on pull request #18143: Use parameters instead of params

2021-09-10 Thread GitBox
mik-laj edited a comment on pull request #18143: URL: https://github.com/apache/airflow/pull/18143#issuecomment-916896880 Should we add notes in CHANGELOG like in google provider?

[airflow] branch v2-1-test updated: Limit the number of queued dagruns created by the Scheduler (#18065)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-1-test by this push: new 203b455 Limit the number of queued

[GitHub] [airflow] ephraimbuddy commented on pull request #18071: Change from dynamic date to fixed date in examples

2021-09-10 Thread GitBox
ephraimbuddy commented on pull request #18071: URL: https://github.com/apache/airflow/pull/18071#issuecomment-916923176 > > would have execution_date stored as "2021-09-12 00:00:00+00". This is when it should run. > > On the execution day, the start_date will still be in the past:

[airflow] branch main updated: Add missing __init__.py files for some test packages (#18142)

2021-09-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3d4bfdc Add missing __init__.py files for some

[GitHub] [airflow] potiuk merged pull request #18142: Add missing __init__.py files for some test packages

2021-09-10 Thread GitBox
potiuk merged pull request #18142: URL: https://github.com/apache/airflow/pull/18142 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch constraints-2-1 updated: Updated to latest version before 2.1.4 release

2021-09-10 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch constraints-2-1 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-1 by this push: new f65e2a5 Updated to latest

[GitHub] [airflow] potiuk commented on pull request #17068: Influxdb Hook

2021-09-10 Thread GitBox
potiuk commented on pull request #17068: URL: https://github.com/apache/airflow/pull/17068#issuecomment-917046569 One more missing ``__init__.py`` - now I think I know where it could come from. Lack of __init__.py creates an implicit namespace and the problem was that from influxdb tests

[GitHub] [airflow] github-actions[bot] commented on pull request #18143: Use parameters instead of params

2021-09-10 Thread GitBox
github-actions[bot] commented on pull request #18143: URL: https://github.com/apache/airflow/pull/18143#issuecomment-916885911 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[airflow] branch v2-1-test updated: Improve dag/task concurrency check (#17786)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-1-test by this push: new 19f32fd Improve dag/task concurrency

[GitHub] [airflow] potiuk commented on a change in pull request #18130: Add script that validate version fields in config.yaml

2021-09-10 Thread GitBox
potiuk commented on a change in pull request #18130: URL: https://github.com/apache/airflow/pull/18130#discussion_r706214672 ## File path: dev/README_RELEASE_AIRFLOW.md ## @@ -745,3 +746,15 @@ EOF ## Update Announcements page Update "Announcements" page at the [Official

[GitHub] [airflow] nathadfield commented on a change in pull request #18064: Fixing Vault AppRole authentication with CONN_URI

2021-09-10 Thread GitBox
nathadfield commented on a change in pull request #18064: URL: https://github.com/apache/airflow/pull/18064#discussion_r706236888 ## File path: airflow/providers/hashicorp/hooks/vault.py ## @@ -148,7 +149,21 @@ def __init__( except ValueError:

[GitHub] [airflow] potiuk commented on a change in pull request #18147: Allow airflow standard images to run in openshift utilising the official helm chart #18136

2021-09-10 Thread GitBox
potiuk commented on a change in pull request #18147: URL: https://github.com/apache/airflow/pull/18147#discussion_r706312705 ## File path: chart/values.yaml ## @@ -1251,7 +1252,7 @@ dags: # Volume size for dags size: 1Gi # If using a custom storageClass, pass

[GitHub] [airflow] jedcunningham commented on a change in pull request #18147: Allow airflow standard images to run in openshift utilising the official helm chart #18136

2021-09-10 Thread GitBox
jedcunningham commented on a change in pull request #18147: URL: https://github.com/apache/airflow/pull/18147#discussion_r706291745 ## File path: chart/templates/rbac/security-context-constraint-rolebinding.yaml ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation

[GitHub] [airflow] potiuk commented on a change in pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-10 Thread GitBox
potiuk commented on a change in pull request #18068: URL: https://github.com/apache/airflow/pull/18068#discussion_r706325427 ## File path: Dockerfile ## @@ -479,6 +479,7 @@ LABEL org.apache.airflow.distro="debian" \ org.opencontainers.image.title="Production Airflow Image"

[airflow] 04/05: Limit colorlog version (6.x is incompatible) (#18099)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 9f98a7d4319f2b1c7cf88f026f7f8442ff7d6560 Author: Malthe Borch AuthorDate: Thu Sep 9 00:05:34 2021 +0200

[airflow] branch v2-1-test updated (6866a7c -> f9c337f)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git. from 6866a7c Fix wrong query on running tis (#17631) new 8ecc710 Increase width for Run column (#17817)

[GitHub] [airflow] msumit opened a new pull request #18143: Use parameters instead of params

2021-09-10 Thread GitBox
msumit opened a new pull request #18143: URL: https://github.com/apache/airflow/pull/18143 These 2 operators - `FacebookAdsReportToGcsOperator` & `GoogleDisplayVideo360RunReportOperator` were (mis)using the `params` keyword & causing a lot of pain in

[GitHub] [airflow] potiuk commented on pull request #18104: Adding corresponding labels to pods created by jobs

2021-09-10 Thread GitBox
potiuk commented on pull request #18104: URL: https://github.com/apache/airflow/pull/18104#issuecomment-916901437 Cool. I think you will need to fix some tests. I run the workflow for you, let's see. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] yavuzmester commented on issue #17507: Task processes killed with WARNING - Recorded pid does not match the current pid

2021-09-10 Thread GitBox
yavuzmester commented on issue #17507: URL: https://github.com/apache/airflow/issues/17507#issuecomment-916909875 I also got the same error and I am using `run_as_user`. I tried changing the BashOperator to PythonOperator and it worked for me. In detail: - Create a function

[airflow] branch v2-1-test updated: Fix deprecation error message rather than silencing it (#18126)

2021-09-10 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a commit to branch v2-1-test in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/v2-1-test by this push: new 54f3a77 Fix deprecation error

[GitHub] [airflow] collinmcnulty edited a comment on pull request #18071: Change from dynamic date to fixed date in examples

2021-09-10 Thread GitBox
collinmcnulty edited a comment on pull request #18071: URL: https://github.com/apache/airflow/pull/18071#issuecomment-916908995 > would have execution_date stored as "2021-09-12 00:00:00+00". This is when it should run. On the execution day, the start_date will still be in the past:

[GitHub] [airflow] ephraimbuddy merged pull request #18071: Change from dynamic date to fixed date in examples

2021-09-10 Thread GitBox
ephraimbuddy merged pull request #18071: URL: https://github.com/apache/airflow/pull/18071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (68d99bc -> 0df31cd)

2021-09-10 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 68d99bc [Airflow 16364] Add conn_timeout and cmd_timeout params to SSHOperator; add conn_timeout param to

[GitHub] [airflow] github-actions[bot] commented on pull request #18142: Add missing __init__.py files for some test packages

2021-09-10 Thread GitBox
github-actions[bot] commented on pull request #18142: URL: https://github.com/apache/airflow/pull/18142#issuecomment-916931855 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] jedcunningham opened a new issue #18146: Deferrable operator not clearing `next_method` and `next_kwargs` on retries

2021-09-10 Thread GitBox
jedcunningham opened a new issue #18146: URL: https://github.com/apache/airflow/issues/18146 ### Apache Airflow version main (development) ### Operating System macOS 11.5.2 ### Versions of Apache Airflow Providers _No response_ ### Deployment

[GitHub] [airflow] potiuk commented on pull request #18068: Switch dumb-init to propagate signals to child process only

2021-09-10 Thread GitBox

[GitHub] [airflow] jedcunningham commented on issue #18136: Allow airflow standard images to run in openshift utilising the official helm chart

2021-09-10 Thread GitBox
jedcunningham commented on issue #18136: URL: https://github.com/apache/airflow/issues/18136#issuecomment-916987782 @nwalens, I'm also happy to help! Ping us on slack in #helm-chart-official if you get stuck or want/need to chat about this . -- This is an automated message from the

[GitHub] [airflow] jedcunningham closed pull request #17211: Chart: Use stable API versions where available

2021-09-10 Thread GitBox
jedcunningham closed pull request #17211: URL: https://github.com/apache/airflow/pull/17211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] jedcunningham commented on pull request #17211: Chart: Use stable API versions where available

2021-09-10 Thread GitBox
jedcunningham commented on pull request #17211: URL: https://github.com/apache/airflow/pull/17211#issuecomment-917000456 > I don't understand the test failure we've currently got.  Most likely a flaky failure, let's try it again. -- This is an automated message from the Apache

[GitHub] [airflow] cablespaghetti opened a new pull request #17211: Chart: Use stable API versions where available

2021-09-10 Thread GitBox
cablespaghetti opened a new pull request #17211: URL: https://github.com/apache/airflow/pull/17211 Fixes #17188 Uses stable versions of Ingress, PodDisruptionBudget and CronJob APIs where available. There is no change for users of Kubernetes versions older than 1.19. Also tidies

[GitHub] [airflow] mik-laj commented on a change in pull request #18130: Add script that validate version fields in config.yaml

2021-09-10 Thread GitBox
mik-laj commented on a change in pull request #18130: URL: https://github.com/apache/airflow/pull/18130#discussion_r706307729 ## File path: dev/README_RELEASE_AIRFLOW.md ## @@ -745,3 +746,15 @@ EOF ## Update Announcements page Update "Announcements" page at the [Official

[GitHub] [airflow] mik-laj commented on a change in pull request #18130: Add script that validate version fields in config.yaml

2021-09-10 Thread GitBox
mik-laj commented on a change in pull request #18130: URL: https://github.com/apache/airflow/pull/18130#discussion_r706329683 ## File path: dev/README_RELEASE_AIRFLOW.md ## @@ -745,3 +746,15 @@ EOF ## Update Announcements page Update "Announcements" page at the [Official

  1   2   3   >