[GitHub] [airflow] yxiao1996 commented on pull request #28686: Update gantt chart UI to display queued state of tasks

2023-01-18 Thread GitBox
yxiao1996 commented on PR #28686: URL: https://github.com/apache/airflow/pull/28686#issuecomment-1387510337 @bbovenzi Hi Brent, could you please help me take another look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] potiuk commented on pull request #28569: Add general-purpose "notifier" concept to DAGs

2023-01-18 Thread GitBox
potiuk commented on PR #28569: URL: https://github.com/apache/airflow/pull/28569#issuecomment-1387507644 > Should we add docs for this feature @ephraimbuddy ? Currently this feature is a "hidden" one Agree we should. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] potiuk merged pull request #28828: Fix helm chart create scheduler service to serve task logs for LocalKubernetesExecutor

2023-01-18 Thread GitBox
potiuk merged PR #28828: URL: https://github.com/apache/airflow/pull/28828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
potiuk commented on PR #29005: URL: https://github.com/apache/airflow/pull/29005#issuecomment-1387503931 This could be retried now: https://github.com/apache/airflow/commit/5ec488f2d03773ebf4c238033d87e5f43e8ad4ac as constraints got updated (and mysql-connector-python is updated). --

[GitHub] [airflow] pierrejeambrun merged pull request #28997: Sync v2-5-stable with v2-5-test to release 2.5.1

2023-01-18 Thread GitBox
pierrejeambrun merged PR #28997: URL: https://github.com/apache/airflow/pull/28997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #27905: listener plugin example added

2023-01-18 Thread GitBox
potiuk commented on PR #27905: URL: https://github.com/apache/airflow/pull/27905#issuecomment-1387493658 Nice one ! @kaxil ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] frankcash opened a new pull request, #29021: [DOCs]change context example to no longer us as variable

2023-01-18 Thread GitBox
frankcash opened a new pull request, #29021: URL: https://github.com/apache/airflow/pull/29021 Due to: https://airflow.apache.org/docs/apache-airflow/stable/release_notes.html#dags-used-in-a-context-manager-no-longer-need-to-be-assigned-to-a-module-variable-23592 DAGs using the

[GitHub] [airflow] kaxil merged pull request #28764: Add support for running a Beam Go pipeline with an executable binary

2023-01-18 Thread GitBox
kaxil merged PR #28764: URL: https://github.com/apache/airflow/pull/28764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #28799: introduce dag processor job fix #27140

2023-01-18 Thread GitBox
potiuk commented on PR #28799: URL: https://github.com/apache/airflow/pull/28799#issuecomment-1387461523 Looks cool @farhan0syakir. But I have two asks: 1. Did you test it with more than one DagFileProcessor running on two different subdirs - each of them should be a potentially

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #28264: Fix helm chart NOTES.txt to show correct URL

2023-01-18 Thread GitBox
boring-cyborg[bot] commented on PR #28264: URL: https://github.com/apache/airflow/pull/28264#issuecomment-1387451731 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk merged pull request #28264: Fix helm chart NOTES.txt to show correct URL

2023-01-18 Thread GitBox
potiuk merged PR #28264: URL: https://github.com/apache/airflow/pull/28264 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #29001: uniformize getting hook through cached property in aws sensors

2023-01-18 Thread GitBox
vandonr-amz commented on code in PR #29001: URL: https://github.com/apache/airflow/pull/29001#discussion_r1073838452 ## airflow/providers/amazon/aws/sensors/eks.py: ## @@ -98,13 +99,15 @@ def __init__( self.region = region super().__init__(**kwargs) -def

[GitHub] [airflow] bolkedebruin commented on a diff in pull request #28980: Ensure non-JSON types go through encoder

2023-01-18 Thread GitBox
bolkedebruin commented on code in PR #28980: URL: https://github.com/apache/airflow/pull/28980#discussion_r1073821432 ## airflow/utils/json.py: ## @@ -60,24 +60,30 @@ def default(self, o: Any) -> Any: if isinstance(data, dict) and DATA in data:

[GitHub] [airflow] bolkedebruin opened a new pull request, #29019: Improve serialization of tuples and sets

2023-01-18 Thread GitBox
bolkedebruin opened a new pull request, #29019: URL: https://github.com/apache/airflow/pull/29019 Sets cannot be encoded into JSON directly and tuples loose information when serialized into JSON directly. We now serialize both into a dict and encode them properly. @uranusjr

[GitHub] [airflow] potiuk merged pull request #29015: Remove erroneous TODO comment

2023-01-18 Thread GitBox
potiuk merged PR #29015: URL: https://github.com/apache/airflow/pull/29015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] snjypl commented on pull request #28394: Fix manual task trigger failing for k8s.

2023-01-18 Thread GitBox
snjypl commented on PR #28394: URL: https://github.com/apache/airflow/pull/28394#issuecomment-1387366314 > @snjypl Tried out your patch in 29012 as well, but still no luck. I'm still getting the same forbidden error. If you want, I can move pod creation specific comments to the discussion

[GitHub] [airflow] vchiapaikeo commented on a diff in pull request #28942: Fix @task.kubernetes to receive input and send output

2023-01-18 Thread GitBox
vchiapaikeo commented on code in PR #28942: URL: https://github.com/apache/airflow/pull/28942#discussion_r1073745616 ## airflow/providers/cncf/kubernetes/decorators/kubernetes.py: ## @@ -82,11 +83,42 @@ def _get_python_source(self): res = remove_task_decorator(res,

[GitHub] [airflow] phanikumv commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
phanikumv commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1073729476 ## airflow/providers/dbt/cloud/operators/dbt.py: ## @@ -91,8 +96,17 @@ def __init__( timeout: int = 60 * 60 * 24 * 7, check_interval: int = 60,

[GitHub] [airflow] inytar commented on issue #28973: Dynamic Task Mapping skips tasks before upstream has started

2023-01-18 Thread GitBox
inytar commented on issue #28973: URL: https://github.com/apache/airflow/issues/28973#issuecomment-1387302162 I tried this both in `main` and #28592 but still see the same issue. Also as @ephraimbuddy I don't see the graph either, but I guess that's an other issue. -- This is an

[GitHub] [airflow] kaxil commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
kaxil commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1073723719 ## airflow/providers/dbt/cloud/operators/dbt.py: ## @@ -91,8 +96,17 @@ def __init__( timeout: int = 60 * 60 * 24 * 7, check_interval: int = 60,

[GitHub] [airflow] josh-fell commented on a diff in pull request #28964: Add transfer operator S3 to (generic) SQL

2023-01-18 Thread GitBox
josh-fell commented on code in PR #28964: URL: https://github.com/apache/airflow/pull/28964#discussion_r1072510110 ## docs/apache-airflow-providers-amazon/operators/transfer/s3_to_sql.rst: ## @@ -0,0 +1,75 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [airflow] MrGeorgeOwl commented on pull request #27776: Add deferrable mode to dataflow operators

2023-01-18 Thread GitBox
MrGeorgeOwl commented on PR #27776: URL: https://github.com/apache/airflow/pull/27776#issuecomment-1387282514 Sorry everyone who was added to this PR as approver. I am not sure why that happened after rebasing to main branch -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] aamster opened a new issue, #29018: Able to pass arguments to `--conf` not specified in `parameters` argument

2023-01-18 Thread GitBox
aamster opened a new issue, #29018: URL: https://github.com/apache/airflow/issues/29018 ### Apache Airflow version 2.5.0 ### What happened Given the following ``` import datetime from airflow.models.dag import dag @dag( dag_id="foo",

[GitHub] [airflow] bbovenzi commented on issue #28973: Dynamic Task Mapping skips tasks before upstream has started

2023-01-18 Thread GitBox
bbovenzi commented on issue #28973: URL: https://github.com/apache/airflow/issues/28973#issuecomment-1387275523 > Working for me in the current main but the graph is not displaying(cc @bbovenzi ). Did it not load at all? Or just incorrectly? I fixed one graph issue yesterday:

[GitHub] [airflow] VladaZakharova closed pull request #29017: Add deferrable mode to KubernetesPodOperator

2023-01-18 Thread GitBox
VladaZakharova closed pull request #29017: Add deferrable mode to KubernetesPodOperator URL: https://github.com/apache/airflow/pull/29017 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] phanikumv commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
phanikumv commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1073695339 ## airflow/providers/dbt/cloud/operators/dbt.py: ## @@ -91,8 +96,17 @@ def __init__( timeout: int = 60 * 60 * 24 * 7, check_interval: int = 60,

[GitHub] [airflow] phanikumv commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
phanikumv commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1073695339 ## airflow/providers/dbt/cloud/operators/dbt.py: ## @@ -91,8 +96,17 @@ def __init__( timeout: int = 60 * 60 * 24 * 7, check_interval: int = 60,

[GitHub] [airflow] vincbeck commented on a diff in pull request #28964: Add transfer operator S3 to (generic) SQL

2023-01-18 Thread GitBox
vincbeck commented on code in PR #28964: URL: https://github.com/apache/airflow/pull/28964#discussion_r1073691669 ## docs/apache-airflow-providers-amazon/operators/transfer/s3_to_sql.rst: ## @@ -0,0 +1,75 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [airflow] IAL32 commented on a diff in pull request #28894: Strip markup from `appbuilder.app_name` if `instance_name_has_markup = True`

2023-01-18 Thread GitBox
IAL32 commented on code in PR #28894: URL: https://github.com/apache/airflow/pull/28894#discussion_r1073627770 ## airflow/www/app.py: ## @@ -81,10 +82,18 @@ def create_app(config=None, testing=False): flask_app.config["PERMANENT_SESSION_LIFETIME"] =

[GitHub] [airflow] arjunanan6 commented on pull request #28394: Fix manual task trigger failing for k8s.

2023-01-18 Thread GitBox
arjunanan6 commented on PR #28394: URL: https://github.com/apache/airflow/pull/28394#issuecomment-1387173786 @snjypl Tried out your patch in 29012 as well, but still no luck. I'm still getting the same forbidden error. -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] IAL32 commented on a diff in pull request #28925: Add log for AWS Glue Job Console URL

2023-01-18 Thread GitBox
IAL32 commented on code in PR #28925: URL: https://github.com/apache/airflow/pull/28925#discussion_r1073614861 ## airflow/providers/amazon/aws/operators/glue.py: ## @@ -144,6 +149,20 @@ def execute(self, context: Context): self.wait_for_completion, )

[GitHub] [airflow] IAL32 commented on a diff in pull request #28925: Add log for AWS Glue Job Console URL

2023-01-18 Thread GitBox
IAL32 commented on code in PR #28925: URL: https://github.com/apache/airflow/pull/28925#discussion_r1073614442 ## airflow/providers/amazon/aws/operators/glue.py: ## @@ -144,6 +149,20 @@ def execute(self, context: Context): self.wait_for_completion, )

[GitHub] [airflow] VladaZakharova commented on pull request #28230: Add deferrable mode to Kubernetes and GKE operators

2023-01-18 Thread GitBox
VladaZakharova commented on PR #28230: URL: https://github.com/apache/airflow/pull/28230#issuecomment-1387151905 @kaxil @MrGeorgeOwl @lwyszomi Hi Team! Please, check separate PR for the KubernetesPodOperator: https://github.com/apache/airflow/pull/29017 Hope that it can really

[GitHub] [airflow] Taragolis commented on a diff in pull request #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
Taragolis commented on code in PR #29014: URL: https://github.com/apache/airflow/pull/29014#discussion_r1073586586 ## airflow/providers/dbt/cloud/hooks/dbt.py: ## @@ -125,6 +128,34 @@ class DbtCloudJobRunException(AirflowException): """An exception that indicates a job run

[GitHub] [airflow] VladaZakharova opened a new pull request, #29017: Add deferrable mode to KubernetesPodOperator

2023-01-18 Thread GitBox
VladaZakharova opened a new pull request, #29017: URL: https://github.com/apache/airflow/pull/29017 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for

[GitHub] [airflow] ecerulm commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-01-18 Thread GitBox
ecerulm commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1387133006 I had to introduce a `ConvertableToDict` abstract base class , in order to keep the `mypy` happy since `k8s.V1EnvVar` is not a dict. Initially I used `hasattr(item, "to_dict") but mypy

[GitHub] [airflow] ecerulm opened a new pull request, #29016: Fix leak sensitive field via V1EnvVar on exception

2023-01-18 Thread GitBox
ecerulm opened a new pull request, #29016: URL: https://github.com/apache/airflow/pull/29016 Currently the KubernetesPodOperator `env_vars` will be printed on the task logs if there is any templating error (like an `UndefinedError`, `TemplateSyntaxError` or `KeyError`) ```

[GitHub] [airflow] obilalk commented on issue #28825: Bad request when triggering dag run with `note` in payload

2023-01-18 Thread GitBox
obilalk commented on issue #28825: URL: https://github.com/apache/airflow/issues/28825#issuecomment-1387123398 In `v1.yaml` you can see setted get-post request for airflow `/dags/{dag_id}/dagRuns`:

[GitHub] [airflow] ephraimbuddy opened a new pull request, #29015: Remove erroneous TODO comment

2023-01-18 Thread GitBox
ephraimbuddy opened a new pull request, #29015: URL: https://github.com/apache/airflow/pull/29015 The issue that this comment was referring to was closed as won't fix. This PR removes the comment. Issue: https://issues.apache.org/jira/browse/AIRFLOW-1455 -- This is an

[GitHub] [airflow] phanikumv opened a new pull request, #29014: Add deferrable mode to `DbtCloudRunJobOperator`

2023-01-18 Thread GitBox
phanikumv opened a new pull request, #29014: URL: https://github.com/apache/airflow/pull/29014 This PR donates the `DbtCloudRunJobOperatorAsync` from [astronomer-providers](https://github.com/astronomer/astronomer-providers) repo --- **^ Add meaningful description above**

[GitHub] [airflow] BasPH closed issue #29009: TaskFlow DAGs don't register implicitly

2023-01-18 Thread GitBox
BasPH closed issue #29009: TaskFlow DAGs don't register implicitly URL: https://github.com/apache/airflow/issues/29009 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] BasPH commented on issue #29009: TaskFlow DAGs don't register implicitly

2023-01-18 Thread GitBox
BasPH commented on issue #29009: URL: https://github.com/apache/airflow/issues/29009#issuecomment-1386992010 Doh! I'll make a PR for the docs to clarify this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] yangguoaws commented on issue #29013: Metrics dagrun.duration.failed. not updated when the dag run failed due to timeout

2023-01-18 Thread GitBox
yangguoaws commented on issue #29013: URL: https://github.com/apache/airflow/issues/29013#issuecomment-1386984602 @o-nikolas Could you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] BasPH commented on issue #29009: TaskFlow DAGs don't register implicitly

2023-01-18 Thread GitBox
BasPH commented on issue #29009: URL: https://github.com/apache/airflow/issues/29009#issuecomment-1386976758 Could you provide a working example? I'm not following your code example. My point is that: ```python with DAG(...): ``` Indeed works, so without having to write

[GitHub] [airflow] snjypl commented on issue #28615: AIP-44 Migrate Dagbag.sync_to_db to internal API.

2023-01-18 Thread GitBox
snjypl commented on issue #28615: URL: https://github.com/apache/airflow/issues/28615#issuecomment-1386973021 @mhenc i would like to work on this issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29013: Metrics dagrun.duration.failed. not updated when the dag run failed due to timeout

2023-01-18 Thread GitBox
boring-cyborg[bot] commented on issue #29013: URL: https://github.com/apache/airflow/issues/29013#issuecomment-1386971612 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] yangguoaws opened a new issue, #29013: Metrics dagrun.duration.failed. not updated when the dag run failed due to timeout

2023-01-18 Thread GitBox
yangguoaws opened a new issue, #29013: URL: https://github.com/apache/airflow/issues/29013 ### Apache Airflow version 2.5.0 ### What happened When the dag was set with `dagrun_timeout`

[GitHub] [airflow] vchiapaikeo commented on a diff in pull request #28942: Fix @task.kubernetes to receive input and send output

2023-01-18 Thread GitBox
vchiapaikeo commented on code in PR #28942: URL: https://github.com/apache/airflow/pull/28942#discussion_r1073463608 ## airflow/providers/cncf/kubernetes/decorators/kubernetes.py: ## @@ -82,11 +82,30 @@ def _get_python_source(self): res = remove_task_decorator(res,

[GitHub] [airflow] vchiapaikeo commented on pull request #28942: Fix @task.kubernetes to receive input and send output

2023-01-18 Thread GitBox
vchiapaikeo commented on PR #28942: URL: https://github.com/apache/airflow/pull/28942#issuecomment-1386962681 Oops - sorry for pinging everyone. Forgot that I had done a rebase on the UI and didn't pull first. Feel free to unassign here! -- This is an automated message from the Apache

[GitHub] [airflow] Taragolis commented on a diff in pull request #29001: uniformize getting hook through cached property in aws sensors

2023-01-18 Thread GitBox
Taragolis commented on code in PR #29001: URL: https://github.com/apache/airflow/pull/29001#discussion_r1073449342 ## airflow/providers/amazon/aws/sensors/glue.py: ## @@ -61,10 +62,13 @@ def __init__( self.errored_states: list[str] = ["FAILED", "STOPPED", "TIMEOUT"]

[GitHub] [airflow] Taragolis commented on a diff in pull request #29001: uniformize getting hook through cached property in aws sensors

2023-01-18 Thread GitBox
Taragolis commented on code in PR #29001: URL: https://github.com/apache/airflow/pull/29001#discussion_r1073444102 ## airflow/providers/amazon/aws/sensors/batch.py: ## @@ -74,16 +73,13 @@ def poke(self, context: Context) -> bool: raise AirflowException(f"Batch sensor

[GitHub] [airflow] potiuk commented on a diff in pull request #28926: Upgrade mypy to 0.991

2023-01-18 Thread GitBox
potiuk commented on code in PR #28926: URL: https://github.com/apache/airflow/pull/28926#discussion_r1073431561 ## airflow/timetables/simple.py: ## @@ -55,6 +55,9 @@ def serialize(self) -> dict[str, Any]: def infer_manual_data_interval(self, *, run_after: DateTime) ->

[GitHub] [airflow] ephraimbuddy commented on pull request #27829: Improving the release process

2023-01-18 Thread GitBox
ephraimbuddy commented on PR #27829: URL: https://github.com/apache/airflow/pull/27829#issuecomment-1386921743 > Please fix those - and if it still does not help - I might take a closer look. On it. Thanks -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] vchiapaikeo commented on a diff in pull request #28942: Fix @task.kubernetes to receive input and send output

2023-01-18 Thread GitBox
vchiapaikeo commented on code in PR #28942: URL: https://github.com/apache/airflow/pull/28942#discussion_r1073416928 ## airflow/providers/cncf/kubernetes/decorators/kubernetes.py: ## @@ -82,15 +84,35 @@ def _get_python_source(self): res = remove_task_decorator(res,

[GitHub] [airflow] potiuk commented on pull request #27829: Improving the release process

2023-01-18 Thread GitBox
potiuk commented on PR #27829: URL: https://github.com/apache/airflow/pull/27829#issuecomment-1386903443 Please fix those - and if it still does not help - I might take a closer look. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on pull request #27829: Improving the release process

2023-01-18 Thread GitBox
potiuk commented on PR #27829: URL: https://github.com/apache/airflow/pull/27829#issuecomment-1386902562 > Hi @potiuk, It seems like there's something wrong with the breeze build on this PR. For example, here:

[GitHub] [airflow] snjypl commented on pull request #28394: Fix manual task trigger failing for k8s.

2023-01-18 Thread GitBox
snjypl commented on PR #28394: URL: https://github.com/apache/airflow/pull/28394#issuecomment-1386891079 > @snjypl That solved the previous issue, and the execution is attempted now. However, manual runs still fail because the service account gets rejected from creating a pod for the

[GitHub] [airflow] snjypl opened a new pull request, #29012: Fix airflow-webserver cannot create resource pods for k8s exceutor deployed using helm chart

2023-01-18 Thread GitBox
snjypl opened a new pull request, #29012: URL: https://github.com/apache/airflow/pull/29012 we see the fellowing error in the webserver log on trying to run the task manually from the UI for k8s executor. ``` Traceback (most recent call last): File

[GitHub] [airflow] potiuk merged pull request #29010: Revert "Remove conn.close() ignores (#29005)"

2023-01-18 Thread GitBox
potiuk merged PR #29010: URL: https://github.com/apache/airflow/pull/29010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28926: Upgrade mypy to 0.991

2023-01-18 Thread GitBox
ephraimbuddy commented on code in PR #28926: URL: https://github.com/apache/airflow/pull/28926#discussion_r1073388347 ## tests/www/views/test_views_connection.py: ## @@ -40,11 +41,10 @@ "password": "admin", } -CONNECTION_WITH_EXTRA = CONNECTION.update( Review Comment:

[GitHub] [airflow] ephraimbuddy opened a new pull request, #29011: Properly test that audit log secrets are masked for connection objects

2023-01-18 Thread GitBox
ephraimbuddy opened a new pull request, #29011: URL: https://github.com/apache/airflow/pull/29011 The previous test was wrong as the connection was updated in place which returned None -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] potiuk merged pull request #29006: Update README_RELEASE_PROVIDER_PACKAGES.md

2023-01-18 Thread GitBox
potiuk merged PR #29006: URL: https://github.com/apache/airflow/pull/29006 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #29010: Revert "Remove conn.close() ignores (#29005)"

2023-01-18 Thread GitBox
potiuk commented on PR #29010: URL: https://github.com/apache/airflow/pull/29010#issuecomment-1386851620 (BTW. This PR should fail as well in static checks so we should merge it regardless) -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] potiuk commented on pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
potiuk commented on PR #29005: URL: https://github.com/apache/airflow/pull/29005#issuecomment-1386849510 The status of constraints refresh can be tracked here @uranusjr: https://github.com/apache/airflow/commits/constraints-main and once it gets refresh you will be able to re-attempt to

[GitHub] [airflow] potiuk commented on pull request #29010: Revert "Remove conn.close() ignores (#29005)"

2023-01-18 Thread GitBox
potiuk commented on PR #29010: URL: https://github.com/apache/airflow/pull/29010#issuecomment-1386845609 The https://github.com/apache/airflow/commit/85f8df7b8a18e1147c7e014a7af7fc4e66aaa8be will break regular PRs so until we get the constraints refreshed we neeed to revert it. -- This

[GitHub] [airflow] potiuk commented on pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
potiuk commented on PR #29005: URL: https://github.com/apache/airflow/pull/29005#issuecomment-1386844240 https://github.com/apache/airflow/pull/29010 reverts it for now. Some more context: The thing with constraints is that they require a successful main build to get

[GitHub] [airflow] potiuk opened a new pull request, #29010: Revert "Remove conn.close() ignores (#29005)"

2023-01-18 Thread GitBox
potiuk opened a new pull request, #29010: URL: https://github.com/apache/airflow/pull/29010 This reverts commit 85f8df7b8a18e1147c7e014a7af7fc4e66aaa8be. --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] Taragolis commented on pull request #28545: Fix Provider Amazon Test Connection when using custom endpoint url

2023-01-18 Thread GitBox
Taragolis commented on PR #28545: URL: https://github.com/apache/airflow/pull/28545#issuecomment-1386839687 @noheton, AFAIK minio doesn't support STS Get Caller Identity, so you can't test connection. Please create separate discussion -- This is an automated message from the

[GitHub] [airflow] BasPH opened a new issue, #29009: TaskFlow DAGs don't register implicitly

2023-01-18 Thread GitBox
BasPH opened a new issue, #29009: URL: https://github.com/apache/airflow/issues/29009 ### Apache Airflow version 2.5.0 ### What happened TaskFlow `@dag`s don't register without explicitly calling the function that defines it. ### What you think should happen

[GitHub] [airflow] ephraimbuddy commented on issue #28973: Dynamic Task Mapping skips tasks before upstream has started

2023-01-18 Thread GitBox
ephraimbuddy commented on issue #28973: URL: https://github.com/apache/airflow/issues/28973#issuecomment-1386837966 Working for me in the current main but the graph is not displaying(cc @bbovenzi ). -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
potiuk commented on PR #29005: URL: https://github.com/apache/airflow/pull/29005#issuecomment-1386837656 This worked accidentally. I believe you broke all people's PRs this way. Constraints were not yet refreshed when you merged and you used old version of mysql-connector-python in this

[GitHub] [airflow] uranusjr merged pull request #28982: Resolve all variables in pickled XCom iterator

2023-01-18 Thread GitBox
uranusjr merged PR #28982: URL: https://github.com/apache/airflow/pull/28982 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] uranusjr commented on a diff in pull request #28926: Upgrade mypy to 0.991

2023-01-18 Thread GitBox
uranusjr commented on code in PR #28926: URL: https://github.com/apache/airflow/pull/28926#discussion_r1073341527 ## airflow/timetables/simple.py: ## @@ -55,6 +55,9 @@ def serialize(self) -> dict[str, Any]: def infer_manual_data_interval(self, *, run_after: DateTime) ->

[GitHub] [airflow] uranusjr merged pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
uranusjr merged PR #29005: URL: https://github.com/apache/airflow/pull/29005 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] noheton commented on pull request #28545: Fix Provider Amazon Test Connection when using custom endpoint url

2023-01-18 Thread GitBox
noheton commented on PR #28545: URL: https://github.com/apache/airflow/pull/28545#issuecomment-1386802509 Is this related to using S3-compatible object stores? Currently our custom X-Com interface using a local minio instance does no longer work. ```python import os from

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28978: Minor improvements to serde helpers

2023-01-18 Thread GitBox
ephraimbuddy commented on code in PR #28978: URL: https://github.com/apache/airflow/pull/28978#discussion_r1073327103 ## airflow/serialization/serde.py: ## @@ -73,18 +71,24 @@ def decode(d: dict[str, str | int | T]) -> tuple: def serialize(o: object, depth: int = 0) -> U |

[GitHub] [airflow] uranusjr opened a new pull request, #29008: Implement XComArg helper function convert()

2023-01-18 Thread GitBox
uranusjr opened a new pull request, #29008: URL: https://github.com/apache/airflow/pull/29008 This takes a callable that would be invoked when the XComArg is resolved. Once implemented an example DAG would look something like this: ```python import json value = ... # Get

[GitHub] [airflow] potiuk commented on pull request #28997: Sync v2-5-stable with v2-5-test to release 2.5.1

2023-01-18 Thread GitBox
potiuk commented on PR #28997: URL: https://github.com/apache/airflow/pull/28997#issuecomment-1386769059 Ah I see that it even works now :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on issue #27328: SFTPOperator throws object of type 'PlainXComArg' has no len() when using with Taskflow API

2023-01-18 Thread GitBox
potiuk commented on issue #27328: URL: https://github.com/apache/airflow/issues/27328#issuecomment-1386767606 > Personally I think the eventual goal would be to ban any logic in `__init__` for custom operators since all logic should be in `execute` anyway, and attribute assignment can be

[GitHub] [airflow] potiuk commented on pull request #28968: Required Param requires None to be explicitly passed as default

2023-01-18 Thread GitBox
potiuk commented on PR #28968: URL: https://github.com/apache/airflow/pull/28968#issuecomment-1386764399 @aamster maybe you can provide some examples of what you think should happen here as the original author of the issue? -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29007: Adding dataset to be discovered by static analyzers

2023-01-18 Thread GitBox
boring-cyborg[bot] commented on PR #29007: URL: https://github.com/apache/airflow/pull/29007#issuecomment-1386758920 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] bharat99k opened a new pull request, #29007: Adding dataset to be discovered by static analyzers

2023-01-18 Thread GitBox
bharat99k opened a new pull request, #29007: URL: https://github.com/apache/airflow/pull/29007 Adding dataset to be discovered by static analyzers (PyDev, PyCharm,) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] uranusjr commented on a diff in pull request #28926: Upgrade mypy to 0.991

2023-01-18 Thread GitBox
uranusjr commented on code in PR #28926: URL: https://github.com/apache/airflow/pull/28926#discussion_r1073304683 ## airflow/example_dags/plugins/workday.py: ## @@ -78,6 +78,9 @@ def next_dagrun_info( # [END howto_timetable_next_dagrun_info] +def validate(self): +

[GitHub] [airflow] potiuk commented on pull request #28724: Use dagrun instead of DagFileProcessor in backfill_job test

2023-01-18 Thread GitBox
potiuk commented on PR #28724: URL: https://github.com/apache/airflow/pull/28724#issuecomment-1386756964 Needs rebase/fixing static checks, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] potiuk commented on pull request #28930: Add json_loads and yaml_loads macro

2023-01-18 Thread GitBox
potiuk commented on PR #28930: URL: https://github.com/apache/airflow/pull/28930#issuecomment-1386754335 > Hmm, looking at other things in the module, I wonder if we should simply import `json` and `yaml` into it, so users can just write `{{ json.loads(...) }}` instead of needing to

[GitHub] [airflow] potiuk closed issue #28803: statsd metric for dataset count

2023-01-18 Thread GitBox
potiuk closed issue #28803: statsd metric for dataset count URL: https://github.com/apache/airflow/issues/28803 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] potiuk merged pull request #28907: add some statsd metrics for dataset

2023-01-18 Thread GitBox
potiuk merged PR #28907: URL: https://github.com/apache/airflow/pull/28907 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy commented on pull request #27829: Improving the release process

2023-01-18 Thread GitBox
ephraimbuddy commented on PR #27829: URL: https://github.com/apache/airflow/pull/27829#issuecomment-1386750234 Hi @potiuk, It seems like there's something wrong with the breeze build on this PR. For example, here:

[GitHub] [airflow] eladkal opened a new pull request, #29006: Update README_RELEASE_PROVIDER_PACKAGES.md

2023-01-18 Thread GitBox
eladkal opened a new pull request, #29006: URL: https://github.com/apache/airflow/pull/29006 adding instructions how to recover from deleted files in svn. It took some time to figure this out so documenting it for easier troubleshoot for next time. --- **^ Add meaningful

[GitHub] [airflow] potiuk commented on issue #28870: Support multiple buttons from `operator_extra_links`

2023-01-18 Thread GitBox
potiuk commented on issue #28870: URL: https://github.com/apache/airflow/issues/28870#issuecomment-1386748498 Updated the description to reflect this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] potiuk commented on issue #28870: Support multiple buttons from `operator_extra_links`

2023-01-18 Thread GitBox
potiuk commented on issue #28870: URL: https://github.com/apache/airflow/issues/28870#issuecomment-1386746732 nice discussion. I see the problem now, and yeah, I agree having support for dynamic collection of links in operator is a better idea -- This is an automated message from the

[GitHub] [airflow] potiuk commented on pull request #25466: Auto ML assets

2023-01-18 Thread GitBox
potiuk commented on PR #25466: URL: https://github.com/apache/airflow/pull/25466#issuecomment-1386729025 REbased - static checks fixed in main (mysql python connector release breaking mypy) -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #28998: write to DB a user who triggered a dag via API endpoint request

2023-01-18 Thread GitBox
ephraimbuddy commented on code in PR #28998: URL: https://github.com/apache/airflow/pull/28998#discussion_r1073259935 ## airflow/api_connexion/endpoints/dag_run_endpoint.py: ## @@ -319,6 +320,27 @@ def post_dag_run(*, dag_id: str, session: Session = NEW_SESSION) ->

[GitHub] [airflow] eladkal commented on a diff in pull request #28970: Fix support for macros with dots in DataProcJobBuilder

2023-01-18 Thread GitBox
eladkal commented on code in PR #28970: URL: https://github.com/apache/airflow/pull/28970#discussion_r1073259126 ## tests/providers/google/cloud/hooks/test_dataproc.py: ## @@ -1006,17 +1006,9 @@ def test_set_python_main(self): self.builder.set_python_main(main)

[GitHub] [airflow] eladkal merged pull request #29003: Update EC2 and EMR system tests to use IMDSv2

2023-01-18 Thread GitBox
eladkal merged PR #29003: URL: https://github.com/apache/airflow/pull/29003 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #28997: Sync v2-5-stable with v2-5-test to release 2.5.1

2023-01-18 Thread GitBox
potiuk commented on PR #28997: URL: https://github.com/apache/airflow/pull/28997#issuecomment-1386661120 The failing "check that image builds quickly" is fine and we can ignore it. The image will not build quickly now until we update constraints because the eventlet>=0.33.3 will conflict

[GitHub] [airflow] uranusjr commented on pull request #29005: Remove conn.close() ignores

2023-01-18 Thread GitBox
uranusjr commented on PR #29005: URL: https://github.com/apache/airflow/pull/29005#issuecomment-1386648436 Uh… this works? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] uranusjr commented on issue #28973: Dynamic Task Mapping skips tasks before upstream has started

2023-01-18 Thread GitBox
uranusjr commented on issue #28973: URL: https://github.com/apache/airflow/issues/28973#issuecomment-1386646579 Can you try if #28592 fixed this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] uranusjr commented on issue #27328: SFTPOperator throws object of type 'PlainXComArg' has no len() when using with Taskflow API

2023-01-17 Thread GitBox
uranusjr commented on issue #27328: URL: https://github.com/apache/airflow/issues/27328#issuecomment-1386626513 Personally I think the eventual goal would be to ban any logic in `__init__` for custom operators since all logic should be in `execute` anyway, and attribute assignment can be

[GitHub] [airflow] potiuk commented on pull request #28997: Sync v2-5-stable with v2-5-test to release 2.5.1

2023-01-17 Thread GitBox
potiuk commented on PR #28997: URL: https://github.com/apache/airflow/pull/28997#issuecomment-1386626425 As discussed in https://github.com/apache/airflow/pull/29004 - eventlet released a fix and we should cherry-pick (now merged) https://github.com/apache/airflow/pull/29004 on top of the

<    1   2   3   4   5   6   7   8   9   10   >