Re: [PR] fix: reduce error irrelevant error logs for pod events. [airflow]

2024-03-06 Thread via GitHub
potiuk commented on PR #37944: URL: https://github.com/apache/airflow/pull/37944#issuecomment-1981676549 Looks good. Two comments: * can you please rebase * can you plesed add a unit test for that one ? -- This is an automated message from the Apache Git Service. To respond to t

[PR] AWS auth manager CLI: persist the policy store description when doing updates [airflow]

2024-03-06 Thread via GitHub
vincbeck opened a new pull request, #37946: URL: https://github.com/apache/airflow/pull/37946 When calling the API `update_policy_store` from the service Amazon Verified Permissions, if the description is not provided, it erases it. This PR fixes that. ---

[PR] Add system test to test the AWS auth manager [airflow]

2024-03-06 Thread via GitHub
vincbeck opened a new pull request, #37947: URL: https://github.com/apache/airflow/pull/37947 Add system test to test the AWS auth manager. These tests are not run as part of the Airflow CI. They are using real AWS credentials to test the AWS auth manager.

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-06 Thread via GitHub
dabla commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1515100619 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, patch,

[PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-03-06 Thread via GitHub
howardyoo opened a new pull request, #37948: URL: https://github.com/apache/airflow/pull/37948 closes #37752 --- This is an PR for [AIP-49](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-49+OpenTelemetry+Support+for+Apache+Airflow) which is Open Telem

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-06 Thread via GitHub
dabla commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515103359 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" Revi

[PR] Fix GCSObjectExistenceSensor xcom push [airflow]

2024-03-06 Thread via GitHub
dmedora opened a new pull request, #37949: URL: https://github.com/apache/airflow/pull/37949 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pu

[PR] Improve output for `check-tests-unittest-testcase` check [airflow]

2024-03-06 Thread via GitHub
Taragolis opened a new pull request, #37950: URL: https://github.com/apache/airflow/pull/37950 Small improvement in output during `check-tests-unittest-testcase` check. Add as prefix `{filename}:{lineno}:` which also detects by IDE (well ... by PyCharm 😸 ) and make a links t

Re: [PR] AWS auth manager CLI: persist the policy store description when doing updates [airflow]

2024-03-06 Thread via GitHub
potiuk merged PR #37946: URL: https://github.com/apache/airflow/pull/37946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-03-06 Thread via GitHub
potiuk commented on code in PR #37948: URL: https://github.com/apache/airflow/pull/37948#discussion_r1515128333 ## airflow/config_templates/config.yml: ## @@ -1130,6 +1130,56 @@ metrics: type: string example: ~ default: "False" +traces: + description: | +

[PR] Add messages to task log about moving to queued [airflow]

2024-03-06 Thread via GitHub
collinmcnulty opened a new pull request, #37951: URL: https://github.com/apache/airflow/pull/37951 --- When troubleshooting a task that failed somewhere other than the main execute function, it is necessary to look across multiple logs and amidst many unre

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-03-06 Thread via GitHub
potiuk commented on code in PR #37948: URL: https://github.com/apache/airflow/pull/37948#discussion_r1515129579 ## airflow/config_templates/config.yml: ## @@ -1130,6 +1130,56 @@ metrics: type: string example: ~ default: "False" +traces: + description: | +

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
jedcunningham commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515139864 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes

[PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
potiuk opened a new pull request, #37952: URL: https://github.com/apache/airflow/pull/37952 …ator (#37928)" This reverts commit db07eb1957a729f333c14ce3c26b5a819736295b. --- **^ Add meaningful description above** Read the **[Pull Request Guidel

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
vincbeck commented on PR #37952: URL: https://github.com/apache/airflow/pull/37952#issuecomment-198113 Thanks! I was going to do it! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
potiuk merged PR #37952: URL: https://github.com/apache/airflow/pull/37952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
potiuk commented on PR #37952: URL: https://github.com/apache/airflow/pull/37952#issuecomment-1981780280 cc: @Taragolis -> the https://github.com/apache/airflow/pull/37928 shoud be re-done and `full tests needed` addded - as it adds a number of `cwd` added in serialization tests and other

Re: [PR] Support adding custom TI Deps to help DagRun make more flexible TI scheduling decisions [airflow]

2024-03-06 Thread via GitHub
XD-DENG commented on PR #37778: URL: https://github.com/apache/airflow/pull/37778#issuecomment-1981784836 > From the description though I'm not sure why this is need? Why can't the dep be set directly on the Operator object? Hi @ashb , The purpose of this change is to control

Re: [PR] Support adding custom TI Deps to help DagRun make more flexible TI scheduling decisions [airflow]

2024-03-06 Thread via GitHub
XD-DENG commented on PR #37778: URL: https://github.com/apache/airflow/pull/37778#issuecomment-1981787008 @ashb , the use case my team is having in mind is: we would want the DagRun to be scheduled/created as normal, but we would like to add custom logics to control if a TI under this DagRu

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
Taragolis commented on PR #37952: URL: https://github.com/apache/airflow/pull/37952#issuecomment-1981808176 Re-revert with fixes 😉 I guess it also might be a good idea add bash operator as full-test selective checks since changes in it might affect other tests -- This is an automated mes

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
ephraimbuddy commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515172594 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes

Re: [I] Scheduler fails to schedule DagRuns due to persistent DAG record lock [airflow]

2024-03-06 Thread via GitHub
gr8web commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1981817654 Hello people. Sorry, it looks like I was wrong. I just saw it again, connections getting stuck in `idle in transaction` state in the database and jobs not progressing.

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
ephraimbuddy commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515179397 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes

Re: [PR] [AIP-49] OpenTelemetry Traces for Apache Airflow [airflow]

2024-03-06 Thread via GitHub
howardyoo commented on code in PR #37948: URL: https://github.com/apache/airflow/pull/37948#discussion_r1515182332 ## airflow/config_templates/config.yml: ## @@ -1130,6 +1130,56 @@ metrics: type: string example: ~ default: "False" +traces: + description: |

Re: [I] Scheduler fails to schedule DagRuns due to persistent DAG record lock [airflow]

2024-03-06 Thread via GitHub
ephraimbuddy commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1981829921 > The only thing what happened to the jobs is that the tasks were cleared for few days in the past. Can you elaborate more on this? Like explain how it was cleared and y

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
jedcunningham commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515190147 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
potiuk commented on PR #37952: URL: https://github.com/apache/airflow/pull/37952#issuecomment-1981839057 > I guess it also might be a good idea add bash operator as full-test triggerer in selective checks since changes in BashOperator might affect other tests Yeah. looks like. We ca

[PR] Make sure that Core and Serialization tests are run for bash changes [airflow]

2024-03-06 Thread via GitHub
potiuk opened a new pull request, #37953: URL: https://github.com/apache/airflow/pull/37953 There are a number of tests using Bash Operator in Core and Serialization test types - so we should force those two test types if bash or it's tests are modified. -

Re: [PR] Revert "Make current working directory as templated field in BashOper… [airflow]

2024-03-06 Thread via GitHub
potiuk commented on PR #37952: URL: https://github.com/apache/airflow/pull/37952#issuecomment-1981872359 PR here: https://github.com/apache/airflow/pull/37953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [I] Triggering a DAG Run from the DAG Graph View does not redirect to the triggered DAG Run page [airflow]

2024-03-06 Thread via GitHub
jscheffl commented on issue #37935: URL: https://github.com/apache/airflow/issues/37935#issuecomment-1981873328 I do not fully understand what you mean with "the triggered DAG Run page" - do you assume the resulting page after triggering should jump to the details of the "new run just trigg

Re: [I] 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte (after upgrade to 2.8.2) [airflow]

2024-03-06 Thread via GitHub
jscheffl commented on issue #37871: URL: https://github.com/apache/airflow/issues/37871#issuecomment-1981876308 So is the problem resolved and the last comment just re-opened? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
potiuk commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515211719 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes +^

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-06 Thread via GitHub
potiuk commented on code in PR #37941: URL: https://github.com/apache/airflow/pull/37941#discussion_r1515212312 ## RELEASE_NOTES.rst: ## @@ -21,6 +21,51 @@ .. towncrier release notes start +Airflow 2.8.3 (2024-03-11) +-- + +Significant Changes +^

Re: [I] Triggering a DAG Run from the DAG Graph View does not redirect to the triggered DAG Run page [airflow]

2024-03-06 Thread via GitHub
pedro-cf commented on issue #37935: URL: https://github.com/apache/airflow/issues/37935#issuecomment-1981895916 > do you assume the resulting page after triggering should jump to the details of the "new run just triggered" Yes, this was working like this up to version 2.8.1 > A

Re: [PR] ECS Executor - add support to adopt orphaned tasks. [airflow]

2024-03-06 Thread via GitHub
ferruzzi merged PR #37786: URL: https://github.com/apache/airflow/pull/37786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

Re: [I] ECS Executor - Adopting Task Instances [airflow]

2024-03-06 Thread via GitHub
ferruzzi closed issue #35491: ECS Executor - Adopting Task Instances URL: https://github.com/apache/airflow/issues/35491 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

Re: [I] Scheduler fails to schedule DagRuns due to persistent DAG record lock [airflow]

2024-03-06 Thread via GitHub
gr8web commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1981908959 > > The only thing what happened to the jobs is that the tasks were cleared for few days in the past. > > Can you elaborate more on this? Like explain how it was cleared and y

Re: [PR] Create metrics to track Scheduled->Queued->Running task state transition times [airflow]

2024-03-06 Thread via GitHub
ferruzzi commented on PR #30612: URL: https://github.com/apache/airflow/pull/30612#issuecomment-1981922602 @htpawel This is quite old, would you please open [a new Issue](https://github.com/apache/airflow/issues/new/choose) with your bug report? -- This is an automated message from the

Re: [PR] Set job labels for traceability [airflow]

2024-03-06 Thread via GitHub
stiak commented on code in PR #37736: URL: https://github.com/apache/airflow/pull/37736#discussion_r1515232407 ## airflow/providers/google/cloud/operators/bigquery.py: ## @@ -2768,11 +2771,26 @@ def prepare_template(self) -> None: with open(self.configuration) as fi

[PR] Gitignore prometheus container volume [airflow]

2024-03-06 Thread via GitHub
o-nikolas opened a new pull request, #37954: URL: https://github.com/apache/airflow/pull/37954 When you run breeze with the statsd or otel integration grafana and prometheus containers are started to provide the metrics exporting and visualization (it all works and is very cool!). Pr

Re: [PR] Gitignore prometheus container volume [airflow]

2024-03-06 Thread via GitHub
o-nikolas commented on PR #37954: URL: https://github.com/apache/airflow/pull/37954#issuecomment-1981980060 CC @ferruzzi -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] fix: reduce irrelevant error logs for pod events. [airflow]

2024-03-06 Thread via GitHub
sudiptob2 commented on PR #37944: URL: https://github.com/apache/airflow/pull/37944#issuecomment-1982004948 @potiuk thanks for looking at the PR. > * can you plesed add a unit test for that one ? Added a unit test. -- This is an automated message from the Apache

Re: [PR] Add calendar view to react [airflow]

2024-03-06 Thread via GitHub
bbovenzi commented on PR #37909: URL: https://github.com/apache/airflow/pull/37909#issuecomment-1982019206 > Cool! > > As it might be confusing that the old calendar is top-down and the new one is bottom-up in ordering of the years... can you remove the old one prior 2.9.0 release cu

[PR] Add timeout to wait_for_crawl sensor task in example_glue [airflow]

2024-03-06 Thread via GitHub
o-nikolas opened a new pull request, #37955: URL: https://github.com/apache/airflow/pull/37955 Historical data shows that wait_for_crawl usually takes around 180s with spikes up to 360 seconds. We recently saw a test failure where this step ran for 3574 seconds (due to a upstream outage in

[PR] This pytest8 is Fiexed [airflow]

2024-03-06 Thread via GitHub
alirezahamami opened a new pull request, #37956: URL: https://github.com/apache/airflow/pull/37956 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.

Re: [PR] This pytest8 is Fiexed [airflow]

2024-03-06 Thread via GitHub
boring-cyborg[bot] commented on PR #37956: URL: https://github.com/apache/airflow/pull/37956#issuecomment-1982053206 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] Add timeout to wait_for_crawl sensor task in example_glue [airflow]

2024-03-06 Thread via GitHub
o-nikolas merged PR #37955: URL: https://github.com/apache/airflow/pull/37955 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] apache-airflow-providers-amazon/8.7.1 uses wrong log group for fargate container [airflow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on issue #37278: URL: https://github.com/apache/airflow/issues/37278#issuecomment-1982079582 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] KPO write_logs does not need to use complicated pod logs reader [airflow]

2024-03-06 Thread via GitHub
github-actions[bot] closed pull request #36228: KPO write_logs does not need to use complicated pod logs reader URL: https://github.com/apache/airflow/pull/36228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] Handling exit of kerberos sidecar container for k8s < 1.28 [airflow]

2024-03-06 Thread via GitHub
github-actions[bot] closed pull request #35748: Handling exit of kerberos sidecar container for k8s < 1.28 URL: https://github.com/apache/airflow/pull/35748 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [I] Marking task state as Failed or Success without taskinstance edit role is resulting into partial empty screen [airflow]

2024-03-06 Thread via GitHub
RodrigoGanancia commented on issue #37451: URL: https://github.com/apache/airflow/issues/37451#issuecomment-1982127419 Hey, can I work on this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Add calendar view to react [airflow]

2024-03-06 Thread via GitHub
phanikumv merged PR #37909: URL: https://github.com/apache/airflow/pull/37909 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] Calendar view infinite loop for `@continuous` DAGs [airflow]

2024-03-06 Thread via GitHub
phanikumv closed issue #37902: Calendar view infinite loop for `@continuous` DAGs URL: https://github.com/apache/airflow/issues/37902 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [I] task instance keeps running when call http url with httpx in PythonOperator [airflow]

2024-03-06 Thread via GitHub
mayuanxiaonong commented on issue #37939: URL: https://github.com/apache/airflow/issues/37939#issuecomment-1982248425 The problem is here . when call `requests.get(url)`, it will check if pass proxy , and the function ` _get_proxy_settings ` stuck . BTW , i call this function in

Re: [PR] fix: reduce irrelevant error logs for pod events. [airflow]

2024-03-06 Thread via GitHub
ketozhang commented on PR #37944: URL: https://github.com/apache/airflow/pull/37944#issuecomment-1982254165 Looks good to me, nice work! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [I] task instance keeps running when call http url with httpx in PythonOperator [airflow]

2024-03-06 Thread via GitHub
mayuanxiaonong commented on issue #37939: URL: https://github.com/apache/airflow/issues/37939#issuecomment-1982285886 skip proxy bypass check with the follow code: ```python ## try with httpx with httpx.Client(trust_env=False, timeout=30) as client: resp = client.get(url=url)

[I] Dag is frequently going into inactive status [airflow]

2024-03-06 Thread via GitHub
pratik4891 opened a new issue, #37959: URL: https://github.com/apache/airflow/issues/37959 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7 ### What happened? Currently we hav

Re: [I] Dag is frequently going into inactive status [airflow]

2024-03-06 Thread via GitHub
boring-cyborg[bot] commented on issue #37959: URL: https://github.com/apache/airflow/issues/37959#issuecomment-1982309186 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] context["task_instance"] does not pickup correct instance if multiple runs (with almost timestamp) [airflow]

2024-03-06 Thread via GitHub
kopalsinha09 opened a new issue, #37960: URL: https://github.com/apache/airflow/issues/37960 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.2.5 ### What happened? If there are

Re: [I] context["task_instance"] does not pickup correct instance if multiple runs (with almost timestamp) [airflow]

2024-03-06 Thread via GitHub
boring-cyborg[bot] commented on issue #37960: URL: https://github.com/apache/airflow/issues/37960#issuecomment-1982481762 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [I] task instance keeps running when call http url with httpx in PythonOperator [airflow]

2024-03-06 Thread via GitHub
Taragolis commented on issue #37939: URL: https://github.com/apache/airflow/issues/37939#issuecomment-1982621066 I guess you have a Segmentation Fault. Try to set environment variable `no_proxy` to `*` -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] task instance keeps running when call http url with httpx in PythonOperator [airflow]

2024-03-06 Thread via GitHub
Taragolis closed issue #37939: task instance keeps running when call http url with httpx in PythonOperator URL: https://github.com/apache/airflow/issues/37939 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [I] Airflow sending negative values [airflow]

2024-03-06 Thread via GitHub
github-actions[bot] commented on issue #29953: URL: https://github.com/apache/airflow/issues/29953#issuecomment-1982642489 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-06 Thread via GitHub
dabla commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1515646723 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, patch,

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-06 Thread via GitHub
dabla commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1515646723 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, patch,

Re: [PR] Move operator color back to a badge vs node background [airflow]

2024-03-06 Thread via GitHub
uranusjr commented on PR #37945: URL: https://github.com/apache/airflow/pull/37945#issuecomment-1982689520 I wonder if we could tweak the color for skipped. If it doesn’t stick out this much in a DAG, I’d suggest using that as the background color like in the thumbnailㄡ -- This is an aut

[PR] Avoid to use some deprecated stuff in tests [airflow]

2024-03-06 Thread via GitHub
Taragolis opened a new pull request, #37964: URL: https://github.com/apache/airflow/pull/37964 Removed some unnecessary usage of deprecated stuff - `TriggerRule.DUMMY` -> `TriggerRule.ALWAYS` - `airflow.operators.python.task` -> `airflow.decorators.task` - `airflo

Re: [PR] This pytest8 is Fiexed [airflow]

2024-03-06 Thread via GitHub
Taragolis closed pull request #37956: This pytest8 is Fiexed URL: https://github.com/apache/airflow/pull/37956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e

Re: [PR] This pytest8 is Fiexed [airflow]

2024-03-06 Thread via GitHub
Taragolis commented on PR #37956: URL: https://github.com/apache/airflow/pull/37956#issuecomment-1982804501 Wondering what a purpose of this pr? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[I] bash: /root/bin/pip: Permission denied [airflow]

2024-03-06 Thread via GitHub
YepXbruce opened a new issue, #37965: URL: https://github.com/apache/airflow/issues/37965 ### Apache Airflow version 2.8.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? when i run docker . `docker -D -l debug run

Re: [I] bash: /root/bin/pip: Permission denied [airflow]

2024-03-06 Thread via GitHub
boring-cyborg[bot] commented on issue #37965: URL: https://github.com/apache/airflow/issues/37965#issuecomment-1982821754 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-06 Thread via GitHub
eladkal commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515703373 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" LO

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
eladkal commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515704404 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" Re

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
eladkal commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515704404 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" Re

Re: [I] Scheduler fails to schedule DagRuns due to persistent DAG record lock [airflow]

2024-03-07 Thread via GitHub
ephraimbuddy commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1982837483 > > > The only thing what happened to the jobs is that the tasks were cleared for few days in the past. > > > > > > Can you elaborate more on this? Like explain how

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
eladkal commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1982841158 @dabla Let me clarify... Your PR is requesting to merge changes on 6 files. One of them is: `tests/providers/microsoft/azure/transfers/test_local_to_adls.py` https://git

[I] In the DAG Graph view, all DAG runs disappear after selecting a Run State [airflow]

2024-03-07 Thread via GitHub
rgelsi opened a new issue, #37966: URL: https://github.com/apache/airflow/issues/37966 ### Apache Airflow version 2.8.2 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? In the DAG Graph view, all DAG runs disappear after

Re: [I] In the DAG Graph view, all DAG runs disappear after selecting a Run State [airflow]

2024-03-07 Thread via GitHub
boring-cyborg[bot] commented on issue #37966: URL: https://github.com/apache/airflow/issues/37966#issuecomment-1982852187 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Make sure that Core and Serialization tests are run for bash changes [airflow]

2024-03-07 Thread via GitHub
Taragolis merged PR #37953: URL: https://github.com/apache/airflow/pull/37953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[PR] Make current working directory as templated field in BashOperato [airflow]

2024-03-07 Thread via GitHub
Taragolis opened a new pull request, #37968: URL: https://github.com/apache/airflow/pull/37968 Attempt 2 (#37928) after reverting previous one by #37952 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/

Re: [PR] Make current working directory as templated field in BashOperato [airflow]

2024-03-07 Thread via GitHub
Taragolis commented on PR #37968: URL: https://github.com/apache/airflow/pull/37968#issuecomment-1982882069 First check with default settings, after that will check with full tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-07 Thread via GitHub
Taragolis commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1515736302 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, pat

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
dabla commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515752141 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" Revi

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
dabla commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515763386 ## tests/providers/microsoft/azure/transfers/test_local_to_adls.py: ## @@ -28,11 +27,9 @@ ) TASK_ID = "test-adls-upload-operator" -FILE_SYSTEM_NAME = "Fabric" LOCA

Re: [PR] Add DataToADLSOperator in Microsoft Azure provider [airflow]

2024-03-07 Thread via GitHub
dabla commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1982974952 > @dabla Let me clarify... > > Your PR is requesting to merge changes on 6 files. One of them is: `tests/providers/microsoft/azure/transfers/test_local_to_adls.py` > > https:

Re: [I] Annotate a Dataset Event in the Source Task [airflow]

2024-03-07 Thread via GitHub
uranusjr commented on issue #37810: URL: https://github.com/apache/airflow/issues/37810#issuecomment-1982982809 Since XCom is just a data storage, it can be used like an external S3 file, or a database the user sets up. It is just a bit more automated and contains some metadata. I feel it i

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-07 Thread via GitHub
Taragolis commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1515771397 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, pat

Re: [PR] Add `ADLSCreateObjectOperator` [airflow]

2024-03-07 Thread via GitHub
eladkal commented on code in PR #37821: URL: https://github.com/apache/airflow/pull/37821#discussion_r1515790176 ## tests/system/providers/microsoft/azure/example_adls_list.py: ## @@ -0,0 +1,54 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribut

Re: [PR] Add `ADLSCreateObjectOperator` [airflow]

2024-03-07 Thread via GitHub
dabla commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1983031334 > @dabla Let me clarify... > > Your PR is requesting to merge changes on 6 files. One of them is: `tests/providers/microsoft/azure/transfers/test_local_to_adls.py` > > https:

[PR] Deferrable mode for RunPipelineJobOperator [airflow]

2024-03-07 Thread via GitHub
e-galan opened a new pull request, #37969: URL: https://github.com/apache/airflow/pull/37969 ### List of changes: - Add the deferrable mode to `RunPipelineJobOperator`; - Move persisting the pipeline job link to the beginning of the operator's execution; - Add new un

[I] The systemd package cannot be installed because this system does not have a merged /usr. [airflow]

2024-03-07 Thread via GitHub
Pooort opened a new issue, #37970: URL: https://github.com/apache/airflow/issues/37970 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.2 ### What happened? I'm using `apache/

Re: [PR] Add the dataset_expression as part of DAGDetailSchema [airflow]

2024-03-07 Thread via GitHub
sunank200 commented on code in PR #37826: URL: https://github.com/apache/airflow/pull/37826#discussion_r1515847888 ## airflow/models/dag.py: ## @@ -3902,6 +3901,16 @@ def get_dataset_triggered_next_run_info(self, *, session=NEW_SESSION) -> dict[st return None

Re: [PR] Add the dataset_expression as part of DAGDetailSchema [airflow]

2024-03-07 Thread via GitHub
sunank200 commented on code in PR #37826: URL: https://github.com/apache/airflow/pull/37826#discussion_r1515849070 ## airflow/models/dag.py: ## @@ -3902,6 +3901,16 @@ def get_dataset_triggered_next_run_info(self, *, session=NEW_SESSION) -> dict[st return None

Re: [PR] Add `ADLSCreateObjectOperator` [airflow]

2024-03-07 Thread via GitHub
eladkal commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1983125487 > Ok @eladkal it's because the branch was based on the other PR with proxy fix where I had to the new operator, so it will be hard to remove that file. What do you want me to do, create

Re: [PR] Add the dataset_expression as part of DAGDetailSchema [airflow]

2024-03-07 Thread via GitHub
sunank200 commented on code in PR #37826: URL: https://github.com/apache/airflow/pull/37826#discussion_r1515910123 ## tests/api_connexion/endpoints/test_dag_endpoint.py: ## @@ -310,6 +310,7 @@ def test_should_respond_200(self, url_safe_serializer): "concurrency": 16

Re: [PR] Add the dataset_expression as part of DAGDetailSchema [airflow]

2024-03-07 Thread via GitHub
sunank200 commented on code in PR #37826: URL: https://github.com/apache/airflow/pull/37826#discussion_r1515910436 ## airflow/models/dag.py: ## @@ -3902,6 +3901,16 @@ def get_dataset_triggered_next_run_info(self, *, session=NEW_SESSION) -> dict[st return None

Re: [PR] Sync v2-8-stable with v2-8-test to release 2.8.3 [airflow]

2024-03-07 Thread via GitHub
SatishChGit commented on PR #37941: URL: https://github.com/apache/airflow/pull/37941#issuecomment-1983219220 Is teradata provider released - https://github.com/apache/airflow/pull/36953 not part of this core release? Changes of teradata provider are not included in this. -- This is an a

Re: [PR] Add the dataset_expression as part of DagModel and DAGDetailSchema [airflow]

2024-03-07 Thread via GitHub
uranusjr commented on code in PR #37826: URL: https://github.com/apache/airflow/pull/37826#discussion_r1515936094 ## airflow/models/dag.py: ## @@ -2009,6 +2011,16 @@ def _get_task_instances( return tis +@property +def get_dataset_expression(self) -> Any | No

[I] Scheduler uses [scheduler]job_heartbeat_sec property for heartbeats instead of [scheduler]scheduler_heartbeat_sec [airflow]

2024-03-07 Thread via GitHub
kosteev opened a new issue, #37971: URL: https://github.com/apache/airflow/issues/37971 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.6.3 ### What happened? Scheduler uses [s

Re: [PR] Add `ADLSCreateObjectOperator` [airflow]

2024-03-07 Thread via GitHub
dabla commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1983320874 > > Ok @eladkal it's because the branch was based on the other PR with proxy fix where I had to the new operator, so it will be hard to remove that file. What do you want me to do, create

<    1   2   3   4   5   6   7   8   9   10   >