Re: [PR] Show if dag page filters are active [airflow]

2024-03-12 Thread via GitHub
bbovenzi commented on code in PR #38080: URL: https://github.com/apache/airflow/pull/38080#discussion_r1522248551 ## airflow/www/static/js/dag/nav/FilterBar.tsx: ## @@ -152,6 +208,7 @@ const FilterBar = () => { background="white" variant="outline"

Re: [PR] Show if dag page filters are active [airflow]

2024-03-12 Thread via GitHub
bbovenzi commented on code in PR #38080: URL: https://github.com/apache/airflow/pull/38080#discussion_r1522248857 ## airflow/www/static/js/dag/nav/FilterBar.tsx: ## @@ -49,9 +63,23 @@ const FilterBar = () => { transformArrayToMultiSelectOptions, } = useFilters(); + //

Re: [PR] Allow sort by on all fields in MappedInstances.tsx [airflow]

2024-03-12 Thread via GitHub
bbovenzi commented on PR #38090: URL: https://github.com/apache/airflow/pull/38090#issuecomment-1992740165 The sort for those fields is disabled because the REST API only supports map index and state right now. We would need to update it there too. See [here](https://github.com/apache/airfl

[PR] Limit importlib_resources as it breaks pytest_rewrites [airflow]

2024-03-12 Thread via GitHub
potiuk opened a new pull request, #38095: URL: https://github.com/apache/airflow/pull/38095 New version of importlib_resources released today - 6.2.0 but also 6.3.0 released few hours later, break pytest_rewrite feature in rather unexpected ways. Until the problem is diagnosed and solved we

[PR] Fix task instances list link [airflow]

2024-03-12 Thread via GitHub
bbovenzi opened a new pull request, #38096: URL: https://github.com/apache/airflow/pull/38096 Fixes: https://github.com/apache/airflow/issues/38030 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/con

Re: [PR] Limit importlib_resources as it breaks pytest_rewrites [airflow]

2024-03-12 Thread via GitHub
potiuk commented on PR #38095: URL: https://github.com/apache/airflow/pull/38095#issuecomment-1992753163 also fixes a typo in skipping images jobs (but that's secondary) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Fix for unhandled exceptions in the Triggerer tests [airflow]

2024-03-12 Thread via GitHub
uranusjr commented on code in PR #37927: URL: https://github.com/apache/airflow/pull/37927#discussion_r1522286709 ## tests/jobs/test_triggerer_job.py: ## @@ -51,6 +52,24 @@ pytestmark = pytest.mark.db_test +class QueueListener_(logging.handlers.QueueListener): +def __in

[PR] Add try number to mapped instance [airflow]

2024-03-12 Thread via GitHub
bbovenzi opened a new pull request, #38097: URL: https://github.com/apache/airflow/pull/38097 Add `tryNumber` to mapped instances table. Also, remove unused `taskFails` from task duration page. --- **^ Add meaningful description above** Read the **[Pull Request Gu

Re: [PR] Fix for unhandled exceptions in the Triggerer tests [airflow]

2024-03-12 Thread via GitHub
uranusjr commented on code in PR #37927: URL: https://github.com/apache/airflow/pull/37927#discussion_r1522291522 ## tests/models/test_trigger.py: ## @@ -142,10 +142,12 @@ def test_submit_failure(session, create_task_instance): assert updated_task_instance.next_method == "_

Re: [PR] Refactor the Design of AzureSynapsePipeline class [airflow]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #36622: URL: https://github.com/apache/airflow/pull/36622#issuecomment-1992783989 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] [AIP-44] Migrate TaskInstance._run_raw_task to Internal API [airflow]

2024-03-12 Thread via GitHub
github-actions[bot] closed pull request #36010: [AIP-44] Migrate TaskInstance._run_raw_task to Internal API URL: https://github.com/apache/airflow/pull/36010 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] feat: SQLExecuteQueryOperator sql_callable [airflow]

2024-03-12 Thread via GitHub
github-actions[bot] commented on PR #35844: URL: https://github.com/apache/airflow/pull/35844#issuecomment-1992784104 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

Re: [PR] Fix for unhandled exceptions in the Triggerer tests [airflow]

2024-03-12 Thread via GitHub
uranusjr commented on code in PR #37927: URL: https://github.com/apache/airflow/pull/37927#discussion_r1522292646 ## tests/jobs/test_triggerer_job.py: ## @@ -398,8 +427,11 @@ def handle_events(self): session.commit() job = Job() +job.prepare_for_execution()

[I] Dagrun "skipped" for timezone aware DAGs with schedules affected by daylight saving time [airflow]

2024-03-12 Thread via GitHub
wolfier opened a new issue, #38098: URL: https://github.com/apache/airflow/issues/38098 ### What do you see as an issue? The issue is that users are unaware of scheduling behaviour between transition of when [daylight saving time](https://en.wikipedia.org/wiki/Daylight_saving_time) i

Re: [PR] Fix for unhandled exceptions in the Triggerer tests [airflow]

2024-03-12 Thread via GitHub
uranusjr commented on code in PR #37927: URL: https://github.com/apache/airflow/pull/37927#discussion_r1522293481 ## tests/jobs/test_triggerer_job.py: ## @@ -361,7 +390,7 @@ class TriggererJob_(TriggererJobRunner): def wait_for_runner_loop(self, runner_loop_count):

Re: [PR] Introduce DAG SLA [airflow]

2024-03-12 Thread via GitHub
michael-robbins-seek commented on PR #36639: URL: https://github.com/apache/airflow/pull/36639#issuecomment-1992960089 Will this DAG SLA be compatible at all with Data-aware scheduled dags? -- This is an automated message from the Apache Git Service. To respond to the message, please log o

[I] Data-aware scheduling SLA support [airflow]

2024-03-12 Thread via GitHub
michael-robbins-seek opened a new issue, #38099: URL: https://github.com/apache/airflow/issues/38099 ### Description Ability to define an SLA for a data aware scheduled DAG. This could take the form of a timedelta since 'the last successful run' or some other more complex expression.

Re: [I] Data-aware scheduling SLA support [airflow]

2024-03-12 Thread via GitHub
boring-cyborg[bot] commented on issue #38099: URL: https://github.com/apache/airflow/issues/38099#issuecomment-1992964377 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-12 Thread via GitHub
CongyueZhang opened a new pull request, #38100: URL: https://github.com/apache/airflow/pull/38100 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.r

Re: [PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-12 Thread via GitHub
boring-cyborg[bot] commented on PR #38100: URL: https://github.com/apache/airflow/pull/38100#issuecomment-1992973362 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] Limit importlib_resources as it breaks pytest_rewrites [airflow]

2024-03-12 Thread via GitHub
edgarrmondragon commented on code in PR #38095: URL: https://github.com/apache/airflow/pull/38095#discussion_r1522385293 ## dev/breeze/pyproject.toml: ## @@ -51,6 +51,9 @@ dependencies = [ "flit>=3.5.0", "gitpython>=3.1.40", "hatch==1.9.1", +# Importib_resourc

Re: [I] 404 Pod not found can be seen often [airflow]

2024-03-12 Thread via GitHub
pankajastro commented on issue #37090: URL: https://github.com/apache/airflow/issues/37090#issuecomment-1993631240 The logs trace you've shared appear to be quite old, and this execution path no longer exists. I believe it would be beneficial if you could try again with the latest version a

Re: [PR] fix: Update SqlToSlackApiFileOperator with new param to check empty output [airflow]

2024-03-12 Thread via GitHub
eladkal commented on code in PR #38079: URL: https://github.com/apache/airflow/pull/38079#discussion_r1522580067 ## airflow/providers/slack/transfers/sql_to_slack.py: ## @@ -134,6 +141,8 @@ def execute(self, context: Context) -> None: output_file_name = fp.name

Re: [PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-12 Thread via GitHub
dirrao commented on code in PR #38100: URL: https://github.com/apache/airflow/pull/38100#discussion_r1522588198 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -257,3 +257,24 @@ In Airflow, sensors wait for specific conditions to be met before proceeding wi

Re: [PR] Prevent large objects from being stored in the RTIF [airflow]

2024-03-12 Thread via GitHub
dirrao commented on code in PR #38094: URL: https://github.com/apache/airflow/pull/38094#discussion_r1522605282 ## airflow/config_templates/config.yml: ## @@ -481,6 +481,14 @@ core: type: string example: ~ default: "Disabled" +max_templated_field_size: +

Re: [PR] Prevent large objects from being stored in the RTIF [airflow]

2024-03-12 Thread via GitHub
dirrao commented on code in PR #38094: URL: https://github.com/apache/airflow/pull/38094#discussion_r1522615945 ## airflow/serialization/helpers.py: ## @@ -37,7 +39,26 @@ def is_jsonable(x): else: return True +max_size = conf.getint("core", "max_templ

Re: [PR] Add executors property to base job [airflow]

2024-03-13 Thread via GitHub
dirrao commented on code in PR #38093: URL: https://github.com/apache/airflow/pull/38093#discussion_r1522681022 ## airflow/jobs/job.py: ## @@ -123,6 +123,10 @@ def __init__(self, executor=None, heartrate=None, **kwargs): def executor(self): return ExecutorLoader.g

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-13 Thread via GitHub
lyuyunbo commented on PR #38008: URL: https://github.com/apache/airflow/pull/38008#issuecomment-1993756566 > I find it as a useful optional feature, but it absolutely cannot be enabled by default. It should be quite a bit more generic also in order to be useful - you should be able to not o

Re: [PR] Improve logging readability with DockerOperator [airflow]

2024-03-13 Thread via GitHub
lyuyunbo commented on PR #38008: URL: https://github.com/apache/airflow/pull/38008#issuecomment-1993759478 > Can you add the test cases for this change? Change has been made in the new PR. Further review will be very welcomed and appreciated. @dirrao -- This is an automated messag

Re: [PR] heartbeat recovery message [airflow]

2024-03-13 Thread via GitHub
Bowrna commented on PR #34457: URL: https://github.com/apache/airflow/pull/34457#issuecomment-1993761467 @potiuk i will add the changes that you have suggested -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

Re: [PR] Limit importlib_resources as it breaks pytest_rewrites [airflow]

2024-03-13 Thread via GitHub
potiuk commented on code in PR #38095: URL: https://github.com/apache/airflow/pull/38095#discussion_r1522709638 ## dev/breeze/pyproject.toml: ## @@ -51,6 +51,9 @@ dependencies = [ "flit>=3.5.0", "gitpython>=3.1.40", "hatch==1.9.1", +# Importib_resources 6.2.0

Re: [PR] Limit importlib_resources as it breaks pytest_rewrites [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38095: URL: https://github.com/apache/airflow/pull/38095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Validate tasks total priority weight in DagBag [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on code in PR #37990: URL: https://github.com/apache/airflow/pull/37990#discussion_r1522726741 ## airflow/utils/dag_parameters_overflow.py: ## @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[PR] Upgrade uv to latest 0.18.0 version [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38101: URL: https://github.com/apache/airflow/pull/38101 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-13 Thread via GitHub
potiuk commented on code in PR #38100: URL: https://github.com/apache/airflow/pull/38100#discussion_r1522762382 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -257,3 +257,24 @@ In Airflow, sensors wait for specific conditions to be met before proceeding wi

Re: [PR] Optimize skipping building images for not "in-workflow" builds [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38091: URL: https://github.com/apache/airflow/pull/38091#issuecomment-1993832960 That's how it looks after the change: ![image](https://github.com/apache/airflow/assets/595491/ed6123f6-dcc6-4d35-9117-b13d6daf1c38) -- This is an automated message from the

[PR] Fix GCS Anonymous user issue because none token [airflow]

2024-03-13 Thread via GitHub
pankajastro opened a new pull request, #38102: URL: https://github.com/apache/airflow/pull/38102 closes: https://github.com/apache/airflow/issues/37834 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apac

[PR] Skip pydantic model round-trip when AIP-44 is disabled [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38103: URL: https://github.com/apache/airflow/pull/38103 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Added kube-prometheus-stack servicemonitor to gather metrics from Airflow statsd exporter [airflow]

2024-03-13 Thread via GitHub
ArieLevs commented on PR #31040: URL: https://github.com/apache/airflow/pull/31040#issuecomment-1993897949 Hi @sudeepgupta90 I also have use case where id like to have the serviceMonitor integrated into the airflow chart (currently i create my own serviceMonitor, airflow chart is a depe

Re: [PR] Add try number to mapped instance [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38097: URL: https://github.com/apache/airflow/pull/38097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[I] No docker-compose and Breeze in Github Codespaces [airflow]

2024-03-13 Thread via GitHub
Lukas1v opened a new issue, #38104: URL: https://github.com/apache/airflow/issues/38104 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Following this documentation on setti

Re: [I] No docker-compose and Breeze in Github Codespaces [airflow]

2024-03-13 Thread via GitHub
boring-cyborg[bot] commented on issue #38104: URL: https://github.com/apache/airflow/issues/38104#issuecomment-1993988724 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add Yandex Query support from Yandex.Cloud [airflow]

2024-03-13 Thread via GitHub
uzhastik commented on PR #37458: URL: https://github.com/apache/airflow/pull/37458#issuecomment-1993994426 @Taragolis @potiuk @eladkal @rozza @ashb @jedcunningham ping again =) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[I] Extra "None" logs from Kubernetes pod while running a KubernetesPodOperator [airflow]

2024-03-13 Thread via GitHub
anteverse opened a new issue, #38105: URL: https://github.com/apache/airflow/issues/38105 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Hello! While running a Kubernete

Re: [PR] Skip pydantic model round-trip when AIP-44 is disabled [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38103: URL: https://github.com/apache/airflow/pull/38103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Add Yandex Query support from Yandex.Cloud [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on code in PR #37458: URL: https://github.com/apache/airflow/pull/37458#discussion_r1522956524 ## airflow/providers/yandex/operators/yandexcloud_yq.py: ## @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor li

Re: [PR] Integrate the run coverage command to the CI [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on PR #37152: URL: https://github.com/apache/airflow/pull/37152#issuecomment-1994093535 @ephraimbuddy I think it worthwhile to check it again after we move impersonations as quarantined -- This is an automated message from the Apache Git Service. To respond to the mess

Re: [PR] fix: Update SqlToSlackApiFileOperator with new param to check empty output [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on code in PR #38079: URL: https://github.com/apache/airflow/pull/38079#discussion_r1523002335 ## airflow/providers/slack/transfers/sql_to_slack.py: ## @@ -134,6 +141,8 @@ def execute(self, context: Context) -> None: output_file_name = fp.name

Re: [PR] fix: Update SqlToSlackApiFileOperator with new param to check empty output [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on code in PR #38079: URL: https://github.com/apache/airflow/pull/38079#discussion_r1523006444 ## airflow/providers/slack/transfers/sql_to_slack.py: ## @@ -134,6 +141,8 @@ def execute(self, context: Context) -> None: output_file_name = fp.name

[PR] Fix spark operator log retrieval from driver [airflow]

2024-03-13 Thread via GitHub
sudohainguyen opened a new pull request, #38106: URL: https://github.com/apache/airflow/pull/38106 Related logs: https://github.com/apache/airflow/assets/22145541/25cb1e79-7d96-4cbf-ae40-b9cef512ead5";> closes: #37681 Overriding default BASE_CONTAINER_NAME as

Re: [PR] Fix spark operator log retrieval from driver [airflow]

2024-03-13 Thread via GitHub
sudohainguyen commented on PR #38106: URL: https://github.com/apache/airflow/pull/38106#issuecomment-1994185179 @eladkal mind taking a look? think this minor change can fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

Re: [I] Getting 410 from spark_kubernetes_operator when a task runs for long [airflow]

2024-03-13 Thread via GitHub
sudohainguyen commented on issue #32744: URL: https://github.com/apache/airflow/issues/32744#issuecomment-1994212173 still have issue in version 7.6.0, which is included the fix #32768 do you know which version resolve this properly? cc: @eladkal @hussein-awala -- This is an automat

Re: [PR] fix: Update SqlToSlackApiFileOperator with new param to check empty output [airflow]

2024-03-13 Thread via GitHub
eladkal commented on code in PR #38079: URL: https://github.com/apache/airflow/pull/38079#discussion_r1523099668 ## airflow/providers/slack/transfers/sql_to_slack.py: ## @@ -134,6 +141,8 @@ def execute(self, context: Context) -> None: output_file_name = fp.name

[PR] Separate cache build to a separate workflow called from the main ci.yml [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38107: URL: https://github.com/apache/airflow/pull/38107 This is another step of decomposing our workflows and introducing more DRY approach for GitHub Actions workflows we have. Building cache started to take more time and space as we started to build 5 i

Re: [I] Getting 410 from spark_kubernetes_operator when a task runs for long [airflow]

2024-03-13 Thread via GitHub
cw-pradyumna commented on issue #32744: URL: https://github.com/apache/airflow/issues/32744#issuecomment-1994239207 Yes, I was facing this issue even after this fix was merged -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] No docker-compose and Breeze in Github Codespaces [airflow]

2024-03-13 Thread via GitHub
potiuk commented on issue #38104: URL: https://github.com/apache/airflow/issues/38104#issuecomment-1994269792 Yes. Because in the codespaces you are already in the breeze environment. You do not need to do anything there to run breeze -- This is an automated message from the Apache Git Se

Re: [I] No docker-compose and Breeze in Github Codespaces [airflow]

2024-03-13 Thread via GitHub
potiuk closed issue #38104: No docker-compose and Breeze in Github Codespaces URL: https://github.com/apache/airflow/issues/38104 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] No docker-compose and Breeze in Github Codespaces [airflow]

2024-03-13 Thread via GitHub
potiuk commented on issue #38104: URL: https://github.com/apache/airflow/issues/38104#issuecomment-1994273615 From https://github.com/apache/airflow/blob/main/contributing-docs/quick-start-ide/contributors_quick_start_codespaces.rst > 3. Once the codespace starts your terminal should

Re: [PR] Upgrade uv to latest 0.18.0 version [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38101: URL: https://github.com/apache/airflow/pull/38101#issuecomment-1994283032 Yep. And They ARM one is not really needed and is removed in the upcoming https://github.com/apache/airflow/pull/38107 -- This is an automated message from the Apache Git Service. To r

Re: [PR] Upgrade uv to latest 0.18.0 version [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38101: URL: https://github.com/apache/airflow/pull/38101 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[I] Scheduler in daemon mode doesn't create PID at the specified location [airflow]

2024-03-13 Thread via GitHub
jitenadav opened a new issue, #38108: URL: https://github.com/apache/airflow/issues/38108 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.8.0 ### What happened? Currently, when

Re: [I] Scheduler in daemon mode doesn't create PID at the specified location [airflow]

2024-03-13 Thread via GitHub
boring-cyborg[bot] commented on issue #38108: URL: https://github.com/apache/airflow/issues/38108#issuecomment-1994274624 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #37937: URL: https://github.com/apache/airflow/pull/37937#issuecomment-1994294008 (as long as tests are fixed that is :) ) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-13 Thread via GitHub
potiuk commented on code in PR #37937: URL: https://github.com/apache/airflow/pull/37937#discussion_r1523156231 ## tests/models/test_baseoperatormeta.py: ## @@ -0,0 +1,64 @@ +import os +from typing import Any +from unittest import TestCase +from unittest.mock import Mock, patch,

Re: [PR] Integrate the run coverage command to the CI [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #37152: URL: https://github.com/apache/airflow/pull/37152#issuecomment-1994298599 > @ephraimbuddy I think it worthwhile to check it again after we move impersonations as quarantined Agree :) -- This is an automated message from the Apache Git Service. To resp

Re: [PR] Add new run type "triggered" to distinguish manual DAG runs and triggered DAG runs [airflow]

2024-03-13 Thread via GitHub
molcay commented on PR #37087: URL: https://github.com/apache/airflow/pull/37087#issuecomment-1994302703 @uranusjr, > Instead of NULL, I would just add a type scheduler instead I think this can be done also. > Should this value be shown anywhere on the UI? We can use differen

Re: [PR] Create GKEStartKueueJobOperator operator [airflow]

2024-03-13 Thread via GitHub
MaksYermak commented on code in PR #37477: URL: https://github.com/apache/airflow/pull/37477#discussion_r1523183823 ## airflow/providers/google/cloud/operators/kubernetes_engine.py: ## @@ -898,3 +898,36 @@ def execute(self, context: Context): ).fetch_cluster_info()

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994335262 (also having it running in CI is ultimate proof that it works - and there is no need for the reviewer to run it locally - they will simply be able to see it in the CI :) -- This is an

[PR] Get rid of lowest-direct resolution in no-constraint upgrades with uv [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38109: URL: https://github.com/apache/airflow/pull/38109 When we fall-back to no-constraint mode in uv, we used so far lowest-direct resolution - but it could cause a really old set of versions to be installed with some strange pyproject.toml errors (for e

Re: [PR] Get rid of lowest-direct resolution in no-constraint upgrades with uv [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38109: URL: https://github.com/apache/airflow/pull/38109#issuecomment-1994381007 Example failure: https://github.com/apache/airflow/actions/runs/8264674751/job/22609047393 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-13 Thread via GitHub
workingDice commented on code in PR #38100: URL: https://github.com/apache/airflow/pull/38100#discussion_r1523243544 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -257,3 +257,24 @@ In Airflow, sensors wait for specific conditions to be met before proceedi

[PR] Bumping minimum version of zenpy to 0.40 [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38110: URL: https://github.com/apache/airflow/pull/38110 The older versions had broken pyproject.toml and caused uv to crash when trying --lowest-direct resolution (which we are going to start using soon to test minimum versions of our dependencies)

Re: [PR] Add `ADLSCreateObjectOperator` [airflow]

2024-03-13 Thread via GitHub
dabla commented on PR #37821: URL: https://github.com/apache/airflow/pull/37821#issuecomment-1994419664 @eladkal I finally managed to make the build pass, I kept the example for the list operator as I found what was causing the issue ( a missing single quote :)) -- This is an automated m

Re: [PR] doc: add more instructions for `up_for_retry` [airflow]

2024-03-13 Thread via GitHub
potiuk commented on code in PR #38100: URL: https://github.com/apache/airflow/pull/38100#discussion_r1523272699 ## docs/apache-airflow/authoring-and-scheduling/deferring.rst: ## @@ -257,3 +257,24 @@ In Airflow, sensors wait for specific conditions to be met before proceeding wi

Re: [PR] Bumping minimum version of zenpy to 0.40 [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38110: URL: https://github.com/apache/airflow/pull/38110 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[PR] Adding MSGraphOperator in Microsoft Azure provider [airflow]

2024-03-13 Thread via GitHub
dabla opened a new pull request, #38111: URL: https://github.com/apache/airflow/pull/38111 As already [discussed ](https://github.com/apache/airflow/discussions/36315)before I did a proposition to add the MS Graph Operator. After discussion on the [airflow dev list](ht

Re: [PR] Get rid of lowest-direct resolution in no-constraint upgrades with uv [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38109: URL: https://github.com/apache/airflow/pull/38109#issuecomment-1994427793 That failure should also be fixed by https://github.com/apache/airflow/pull/38110, but anyhow it's likely better to keep default resolution strategy for those upgrades. `--lowest-direct`

Re: [PR] Create GKEStartKueueJobOperator operator [airflow]

2024-03-13 Thread via GitHub
MaksYermak commented on code in PR #37477: URL: https://github.com/apache/airflow/pull/37477#discussion_r1523287350 ## airflow/providers/google/cloud/operators/kubernetes_engine.py: ## @@ -898,3 +898,36 @@ def execute(self, context: Context): ).fetch_cluster_info()

Re: [PR] Fix task instances list link [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38096: URL: https://github.com/apache/airflow/pull/38096#issuecomment-1994443936 Breeze `importlib-resources` test failure fixed in main. Merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] Incorrect link to task detail from task instance list [airflow]

2024-03-13 Thread via GitHub
potiuk closed issue #38030: Incorrect link to task detail from task instance list URL: https://github.com/apache/airflow/issues/38030 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

Re: [PR] Fix task instances list link [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38096: URL: https://github.com/apache/airflow/pull/38096 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-13 Thread via GitHub
dabla commented on PR #37937: URL: https://github.com/apache/airflow/pull/37937#issuecomment-1994447371 > (as long as tests are fixed that is :) ) I'm working on that I hope I will have them fix today. But now my question remains, do we really throw an Exception when bad mixing

Re: [PR] Fix spark operator log retrieval from driver [airflow]

2024-03-13 Thread via GitHub
sudohainguyen commented on PR #38106: URL: https://github.com/apache/airflow/pull/38106#issuecomment-1994490042 > LGTM. Can you add the test case for this? sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-13 Thread via GitHub
Taragolis commented on PR #37937: URL: https://github.com/apache/airflow/pull/37937#issuecomment-1994562340 > (as long as tests are fixed that is :) ) That maybe good example when use `dag_maker` and move as db-tests is better than use try to mock everything in sqlalchemy and TaskInst

[PR] Bump Cryptography to > 39.0.0 [airflow]

2024-03-13 Thread via GitHub
potiuk opened a new pull request, #38112: URL: https://github.com/apache/airflow/pull/38112 We had a very old cryptography version specified as min requirement. Bumping cryptography to 39.0.0 (released Jan 2023) should help with better resolutions and should keep Airflow secure.

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
amoghrajesh commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994594722 > we should remove the dev/* script in the same PR Yeah I wanted to, but wasn't sure whether to keep this in case it doesn't work. -- This is an automated message from the

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
amoghrajesh commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994595981 > Also I guess there will be a follow up PR where we - possibly - do some refactor and extract the common parts of generate_issue_content_* methods ? It looks like there are quite

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
amoghrajesh commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994592809 > We already do that in case of providers: Understood. Will make the changes for this -- This is an automated message from the Apache Git Service. To respond to the message, ple

Re: [PR] Get rid of lowest-direct resolution in no-constraint upgrades with uv [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38109: URL: https://github.com/apache/airflow/pull/38109 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] refactor: Refactored __new__ magic method of BaseOperatorMeta to avoid bad mixing classic and decorated operators [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #37937: URL: https://github.com/apache/airflow/pull/37937#issuecomment-1994608453 > > (as long as tests are fixed that is :) ) > > I'm working on that I hope I will have them fix today. > > But now my question remains, do we really throw an Exception when

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
amoghrajesh commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994614919 @potiuk @eladkal I added it to CI as well, wasn't so sure what version to use, so I used some random versions. WDYT? -- This is an automated message from the Apache Git Service. T

Re: [PR] Bump Cryptography to > 39.0.0 [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38112: URL: https://github.com/apache/airflow/pull/38112#issuecomment-1994623316 > LGTM. One qn, its a major bump, are we sure it wont break anything? It's just minium version. Everyone who installs airfllow from cratch gets latest version - also in the constra

Re: [PR] Bump Cryptography to > 39.0.0 [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38112: URL: https://github.com/apache/airflow/pull/38112#issuecomment-1994625626 > I mean Ci will catch it, so we are good In CI nothing will change as we have 41.07 already :) -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Bump Cryptography to > 39.0.0 [airflow]

2024-03-13 Thread via GitHub
potiuk merged PR #38112: URL: https://github.com/apache/airflow/pull/38112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Optimize skipping building images for not "in-workflow" builds [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38091: URL: https://github.com/apache/airflow/pull/38091#issuecomment-1994629775 Another from the series of optimization and streamlining our CI workflows -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
potiuk commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994639723 > @potiuk @eladkal I added it to CI as well, wasn't so sure what version to use, so I used some random versions. WDYT? It will get longer and longer over time. I deally, we should

Re: [PR] Migrating issue generation to breeze release-management [airflow]

2024-03-13 Thread via GitHub
amoghrajesh commented on PR #38062: URL: https://github.com/apache/airflow/pull/38062#issuecomment-1994644567 Ok understood, I can take it up in a follow up once this is stabilised. Your call, let me know -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [I] Task Killed because of Recorded pid does not match the current pid [airflow]

2024-03-13 Thread via GitHub
boring-cyborg[bot] commented on issue #38113: URL: https://github.com/apache/airflow/issues/38113#issuecomment-1994674619 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[I] Task Killed because of Recorded pid does not match the current pid [airflow]

2024-03-13 Thread via GitHub
keysersoza opened a new issue, #38113: URL: https://github.com/apache/airflow/issues/38113 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.5.3 ### What happened? The task is be

Re: [PR] Add a task instance dependency for mapped dependencies (#37091) [airflow]

2024-03-13 Thread via GitHub
stevenschaerer commented on code in PR #37498: URL: https://github.com/apache/airflow/pull/37498#discussion_r1523497837 ## airflow/models/baseoperator.py: ## @@ -826,6 +827,8 @@ def __init__( self.task_id = task_group.child_id(task_id) if task_group else task_id

<    5   6   7   8   9   10   11   12   13   14   >