(airflow) branch main updated: Ensure internal api command get the "ready" prefix on start (#38550)

2024-04-02 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new b255012cf0 Ensure internal api command get the

Re: [PR] Ensure internal api command get the "ready" prefix on start [airflow]

2024-04-02 Thread via GitHub
dstandish merged PR #38550: URL: https://github.com/apache/airflow/pull/38550 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Provide option to cancel job runs in DatabricksRunNowOperator [airflow]

2024-04-02 Thread via GitHub
boring-cyborg[bot] commented on issue #38695: URL: https://github.com/apache/airflow/issues/38695#issuecomment-2033533887 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Provide option to cancel job runs in DatabricksRunNowOperator [airflow]

2024-04-02 Thread via GitHub
subham611 opened a new issue, #38695: URL: https://github.com/apache/airflow/issues/38695 ### Description We are using apache airflow databricks operator to manage spark streaming jobs. When we submit a new streaming job after code update, DatabricksRunNowOperator simply queue new

Re: [PR] Amazon Bedrock -Model Customization Jobs [airflow]

2024-04-02 Thread via GitHub
ferruzzi commented on PR #38693: URL: https://github.com/apache/airflow/pull/38693#issuecomment-2033530120 I also didn't add a BedrockDeleteCustomModelOperator because it is literally just a hook call: `BedrockHook().conn.delete_custom_model(modelIdentifier=custom_model_arn)`. It's

Re: [PR] Fix message raised on excepion in SFTP deferrable operator [airflow]

2024-04-02 Thread via GitHub
shahar1 commented on PR #38525: URL: https://github.com/apache/airflow/pull/38525#issuecomment-2033513027 > Ah I see we also have https://github.com/apache/airflow/pull/38518 with the same changes @shahar1 - do you want to proceed with yours? I prefer so :) @Bowrna no

Re: [PR] Fix message raised on excepion in SFTP deferrable operator [airflow]

2024-04-02 Thread via GitHub
Bowrna commented on PR #38525: URL: https://github.com/apache/airflow/pull/38525#issuecomment-2033511597 > Ah I see we also have #38518 with the same changes @shahar1 - do you want to proceed with yours? I didn't know @shahar1 is working on this changes, my bad! :) --

Re: [PR] Amazon Bedrock -Model Customization Jobs [airflow]

2024-04-02 Thread via GitHub
ferruzzi commented on PR #38693: URL: https://github.com/apache/airflow/pull/38693#issuecomment-2033487899 Just realized I didn't finish making the sensor deferrable, I'll add the execute_complete() to that tomorrow. -- This is an automated message from the Apache Git Service. To respond

Re: [I] Parsing error of email_conn_id on utils/email.py [airflow]

2024-04-02 Thread via GitHub
hegdeadithyak commented on issue #23262: URL: https://github.com/apache/airflow/issues/23262#issuecomment-2033448735 @potiuk Please let me know if this is still unassigned and if you'd be happy for me to take it on. I'm interested in working on this issue. Before I dive in, I wanted

Re: [I] Use dynamic task mapping in TriggerDagRunOperator may generate the same run_id [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on issue #28868: URL: https://github.com/apache/airflow/issues/28868#issuecomment-2033441350 One possible approach would be to use Dataset to trigger the DAG instead. This would allow the scheduler to decide what the logical date (and run ID) is, and deduplicate the date

Re: [PR] Implement Metadata to emit runtime extra [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on code in PR #38650: URL: https://github.com/apache/airflow/pull/38650#discussion_r1548815506 ## airflow/operators/python.py: ## @@ -232,7 +232,11 @@ def execute(self, context: Context) -> Any: context_merge(context, self.op_kwargs,

Re: [I] AWS Batch job stuck in runnable state with no retry [airflow]

2024-04-02 Thread via GitHub
aiqc commented on issue #38298: URL: https://github.com/apache/airflow/issues/38298#issuecomment-2033387481 Circling back here. The root problem was service quota limits on AWS Fargate. On Tue, Mar 19, 2024 at 4:22 PM, Andrey Anshin ***@***.***(mailto:On Tue, Mar 19, 2024 at 4:22 PM,

Re: [PR] Fix dataset serialization compatibility [airflow]

2024-04-02 Thread via GitHub
uranusjr closed pull request #38148: Fix dataset serialization compatibility URL: https://github.com/apache/airflow/pull/38148 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Fix dataset serialization compatibility [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on PR #38148: URL: https://github.com/apache/airflow/pull/38148#issuecomment-2033375470 Continuing in #38694. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Amazon Bedrock -Model Customization Jobs [airflow]

2024-04-02 Thread via GitHub
ferruzzi opened a new pull request, #38693: URL: https://github.com/apache/airflow/pull/38693 Adds support for Model Customization Jobs. Includes changes to the hook(s), new Operator, Sensor, Trigger, Waiter, unit tests for the above, updates to the doc page, and additions to the system

Re: [PR] Support adding custom TI Deps to help DagRun make more flexible TI scheduling decisions [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on PR #37778: URL: https://github.com/apache/airflow/pull/37778#issuecomment-2033363322 The scenarios do make sense to me (although No. 2 is a bit weak; we should probably instead improve Dataset and Triggerer to cover the use case instead). But in all scenarios we’d

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-02 Thread via GitHub
tanvn commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2033357051 I has been using Airflow with statsd-exporter for quite a long time and to be honest, while I found the document about [the timers

Re: [PR] Data aware scheduling docs edits [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on code in PR #38687: URL: https://github.com/apache/airflow/pull/38687#discussion_r1548779735 ## docs/apache-airflow/authoring-and-scheduling/datasets.rst: ## @@ -51,38 +51,38 @@ In addition to scheduling DAGs based upon time, they can also be scheduled

Re: [PR] Use async db calls in WorkflowTrigger [airflow]

2024-04-02 Thread via GitHub
hussein-awala commented on PR #38689: URL: https://github.com/apache/airflow/pull/38689#issuecomment-206828 Also some async features are only available in sqlalchemy 2.0, so maybe we need to focus on this migration first. -- This is an automated message from the Apache Git Service.

Re: [PR] Data aware scheduling docs edits [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on code in PR #38687: URL: https://github.com/apache/airflow/pull/38687#discussion_r1548778668 ## docs/apache-airflow/authoring-and-scheduling/datasets.rst: ## @@ -51,38 +51,38 @@ In addition to scheduling DAGs based upon time, they can also be scheduled

Re: [PR] Use async db calls in WorkflowTrigger [airflow]

2024-04-02 Thread via GitHub
hussein-awala commented on PR #38689: URL: https://github.com/apache/airflow/pull/38689#issuecomment-205138 > I don’t think we can just remove TaskStateTrigger, but let’s deprecate it for future removal. > > > > It would also be a good idea to implement an version of

Re: [PR] Bump min packaging version for both Airflow and providers. [airflow]

2024-04-02 Thread via GitHub
github-actions[bot] closed pull request #37335: Bump min packaging version for both Airflow and providers. URL: https://github.com/apache/airflow/pull/37335 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

Re: [PR] fix: airflow dag parser logic [airflow]

2024-04-02 Thread via GitHub
github-actions[bot] closed pull request #37343: fix: airflow dag parser logic URL: https://github.com/apache/airflow/pull/37343 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [I] Getting dags from dagbag is failing [airflow]

2024-04-02 Thread via GitHub
github-actions[bot] commented on issue #38235: URL: https://github.com/apache/airflow/issues/38235#issuecomment-2033310714 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] Triggering a inactive dag manually makes the dag active which leads to making the dag, and the subsequent run is also triggered at the next scheduled time [airflow]

2024-04-02 Thread via GitHub
github-actions[bot] commented on issue #38242: URL: https://github.com/apache/airflow/issues/38242#issuecomment-2033310692 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [I] AWS Batch job stuck in runnable state with no retry [airflow]

2024-04-02 Thread via GitHub
github-actions[bot] commented on issue #38298: URL: https://github.com/apache/airflow/issues/38298#issuecomment-2033310670 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] Use async db calls in WorkflowTrigger [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on PR #38689: URL: https://github.com/apache/airflow/pull/38689#issuecomment-2033293112 I don’t think we can just remove TaskStateTrigger, but let’s deprecate it for future removal. It would also be a good idea to implement an version of `_get_count` to use the

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
uranusjr commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548745509 ## airflow/jobs/job.py: ## @@ -104,12 +105,13 @@ class Job(Base, LoggingMixin): Only makes sense for SchedulerJob and BackfillJob instances. """ -

Re: [PR] Add type annotations to `conftest.py` [airflow]

2024-04-02 Thread via GitHub
uranusjr merged PR #38691: URL: https://github.com/apache/airflow/pull/38691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Add type annotations to `conftest.py` (#38691)

2024-04-02 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 26b5bc9801 Add type annotations to `conftest.py`

[PR] Fix minor typo in Breeze cheatsheet [airflow]

2024-04-02 Thread via GitHub
syedahsn opened a new pull request, #38692: URL: https://github.com/apache/airflow/pull/38692 Fixing a minor typo in breeze visual --- **^ Add meaningful description above** Read the **[Pull Request

(airflow) branch v2-9-test updated (b8b6ecbc9f -> c8c1863289)

2024-04-02 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit b8b6ecbc9f Remove google-cloud-aiplatform when tests without pydantic are run (#38617) omit

(airflow) branch constraints-2-9 updated: Updating constraints. Github run id:8530005081

2024-04-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-9 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-9 by this push: new ef3ac934f0 Updating

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548704690 ## airflow/jobs/scheduler_job_runner.py: ## @@ -819,19 +821,20 @@ def _execute(self) -> int | None: ) try: -

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
o-nikolas commented on PR #38514: URL: https://github.com/apache/airflow/pull/38514#issuecomment-2033219581 > on executor end... i looked at the way executors have implemented end... i was surprised to see that they wait for all tasks to finish. given that we have re-adoption logic, i

Re: [PR] Add type annotations to `conftest.py` [airflow]

2024-04-02 Thread via GitHub
hussein-awala commented on code in PR #38691: URL: https://github.com/apache/airflow/pull/38691#discussion_r1548700505 ## tests/system/conftest.py: ## @@ -40,18 +39,18 @@ def provider_env_vars(): @pytest.fixture(autouse=True) -def

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548690491 ## airflow/jobs/job.py: ## @@ -104,12 +105,13 @@ class Job(Base, LoggingMixin): Only makes sense for SchedulerJob and BackfillJob instances. """ -

Re: [PR] Update `pytest_collection_modifyitems` to use correct properties [airflow]

2024-04-02 Thread via GitHub
syedahsn commented on code in PR #38665: URL: https://github.com/apache/airflow/pull/38665#discussion_r1548695597 ## tests/system/conftest.py: ## @@ -49,9 +49,9 @@ def skip_if_env_var_not_set(provider_env_vars): def pytest_collection_modifyitems(config, items): """Add

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548693252 ## airflow/jobs/scheduler_job_runner.py: ## @@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType | None) -> None:

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548690491 ## airflow/jobs/job.py: ## @@ -104,12 +105,13 @@ class Job(Base, LoggingMixin): Only makes sense for SchedulerJob and BackfillJob instances. """ -

[PR] Add type annotations to `conftest.py` [airflow]

2024-04-02 Thread via GitHub
syedahsn opened a new pull request, #38691: URL: https://github.com/apache/airflow/pull/38691 Remove redundant Path casting, and add type annotations to `conftest.py` --- **^ Add meaningful description above** Read the **[Pull Request

[PR] bump uv to 0.1.28 [airflow]

2024-04-02 Thread via GitHub
hussein-awala opened a new pull request, #38690: URL: https://github.com/apache/airflow/pull/38690 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Improve XComObjectStorageBackend implementation (#38608)

2024-04-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 62f948cd30 Improve XComObjectStorageBackend

Re: [PR] Improve XComObjectStoreBackend implementation [airflow]

2024-04-02 Thread via GitHub
jedcunningham merged PR #38608: URL: https://github.com/apache/airflow/pull/38608 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch v2-9-test updated (15e739197a -> b8b6ecbc9f)

2024-04-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git from 15e739197a Update RELEASE_NOTES.rst add b8b6ecbc9f Remove google-cloud-aiplatform when tests without

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
dstandish commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548639831 ## airflow/jobs/scheduler_job_runner.py: ## @@ -819,19 +821,20 @@ def _execute(self) -> int | None: ) try: -

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
dstandish commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548638986 ## airflow/jobs/scheduler_job_runner.py: ## @@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType | None) -> None:

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
dstandish commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548638498 ## airflow/jobs/scheduler_job_runner.py: ## @@ -270,8 +270,10 @@ def _debug_dump(self, signum: int, frame: FrameType | None) -> None:

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-02 Thread via GitHub
dstandish commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1548637266 ## airflow/jobs/job.py: ## @@ -104,12 +105,13 @@ class Job(Base, LoggingMixin): Only makes sense for SchedulerJob and BackfillJob instances. """ -

Re: [PR] inherit key "verify" from env into session so that kv_client can read it properly [airflow]

2024-04-02 Thread via GitHub
chz8494 commented on PR #38614: URL: https://github.com/apache/airflow/pull/38614#issuecomment-2033101077 @hussein-awala not sure if I'm doing it correctly, just added ssl case for v1 test. Also noticed a corner case problem with condition check, so I also change code to be: ```pythton

Re: [PR] Refactor DAG tabs navigation [airflow]

2024-04-02 Thread via GitHub
jscheffl commented on PR #38592: URL: https://github.com/apache/airflow/pull/38592#issuecomment-2033089071 @bbovenzi Meaning closing in (1) favor _not_ blocking 2.9 release and make the rework in 2.10 or (2) another PR that you open (soon)? -- This is an automated message from the Apache

[PR] Use async db calls in WorkflowTrigger [airflow]

2024-04-02 Thread via GitHub
stevenschaerer opened a new pull request, #38689: URL: https://github.com/apache/airflow/pull/38689 This closes: #38672 I opted to slightly extend some existing unit tests rather than write mostly identical new ones. If the `@sync_to_async` decorator and the `await`s before

(airflow) branch v2-9-test updated (5a7cf9bb73 -> 15e739197a)

2024-04-02 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git discard 5a7cf9bb73 Undeprecating `DBApiHookForTests._make_common_data_structure` (#38573) discard 506c7e6415

Re: [PR] Fix kubeconform pre-commit to not leave around containers [airflow]

2024-04-02 Thread via GitHub
jedcunningham merged PR #38688: URL: https://github.com/apache/airflow/pull/38688 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated (3dcd17a922 -> d212b119e1)

2024-04-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 3dcd17a922 Add entry to INTHEWILD.md (#38686) add d212b119e1 Fix kubeconform pre-commit to not leave

Re: [I] apache-airflow-providers-snowflake: 5.3.1 [airflow]

2024-04-02 Thread via GitHub
mtsadler-branch commented on issue #38681: URL: https://github.com/apache/airflow/issues/38681#issuecomment-2033065796 Sorry, I mis-spoke when opening the issue. It looks like `apache-airflow-providers-snowflake==5.3.1` requires `pyarrow==10.0.1` However,

[PR] Fix kubeconform pre-commit to not leave around containers [airflow]

2024-04-02 Thread via GitHub
jedcunningham opened a new pull request, #38688: URL: https://github.com/apache/airflow/pull/38688 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow) branch main updated: Add entry to INTHEWILD.md (#38686)

2024-04-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 3dcd17a922 Add entry to INTHEWILD.md (#38686)

Re: [PR] Added BNY Mellon to the list of companies using Apache Airflow [airflow]

2024-04-02 Thread via GitHub
boring-cyborg[bot] commented on PR #38686: URL: https://github.com/apache/airflow/pull/38686#issuecomment-2032990280 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

Re: [PR] Added BNY Mellon to the list of companies using Apache Airflow [airflow]

2024-04-02 Thread via GitHub
potiuk merged PR #38686: URL: https://github.com/apache/airflow/pull/38686 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Raise deprecated warning in `airflow dags backfill` only if `-I`/`--ignore-first-depends-on-past` provided (#38676)

2024-04-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e58cd4b519 Raise deprecated warning in `airflow

Re: [PR] Raise deprecated warning in `airflow dags backfill` only if `-I`/`--ignore-first-depends-on-past` provided [airflow]

2024-04-02 Thread via GitHub
potiuk merged PR #38676: URL: https://github.com/apache/airflow/pull/38676 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] pt012 rule enabled amazon [airflow]

2024-04-02 Thread via GitHub
potiuk commented on code in PR #38603: URL: https://github.com/apache/airflow/pull/38603#discussion_r1548483297 ## tests/providers/amazon/aws/hooks/test_base_aws.py: ## @@ -1123,13 +1123,13 @@ def test_raise_no_creds_default_credentials_strategy(tmp_path_factory, monkeypat

Re: [PR] Resolve `PT012` in `SFTP` provider tests [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38518: URL: https://github.com/apache/airflow/pull/38518#issuecomment-2032877859 Or we can merge #38525 instead - @eladkal ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix message raised on excepion in SFTP deferrable operator [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38525: URL: https://github.com/apache/airflow/pull/38525#issuecomment-2032870739 Ah I see we also have https://github.com/apache/airflow/pull/38518 with the same changes @shahar1 - do you want to proceed with yours? -- This is an automated message from the

Re: [PR] pt012 rule enabled for sftp [airflow]

2024-04-02 Thread via GitHub
potiuk commented on code in PR #38525: URL: https://github.com/apache/airflow/pull/38525#discussion_r1548448354 ## tests/providers/sftp/triggers/test_sftp.py: ## @@ -176,12 +176,11 @@ async def test_sftp_trigger_run_trigger_failure_state(self, mock_get_files_by_pa

Re: [PR] pt012 rule enabled for sftp [airflow]

2024-04-02 Thread via GitHub
potiuk commented on code in PR #38525: URL: https://github.com/apache/airflow/pull/38525#discussion_r1548447743 ## airflow/providers/sftp/triggers/sftp.py: ## @@ -134,7 +134,7 @@ async def run(self) -> AsyncIterator[TriggerEvent]: # Break loop to avoid infinite

Re: [PR] pt012 rule enabled for sftp [airflow]

2024-04-02 Thread via GitHub
potiuk commented on code in PR #38525: URL: https://github.com/apache/airflow/pull/38525#discussion_r1548435066 ## airflow/providers/sftp/triggers/sftp.py: ## @@ -134,7 +134,7 @@ async def run(self) -> AsyncIterator[TriggerEvent]: # Break loop to avoid infinite

(airflow) branch main updated: Extract checkout target commit to a composite action (#38682)

2024-04-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 76c92c3e0a Extract checkout target commit to a

Re: [PR] Extract checkout target commit to a composite action [airflow]

2024-04-02 Thread via GitHub
potiuk merged PR #38682: URL: https://github.com/apache/airflow/pull/38682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2032822547 > But this failure also show that docker compose tests can't show useful information in case if API failed because it just fail before show logs from the container. I will add

(airflow) branch v2-9-test updated (506c7e6415 -> 5a7cf9bb73)

2024-04-02 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git from 506c7e6415 Update RELEASE_NOTES.rst add 5a7cf9bb73 Undeprecating

(airflow) branch constraints-main updated: Updating constraints. Github run id:8526749915

2024-04-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 4a46ad47c0 Updating

Re: [PR] Respect `@pytest.mark.filterwarnings` in prohibited warnings [airflow]

2024-04-02 Thread via GitHub
Taragolis commented on PR #38684: URL: https://github.com/apache/airflow/pull/38684#issuecomment-2032787340 An unpleasant side effect: now it also affects warnings in fixtures, e.g. DagBag which collect example DAGs skip all DAGs which contain deprecated stuff, e.g. SubDagOperator. Need to

Re: [PR] Deprecation of AutoML services: Add deprecation warnings and raise exceptions for already deprecated ones [airflow]

2024-04-02 Thread via GitHub
eladkal commented on code in PR #38673: URL: https://github.com/apache/airflow/pull/38673#discussion_r1548384082 ## airflow/providers/google/cloud/hooks/vertex_ai/prediction_service.py: ## @@ -0,0 +1,86 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] remove display of param from the UI [airflow]

2024-04-02 Thread via GitHub
hussein-awala commented on PR #38660: URL: https://github.com/apache/airflow/pull/38660#issuecomment-2032783818 > > Ah - so does it need to be displayed ? > > > > Yes, but it's a bit broken. So I can work on a real fix for this Waiting for your PR to understand why

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-04-02 Thread via GitHub
Taragolis commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2032778319 But this failure also show that docker compose tests can't show useful information in case if API failed because it just fail before show logs from the container. I will add

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-04-02 Thread via GitHub
Taragolis commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2032763101 > That's already fixed in https://github.com/apache/airflow/pull/38678 -> I rebased your PR @Taragolis Hehe, I've also rebased it. DOUBLE REBASE 藍 -- This is an automated

Re: [PR] Data aware scheduling docs edits [airflow]

2024-04-02 Thread via GitHub
lzdanski commented on PR #38687: URL: https://github.com/apache/airflow/pull/38687#issuecomment-2032743427 @sunank200 - Docs copyediting for https://github.com/apache/airflow/pull/37101. I took a look at the PR's you shared with me, https://github.com/apache/airflow/pull/37771 and

[PR] Data aware scheduling docs edits [airflow]

2024-04-02 Thread via GitHub
lzdanski opened a new pull request, #38687: URL: https://github.com/apache/airflow/pull/38687 --- Copy edits for 2.9 Data Aware Scheduling feature docs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Replace to broad exceptions into the Dev [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38489: URL: https://github.com/apache/airflow/pull/38489#issuecomment-2032734953 Should we just complete thaat one :) ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Build helm values schemas with kubernetes 1.29 resources [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38460: URL: https://github.com/apache/airflow/pull/38460#issuecomment-2032726289 In the meantime we had some conflicts so if you could rebase and fix them @mathiaHT - that woudl give chance for the others to take a look and merge it. -- This is an automated

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2032719082 Sounds like a breaking bug-fix to me . I.e. yes - we know it will be breaking, but it was broken in the first place anyway. @ferruzzi - WDYT? I'd merge it and add significant note about

Re: [PR] Give `on_task_instance_failed` access to the error that caused the failure [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38155: URL: https://github.com/apache/airflow/pull/38155#issuecomment-2032715168 > > using a new public API call (`get_last_task_failure`- and add documentation about it). > > by "new API" you meant just a method on the class, right ? Is there more doc to

Re: [PR] Added BNY Mellon to the list of companies using Apache Airflow [airflow]

2024-04-02 Thread via GitHub
boring-cyborg[bot] commented on PR #38686: URL: https://github.com/apache/airflow/pull/38686#issuecomment-2032709812 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

Re: [I] [Backfill] Support Partial DAG Backfills while still honoring dependencies [airflow]

2024-04-02 Thread via GitHub
potiuk commented on issue #38407: URL: https://github.com/apache/airflow/issues/38407#issuecomment-2032699912 That sounds like something that needs a devlist discussion - not GitHuyb issue I am afraid - it's not that straightforward and I have a feeling this one will not play well with

Re: [PR] heartbeat recovery message [airflow]

2024-04-02 Thread via GitHub
potiuk commented on code in PR #34457: URL: https://github.com/apache/airflow/pull/34457#discussion_r1548320952 ## airflow/jobs/job.py: ## @@ -131,22 +146,20 @@ def executors(self): def heartrate(self) -> float: return Job._heartrate(self.job_type) -def

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2032677469 (and even better - extraction of the common code added then is in #38682 - ready for review. -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] Use `methodtools.lru_cache` instead of `functools.lru_cache` in class methods [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #37757: URL: https://github.com/apache/airflow/pull/37757#issuecomment-2032674265 > The problem here with tests that Prod images build do not reflect changes into requirements in `hatch_build.py` and it doesn't install `methodtools` That's already fixed in

Re: [I] Momento Warning [airflow]

2024-04-02 Thread via GitHub
potiuk commented on issue #37281: URL: https://github.com/apache/airflow/issues/37281#issuecomment-2032669417 > Not directly related to this, but Airflow should move off of `momentjs` altogether at some point. The project is long considered legacy and developers are encouraged not to use

Re: [I] GlueJobOperator failing with Invalid type for parameter GlueVersion for provider version >= 7.1.0. [airflow]

2024-04-02 Thread via GitHub
don1uppa commented on issue #32404: URL: https://github.com/apache/airflow/issues/32404#issuecomment-2032657099 If it fix the value here back to a string under debug it works -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] remove display of param from the UI [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38660: URL: https://github.com/apache/airflow/pull/38660#issuecomment-2032653376 :eyes: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [I] GlueJobOperator failing with Invalid type for parameter GlueVersion for provider version >= 7.1.0. [airflow]

2024-04-02 Thread via GitHub
don1uppa commented on issue #32404: URL: https://github.com/apache/airflow/issues/32404#issuecomment-2032656200 well - its corrupted upfront. but I can't pin it down![image](https://github.com/apache/airflow/assets/33661098/26a0ae62-cb42-4106-a400-9c5438268773) -- This is an automated

Re: [PR] remove display of param from the UI [airflow]

2024-04-02 Thread via GitHub
bbovenzi commented on PR #38660: URL: https://github.com/apache/airflow/pull/38660#issuecomment-2032640007 > Ah - so does it need to be displayed ? Yes, but it's a bit broken. So I can work on a real fix for this -- This is an automated message from the Apache Git Service. To

Re: [PR] remove display of param from the UI [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38660: URL: https://github.com/apache/airflow/pull/38660#issuecomment-2032634406 Ah - so does it need to be displayed ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] Extract checkout target commit to a composite action [airflow]

2024-04-02 Thread via GitHub
potiuk commented on PR #38682: URL: https://github.com/apache/airflow/pull/38682#issuecomment-2032628203 Tested in : * https://github.com/potiuk/airflow/pull/311 (committer build) * https://github.com/potiuk/airflow/pull/312 (non-committer build) -- This is an automated

Re: [PR] remove display of param from the UI [airflow]

2024-04-02 Thread via GitHub
bbovenzi commented on PR #38660: URL: https://github.com/apache/airflow/pull/38660#issuecomment-2032626711 `root` allows a user to filter the grid+graph by only showing what's upstream/downstream of a task. The UX needs to be improved, but it is still used! -- This is an automated

(airflow) branch v2-9-test updated (fb21392124 -> 506c7e6415)

2024-04-02 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-9-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit fb21392124 Update RELEASE_NOTES.rst add 2ce6d1c84d Update UI doc screenshots (#38680) add

Re: [PR] Update UI doc screenshots [airflow]

2024-04-02 Thread via GitHub
bbovenzi merged PR #38680: URL: https://github.com/apache/airflow/pull/38680 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch flask-session-060 deleted (was 3f72f68474)

2024-04-02 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch flask-session-060 in repository https://gitbox.apache.org/repos/asf/airflow.git was 3f72f68474 [PoC] Flask Session 0.6 The revisions that were on this branch are still contained in

  1   2   3   >