shahar1 commented on code in PR #77245:
URL: https://github.com/apache/airflow/pull/77245#discussion_r4619830407
##########
providers/hashicorp/docs/changelog.rst:
##########
@@ -27,13 +27,24 @@
Changelog
---------
+4.8.5
Review Comment:
The `use_team_secrets_path` warning that now sits under 4.8.5 does not
belong to this release. It was added by #73789 (`2a274ab3c93`), which is
already listed under 4.8.3 Bug Fixes ("Refuse nested secret ids looked up with
no team in multi-team mode") and is contained in the
`providers-hashicorp/4.8.3` tag. On `main` the block was left above the `4.8.4`
heading, so the new heading landed on top of it.
As it stands, 4.8.5 presents a behaviour change as part of a patch release
whose only entry is a boto3 floor. Please move the whole `.. warning::` block
down to sit under the `4.8.3` heading, before its `Bug Fixes`.
This is the same kind of case as the cncf.kubernetes thread, but there the
notes really do belong to the new version. I checked the other moved notes in
this wave and hashicorp is the only one that belongs to a released version.
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/amazon/docs/changelog.rst:
##########
@@ -35,6 +38,37 @@ Changelog
submitting a new job on every try. Reconnecting needs Airflow 3.3 or
later; below that the
operator behaves as with ``durable=False``.
+Features
+~~~~~~~~
+
+* ``Redirect to the requested page after AWS SAML login (#77209)``
+* ``Add S3ArchiveDagBundle for single-archive staging (#73015)``
+* ``Support Athena managed query results in AthenaHook (#77067)``
+* ``Add deferrable support to AthenaSparkOperator (#75915)``
+* ``Support durable execution in SageMakerTrainingOperator (#76036, #76964)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix templated page_size staying a string in mapped SqlToS3Operator tasks
(#77153)``
+* ``Template the Neptune cluster operators' db_cluster_id argument under its
own name (#70491)``
Review Comment:
This entry reads as a templating fix, but the commit renames a public
attribute. On `NeptuneStartDbClusterOperator` and
`NeptuneStopDbClusterOperator`, `self.cluster_id` becomes `self.db_cluster_id`
and the template field changes from `cluster_id` to `db_cluster_id`. The
`db_cluster_id=` constructor argument is unchanged, so most Dags are fine, but
a subclass or callback that reads `operator.cluster_id` now gets
`AttributeError`.
```suggestion
* ``Rename the cluster_id attribute and template field of the Neptune
cluster operators to db_cluster_id, matching the constructor argument (#70491)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/fab/docs/changelog.rst:
##########
@@ -20,6 +20,26 @@
Changelog
---------
+3.11.1
+......
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix FAB role rename to an existing name returning 200 (#76972)``
+* ``Fix FAB userinfo page breaking after viewing a user (#76473)``
+* ``Fix FAB user auto-registration failing with a 500 when add_user fails
(#76746)``
Review Comment:
The fix changes the contract of
`FabAirflowSecurityManagerOverride.add_user`: on a save failure it used to
raise `FabException`, and it now rolls back and returns `None` (return type
`User | None`). A custom security manager or script that calls `add_user` and
relies on the exception will carry on with `None`. I think the entry should say
so:
```suggestion
* ``Fix FAB user auto-registration failing with a 500 when add_user fails;
add_user now returns None instead of raising FabException (#76746)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/dbt/cloud/docs/changelog.rst:
##########
@@ -28,6 +28,17 @@
Changelog
---------
+4.12.0
+......
+
+Features
+~~~~~~~~
+
+* ``Support durable execution for dbt cloud job runs (#76918)``
Review Comment:
Same point as databricks: #76918 makes `DbtCloudRunJobOperator` durable by
default (its docstring says "When ``True`` (the default)"). On Airflow 3.3+
with `wait_for_termination=True`, a retry reconnects to the stored dbt Cloud
run instead of triggering a new one. The operator docs added by the PR also
describe a case users can trip over: before Airflow 3.5, clearing the whole Dag
run keeps the stored run id, so a task whose run succeeded returns without
running dbt again.
Suggested note above `Features`, in the style of the microsoft.azure one:
```rst
.. note::
On Airflow 3.3 and later, ``DbtCloudRunJobOperator`` is durable by
default when
``wait_for_termination`` is True: a retry reconnects to the dbt Cloud
run the previous try
triggered instead of triggering a new one. Before Airflow 3.5, clearing
the whole Dag run keeps
the stored run id, so a task whose run succeeded does not run dbt again.
Clear the task instance
instead, or set ``durable=False`` to trigger a new run on every try.
Below Airflow 3.3 nothing
changes.
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/databricks/docs/changelog.rst:
##########
@@ -26,6 +26,31 @@
Changelog
---------
+7.25.0
+......
+
+Features
+~~~~~~~~
+
+* ``Support durable execution in Databricks task operators (#76295)``
Review Comment:
#76295 makes `DatabricksNotebookOperator` and `DatabricksTaskOperator`
durable by default (`DatabricksTaskBaseOperator` now inherits
`ResumableJobMixin`, whose `durable` defaults to `True`). On Airflow 3.3+ a
retry of these two operators reconnects to the stored Databricks run, and
returns without resubmitting when that run already succeeded, where 7.24.0
submitted a new run on every try. `DatabricksSubmitRunOperator` and
`DatabricksRunNowOperator` were already durable before this commit, so they are
not affected.
amazon (SageMaker) and microsoft.azure (Data Factory) carry a note for the
same change of default in this wave; databricks has none. Suggested note above
`Features`:
```rst
.. note::
On Airflow 3.3 and later, ``DatabricksNotebookOperator`` and
``DatabricksTaskOperator`` are durable
by default: a retry reconnects to the Databricks run the previous try
submitted instead of
submitting a new one, and returns without resubmitting when that run has
already succeeded. Set
``durable=False`` to submit a new run on every try. Below Airflow 3.3
nothing changes.
```
and the entry could name the operators:
```suggestion
* ``Support durable execution in DatabricksNotebookOperator and
DatabricksTaskOperator (#76295)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/ssh/docs/changelog.rst:
##########
@@ -27,6 +27,17 @@
Changelog
---------
+7.0.3
+.....
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix SSHHookAsync deadlock on the extra and compress like SSHHook (#76528)``
Review Comment:
#76528 does two things the entry leaves out. It raises the `asyncssh` floor
from 2.12.0 to 2.18.0, and `SSHHookAsync` now compresses traffic by default, as
`SSHHook` does (`self.compress = True`, switched off with `compress: false` in
the connection extra). The PR description covers the `common-compat>=1.21.0`
floor, but the changelog mentions neither floor.
```suggestion
* ``Fix SSHHookAsync deadlock on the extra and compress by default like
SSHHook; raise asyncssh minimum to 2.18.0 (#76528)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/git/docs/changelog.rst:
##########
@@ -19,6 +19,23 @@
Changelog
---------
+1.1.0
+.....
+
+Features
+~~~~~~~~
+
+* ``Keep the previous Dag source when a GitDagBundle refresh fails (#74789)``
Review Comment:
This behaviour depends on the core class `DagBundleRefreshError`, which the
provider looks up with `getattr(bundle_base, "DagBundleRefreshError", None)`.
The class exists on `main` (3.5.0) only; it is not in 3.4.0 or on `v3-4-test`.
On every Airflow version released today the provider re-raises the original
exception, so nothing changes for users of 1.1.0 yet.
```suggestion
* ``Keep the previous Dag source when a GitDagBundle refresh fails, on
Airflow 3.5+ (#74789)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/common/ai/docs/changelog.rst:
##########
@@ -89,6 +92,63 @@ Changelog
typed ``str``; in 1.1.0 it defaulted to ``None`` and was typed ``str |
None``. This changes
only the signature and the annotated type, not the behaviour.
+Features
+~~~~~~~~
+
+* ``Show common.ai retry policies why each fallback connection failed
(#76999)``
+* ``Let SQLToolset agents see table comments and search large schemas
(#76799)``
+* ``Park HITL reviews between rounds and let durable agent tasks use them
(#76916)``
+* ``Let durable agent tasks pause for tool approval (#76882)``
+* ``Let durable agent tasks use toolsets without an id (#76877)``
+* ``Let durable agent tasks run chosen tools on every attempt instead of
replaying them (#76864)``
+* ``Make common.ai tool_prefix and output_type signatures consistent (#74695)``
Review Comment:
"Signatures consistent" undersells #74695. It is the PR behind the two notes
above ("Significant change: output serialization of the LLM operators" and the
MCP `tool_prefix` one): `LLMOperator`, `@task.llm` and
`LLMFileAnalysisOperator` now push some models to XCom as dicts where 1.1.0
pushed model instances, so downstream attribute access breaks. It also adds a
`tool_prefix` parameter to `SQLToolset` and `DataFusionToolset`.
```suggestion
* ``Push LLM operator models as dicts where the worker cannot register their
class, and add tool_prefix to SQLToolset and DataFusionToolset (#74695)``
```
Separately, a question for the release manager: the note itself calls this a
significant change and it can break downstream tasks. Is a minor bump with the
entry under Features what we want here, or should it be listed as breaking?
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/standard/docs/changelog.rst:
##########
@@ -47,19 +47,25 @@ Changelog
Features
~~~~~~~~
+* ``Restore clearing 'ExternalTaskMarker' targets in other Dags (#75874)``
Review Comment:
Same situation as git #74789. The marker builds its clear target from
`airflow.sdk.ExternalClearTarget`, imported inside `try/except ImportError`.
That class is on `main` (3.5.0) only, not in 3.4.0 or on `v3-4-test`, so on
Airflow 3.0 to 3.4 clearing an `ExternalTaskMarker` still does not clear the
target in the other Dag. "Restore" without the version reads as if upgrading
the provider is enough.
```suggestion
* ``Restore clearing 'ExternalTaskMarker' targets in other Dags on Airflow
3.5+ (#75874)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/common/ai/docs/changelog.rst:
##########
@@ -89,6 +92,63 @@ Changelog
typed ``str``; in 1.1.0 it defaulted to ``None`` and was typed ``str |
None``. This changes
only the signature and the annotated type, not the behaviour.
+Features
+~~~~~~~~
+
+* ``Show common.ai retry policies why each fallback connection failed
(#76999)``
+* ``Let SQLToolset agents see table comments and search large schemas
(#76799)``
+* ``Park HITL reviews between rounds and let durable agent tasks use them
(#76916)``
+* ``Let durable agent tasks pause for tool approval (#76882)``
+* ``Let durable agent tasks use toolsets without an id (#76877)``
+* ``Let durable agent tasks run chosen tools on every attempt instead of
replaying them (#76864)``
Review Comment:
#76864 also raises the provider's main dependency floor:
`pydantic-ai-slim>=2.36.0` becomes `>=2.46.0` (and `pydantic-ai-slim[mcp]` with
it). No entry in this section mentions it.
```suggestion
* ``Let durable agent tasks run chosen tools on every attempt instead of
replaying them; raise pydantic-ai-slim minimum to 2.46.0 (#76864)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/google/docs/changelog.rst:
##########
@@ -27,12 +27,55 @@
Changelog
---------
+23.0.0
+......
+
+.. warning::
+ ``BigQueryGetDataOperator`` no longer accepts ``project_id``, which has
been deprecated since
+ 19.4.0. Use ``table_project_id`` instead. Dags that still pass
``project_id`` fail at parse time
+ with ``TypeError``.
+
.. note::
The provider no longer installs ``litellm``. It now depends on
``google-cloud-aiplatform`` and the
dependencies of its ``evaluation`` extra except ``litellm``, which the
extra caps below the fix for
a critical advisory. If your tasks call ``vertexai.Client().evals`` with a
model that LiteLLM
serves, install ``litellm`` yourself; without it those calls raise
``ImportError``.
+Breaking changes
+~~~~~~~~~~~~~~~~
+
+* ``Remove deprecated project_id from BigQueryGetDataOperator (#76115)``
+
+Features
+~~~~~~~~
+
+* ``Add Cloud SQL Auth Proxy v2 support to CloudSqlProxyRunner (#73646)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fix an issue with getting dataflow_job_id for Dataflow Runner when logging
is disabled in Pipeline (#72809)``
+* ``Fix task state lookup for mapped tasks in Google triggers (#76895)``
+* ``Honor cancel_on_kill in deferred GCS to BigQuery loads (#74358)``
+* ``Fix GcpBodyFieldValidator warning on valid version-specific fields
(#74930)``
+
+Misc
+~~~~
+
+* ``Raise google-api-python-client minimum to 2.196.0, and
google-cloud-bigquery-storage and google-cloud-bigtable minimum to 2.38.0
(#76334)``
Review Comment:
The entry names 3 of the 13 floors that #76334 raises in
`providers/google/pyproject.toml`. The other ten: `google-cloud-dataplex`
2.19.0, `google-cloud-kms` 3.13.0, `google-cloud-orchestration-airflow` 1.21.0,
`google-cloud-os-login` 2.21.0, `google-cloud-pubsub` 2.38.0,
`google-cloud-secret-manager` 2.28.0, `google-cloud-spanner` 3.66.0,
`google-cloud-speech` 2.39.0, `google-cloud-vision` 3.14.0,
`google-cloud-workflows` 1.22.0.
Either list them all or stop naming individual packages, since a partial
list reads as complete:
```suggestion
* ``Raise the minimum versions of google-api-python-client (2.196.0) and 12
google-cloud-* client libraries: bigquery-storage, bigtable, dataplex, kms,
orchestration-airflow, os-login, pubsub, secret-manager, spanner, speech,
vision and workflows (#76334)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/common/ai/docs/changelog.rst:
##########
@@ -89,6 +92,63 @@ Changelog
typed ``str``; in 1.1.0 it defaulted to ``None`` and was typed ``str |
None``. This changes
only the signature and the annotated type, not the behaviour.
+Features
+~~~~~~~~
+
+* ``Show common.ai retry policies why each fallback connection failed
(#76999)``
+* ``Let SQLToolset agents see table comments and search large schemas
(#76799)``
Review Comment:
The PR description explains that the `sql` extra moves to
`common-sql>=3.1.0` for the new `DbApiHook` methods, but the changelog does not
say it, and that is a jump from `>=2.2.0` across a major version of common.sql.
```suggestion
* ``Let SQLToolset agents see table comments and search large schemas; the
sql extra now needs common-sql 3.1.0 (#76799)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/common/ai/docs/changelog.rst:
##########
@@ -89,6 +92,63 @@ Changelog
typed ``str``; in 1.1.0 it defaulted to ``None`` and was typed ``str |
None``. This changes
only the signature and the annotated type, not the behaviour.
+Features
+~~~~~~~~
+
+* ``Show common.ai retry policies why each fallback connection failed
(#76999)``
+* ``Let SQLToolset agents see table comments and search large schemas
(#76799)``
+* ``Park HITL reviews between rounds and let durable agent tasks use them
(#76916)``
+* ``Let durable agent tasks pause for tool approval (#76882)``
+* ``Let durable agent tasks use toolsets without an id (#76877)``
+* ``Let durable agent tasks run chosen tools on every attempt instead of
replaying them (#76864)``
+* ``Make common.ai tool_prefix and output_type signatures consistent (#74695)``
+* ``Accept '.filtered()', '.prefixed()' and '.renamed()' toolsets in the agent
framework adapters (#76752)``
+* ``Pause an OpenAI Agents SDK durable run for tool approval in Required
Actions, and fail the task where that is not available (#76785, #76786)``
+* ``Add Ollama model readiness check to PydanticAIHook (#75369)``
+* ``Support multimodal prompts with enable_hitl_review in AgentOperator
(#74389)``
+* ``Support fail_on_reject on every LLM approval operator (#74493)``
+* ``Suggest common model providers in the pydanticai connection form (#76053)``
+* ``Show which skills an agent was offered and used in the AI Model tab
(#76309, #76541)``
+
+Bug Fixes
+~~~~~~~~~
+
+* ``Fail the attempt when 'SQLToolset' cannot reach its database (#76783)``
+* ``Fix durable agent retries replaying the same failing tool call on every
attempt (#76862)``
+* ``Stop ObjectStorage journal write failures from failing durable tasks
(#75916)``
+* ``Fix durable agent tasks failing when a tool returns bytes (#76861)``
+* ``Count replayed tool errors against the retry limit in OpenAI Agents
durable runs (#76790)``
+* ``Stop durable agent tasks failing when pydantic-ai 2.66 cannot fingerprint
a tool return (#76334)``
+* ``Parse HookToolset :param descriptions by indentation (#76701)``
+* ``Say in the AI Model tab which values are kept in external storage
(#76747)``
+* ``Keep SQL deep scan on for custom read-only allowed_types (#76688)``
+* ``Keep CSV quoting in LLMFileAnalysisOperator sample rows (#76389)``
+* ``Keep parallel tool results replayable across retries in OpenAI Agents
durable runs (#76015)``
+* ``Key LLM batch and durable Strands state on the task loop iteration
(#76201)``
+* ``Render only the returned value in TaskFlow decorators that template it
(#76244)``
+
+Misc
+~~~~
+
+* ``Reduce task state store calls made by durable agent tasks (#76863)``
+* ``Name the caller's own MCP client and durable setup in Common AI errors
(#76495)``
+
+Doc-only
+~~~~~~~~
+
+* ``Correct what clearing a durable agent task replays in common.ai docs
(#76860)``
+* ``Correct sbx swap guidance in Common AI sandbox docs (#76879)``
+* ``Split the Common AI sandbox backends page into one page per backend
(#76853)``
+* ``Document that the llamaindex and openai extras cannot be installed
together (#76503)``
+* ``Rewrite the OpenAI Agents SDK guide to stand on its own (#76340)``
+
+.. Below changes are excluded from the changelog. Move them to
+ appropriate section above if needed. Do not delete the lines(!):
+ * ``Keep common.ai tripwires and module_loading types working after
dependency upgrades (#76715)``
+ * ``Stop pydantic-ai providers that models cannot use from failing the
common.ai vendor check (#76312)``
+ * ``Add screenshots and a review loop example to the task loops docs
(#76435)``
Review Comment:
Minor: the common.ai slice of #76435 is a new example Dag shipped in the
package (`example_dags/example_llm_review_loop.py`), so it is not a
tests/CI-only change. Elsewhere in this wave an example Dag change is listed
under Doc-only (amazon #77173). Suggest moving it there as:
```rst
* ``Add a review loop example Dag for task loops (#76435)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
##########
providers/openlineage/docs/changelog.rst:
##########
@@ -26,6 +26,23 @@
Changelog
---------
+2.21.1
+......
+
+Misc
+~~~~
+
+* ``Bump openlineage libraries to 1.55 in openlineage provider (#77031)``
Review Comment:
Minor: this is a raised minimum for two packages (`openlineage-python` and
`openlineage-integration-common`, both `>=1.52.0` to `>=1.55.0`), which "Bump
... libraries" does not make clear.
```suggestion
* ``Raise openlineage-python and openlineage-integration-common minimum to
1.55.0 (#77031)``
```
---
Drafted-by: Claude Code (Fable 5.2); reviewed by `shahar1` before posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]