SEPURI-SAI-KRISHNA commented on issue #74741:
URL: https://github.com/apache/airflow/issues/74741#issuecomment-6229586703
Verified dbt.cloud 4.11.0rc2 for #74199 on Airflow 3.4.0, Python 3.12.3,
tenacity 9.1.4.
One script run unchanged against three provider versions, with
`DbtCloudRunJobOperator(deferrable=True, hook_params={"retry_limit": 7,
"retry_args": {"stop": stop_after_attempt(3)}})` and the hook mocked, so this
covers the serialization boundary rather than a live dbt Cloud run. In all
three the task defers and the job is recorded as started before it does.
- 4.10.0: the trigger receives no `hook_params`, so the hook in the trigger
is built with none of them. Serializing the trigger arguments succeeds.
- 4.11.0rc1: the trigger receives `retry_args` and `retry_limit`, and
putting the trigger arguments through `airflow.sdk.serde.serialize`, which is
what `Trigger.encrypt_kwargs` uses to store them, raises `TypeError: Cannot
serialize object of type <class 'tenacity.stop.stop_after_attempt'>`. The job
is already started at that point, which is the orphaned run that kept dbt.cloud
out of the previous wave.
- 4.11.0rc2: the trigger keeps `retry_limit`, logs a warning naming
`retry_args` as dropped, and the trigger arguments serialize. `DbtCloudHook`
builds from what survived.
Only the values that fail the JSON check are dropped. Given `retry_limit`,
`retry_delay` and a tenacity `retry_args` together, rc2 keeps `retry_limit` and
`retry_delay` and drops only `retry_args`. The operator half keeps all three,
since the operator builds its own hook from the unfiltered mapping. The trigger
docstring describes this.
`operators/dbt.py` and `triggers/dbt.py` in the rc2 wheel are byte identical
by sha256 to the same files at current main `20ccfcd7457a`, and they are the
only source files #74199 and #74489 changed between them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
