boraberke opened a new pull request, #38868: URL: https://github.com/apache/airflow/pull/38868
This PR adds a new `retry_from_failure` parameter to the `DbtRunJobOperator` to retry a failed run of a dbt Cloud job from the point of failure. The implementation uses the new [rerun](https://docs.getdbt.com/dbt-cloud/api-v2#/operations/Retry%20Failed%20Job) endpoint in the `dbt API` which handles the lookup of the last run for a given job itself and decides whether to start a new run of the job or not. I am not sure about the current implementation, but wanted to point out the problems and discuss it over this PR: 1. `rerun` endpoint does not accept body, which means parameters like `steps_override`, `schema_override`, `threads_override`, `cause` cannot be passed. Current implementation always uses `rerun` endpoint if `retry_from_failure` is set to True. To overcome this issue, `rerun` endpoint can be used only if the task is retried (i.e. `ti.try_number !=1`). Closes: #35772 See also: #38001 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
