Birne94 opened a new pull request, #38921:
URL: https://github.com/apache/airflow/pull/38921
This pull requests adds functionality to restore databricks run-ids in the
`on_kill` callback if it is not set.
**Background**
We encountered an issue in our MWAA (2.7.2) environment after enabling the
deferred flag our databricks operators. If a deferred databricks task timed
out, it will try to cancel the tasks but is unable to do so because of missing
`run_id`:
```
{{databricks.py:412}} ERROR - Error: Task: my-task-id with invalid run_id
was requested to be cancelled.
```
Since the run-id is only [set as an instance
variable](https://github.com/apache/airflow/blob/4f169bd2f3e27b8530da4b82d0d3b25b796eff39/airflow/providers/databricks/operators/databricks.py#L560)
during execution, it is no longer available when the task is restored after
the trigger has finished or is interrupted.
With this PR
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
