amoghrajesh commented on code in PR #76295: URL: https://github.com/apache/airflow/pull/76295#discussion_r4453333445
########## providers/databricks/docs/operators/notebook.rst: ########## @@ -80,3 +81,35 @@ attempt is observed, so downstream failure handling is not delayed by long-runni Only unlimited retries (``max_retries=-1``) keep the Airflow task waiting (or deferring) until the parent workflow run reaches a terminal state, because Databricks may still launch a retry attempt under the same ``task_key`` until then. Sibling tasks in the run continue independently. + + +Durable execution +----------------- + +Outside a +:class:`~airflow.providers.databricks.operators.databricks_workflow.DatabricksWorkflowTaskGroup`, +this operator submits a run and polls it to completion on the worker. By default it runs in a Review Comment: It was intended. `execute` goes through `execute_resumable` whether or not deferrable is set, and the defer happens inside `monitor_databricks_job`, after the mixin has persisted the run id and added a paragraph to both pages saying durable covers deferrable mode here, and calling out that it differs from the sibling operators. Also fixed "polls it to completion on the worker", which was wrong once it defers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
