kaxil commented on code in PR #77158:
URL: https://github.com/apache/airflow/pull/77158#discussion_r4604268697


##########
providers/amazon/docs/operators/batch.rst:
##########
@@ -51,6 +51,22 @@ use 
:class:`~airflow.providers.amazon.aws.operators.batch.BatchOperator`.
     :start-after: [START howto_operator_batch]
     :end-before: [END howto_operator_batch]
 
+Durable execution
+=================
+
+By default ``BatchOperator`` saves the Batch job id to the
+:doc:`task state store <apache-airflow:core-concepts/task-state-store>` right 
after submitting the
+job. If the worker crashes and the task is retried, the operator checks that 
job first:
+
+* still active (``SUBMITTED`` through ``RUNNING``): reconnects and keeps 
waiting on it
+* already ``SUCCEEDED``: returns the job id without submitting again

Review Comment:
   On 3.3 and 3.4 a clear can keep `batch_job_id` as well: every clear on 3.3, 
and on 3.4 every path except clearing task instances through the UI or REST API 
(Clear Run, the CLI, downstream clears from marking a task). With 
`clear_on_success` off by default, someone who fixes upstream data and clears 
the run gets a green task and no new Batch job, where before this change every 
clear resubmitted. Could this section and the changelog note say so, like the 
clearing paragraph in the Databricks `submit_run.rst` docs? #76806 fixes it on 
main, so it only affects the released versions.



##########
providers/amazon/tests/unit/amazon/aws/operators/test_batch.py:
##########
@@ -730,6 +731,118 @@ def test_execute_complete_failure_without_logs(
             mock_get_job_all_awslogs_info.assert_called_once_with("12345")
 
 
[email protected](not AIRFLOW_V_3_3_PLUS, reason="durable execution requires 
Airflow 3.3+")
+class TestBatchOperatorDurable:
+    NEW_JOB_ID = "11111111-2222-3333-4444-555555555555"
+
+    @pytest.fixture
+    def client(self):
+        with 
mock.patch("airflow.providers.amazon.aws.hooks.batch_client.AwsBaseHook.get_client_type")
 as m:
+            m.return_value.submit_job.return_value = {"jobName": JOB_NAME, 
"jobId": self.NEW_JOB_ID}
+            yield m.return_value
+
+    @pytest.fixture
+    def task_store(self):
+        store = mock.MagicMock(spec_set=["get", "set"])
+        store.get.return_value = None
+        return store
+
+    @staticmethod
+    def _make_operator(operator_class=BatchOperator, **kwargs):
+        return operator_class(
+            task_id="task",
+            job_name=JOB_NAME,
+            job_queue="queue",
+            job_definition="hello-world",
+            do_xcom_push=False,
+            **kwargs,
+        )
+
+    @staticmethod
+    def _context(task_store):
+        return {"ti": mock.MagicMock(spec_set=["stats_tags"], stats_tags={}), 
"task_state_store": task_store}
+
+    
@mock.patch("airflow.providers.amazon.aws.links.batch.BatchJobDetailsLink.persist")
+    @mock.patch.object(BatchOperator, "monitor_job", autospec=True)
+    def test_fresh_submit_persists_job_id(self, monitor_mock, link_mock, 
client, task_store):
+        result = self._make_operator().execute(self._context(task_store))
+
+        client.submit_job.assert_called_once()
+        link_mock.assert_called_once()
+        task_store.set.assert_called_once_with("batch_job_id", self.NEW_JOB_ID)
+        monitor_mock.assert_called_once()
+        assert result == self.NEW_JOB_ID
+
+    
@mock.patch("airflow.providers.amazon.aws.links.batch.BatchJobDetailsLink.persist")
+    @mock.patch.object(BatchOperator, "monitor_job", autospec=True)
+    def test_reconnects_to_running_job(self, monitor_mock, link_mock, client, 
task_store):
+        task_store.get.return_value = JOB_ID
+        client.describe_jobs.return_value = {"jobs": [{"jobId": JOB_ID, 
"status": "RUNNING"}]}

Review Comment:
   Could this be parametrized over all of `INTERMEDIATE_STATES` (`SUBMITTED`, 
`PENDING`, `RUNNABLE`, `STARTING`, `RUNNING`) and assert `describe_jobs` was 
called with `jobs=[JOB_ID]`? As written, narrowing `is_job_active` to `RUNNING` 
only, or looking up `self.job_id` instead of the stored id, both still pass.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to