AlexanderBLR commented on code in PR #71380:
URL: https://github.com/apache/airflow/pull/71380#discussion_r3763868675
##########
providers/amazon/src/airflow/providers/amazon/aws/executors/batch/batch_executor.py:
##########
@@ -268,11 +268,30 @@ def sync_running_jobs(self):
self.log.debug("Active Workers: %s", describe_job_response)
for job in describe_job_response:
- if job.get_job_state() == State.FAILED:
- self._handle_failed_job(job)
- elif job.get_job_state() == State.SUCCESS:
- workload_key = self.active_workers.pop_by_id(job.job_id)
- self.success(workload_key)
+ try:
+ if job.get_job_state() == State.FAILED:
+ self._handle_failed_job(job)
+ elif job.get_job_state() == State.SUCCESS:
+ workload_key = self.active_workers.pop_by_id(job.job_id)
+ self.success(workload_key)
+ except Exception:
+ self.log.exception(
+ "Evicting Batch job %s after an unexpected error while
syncing it, "
+ "so that one broken job cannot abort the sync cycle and
stall task "
+ "submission for the whole executor.",
Review Comment:
Agreed, trimmed the message to just the action in the follow-up commit.
---Drafted-by: Claude Code (Fable 5); reviewed by @AlexanderBLR before
posting
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
