fat-catTW commented on code in PR #71546:
URL: https://github.com/apache/airflow/pull/71546#discussion_r3788442726
##########
providers/amazon/src/airflow/providers/amazon/aws/operators/glue.py:
##########
@@ -447,8 +452,10 @@ def _find_previous_job_run(self, context: Context,
task_uuid: str) -> str | None
self.log.info("Previous Glue job_run_id: %s, state: %s",
previous_job_run_id, state)
if self.is_job_active(state):
return previous_job_run_id
- except Exception:
- self.log.warning("Failed to get previous Glue job run state",
exc_info=True)
+ except ClientError:
Review Comment:
Yes, this was intentional. AWS lookup failures are still handled via
`ClientError`, so a Glue API failure during reconnect lookup still logs the
exception and falls back to submitting a fresh run.
The behavior change is for non-AWS exceptions. Those are more likely to be
programming errors or unexpected local assumptions, and silently converting
them into a fresh Glue job submission can hide the real bug and potentially
create a duplicate run. That was one of the issues this PR is trying to address.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
