o-nikolas commented on a change in pull request #19195:
URL: https://github.com/apache/airflow/pull/19195#discussion_r736845819
##########
File path: airflow/providers/amazon/aws/hooks/sagemaker.py
##########
@@ -954,3 +954,13 @@ def _list_request(
return results
else:
next_token = response["NextToken"]
+
+ def find_processing_job_by_name(self, processing_job_name: str) -> bool:
+ """
+ Query processing job existence by describe_processing_job, rather than
iterate all the jobs.
+ """
+ try:
+ res =
self.get_conn().describe_processing_job(ProcessingJobName=processing_job_name)
+ return True
+ except ClientError as e:
Review comment:
While this API specifically throws only
[ResourceNotFound](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_DescribeProcessingJob.html#API_DescribeProcessingJob_Errors)
there are many errors that are common to all sagemaker APIs that could
technically be thrown (see the list
[here](https://docs.aws.amazon.com/sagemaker/latest/APIReference/CommonErrors.html)).
For completeness I think it's worth checking directly for ResourceNotFound. It
is also more future proof that way. Just my 2c
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
