Jacob Marble created BEAM-3638:
----------------------------------
Summary: S3 occasionally fails to delete
Key: BEAM-3638
URL: https://issues.apache.org/jira/browse/BEAM-3638
Project: Beam
Issue Type: Bug
Components: sdk-java-extensions
Affects Versions: 2.3.0
Reporter: Jacob Marble
Assignee: Jacob Marble
Fix For: 2.3.0
When AmazonS3Client.deleteObjects() is called, the S3 service occasionally
returns a retryable error, and also deletes the object. Beam retries the
request, and the S3 service correctly returns a "404 Not Found" error. The job
fails needlessly.
I propose to fix this by logging and allowing "404 Not Found" errors; if the
object doesn't exist then the current state is also the desired state. Should
have a PR later today.
The first error looks like this:
{{java.io.IOException:
com.amazonaws.services.s3.model.MultiObjectDeleteException: One or more objects
could not be deleted (Service: null; Status Code: 200; Error Code: null;
Request ID: [redacted]; S3 Extended Request ID: [redacted]), S3 Extended
Request ID: [redacted]}}
Subsequent errors:
{{java.io.IOException: com.amazonaws.services.s3.model.AmazonS3Exception: Not
Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request
ID: [redacted]; S3 Extended Request ID: [redacted]), S3 Extended Request ID:
[redacted]}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
