carloea2 commented on code in PR #8262:
URL: https://github.com/apache/texera/pull/8262#discussion_r4059712761
##########
amber/src/test/python/pytexera/storage/test_large_binary_manager.py:
##########
@@ -128,6 +129,33 @@ def
test_ensure_bucket_exists_creates_bucket_when_missing(self):
mock_client.head_bucket.assert_called_once_with(Bucket="test-bucket")
mock_client.create_bucket.assert_called_once_with(Bucket="test-bucket")
+ def test_ensure_bucket_exists_creates_bucket_on_head_404(self):
Review Comment:
Retitled to feat(pyamber): create missing large binary buckets. Keeping feat
because the generic 404 path never created the bucket. No release backport is
requested.
##########
amber/src/main/python/pytexera/storage/large_binary_manager.py:
##########
@@ -84,7 +85,15 @@ def _ensure_bucket_exists(self, bucket: str):
s3 = self._get_s3_client()
try:
s3.head_bucket(Bucket=bucket)
- except s3.exceptions.NoSuchBucket:
+ except (s3.exceptions.NoSuchBucket, ClientError) as error:
+ if isinstance(error, ClientError) and error.response["Error"][
+ "Code"
+ ] not in (
+ "404",
+ "NoSuchBucket",
+ "NotFound",
+ ):
+ raise
logger.debug(f"Bucket {bucket} not found, creating it")
s3.create_bucket(Bucket=bucket)
Review Comment:
This is a fallback behind FileService. Added coverage and handling for
BucketAlreadyOwnedByYou. BucketAlreadyExists and other creation errors still
propagate.
##########
amber/src/main/python/pytexera/storage/large_binary_manager.py:
##########
@@ -84,7 +85,15 @@ def _ensure_bucket_exists(self, bucket: str):
s3 = self._get_s3_client()
try:
s3.head_bucket(Bucket=bucket)
- except s3.exceptions.NoSuchBucket:
+ except (s3.exceptions.NoSuchBucket, ClientError) as error:
+ if isinstance(error, ClientError) and error.response["Error"][
+ "Code"
+ ] not in (
+ "404",
+ "NoSuchBucket",
+ "NotFound",
+ ):
+ raise
Review Comment:
Simplified to except ClientError and replaced the fabricated exception with
botocore's actual NoSuchBucket class. Storage tests pass.
##########
amber/src/test/python/pytexera/storage/test_large_binary_manager.py:
##########
@@ -128,6 +129,33 @@ def
test_ensure_bucket_exists_creates_bucket_when_missing(self):
mock_client.head_bucket.assert_called_once_with(Bucket="test-bucket")
mock_client.create_bucket.assert_called_once_with(Bucket="test-bucket")
+ def test_ensure_bucket_exists_creates_bucket_on_head_404(self):
+ mock_client = MagicMock()
+ mock_client.exceptions.NoSuchBucket = type("NoSuchBucket",
(Exception,), {})
+ mock_client.head_bucket.side_effect = ClientError(
+ {"Error": {"Code": "404", "Message": "Not Found"}}, "HeadBucket"
+ )
+ large_binary_manager._s3_client = mock_client
Review Comment:
Both tests now use monkeypatch so the previous singleton client is restored
afterward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
