[
https://issues.apache.org/jira/browse/ASTERIXDB-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ritik Raj updated ASTERIXDB-3552:
---------------------------------
Description:
All major object storage like S3, GCS and Azure Blob Storage provide an API for
performing a batch operation which can be clubbed in a single request, the
catch with the API is unless the request payload is malformed, it will give
status code 200 every time,
even if some or all the clubbed requests failed on the object storage server.
[GCS batch
Operations|https://cloud.google.com/storage/docs/batch#:~:text=If%20the%20overall%20request%20isn%27t%20formatted%20correctly%20and%20Cloud%20Storage%20is%20unable%20to%20parse%20it%20into%20sub%2Drequests%2C%20you%20receive%20a%20400%20error.%20Otherwise%2C%20Cloud%20Storage%20returns%20a%20200%20status%20code%2C%20even%20if%20some%20or%20all%20of%20the%20sub%2Drequests%20fail]
[AWS S3 Batch Delete
Objects|https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_RequestSyntax:~:text=The%20request%20can,result%20as%20deleted]
[Azure Blob Batch
Operations|https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.specialized.blobbatchclient.deleteblobs?view=azure-dotnet]
So in batch requests, the API responds to the status of each request in the
response except in the case of S3 where it gives a list of the encountered
errors. We should check the individual response and fail if either of them
fails.
was:
All major object storage like S3, GCS and Azure Blob Storage provide an api for
performing batch operation which can be clubbed in a single request, the catch
with the api is unless the request payload is malformed, it will give status
code 200 everytime,
even if some or all the clubbed requested failed on the object storage server.
[GCS batch
Operations|https://cloud.google.com/storage/docs/batch#:~:text=If%20the%20overall%20request%20isn%27t%20formatted%20correctly%20and%20Cloud%20Storage%20is%20unable%20to%20parse%20it%20into%20sub%2Drequests%2C%20you%20receive%20a%20400%20error.%20Otherwise%2C%20Cloud%20Storage%20returns%20a%20200%20status%20code%2C%20even%20if%20some%20or%20all%20of%20the%20sub%2Drequests%20fail]
[AWS S3 Batch Delete
Objects|https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_RequestSyntax:~:text=The%20request%20can,result%20as%20deleted]
[Azure Blob Batch
Operations|https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.specialized.blobbatchclient.deleteblobs?view=azure-dotnet]
so in batch request, the api responds the status of each request in the
response except in case of S3 where it gives a list consisting of the
encountered errors. We should check the individual response and fail if either
of them fails.
> Missing status check on cloud deletes
> -------------------------------------
>
> Key: ASTERIXDB-3552
> URL: https://issues.apache.org/jira/browse/ASTERIXDB-3552
> Project: Apache AsterixDB
> Issue Type: Bug
> Components: STO - Storage
> Affects Versions: 0.9.10
> Reporter: Ritik Raj
> Priority: Critical
> Fix For: 0.9.10
>
>
> All major object storage like S3, GCS and Azure Blob Storage provide an API
> for performing a batch operation which can be clubbed in a single request,
> the catch with the API is unless the request payload is malformed, it will
> give status code 200 every time,
> even if some or all the clubbed requests failed on the object storage server.
> [GCS batch
> Operations|https://cloud.google.com/storage/docs/batch#:~:text=If%20the%20overall%20request%20isn%27t%20formatted%20correctly%20and%20Cloud%20Storage%20is%20unable%20to%20parse%20it%20into%20sub%2Drequests%2C%20you%20receive%20a%20400%20error.%20Otherwise%2C%20Cloud%20Storage%20returns%20a%20200%20status%20code%2C%20even%20if%20some%20or%20all%20of%20the%20sub%2Drequests%20fail]
> [AWS S3 Batch Delete
> Objects|https://docs.aws.amazon.com/AmazonS3/latest/API/API_DeleteObjects.html#API_DeleteObjects_RequestSyntax:~:text=The%20request%20can,result%20as%20deleted]
> [Azure Blob Batch
> Operations|https://learn.microsoft.com/en-us/dotnet/api/azure.storage.blobs.specialized.blobbatchclient.deleteblobs?view=azure-dotnet]
> So in batch requests, the API responds to the status of each request in the
> response except in the case of S3 where it gives a list of the encountered
> errors. We should check the individual response and fail if either of them
> fails.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
