[
https://issues.apache.org/jira/browse/HDDS-16363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Prince Raj updated HDDS-16363:
------------------------------
Description:
h2. Description
Recon's storage distribution information does not appear to account for the
storage consumed by parts of an incomplete S3 multipart upload.
When a multipart upload is initiated and one or more parts are uploaded without
completing the MPU, the uploaded parts remain as open/incomplete data and
should be reflected in Recon's {{multipartOpenKeyBytes}} metric.
Currently, {{multipartOpenKeyBytes}} can remain unchanged even after a
multipart upload part has been successfully written.
h3. Expected behavior
For an incomplete multipart upload:
# An MPU is initiated for an object.
# One or more parts are uploaded successfully.
# The MPU is intentionally left incomplete.
# Recon's storage distribution API should reflect the replicated storage
consumed by the uploaded MPU parts in {{{}multipartOpenKeyBytes{}}}.
# Once the corresponding MPU data is cleaned up, {{multipartOpenKeyBytes}}
should decrease accordingly.
For example, if a 16 MiB part is uploaded with RATIS replication factor 3, the
corresponding replicated storage contribution should be approximately:
{{16 MiB × 3 = 50,331,648 bytes}}
The exact value should account for the existing baseline and the configured
replication factor.
h3. Actual behavior
After successfully uploading a part to an incomplete MPU, Recon's
{{multipartOpenKeyBytes}} does not increase as expected and may continue
reporting the previous value.
This suggests that the storage associated with incomplete MPU parts may not be
correctly propagated into Recon's multipart open-key accounting.
h3. Reproduction scenario
A minimal reproduction should:
# Create a bucket and object.
# Initiate a multipart upload.
# Upload at least one part.
# Do not complete the MPU.
# Query Recon's storage distribution information.
# Verify the value reported for {{{}multipartOpenKeyBytes{}}}.
# Clean up the incomplete MPU/container data.
# Query Recon again and verify that the corresponding storage is removed.
h3.
was:
h2. Description
Recon's storage distribution information does not appear to account for the
storage consumed by parts of an incomplete S3 multipart upload.
When a multipart upload is initiated and one or more parts are uploaded without
completing the MPU, the uploaded parts remain as open/incomplete data and
should be reflected in Recon's {{multipartOpenKeyBytes}} metric.
Currently, {{multipartOpenKeyBytes}} can remain unchanged even after a
multipart upload part has been successfully written.
h3. Expected behavior
For an incomplete multipart upload:
# An MPU is initiated for an object.
# One or more parts are uploaded successfully.
# The MPU is intentionally left incomplete.
# Recon's storage distribution API should reflect the replicated storage
consumed by the uploaded MPU parts in {{{}multipartOpenKeyBytes{}}}.
# Once the corresponding MPU data is cleaned up, {{multipartOpenKeyBytes}}
should decrease accordingly.
For example, if a 16 MiB part is uploaded with RATIS replication factor 3, the
corresponding replicated storage contribution should be approximately:
{{16 MiB × 3 = 50,331,648 bytes}}
The exact value should account for the existing baseline and the configured
replication factor.
h3. Actual behavior
After successfully uploading a part to an incomplete MPU, Recon's
{{multipartOpenKeyBytes}} does not increase as expected and may continue
reporting the previous value.
This suggests that the storage associated with incomplete MPU parts may not be
correctly propagated into Recon's multipart open-key accounting.
h3.
> Recon does not report storage used by incomplete multipart uploads in
> multipartOpenKeyBytes
> -------------------------------------------------------------------------------------------
>
> Key: HDDS-16363
> URL: https://issues.apache.org/jira/browse/HDDS-16363
> Project: Apache Ozone
> Issue Type: Bug
> Reporter: Prince Raj
> Assignee: Priyesh Karatha
> Priority: Major
> Labels: pull-request-available
>
> h2. Description
> Recon's storage distribution information does not appear to account for the
> storage consumed by parts of an incomplete S3 multipart upload.
> When a multipart upload is initiated and one or more parts are uploaded
> without completing the MPU, the uploaded parts remain as open/incomplete data
> and should be reflected in Recon's {{multipartOpenKeyBytes}} metric.
> Currently, {{multipartOpenKeyBytes}} can remain unchanged even after a
> multipart upload part has been successfully written.
> h3. Expected behavior
> For an incomplete multipart upload:
> # An MPU is initiated for an object.
> # One or more parts are uploaded successfully.
> # The MPU is intentionally left incomplete.
> # Recon's storage distribution API should reflect the replicated storage
> consumed by the uploaded MPU parts in {{{}multipartOpenKeyBytes{}}}.
> # Once the corresponding MPU data is cleaned up, {{multipartOpenKeyBytes}}
> should decrease accordingly.
> For example, if a 16 MiB part is uploaded with RATIS replication factor 3,
> the corresponding replicated storage contribution should be approximately:
>
> {{16 MiB × 3 = 50,331,648 bytes}}
> The exact value should account for the existing baseline and the configured
> replication factor.
> h3. Actual behavior
> After successfully uploading a part to an incomplete MPU, Recon's
> {{multipartOpenKeyBytes}} does not increase as expected and may continue
> reporting the previous value.
> This suggests that the storage associated with incomplete MPU parts may not
> be correctly propagated into Recon's multipart open-key accounting.
> h3. Reproduction scenario
> A minimal reproduction should:
> # Create a bucket and object.
> # Initiate a multipart upload.
> # Upload at least one part.
> # Do not complete the MPU.
> # Query Recon's storage distribution information.
> # Verify the value reported for {{{}multipartOpenKeyBytes{}}}.
> # Clean up the incomplete MPU/container data.
> # Query Recon again and verify that the corresponding storage is removed.
>
> h3.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
