[ 
https://issues.apache.org/jira/browse/HDDS-17832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Layam Pavan Venkat reassigned HDDS-17832:
-----------------------------------------

    Assignee: Layam Pavan Venkat

> Recon MPU size totals are not reduced for split-layout MPUs completed or 
> aborted after a Recon restart
> ------------------------------------------------------------------------------------------------------
>
>                 Key: HDDS-17832
>                 URL: https://issues.apache.org/jira/browse/HDDS-17832
>             Project: Apache Ozone
>          Issue Type: Bug
>            Reporter: Layam Pavan Venkat
>            Assignee: Layam Pavan Venkat
>            Priority: Major
>
> {\{MultipartInfoInsightHandler}} (added in HDDS-16363) computes the 
> replicated size of \{{multipartPartsTable}} events using the parent MPU's 
> replication config, because \{{OmMultipartPartInfo}} does not carry one. The 
> config is kept in an in-memory cache keyed by uploadId, filled only from 
> \{{multipartInfoTable}} PUT and UPDATE events.
> After a Recon restart the cache is empty. When an MPU initiated before the 
> restart is completed or aborted, OM deletes the \{{multipartInfoTable}} row 
> before its \{{multipartPartsTable}} rows in the same batch. The DELETE event 
> does not fill the cache, so every part delete logs \{{No cached replication 
> config for upload ...; skipping part size update}}, and the upload's bytes 
> stay in \{{/keys/open/mpu/summary}} until the next full reprocess.
> *Steps to reproduce*
> # Initiate an MPU on a bucket that uses the split parts table layout, and 
> commit a few parts.
> # Restart Recon.
> # Complete or abort the MPU.
> # \{{/keys/open/mpu/summary}} still reports the upload's size.
> *Possible fix*
> Also cache the replication config on the \{{multipartInfoTable}} DELETE event.
> *To review as part of this issue*
> * The cache is bounded at 10,000 uploads (least recently used). On a miss, 
> the part size update is skipped, so the totals drift until the next 
> reprocess. Check whether eviction can drop configs still needed by active 
> uploads, and whether a miss should fall back to looking up the config, or 
> flag the totals for recompute, instead of skipping.
> * Check that every add and remove path (part commit, part re-upload, 
> complete, abort, expired MPU cleanup) finds the config it needs.
> Found during the review of HDDS-16364 
> (https://github.com/apache/ozone/pull/11685).



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to