Layam Pavan Venkat created HDDS-17832:
-----------------------------------------

             Summary: Recon MPU size totals are not reduced for split-layout 
MPUs completed or aborted after a Recon restart
                 Key: HDDS-17832
                 URL: https://issues.apache.org/jira/browse/HDDS-17832
             Project: Apache Ozone
          Issue Type: Bug
            Reporter: Layam Pavan Venkat


{\{MultipartInfoInsightHandler}} (added in HDDS-16363) computes the replicated 
size of \{{multipartPartsTable}} events using the parent MPU's replication 
config, because \{{OmMultipartPartInfo}} does not carry one. The config is kept 
in an in-memory cache keyed by uploadId, filled only from 
\{{multipartInfoTable}} PUT and UPDATE events.

After a Recon restart the cache is empty. When an MPU initiated before the 
restart is completed or aborted, OM deletes the \{{multipartInfoTable}} row 
before its \{{multipartPartsTable}} rows in the same batch. The DELETE event 
does not fill the cache, so every part delete logs \{{No cached replication 
config for upload ...; skipping part size update}}, and the upload's bytes stay 
in \{{/keys/open/mpu/summary}} until the next full reprocess.

*Steps to reproduce*
# Initiate an MPU on a bucket that uses the split parts table layout, and 
commit a few parts.
# Restart Recon.
# Complete or abort the MPU.
# \{{/keys/open/mpu/summary}} still reports the upload's size.

*Possible fix*
Also cache the replication config on the \{{multipartInfoTable}} DELETE event.

*To review as part of this issue*
* The cache is bounded at 10,000 uploads (least recently used). On a miss, the 
part size update is skipped, so the totals drift until the next reprocess. 
Check whether eviction can drop configs still needed by active uploads, and 
whether a miss should fall back to looking up the config, or flag the totals 
for recompute, instead of skipping.
* Check that every add and remove path (part commit, part re-upload, complete, 
abort, expired MPU cleanup) finds the config it needs.

Found during the review of HDDS-16364 
(https://github.com/apache/ozone/pull/11685).



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to