[
https://issues.apache.org/jira/browse/HDDS-17503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-17503:
----------------------------------
Labels: pull-request-available (was: )
> Recon: report OM pending deletion as purgeable vs snapshot-held using live
> OM data
> -----------------------------------------------------------------------------------
>
> Key: HDDS-17503
> URL: https://issues.apache.org/jira/browse/HDDS-17503
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: Ozone Recon
> Reporter: Priyesh Karatha
> Assignee: Priyesh Karatha
> Priority: Major
> Labels: pull-request-available
>
> /api/v1/pendingDeletion?component=om currently builds its numbers from
> Recon's own copy of the OM DB
> (ReconGlobalMetricsService.calculatePendingSizes()). That data:
> - lags behind OM
> - can't tell purgeable data apart from data still held by snapshots
> OM now offers GetBucketDeletedBytes, which classifies deleted data per bucket
> using the snapshot reclaimable filters.
> Proposed change:
> - New BucketDeletedBytesMetricsService: a singleton that calls
> GetBucketDeletedBytes once per bucket in Recon's bucket table, using a thread
> pool. One BucketDeletedBytesCollectionTask runs per bucket, and the service
> adds up:
> - purgeable bytes and snapshot-held bytes
> - purgeable key and directory counts
> - purgeable key size and purgeable directory size
> - Rate limiting and timeout: collection runs asynchronously and is rate
> limited. New configs:
> - ozone.recon.bucket.deleted.bytes.collection.minimum.api.delay (default
> 30s)
> - ozone.recon.bucket.deleted.bytes.collection.timeout (default 10m)
> - ozone.recon.bucket.deleted.bytes.collection.thread.count
> - Failure handling:
> - A bucket that fails is counted in totalBucketQueriesFailed and left out
> of the totals.
> - If the collection run itself fails, the status is FAILED.
> - If Recon's OM DB copy isn't loaded yet, the request fails and the next
> request retries, instead of staying stuck in NOT_STARTED.
> - /api/v1/pendingDeletion?component=om:
> - Returns 202 with \{status, message} while the status is NOT_STARTED,
> IN_PROGRESS or FAILED.
> - Returns 200 once FINISHED, with totalPurgeableSize,
> totalSnapshotTrappedSize, totalPurgeableKeys, totalPurgeableDirs,
> totalPurgeableKeySize, totalPurgeableDirSize, totalBucketsQueried and
> totalBucketQueriesFailed.
> - Older OM: if OM doesn't report the key/dir split, totalPurgeableKeySize and
> totalPurgeableDirSize are -1.
> - /api/v1/buckets: adds per-bucket trappedBytes and trappedKeys (-1 until
> they are collected) and trappedBytesCollectionStatus.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
