[
https://issues.apache.org/jira/browse/HDDS-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devesh Kumar Singh updated HDDS-10466:
--------------------------------------
Description:
Ozone snapshots can be taken at the bucket level. Currently Recon doesn't show
any snapshot related information. Since Snapshot state includes all of ozone
metadata starting at the given snapshot scope e.g. a bucket snapshot would
include all of the OM metadata pertaining to that bucket. It also includes all
the keys in the given bucket and all the data blocks that belong to those keys.
h4. So Recon can have few tasks to create various summaries related to
snapshots.
* We can calculate how many data blocks are shared across different snapshots.
We can have a quick summary of number of keys
* That are present in only one snapshot
* Are present only in active OM DB
* Are present in 2 or more snapshots
* How much space is tied up with the snapshots.
* Impact of a specific snapshot deletion on available free space in Ozone.
* A summary/distribution of snapshots across a number of buckets.
* A drill down into hot buckets that are consuming a lot of snapshots
* Going forward, we should have an optional field called application-key
during snap-create operation so that we can attribute which application is
creating the snapshot.
* A summary of snapshot create/delete rate. How often the snapshots are
created/destroyed on various buckets.
* A summary of how many pre-calculate snapdiffs are available and how often
precalculated diffs are used. This can be useful in tuning snapdiff related
parameters.
* A summary of how large the snapdiff sizes are on various buckets. This can
be useful in tuning snapdiff parameters.
* Recon job can also auto-tune some of these snapshot/snapdiff parameters
based on summaries calculated.
h5. Solves below user's pain points:
* Bringing snapshots info in recon tells how much space is tied up with
snapshots.
* How much space can be reclaimed on a snapshot deletion.
was:
Ozone snapshots can be taken at the bucket level. Currently Recon doesn't show
any snapshot related information. Since Snapshot state includes all of ozone
metadata starting at the given snapshot scope e.g. a bucket snapshot would
include all of the OM metadata pertaining to that bucket. It also includes all
the keys in the given bucket and all the data blocks that belong to those keys.
h4. So Recon can have few tasks to create various summaries related to
snapshots.
* We can calculate how many data blocks are shared across different snapshots.
We can have a quick summary of number of keys
* That are present in only one snapshot
* Are present only in active OM DB
* Are present in 2 or more snapshots
* How much space is tied up with the snapshots.
* Impact of a specific snapshot deletion on available free space in Ozone.
* A summary/distribution of snapshots across a number of buckets.
* A drill down into hot buckets that are consuming a lot of snapshots
* Going forward, we should have an optional field called application-key
during snap-create operation so that we can attribute which application is
creating the snapshot.
* A summary of snapshot create/delete rate. How often the snapshots are
created/destroyed on various buckets.
* A summary of how many pre-calculate snapdiffs are available and how often
precalculated diffs are used. This can be useful in tuning snapdiff related
parameters.
* A summary of how large the snapdiff sizes are on various buckets. This can
be useful in tuning snapdiff parameters.
* Recon job can also auto-tune some of these snapshot/snapdiff parameters
based on summaries calculated.
h5. Solves customer pain points:
* Bringing snapshots info in recon tells how much space is tied up with
snapshots.
* How much space can be reclaimed on a snapshot deletion.
> Ozone Recon - Bring Ozone Snapshot summaries and related info in Recon
> ----------------------------------------------------------------------
>
> Key: HDDS-10466
> URL: https://issues.apache.org/jira/browse/HDDS-10466
> Project: Apache Ozone
> Issue Type: Improvement
> Components: Ozone Recon
> Reporter: Devesh Kumar Singh
> Priority: Major
>
> Ozone snapshots can be taken at the bucket level. Currently Recon doesn't
> show any snapshot related information. Since Snapshot state includes all of
> ozone metadata starting at the given snapshot scope e.g. a bucket snapshot
> would include all of the OM metadata pertaining to that bucket. It also
> includes all the keys in the given bucket and all the data blocks that belong
> to those keys.
> h4. So Recon can have few tasks to create various summaries related to
> snapshots.
> * We can calculate how many data blocks are shared across different
> snapshots. We can have a quick summary of number of keys
> * That are present in only one snapshot
> * Are present only in active OM DB
> * Are present in 2 or more snapshots
> * How much space is tied up with the snapshots.
> * Impact of a specific snapshot deletion on available free space in Ozone.
> * A summary/distribution of snapshots across a number of buckets.
> * A drill down into hot buckets that are consuming a lot of snapshots
> * Going forward, we should have an optional field called application-key
> during snap-create operation so that we can attribute which application is
> creating the snapshot.
> * A summary of snapshot create/delete rate. How often the snapshots are
> created/destroyed on various buckets.
> * A summary of how many pre-calculate snapdiffs are available and how often
> precalculated diffs are used. This can be useful in tuning snapdiff related
> parameters.
> * A summary of how large the snapdiff sizes are on various buckets. This can
> be useful in tuning snapdiff parameters.
> * Recon job can also auto-tune some of these snapshot/snapdiff parameters
> based on summaries calculated.
>
> h5. Solves below user's pain points:
> * Bringing snapshots info in recon tells how much space is tied up with
> snapshots.
> * How much space can be reclaimed on a snapshot deletion.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
