[
https://issues.apache.org/jira/browse/HDDS-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16685740#comment-16685740
]
Arpit Agarwal commented on HDDS-816:
------------------------------------
Approach #4 (traverse all keys) cannot work in practice. It may take many hours
to iterate over billions of keys.
I am also hesitant about approach #3 without knowing the implementation details
of how the estimate is done. If RocksDB is simply counting number of writes
into the log then the estimate could be very wrong e.g. if there are frequent
overwrites. LSMs typically do not know if a putKey is an overwrite or not when
the operation happens.
I like Bharat's proposed approach. Let's just persist the current key count
periodically to a separate file (don't store it in RocksDB), and also on
shutdown. This let's us limit the 'staleness' of the metric to a few
seconds/minutes.
> Create OM metrics for bucket, volume, keys
> ------------------------------------------
>
> Key: HDDS-816
> URL: https://issues.apache.org/jira/browse/HDDS-816
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Attachments: HDDS-816.00.patch, Metrics for number of volumes,
> buckets, keys.pdf
>
>
> This Jira is used to create the following metrics in Ozone manager.
> # number of volumesÂ
> # number of buckets
> # number of keys
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
