[
https://issues.apache.org/jira/browse/HDDS-816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16688069#comment-16688069
]
Elek, Marton commented on HDDS-816:
-----------------------------------
Didn't check the code just read the proposed approach. I am not against this
approach but -- to be honest -- I don't know how is it better than the
estimated count by rocks db.
With this approach after an unclear shutdown the key count will be invalid
forever and the difference depends from the persisting time period + writing
speed (at the time of the crash).
With the estimated number of the rocksdb the count will be adjusted and exact
after every compaction. Between the compaction it is approximate and the
difference depends from the actual usage pattern.
{quote}
Q: How to know the number of keys stored in a RocksDB database?
A: Use GetIntProperty(cf_handle, “rocksdb.estimate-num-keys") to obtain an
estimated number of keys stored in a column family, or use
GetAggregatedIntProperty(“rocksdb.estimate-num-keys", &num_keys) to obtain an
estimated number of keys stored in the whole RocksDB database.
Q: Why GetIntProperty can only returns an estimated number of keys in a RocksDB
database?
A: Obtaining an accurate number of keys in any LSM databases like RocksDB is a
challenging problem as they have duplicate keys and deletion entries (i.e.,
tombstones) that will require a full compaction in order to get an accurate
number of keys. In addition, if the RocksDB database contains merge operators,
it will also make the estimated number of keys less accurate.
{quote}
https://github.com/facebook/rocksdb/wiki/RocksDB-FAQ
> Create OM metrics for bucket, volume, keys
> ------------------------------------------
>
> Key: HDDS-816
> URL: https://issues.apache.org/jira/browse/HDDS-816
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Bharat Viswanadham
> Assignee: Bharat Viswanadham
> Priority: Major
> Attachments: HDDS-816.00.patch, HDDS-816.01.patch, HDDS-816.03.patch,
> Metrics for number of volumes, buckets, keys.pdf, Proposed Approach.pdf
>
>
> This Jira is used to create the following metrics in Ozone manager.
> # number of volumes
> # number of buckets
> # number of keys
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
