symious commented on code in PR #3875:
URL: https://github.com/apache/ozone/pull/3875#discussion_r1045097934
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -61,22 +69,70 @@ public class RocksDBStoreMBean implements DynamicMBean,
MetricsSource {
LoggerFactory.getLogger(RocksDBStoreMBean.class);
public static final String ROCKSDB_CONTEXT_PREFIX = "Rocksdb_";
+ public static final String ROCKSDB_PROPERTY_PREFIX = "rocksdb.";
+
+ // RocksDB properties
+ private final String[][] cfPros = {
+ // 1 if a memtable flush is pending; otherwise, returns 0
+ {"mem-table-flush-pending", "false", ""},
+ // estimated total number of bytes compaction needs to rewrite to get
+ // all levels down to under target size.
+ {"estimate-pending-compaction-bytes", "true", ""},
+ // 1 if at least one compaction is pending; otherwise, returns 0.
+ {"compaction-pending", "false", ""},
+ // block cache capacity
+ {"block-cache-capacity", "true", ""},
+ // the memory size for the entries residing in block cache
+ {"block-cache-usage", "true", ""},
+ // the memory size for the entries being pinned
+ {"block-cache-pinned-usage", "true", ""},
+ // number of level to which L0 data will be compacted.
+ {"base-level", "false", ""},
+ // approximate active mem table size (bytes)
+ {"cur-size-active-mem-table", "true", ""},
+ // approximate size of active and unflushed immutable (bytes)
+ {"cur-size-all-mem-tables", "true", ""},
+ // approximate size of active, unflushed immutable, and pinned immutable
+ // memtables (bytes)
+ {"size-all-mem-tables", "true", ""},
+ // number of immutable memtables that have not yet been flushed
+ {"num-immutable-mem-table", "true", ""},
+ // total size (bytes) of all SST files belong to the latest LSM tree
+ {"live-sst-files-size", "true", ""},
+ // estimated number of total keys in memtables and storage(Can be very
+ // wrong according to RocksDB document)
+ {"estimate-num-keys", "true", ""},
+ // estimated memory used for reading SST tables, excluding memory used
+ // in block cache (e.g., filter and index blocks)
+ {"estimate-table-readers-mem", "true", ""}
+ };
- public RocksDBStoreMBean(Statistics statistics, String dbName) {
+ // level-x sst file info (Global)
+ private static final String NUM_FILES_AT_LEVEL = "num_files_at_level";
+ private static final String SIZE_AT_LEVEL = "size_at_level";
+
+ public RocksDBStoreMBean(Statistics statistics, RocksDatabase db,
+ String dbName) {
this.contextName = ROCKSDB_CONTEXT_PREFIX + dbName;
this.statistics = statistics;
+ this.rocksDB = db;
histogramAttributes.add("Average");
histogramAttributes.add("Median");
histogramAttributes.add("Percentile95");
histogramAttributes.add("Percentile99");
histogramAttributes.add("StandardDeviation");
+ histogramAttributes.add("Max");
+
+ // To get the metric name complied with prometheus naming rule
+ for (String[] property : cfPros) {
+ property[2] = property[0].replace("-", "_");
Review Comment:
The metrics of `HistogramData` and `TickerTypeData` are in upper-case,
should we also keep the format for the new metrics?
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -61,22 +69,70 @@ public class RocksDBStoreMBean implements DynamicMBean,
MetricsSource {
LoggerFactory.getLogger(RocksDBStoreMBean.class);
public static final String ROCKSDB_CONTEXT_PREFIX = "Rocksdb_";
+ public static final String ROCKSDB_PROPERTY_PREFIX = "rocksdb.";
+
+ // RocksDB properties
Review Comment:
I think it might be more readable to add some comments showing the structure
of the `String[][]`, for example, column 1 means ..., column 2 means.... .
##########
hadoop-hdds/framework/src/main/java/org/apache/hadoop/hdds/utils/RocksDBStoreMBean.java:
##########
@@ -179,6 +234,7 @@ public void getMetrics(MetricsCollector metricsCollector,
boolean b) {
MetricsRecordBuilder rb = metricsCollector.addRecord(contextName);
getHistogramData(rb);
getTickerTypeData(rb);
+ getDBPropertyData(rb);
Review Comment:
When `ozone.metastore.rocksdb.statistics` is set to `OFF` (by default),
these metrics won't be available.
Understand that `HistogramData` and `TickerTypeData` are retrieved from
`rocksdb.statistics`, so when config if set to "OFF", metrics won't be visible.
But seems `DBPropertyData` are independent, should we skip the check on the
config?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
