Guozhang Wang commented on KAFKA-6498:

[~james.c] Thanks for your interest and sorry for late response. Let me 
summarize the scope of this ticket before you commit to contributing on it.

RocksDB JNI's Statistics object contains multiple metrics, and moving forward 
it may add more. So to maintain compatibility, we need to do two things:

1) Select what metrics (a subset of all metrics from Statistics) to be exposed 
directly via Streams metrics as built-in streams metrics; note we cannot just 
enumerate over all metrics and add them all into built-in metrics, since there 
will just be more and more of them
2) More importantly, design a set of interfaces to allow users to manually add 
more rocksDB metrics in a custom way (in StreamMetrics we already have these 
addXXXSensor to let users to build their own metrics, the question then is how 
to allow users to hook up more rocksDB metrics)

So this ticket involves design and implementation, not merely straight forward 
implementations. And the contributor is supposed to write a KIP about his 
design on this. If you are interested, could you start by doing the research on 
the above mentioned interfaces and write a KIP proposing your design?

> Add RocksDB statistics via Streams metrics
> ------------------------------------------
>                 Key: KAFKA-6498
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6498
>             Project: Kafka
>          Issue Type: Improvement
>          Components: metrics, streams
>            Reporter: Guozhang Wang
>            Assignee: james chien
>            Priority: Major
>              Labels: needs-kip
> RocksDB's own stats can be programmatically exposed via 
> {{Options.statistics()}} and the JNI `Statistics` has indeed implemented many 
> useful settings already. However these stats are not exposed directly via 
> Streams today and hence for any users who wants to get access to them they 
> have to manually interact with the underlying RocksDB directly, not through 
> Streams.
> We should expose such stats via Streams metrics programmatically for users to 
> investigate them without trying to access the rocksDB directly.

This message was sent by Atlassian JIRA

Reply via email to