Jayesh, Most of the metadata you mentioned already exist. For example, ISR is already in the TopicMetadataResponse and the high water mark can be returned from the OffsetRequest. The details of the current wire protocol can be found at https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol
We have another KIP ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-24+-+Remove+ISR+information+from+TopicMetadataRequest+and+add+broker+level+metadata+request) to add a new request for collecting some broker level info. There will be a discussion thread on this at some point that you can chime in. Thanks, Jun On Wed, Jun 10, 2015 at 8:42 AM, Jayesh Thakrar <j_thak...@yahoo.com.invalid > wrote: > Our company (Conversant Media) became a Kafka user pretty recently (about > 3 months ago) and I have been following the user and dev group for the past > month, so could have missed the discussions on KIP-4. > In looking up the current command line capabilities in Kafka 0.8.2.1 (not > the latest release) it seems we are missing a couple of pieces. > Ideally, I am looking at getting the following kind of information > Describe a "Kafka cluster" (example) -kafka cluster metadata = { > "brokers" : [ { "id" : 0, attributes from broker Json schema}, { "id" : > 1, attributes from broker Json schema}, ...] "topics" : [ { "name" : > "topic1", attributes from topic and parition and ISR and leader information > }, { "name" : "topic2", attributes from topic and parition and ISR and > leader information } ] } > > Topic metrics = atleast some kind of message high water mark from the > leaders for each topic partition > Describe consumers (example) -Consumer metadata = { [ "group" : > "group_id1", "client" : "client_id1", [ list of subscribed topics ] ], > ......} > Having such information will make it easy to build monitoring tools and > also aid troubleshooting and sizing/configuration. > Is discussing these things as part of KIP the right process or is there > some other forum? > Thanks,Jayesh Thakrar > > > From: Gwen Shapira <gshap...@cloudera.com> > To: "dev@kafka.apache.org" <dev@kafka.apache.org> > Sent: Tuesday, June 9, 2015 1:19 PM > Subject: KIP Call Notes > > Shortest KIP discussion ever :) > > Here are my notes. Feel free to correct or expand: > > SSL: Harsh is rebasing after Kafka-1928, and resolving an issue with > support for specific API > > KIP-4 - ISR in Metadata request: We prefer to preserve this information. > > KIP-25 - System Tests: Geoffrey posted an update with discussion > summary, there were no objections to the proposal. We will wait a day > before starting a vote. > > New consumer - Separate the API changes for metadata to a new JIRA. > Since the new consumer is still work in progress, there's no need for > KIP for every modification. > > > >
