I agree with Jun here, that it would make it easier to do lag checking. However, for individual checks it's really not that much trouble to do the second request. If you're doing a lot of lag checking (like every consumer and every topic) where the scale would start to make a difference, I would argue that you should not be using individual OffsetFetchRequests to do it. You should instead consume the __consumer_offsets topic to get the committed offsets, in which case you're still back to getting the high watermark another way (either through an OffsetRequest or through JMX).
-Todd On Thu, Mar 26, 2015 at 7:54 AM, Jun Rao <j...@confluent.io> wrote: > Grant, > > In addition to FetchRequest, currently we have another way to get the high > watermark through OffsetRequest ( > > https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetRequest > ). > OffsetRequest is a read-only request and is much lighter than FetchRequest. > This is what monitoring tools like ConsumerOffsetChecker is using now. > > By returning the high watermark in the OffsetFetchRequest, we can implement > tools like ConsumerOffsetChecker a bit simpler: instead of making two > requests, the tool just needs to make one request. However, I am not sure > if it makes a big difference. > > Thanks, > > Jun > > > On Tue, Mar 24, 2015 at 8:13 PM, Grant Henke <ghe...@cloudera.com> wrote: > > > Here is an initial proposal to add HighwaterMarkOffset to the > > OffsetFetchResponse: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-17+-+Add+HighwaterMarkOffset+to+OffsetFetchResponse > > > > I can add a jira and more implementation details if the > > initial proposal has interest. > > > > Thanks, > > Grant > > -- > > Grant Henke > > Solutions Consultant | Cloudera > > ghe...@cloudera.com | 920-980-8979 > > twitter.com/ghenke <http://twitter.com/gchenke> | > > linkedin.com/in/granthenke > > >
