Just to add to Jagasish's suggestion - you can configure MetricsSnapshotRecorder to output the metrics into a stream and read them from there if it works better for you.
Boris. On Mon, Nov 16, 2015 at 12:32 PM, Jordan Shaw <jor...@pubnub.com> wrote: > Michael, > I should have added if your using Burrow in the context of samza consumers > it probably won't work because samza does it's own offset tracking (see > checkpoint topics). The messages-behind-high-watermark is probably your > best bet if you just want something out of the box and don't care about lag > time only message count behind. > -Jordan > > On Mon, Nov 16, 2015 at 11:40 AM, Michael Ravits <michaelr...@gmail.com> > wrote: > > > Thanks Jagadish! I'll look further into this. > > > > Jordan, I tested Burrow with 0.8.3-SNAPSHOT and set it to read consumer > > offsets from zookeeper because I assumed that it's the default Kafka > config > > for commiting offsets. Will try again with Burrow set to read from > > __consumer_offsets. > > > > Thanks > > > > On Mon, Nov 16, 2015 at 8:04 PM, Jordan Shaw <jor...@pubnub.com> wrote: > > > > > Michael, > > > It depends on how you define lag. > > > > > > 1) If you define lag as the total number of messages behind then burrow > > is > > > a good tool as long as all your infrastructure is on 0.8.2, it > basically > > > works by inspecting the __consumer_offsets topic which was introduced > in > > > 0.8.2 (they said they were going to support <0.8.2 but i don't think > > that's > > > a thing yet). > > > > > > 2. If you define lag in time units I would recommend sending a > timestamp > > > with the msg and doing some "manual" inspecting on the consumer end. > > > -Jordan > > > > > > On Mon, Nov 16, 2015 at 9:16 AM, Michael Ravits <michaelr...@gmail.com > > > > > wrote: > > > > > > > Hi, > > > > > > > > I'd like to monitor consumer's lag. > > > > Found this tool https://github.com/linkedin/Burrow. > > > > But now realized that Samza is using it's own checkpointing > mechanism. > > > > > > > > So question is what's the best way to monitor whether and how much > the > > > > consumer is lagging? > > > > > > > > On a related subject, I'd also like to monitor throughput per topic > in > > > > terms of messages per second and bytes per second. Should I query > > brokers > > > > periodically, or maybe there is a better way? > > > > > > > > Thanks, > > > > Michael > > > > > > > > > > > > > > > > -- > > > Jordan Shaw > > > Full Stack Software Engineer > > > PubNub Inc > > > 1045 17th St > > > San Francisco, CA 94107 > > > > > > > > > -- > Jordan Shaw > Full Stack Software Engineer > PubNub Inc > 1045 17th St > San Francisco, CA 94107 >
