Re: Total Volume metrics of Kafka

2019-01-16 Thread Amitav Mohanty
; account deletion after the retention period. > > If you want metrics on how much space a topic occupies on disk, I’d > suggest using collectd and this plugin: > https://github.com/HubSpot/collectd-kafka-disk > > — > Peter > > > On Jan 16, 2019, at 1:12 PM, Amitav Mohanty > w

Total Volume metrics of Kafka

2019-01-16 Thread Amitav Mohanty
Hi I am interested in getting total volume of data that a topic ingested in a period of time. Does Kafka collect any such metrics? I check JMX console but I only found rate metrics. Regards, Amitav

Re: measuring incoming bytes

2019-02-06 Thread Amitav Mohanty
Which part? -Amitav On Mon, Feb 4, 2019 at 8:42 PM Christopher Bogan wrote: > Correct > > On Sat, Feb 2, 2019, 10:09 AM Amitav Mohanty wrote: > > > Hi > > > > I am trying to measure incoming bytes over time. I am trying collect the > > following metric an

measuring incoming bytes

2019-02-02 Thread Amitav Mohanty
Hi I am trying to measure incoming bytes over time. I am trying collect the following metric and apply integral function over a set of data points on a time series. kafka.server:type=BrokerTopicMetrics,name=BytesInPerSec,topic=my_topic FifteenMinuteRate It seems that the number I am getting is

Re: measuring incoming bytes

2019-02-04 Thread Amitav Mohanty
15MinuteRate` to find out the total volume > of bytes that came in a topic for last 15 mins by taking diff of the two > points. > > > > On Sat, Feb 2, 2019 at 8:39 PM Amitav Mohanty > wrote: > > > Hi > > > > I am trying to measure incoming bytes over time.

Re: Understanding Kafka controller log

2020-05-20 Thread Amitav Mohanty
t; ever adds up to more than 1 across your cluster, you've got problems). > > > To follow the leader of any given partition, do we need to correlate > across controller logs of all brokers ? > > What do you mean by follow the leader? > > Cheers, > > Liam Clarke-Hutchins

Effect of changing rack id

2020-05-19 Thread Amitav Mohanty
Hey I would like to understand what is the effect of changing the rack ids of the brokers of an existing cluster. Let's say currently the brokers have rack values "even" and "odd". If I change the rack ids to "rack_1" and "rack_2" and do a rolling restart will there be any issues in the following

Understanding Kafka controller log

2020-05-19 Thread Amitav Mohanty
Hi I am trying to understand the right way of viewing controller logs. As the controller logs are written in each broker, do we need to see all of them to know the state of the cluster at any given point in time ? To follow the leader of any given partition, do we need to correlate across