Peter, Thanks for the inputs. I am interested in aggregate bytes published into a topic. The approach of metrics collector along with graphing tool seems appealing. I can volume ingested over arbitrary periods of time which is exactly what I am looking for. Can you please point to some metrics collector that I can use? Is it sort of a cron-job that notes the rate every minute or every 15 mins?
Regards, Amitav On Thu, Jan 17, 2019 at 3:23 AM Peter Bukowinski <pmb...@gmail.com> wrote: > Amitav, > > When you say total volume, do you want a topic’s size on disk, taking into > account replication and retention, or do you want the aggregate bytes > published into a topic? If you have a metrics collector and a graphing tool > such as grafana, you can transform the rate metrics to a byte sum by > applying an integral function, but those will always grow and not take into > account deletion after the retention period. > > If you want metrics on how much space a topic occupies on disk, I’d > suggest using collectd and this plugin: > https://github.com/HubSpot/collectd-kafka-disk > > — > Peter > > > On Jan 16, 2019, at 1:12 PM, Amitav Mohanty <amitavmohant...@gmail.com> > wrote: > > > > Hi > > > > I am interested in getting total volume of data that a topic ingested in > a > > period of time. Does Kafka collect any such metrics? I check JMX console > > but I only found rate metrics. > > > > Regards, > > Amitav > >
