I would also encourage looking into collectd which has a great JMX input plugin, the java bindings for collectd also allow very easy integration of kafka for the output (an interface with a single method must be implemented, which gets a list of values).
See: * http://collectd.org/wiki/index.php/Plugin:GenericJMX (how to query JMX values) * http://collectd.org/wiki/index.php/Plugin:Java (how to register a kafka plugin) * http://collectd.org/documentation/javadoc/ (java interfaces to implement) * http://collectd.org (project home page, much more than jmx can be fed to kafka) On Mon, Apr 16, 2012 at 5:16 AM, liu brent <liubaoc...@gmail.com> wrote: > Thanks guys, I think tcollector meets our needs now. :) > > On Tue, Apr 10, 2012 at 12:43 AM, Dave Barr <dave.b...@gmail.com> wrote: > >> We have an agent (tcollector) that collects various metrics and sends >> them to OpenTSDB. We then have a simple python script that lets you >> query OpenTSDB for a given metric and alert on a threshold. We >> currently gather about 1300 metrics and about 10000 datapoints/sec. >> >> https://github.com/stumbleupon/tcollector >> http://opentsdb.net/nagios.html >> >> We're just starting to use Kafka here. I'm not aware yet of a Kafka >> collector, but I'm sure one will be coming soon. They're really >> simple to write. You can look at either the elasticsearch collector >> or the HBase collector as an example of different styles. The HBase >> one forks a jmx process to read metrics from the running JVM, the >> elasticsearch one queries the running process for metrics via an >> ElasticSearch endpoint. >> >> --Dave >> On Mon, Apr 9, 2012 at 12:30 AM, liu brent <liubaoc...@gmail.com> wrote: >> > Hi, >> > Currently, there are 20+ kinds of web service running on hundreds of >> > machines in 4 data centers, which are written in java. We want to collect >> > and store the statics of JVM, MBeans of JMX, cpu usage etc of the service >> > and machines, and then send the waring to Nagios. Does anyone has the >> > experience in using Kafka to do so? >> > The direct solution seems like that we install a client on every >> machine, >> > which works as JMX client, collect the information and work as kafka >> > producer. Anyone has better ideas or suggestions. >> > >> > Thanks, >> > Liu >>
