I am really skeptical about finding the graal of the statistic reporting protocol, I made many attempts and everytime there is some things that works in one case but not in another. Anyway I like the idea :)
The network protocol used by collectd is the cleanest thing I have seen until now and it is well described which is a really good thing: http://collectd.org/wiki/index.php/Binary_protocol I have implemented parsers for it in maybe 3 languages and each time they were really easy to write. The collectd protocol allows: - multiple statistics per network packet - multiple values per statistics (uptime for example could have 3 for 1min, 5min, 15min) - support for doubles and integers (64bits) - extensible - transport of notifications (severity + string) - encryption I started working on zeromq collectd plugin which allowed external applications to send statistics to the daemon but never finished it, I think there is another one somewhere. For the graphing part there are some solutions based on rrd which can be written by collectd. Overall Collectd is the best thing I have seen to collect/store statistics, I am not fully satisfied with it but nothing else come closer to it, just look at the plugin list On 28 May 2012 10:17, Pieter Hintjens <[email protected]> wrote: > Hi Paul, > > This is a great idea. My advice is start simple and add functionality > as needed over time. If you want an easy reusable legal template for > protocols, see the rfc.zeromq.org site. > > -Pieter > > On Sun, May 27, 2012 at 4:38 PM, Paul Colomiets <[email protected]> > wrote: > > Hi, > > > > Every time I develop something using zeromq or libxs I come up with a > > problem of how to report statistics. Pub/Sub pattern is suited for > > statistics reporting very well, but the format of messages is unclear. > > The result is ugly and diverse solutions for reporting real-time > > statistics. > > > > I think the community would benefit from having common format for > > reporting statistics, so that all open-source software could use the > > format. And various plugins can be developed for existing statistics > > and monitoring systems. > > > > > > The protocol should meet the following requirements: > > > > 1. Simple compact format. Probably textual or fixed-size binary. It > > should be easy to craft messages without any dependencies. > > > > 2. Each message should bring compact representation of the type > > information in itself (so that centralized system doesn't need to be > > reconfigured for each new reported value and each software upgrade on > > remote system) > > > > 3. Subscriber should be able to filter statistics data by pub/sub > subscriptions > > > > 4. There should be solution to collect that statistics (draw graphs, > etc.) > > > > > > So I have the following questions: > > > > 1. What formats are available to (re)use? > > > > 2. What protocols and software do you use for your projects? > > > > 3. Would you like to participate in standardization of the protocol? > > > > > > > > At our company we use collectd. And as zeromq plugin is not in the > > mainstream for collectd, and it has complex binary format, we report > > using some custom formats and a special application that translates > > that into collectd protocol for unix sockets (which in turn nice > > textual protocol). Same is done in zerogw (see zerogwstat utility). > > This kind of proxying, and the point #2 from the requirements makes me > > incomfortable. > > > > -- > > Paul > > _______________________________________________ > > zeromq-dev mailing list > > [email protected] > > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
