We have at Entwine done quite a bit of work towards centralizing Matterhorn logs leveraging one of the available Open source tools, we looked at rsyslog,losgstash,graylog2 and later on kibana. Original choice was Logstash but the fact we had to go to Graylog2 or Kibana for a usable Web UI ultimately was a deal breaker as we picked GrayLog2 which is a more "all in one" solution (message broker,stats,web ui,...). As a matter of fact Graylog2 has a number of libraries available to deal with log4j which is what Matterhorn uses.
Thus in a effort to limit the number of moving parts we are now deploying Graylog2 with a modified version of gelf4j to comply with OSGi model. That said logstash+kibana is very decent setup in particular using redis as the message broker for Logstash worked pretty smoothly during my testing, while setting up graylog2 is..well...how shoud I put it..."challenging". So our setup is "fairly" simple we setup a gelf appender in log4j properties for each Matterhorn node and that appender is talking directly to the Graylog2 server which does all the magic including a pretty decent web UI with all the bells and whistles (streams, stats, alerts,...) Anything "syslog" was discarded because of the dealing with stack trace issues. Also some folks are pointing at the perfomance limits of MongoDB required by graylog2. ************* Jaime Gago Systems Engineer [email protected] @JaimeGagoTech Entwine - Knowledge In Motion www.entwinemedia.com @entwinemedia On Aug 28, 2012, at 1:12 AM, Kristof Keppens wrote: > Hi Stephen, > > We ( Ghent university ) are considering logstash + Kibana for that. Already > did some tests but nothing shareable yet. If we do plan to go ahead with this > we will place the configs etc. on github. If anyone is considering different > solutions we would be interested in hearing them as well. > > Kristof > > On 08/28/2012 09:39 AM, Stephen Marquard wrote: >> Hi all, >> >> Has anyone implemented centralized logging from CAs, i.e. using syslog >> or similar to get the contents of what usually goes into matterhorn.log >> for each capture agent logged on one central server? >> >> If so please could you share configs? >> >> Thanks >> Stephen >> >> >> Stephen Marquard, Learning Technologies Co-ordinator >> Centre for Educational Technology, University of Cape Town >> http://www.cet.uct.ac.za >> Email/IM/XMPP: [email protected] >> Phone: +27-21-650-5037 Cell: +27-83-500-5290 >> >> >> >> >> >> ### >> >> UNIVERSITY OF CAPE TOWN >> >> This e-mail is subject to the UCT ICT policies and e-mail disclaimer >> published on our website at >> http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from >> +27 21 650 9111. This e-mail is intended only for the person(s) to whom >> it is addressed. If the e-mail has reached you in error, please notify >> the author. If you are not the intended recipient of the e-mail you may >> not use, disclose, copy, redirect or print the content. If this e-mail >> is not related to the business of UCT it is sent by the sender in the >> sender's individual capacity. >> >> ### >> >> ________________________________ >> UNIVERSITY OF CAPE TOWN >> >> This e-mail is subject to the UCT ICT policies and e-mail disclaimer >> published on our website at >> http://www.uct.ac.za/about/policies/emaildisclaimer/ or obtainable from +27 >> 21 650 9111. This e-mail is intended only for the person(s) to whom it is >> addressed. If the e-mail has reached you in error, please notify the author. >> If you are not the intended recipient of the e-mail you may not use, >> disclose, copy, redirect or print the content. If this e-mail is not related >> to the business of UCT it is sent by the sender in the sender's individual >> capacity. >> >> _______________________________________________ >> Matterhorn-users mailing list >> [email protected] >> http://lists.opencastproject.org/mailman/listinfo/matterhorn-users > > _______________________________________________ > Matterhorn-users mailing list > [email protected] > http://lists.opencastproject.org/mailman/listinfo/matterhorn-users _______________________________________________ Matterhorn-users mailing list [email protected] http://lists.opencastproject.org/mailman/listinfo/matterhorn-users
