Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-04-02 Thread Adrian Sevcenco
On 03/27/2014 10:07 PM, Daniel Pocock wrote: I made up a rough diagram about how Ganglia 4.x could look: https://raw.githubusercontent.com/ganglia/monitor-core/master/doc/planning/ganglia-4.x.png The biggest change is the introduction of MongoDB Instead of having the gmetad serve up

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Maxime Brugidou
I don't understand why all this is necessary. I strongly disagree with the horizontal scalability of mongoDB ( i run a very large mongodb cluster in production in addition to other databases) and would rather suggest a pluggable backend with a simpler default (like text files or maybe postgresql

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Alexander Karner
mobility? Mit freundlichen Grüßen / Kind regards Alexander Karner From: Daniel Pocock dan...@pocock.com.au To: ganglia-general@lists.sourceforge.net, Date: 27.03.2014 21:08 Subject:[Ganglia-general] Ganglia 4.x architecture planning I made up a rough diagram about how

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Daniel Pocock
On 28/03/14 09:07, Alexander Karner wrote: Hi! I think we should continue to put an emphasis on portability: Ganglia is not only used in Linux environments but also on AIX, HP-UX, Solaris etc. This includes both, gmond and gmetad (+webserver). In the earlier reply from Adam, the idea of a

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Daniel Pocock
On 27/03/14 21:43, Alex Dean wrote: On Mar 27, 2014, at 3:07 PM, Daniel Pocock dan...@pocock.com.au wrote: The introduction of RabbitMQ is an optional dependency. It would allow users to send commands from the web interface. 1. Why add the extra dependency on rabbitmq? As long as you're

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Michael Perzl
Hi Daniel, with the introduction of MongoDB you would exclude all big-endian architectures immediately as MongoDB is little-endian only. Although there has been big-endian support requested for MongoDB and some attempts have been made in this direction but at the moment MongoDB is still

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Daniel Pocock
On 28/03/14 11:41, Michael Perzl wrote: Hi Daniel, with the introduction of MongoDB you would exclude all big-endian architectures immediately as MongoDB is little-endian only. Although there has been big-endian support requested for MongoDB and some attempts have been made in this direction

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Daniel Pocock
On 28/03/14 14:39, Aaron Nichols wrote: On Fri, Mar 28, 2014 at 1:45 AM, Maxime Brugidou maxime.brugi...@gmail.com mailto:maxime.brugi...@gmail.com wrote: I don't understand why all this is necessary. I strongly disagree with the horizontal scalability of mongoDB ( i run a very

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Khrist Hansen
mobility? Mit freundlichen Grüßen / Kind regards Alexander Karner From:Daniel Pocock dan...@pocock.com.au To:ganglia-general@lists.sourceforge.net, Date:27.03.2014 21:08 Subject:[Ganglia-general] Ganglia 4.x architecture planning _ I made

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Vladimir Vuksan
We may want to clarify what services from Mongo are we gonna use. Is the intention to use it as a key/value store or use it for things like capped collections (http://docs.mongodb.org/manual/tutorial/use-capped-collections-for-fast-writes-and-reads/). Currently

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Aaron Nichols
On Fri, Mar 28, 2014 at 1:45 AM, Maxime Brugidou maxime.brugi...@gmail.comwrote: I don't understand why all this is necessary. I strongly disagree with the horizontal scalability of mongoDB ( i run a very large mongodb cluster in production in addition to other databases) and would rather

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-28 Thread Aaron Nichols
On Fri, Mar 28, 2014 at 7:51 AM, Daniel Pocock dan...@pocock.com.au wrote: That has already been discussed in the thread - are you OK with the plugin idea? I think so - having an easy built-in default would be ideal. Knowing you have 180k metrics is only one factor How many users (both

[Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Daniel Pocock
I made up a rough diagram about how Ganglia 4.x could look: https://raw.githubusercontent.com/ganglia/monitor-core/master/doc/planning/ganglia-4.x.png The biggest change is the introduction of MongoDB Instead of having the gmetad serve up an XML every time somebody asks to see the web page,

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Daniel Pocock
On 27/03/14 21:16, Adam Compton wrote: I'm in favor of teaching gmetad how to send the metrics it collects to a wider variety of things, particularly if there's a plugin interface for writing them. That is how rsyslog does it actually - MongoDB is just one of their output modules, called

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Adam Compton
On 3/27/14 1:20 PM, Daniel Pocock wrote: On 27/03/14 21:16, Adam Compton wrote: I'm in favor of teaching gmetad how to send the metrics it collects to a wider variety of things, particularly if there's a plugin interface for writing them. That is how rsyslog does it actually - MongoDB is

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Adam Compton
I'm in favor of teaching gmetad how to send the metrics it collects to a wider variety of things, particularly if there's a plugin interface for writing them. - Adam On 3/27/14 1:07 PM, Daniel Pocock wrote: I made up a rough diagram about how Ganglia 4.x could look:

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Alex Dean
On Mar 27, 2014, at 3:07 PM, Daniel Pocock dan...@pocock.com.au wrote: The introduction of RabbitMQ is an optional dependency. It would allow users to send commands from the web interface. 1. Why add the extra dependency on rabbitmq? As long as you're adding a persistent data store (mongo),

Re: [Ganglia-general] Ganglia 4.x architecture planning

2014-03-27 Thread Bernard Li
Hi Adam: This already exists as gmetad-python -- I have been told that it's not as scalable as gmetad-C but I would really like someone to pick it up and run with it: https://github.com/ganglia/monitor-core/tree/master/gmetad-python Cheers, Bernard On Thu, Mar 27, 2014 at 1:16 PM, Adam