Re: [Ganglia-developers] gmond python module interface
Hey, Yeah I agree third party daemons are not the way to go long term. And yeah if the python module interface could be improved and then it would likely be easier and more fun to use... Yeah my proposal wasn't really well thought out... The idea seemed more attractive when the daemon was threading... then it would scale well maybe. I hear tell Python threads have some limitations... but for this application it probably wouldn't matter. I think the main limitation I hear about Python threads has to do with the Global Interpretor Lock... Only one thread at a time has access to Python objects... Anyway I might continue to write some modules using gmond interface... But for a simple module I might sometimes use a gmetric script or possibly my proposed daemon. Eventually I want to look at more code and figure out how to use Ganglia better... ~ David On Sat, Jan 31, 2009 at 9:48 PM, Spike Spiegel fsm...@gmail.com wrote: Hi, provided that I haven't had the time to look at this part of the code yet and that I agree it would be much nicer to have a gmetric-like behavior, On Sun, Feb 1, 2009 at 12:21 AM, David Stainton dstainton...@gmail.com wrote: I like using gmetric to monitor... so I wrote gmetric-daemon which is my attempt at a forking standalone daemon which runs Python metric modules and calls gmetric for each metric... in a previous email you call upon a most scalable, most correct and most reliable/highly available design, which is certainly a valuable goal that I don't see met by this proposal. A gmetric-daemon as far as I understand gmetric would defy caching and directives like threshold and timeout, which are very important at least as far as scalability goes. Furthermore as long as there are built-int plugins with collection groups and so on a third party daemon sounds like the wrong approach to me, so as much easier as it might be at first I'd believe that the most scalable, most correct and most reliable design is the one Brad proposes cavia the fact that figuring it all out will take more time. I wanted a slightly different multithreaded approach to monitoring... but it turns out that Python threads really suck. care to share in which way python threads really suck? So I made this a forking daemon. One process per module. Not very memory effecient. But then I don't expect to need many modules... *I* don't? what if somebody else does? what if you do tomorrow/at another job? I don't see how you'd fix something like that at later stage without having to throw everything away. And how does this meet the most scalable design goal? Don't get me wrong, I'm sure everybody agrees on the problems and appreciate the effort, I'm merely pointing out that from my perspective this proposal doesn't meet the design goals and is unlikely to get traction upstream or in the HPC community, even tho it might be just perfect for you and other people. And just in case, I've no affiliation with ganglia and these are my own opinions, maybe upstream folks have completely different thoughts. time and skills permitting I'd be happy to help out with improving the python interface especially since it's something we'd like to heavily leverage at work. thanks -- Behind every great man there's a great backpack - B. -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] gmond python module interface
Hi, provided that I haven't had the time to look at this part of the code yet and that I agree it would be much nicer to have a gmetric-like behavior, On Sun, Feb 1, 2009 at 12:21 AM, David Stainton dstainton...@gmail.com wrote: I like using gmetric to monitor... so I wrote gmetric-daemon which is my attempt at a forking standalone daemon which runs Python metric modules and calls gmetric for each metric... in a previous email you call upon a most scalable, most correct and most reliable/highly available design, which is certainly a valuable goal that I don't see met by this proposal. A gmetric-daemon as far as I understand gmetric would defy caching and directives like threshold and timeout, which are very important at least as far as scalability goes. Furthermore as long as there are built-int plugins with collection groups and so on a third party daemon sounds like the wrong approach to me, so as much easier as it might be at first I'd believe that the most scalable, most correct and most reliable design is the one Brad proposes cavia the fact that figuring it all out will take more time. I wanted a slightly different multithreaded approach to monitoring... but it turns out that Python threads really suck. care to share in which way python threads really suck? So I made this a forking daemon. One process per module. Not very memory effecient. But then I don't expect to need many modules... *I* don't? what if somebody else does? what if you do tomorrow/at another job? I don't see how you'd fix something like that at later stage without having to throw everything away. And how does this meet the most scalable design goal? Don't get me wrong, I'm sure everybody agrees on the problems and appreciate the effort, I'm merely pointing out that from my perspective this proposal doesn't meet the design goals and is unlikely to get traction upstream or in the HPC community, even tho it might be just perfect for you and other people. And just in case, I've no affiliation with ganglia and these are my own opinions, maybe upstream folks have completely different thoughts. time and skills permitting I'd be happy to help out with improving the python interface especially since it's something we'd like to heavily leverage at work. thanks -- Behind every great man there's a great backpack - B. -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] gmond python module interface
On 1/28/2009 at 7:59 AM, in message 20090128145933.ga19...@imperial.ac.uk,
Kostas Georgiou k.georg...@imperial.ac.uk wrote:
On Wed, Jan 28, 2009 at 06:09:48AM -0800, Gilad Raphaelli wrote:
I think this is a well thought out email and I'm a little surprised at
the lack of response to it. Is it because no one is actually using
the gmond python module interface and hasn't had to make these types
of decisions? I don't have a single gmetric script/module that only
collects one metric and have used both the mutli-threaded collector
approach and the single-thread hopeful/naive approach (having written
the mysql metric module you mention). I agree that the multi-threaded
design seems less prone to problems but in practice I haven't had any
problems either way. That being said, I will be trying some of your
suggestions or moving to threads for that mysql module when time
permits.
You could also modify gmond to run each plugin in a new thread, no sure
if it is easy/possible though since I haven't looked at that part of the
code yet.
My frustration with the gmond python module interface is that it's not
actually a complete replacement for gmetric scripts as I use them.
Needing to know all of the metrics that a module will report before
runtime makes for a lot of upfront work creating .pyconf files and
doesn't allow for adding new metrics without restarting gmond. Being
able to deploy one gmetric script that conditionally reports gmetrics
based on what's running/hardware installed/etc on a box is a big
advantage for a gmetric script over conditionally generating pyconf
files and then still having to conditionally collect metrics in the
actual gmond module. I expect most users just stick with the gmetric
script in this case and handle scheduling themselves?
I agree here, my suggestion in the Wildcard Configuration was to add a
metric_autoconf function in the plugin and get gmond to use that to get
the collection_group from there. In any case for the modules that I
develop I call this function from main so I can do python module.py -t
module.pyconf with a similar effect.
This doesn't solve all problems though, I would like to be able to just
start mysql in a host or add a new disk for example and have metrics for
them without touching the configuration at all. Maybe extending the
configuration so in a .pyconf you can write something like
collection_group {
autoconf = 300
}
to have gmond interogate the plugin every 5 mins to get a new collection
group will be enough, I need to think about it a bit more...
The issue is more about the internal hash tables and arrays that are
initialized at startup directly from the information that is found in the .conf
files. In order to make gmond recognize metrics without them being explicitly
configured in the .conf file, gmond needs to be changed so that the internal
tables can be allocated on the fly. It can certainly be done, it is just a
matter of taking the time to figure it all out and make the changes compatible
with the current functionality.
Brad
--
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] gmond python module interface
On Wed, Jan 28, 2009 at 06:09:48AM -0800, Gilad Raphaelli wrote:
I think this is a well thought out email and I'm a little surprised at
the lack of response to it. Is it because no one is actually using
the gmond python module interface and hasn't had to make these types
of decisions? I don't have a single gmetric script/module that only
collects one metric and have used both the mutli-threaded collector
approach and the single-thread hopeful/naive approach (having written
the mysql metric module you mention). I agree that the multi-threaded
design seems less prone to problems but in practice I haven't had any
problems either way. That being said, I will be trying some of your
suggestions or moving to threads for that mysql module when time
permits.
You could also modify gmond to run each plugin in a new thread, no sure
if it is easy/possible though since I haven't looked at that part of the
code yet.
My frustration with the gmond python module interface is that it's not
actually a complete replacement for gmetric scripts as I use them.
Needing to know all of the metrics that a module will report before
runtime makes for a lot of upfront work creating .pyconf files and
doesn't allow for adding new metrics without restarting gmond. Being
able to deploy one gmetric script that conditionally reports gmetrics
based on what's running/hardware installed/etc on a box is a big
advantage for a gmetric script over conditionally generating pyconf
files and then still having to conditionally collect metrics in the
actual gmond module. I expect most users just stick with the gmetric
script in this case and handle scheduling themselves?
I agree here, my suggestion in the Wildcard Configuration was to add a
metric_autoconf function in the plugin and get gmond to use that to get
the collection_group from there. In any case for the modules that I
develop I call this function from main so I can do python module.py -t
module.pyconf with a similar effect.
This doesn't solve all problems though, I would like to be able to just
start mysql in a host or add a new disk for example and have metrics for
them without touching the configuration at all. Maybe extending the
configuration so in a .pyconf you can write something like
collection_group {
autoconf = 300
}
to have gmond interogate the plugin every 5 mins to get a new collection
group will be enough, I need to think about it a bit more...
Kostas
--
This SF.net email is sponsored by:
SourcForge Community
SourceForge wants to tell your story.
http://p.sf.net/sfu/sf-spreadtheword
___
Ganglia-developers mailing list
Ganglia-developers@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/ganglia-developers
