Re: [Ganglia-developers] [Ganglia-svn] SF.net SVN: ganglia:[1538]trunk/monitor-core/Makefile.am
On Thu, Jul 10, 2008 at 02:12:46PM -0600, Brad Nicholes wrote: On 7/10/2008 at 2:06 PM, in message [EMAIL PROTECTED], Jesse Becker [EMAIL PROTECTED] wrote: On Thu, Jul 10, 2008 at 15:48, Brad Nicholes [EMAIL PROTECTED] wrote: On 7/10/2008 at 12:52 PM, in message [EMAIL PROTECTED], Jesse Becker [EMAIL PROTECTED] wrote: This is exactly what contrib directories are for: things that are useful and worth distributing as a courtesy, but are *not* directly So what does that mean? Should contrib/ be part of the tarballs, snapshots, releases or just an SVN repository location for misc. stuff? Yes. There usually isn't a lot of churn in contrib/ directories, nor are the contents large. Having them in SVN makes sure that they aren't lost, or mistakenly omitted. Putting them in tarballs/releases makes sure that they are distributed (which is the whole point). So are we OK with distributing everything that is currently in contrib/ directory? If it gets voted for backport as explained in the STATUS page for next release (not 3.1.0) Any package from trunk will include contrib in the package since r1538 anyway. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On Thu, Jul 10, 2008 at 11:37:23AM -0700, Bernard Li wrote: Here's the release plan for the upcoming 3.1 release. do you mean 3.1.0? I believe all the important, show-stopping backport proposals in the STATUS file for 3.1 branch have already been processed and voted on. actually there is 1 showstopper reported there which has not yet a resolution, and that is the probable licensing issue between the BSD ganglia-webfrontend and the GPL templatePower class. sadly, I hadn't heard back from Ron (the author of templatePower) on the alternatives we might be able to go with and so can't comment in that. but checking again all legalese that seems to be tied into the files in the web frontend, it might seem we could be OK after all as the terms of the BSD license there (which actually look more like a MIT license to me) seem compatible with the terms of GPLv2. but of course IANAL and we should probably seek advice from one (maybe debian legal or fedora legal could help there). So all the remaining backport proposals will be moved from BACKPORT PROPOSALS to BACKPORT PROPOSALS NEXT VERSION except for documentation patches. 2 of them still needing votes, and most likely some other ones still not proposed for backport or committed and dealing with the issues that we had been saying will be put in documentation like the one proposed below, the upgrading instructions or building/packaging recommendations for CentOS 4 users (including dependencies that are not available in the official repositories). As for... * gmond: avoid latency and timeouts when using the tcpconn python module If this causes issues, we could just turn it off by default and put in documentation about its potential pitfalls on certain platforms. It is definitely unstable and not likely to be fixed before the freeze, so IMHO would be better deleted (not turned off by default) as there is no way to do that reliably in a clean way AFAIK. If we would have contrib for 3.1.0, adding it back there in both versions (the python 2.3 compatible one, and the more reliable python 2.4 compatible version) might be a good idea, so that users can use them and configure them as needed (if they agree to the annoyances/risks), but since that is very likely to delay tagging the beta since today is already the release date proposed, will be most likely better to just cut it clean. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On Friday 11 July 2008 07:02:04 am Carlo Marcelo Arenas Belon wrote: On Thu, Jul 10, 2008 at 11:37:23AM -0700, Bernard Li wrote: Here's the release plan for the upcoming 3.1 release. do you mean 3.1.0? I believe all the important, show-stopping backport proposals in the STATUS file for 3.1 branch have already been processed and voted on. actually there is 1 showstopper reported there which has not yet a resolution, and that is the probable licensing issue between the BSD ganglia-webfrontend and the GPL templatePower class. sadly, I hadn't heard back from Ron (the author of templatePower) on the alternatives we might be able to go with and so can't comment in that. but checking again all legalese that seems to be tied into the files in the web frontend, it might seem we could be OK after all as the terms of the BSD license there (which actually look more like a MIT license to me) seem compatible with the terms of GPLv2. but of course IANAL and we should probably seek advice from one (maybe debian legal or fedora legal could help there). Tom Callaway is Fedora's first line of defense when it comes to licensing questions, and either he knows the answer from looking into tons of package licensing issues already, or knows who to talk to. -- Jarod Wilson [EMAIL PROTECTED] - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On 7/11/2008 at 5:02 AM, in message [EMAIL PROTECTED], Carlo Marcelo Arenas Belon [EMAIL PROTECTED] wrote: On Thu, Jul 10, 2008 at 11:37:23AM -0700, Bernard Li wrote: So all the remaining backport proposals will be moved from BACKPORT PROPOSALS to BACKPORT PROPOSALS NEXT VERSION except for documentation patches. 2 of them still needing votes, and most likely some other ones still not proposed for backport or committed and dealing with the issues that we had been saying will be put in documentation like the one proposed below, the upgrading instructions or building/packaging recommendations for CentOS 4 users (including dependencies that are not available in the official repositories). As for... * gmond: avoid latency and timeouts when using the tcpconn python module If this causes issues, we could just turn it off by default and put in documentation about its potential pitfalls on certain platforms. It is definitely unstable and not likely to be fixed before the freeze, so IMHO would be better deleted (not turned off by default) as there is no way to do that reliably in a clean way AFAIK. Disabling it is just a matter of a file name change from tcpconn.py to tcpconn.pyoff or something like that. The same thing would have to be done for the tcpconn.pyconf file as well (tcpconn.pyoff). I would suggest we just make the file name change and still distribute it for those that want to use it anyway. It still works reliably, it just has a wait timeout issue that is really only noticeable when using the -m parameter. Brad - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On Fri, Jul 11, 2008 at 09:29:06AM -0600, Brad Nicholes wrote: On 7/11/2008 at 5:02 AM, in message [EMAIL PROTECTED], Carlo Marcelo Arenas Belon [EMAIL PROTECTED] wrote: It is definitely unstable and not likely to be fixed before the freeze, so IMHO would be better deleted (not turned off by default) as there is no way to do that reliably in a clean way AFAIK. Disabling it is just a matter of a file name change from tcpconn.py to tcpconn.pyoff or something like that. The same thing would have to be done for the tcpconn.pyconf file as well (tcpconn.pyoff). That is what I meant by not in a clean way, as it will leave dead code around and will get most likely people confused by the funny names and will require them to rename files (which are under a package manager and then will complain as being missing and won't be removed at uninstall time obstructing the removal for other directories as well). in any case, if documented clearly I have no reason to object but that is just because I won't be affected anyway as I don't use our provided RPM packages. but on that line, remember that it might not be implemented the way you envision for all available packages (which is what I meant by unreliably) as the copying of the files is done now by the SPEC and that could result in even more confusion. I would suggest we just make the file name change and still distribute it for those that want to use it anyway. My suggestion was to make a file name change as well into the contrib directory, where it won't get in the way and will be also available for those that want to use it, but since there is no contrib yet distributed then cleanly removing it (it will be available from our repository in the web anyway for whoever wants to install it) looks like the best next option. It still works reliably, it just has a wait timeout issue that is really only noticeable when using the -m parameter. but that would result in some metric samples failing silently and therefore in some wholes in the RRD values that could then result in mysterious drops in the graphs or flat lines. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On Fri, Jul 11, 2008 at 12:10:36PM -0400, Tom spot Callaway wrote: Now, the web front-end is composed of MIT licensed pages and the one GPLv2+ licensed page. MIT and GPLv2+ are compatible, so this is not a problem. In my opinion, the web front-end is not a derived work of the libganglia code (doesn't take code from it, doesn't link to the library), so there is no concern around licensing incompatibility between the ASL 1.1 portion of the libganglia license and the GPLv2+ php page. This is great news, and that is also supported by the fact that the ganglia web frontend was originally and independent package (before 3.0) and so has cleared at least for me any doubts about the legality of distributing it with the upcoming 3.1 release. However, if you disagree and think that the web front-end is a derived work, you would need to either relicense (or replace) the code under the ASL 1.1 license or the GPLv2+ license to resolve the conflict. Probably a nice thing to do for a future release and just so every possible interpretation of our license mix is covered as you suggested. A few additional points worth mentioning: 1. A large chunk of the code in that tarball does not have license attribution in the code itself. The only reliable way to determine the license of code is to have the license attribution in the source file itself (usually in the initial comment header). I would highly encourage you to do this for all of your source code as soon as possible. Remember that code moves often, and people forget what COPYING said (or even which COPYING it came from). Agree, and definitely something I was looking forward to after we are done with this release. 2. You should correct the BSD license references in your code, Agree, using MIT is definitely more accurate, but in our defense BSD is a confusing license name anyway as it can really mean different things, some of which are functional equivalent to a MIT license, like the 2 clause BSD and MIT was after all based in BSD. There is also the fact that this all was started as part of a UC Berkeley project and therefore Matt might had been playing the regents a prank when he used instead a MIT license and put the regent names inside ;) and so, since he is still at shooting distance from Berkeley, calling it BSD helps avoid any animosities directed at him or us. In any case since the original intent was to use a 3 clause BSD license from what I recall and that is functionally equivalent to a MIT license I don't think that to be considered a showstopper anyway but sometime to work for in the near future. it is clearly not BSD licensed (with the exception of the freebsd metrics code). and the other BSD metric code which is also under the Original BSD (AKA 4 clause BSD) license and that we will hopefully replace soon with something more modern as well. Hope that helps, Thanks a lot for your great advice, we surely own you one, and take for granted the next time we meet that beer is on me. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] More info about the tcpconn latency issue...
On Fri, Jul 11, 2008 at 12:24:01PM -0600, Brad Nicholes wrote: On 7/11/2008 at 11:15 AM, in message [EMAIL PROTECTED], Carlo Marcelo Arenas Belon [EMAIL PROTECTED] wrote: I guess I would just rather see it distributed so that the user can decide what they want to do rather than us making the decision for them. and I agree with you on that, the only difference of opinions comes on how to distribute that and if that is feasible now (see below). My suggestion was to make a file name change as well into the contrib directory, where it won't get in the way and will be also available for those that want to use it, but since there is no contrib yet distributed then cleanly removing it (it will be available from our repository in the web anyway for whoever wants to install it) looks like the best next option. I would agree as well if we had a contrib/ directory. But just because we don't should not mean that we remove it completely and make it unavailable for those that would still like to use it. there is also the possibility of just adding the contrib into this first release and using instead that (which should be safe enough) and has been already voted for backport (but for the next release). feel free to commit that then and base disabling this metric / documentation on the contrib directory which should satisfy all raised concerns. if you are going that route, it might be also a good idea to backport including ganglia-rrd-modify.pl into the contrib which has been approved also and was dependent on that first backport. but if you are going that route (and this is where this starts becoming a risky proposition) is that would be also nice to backport the original python 2.4 compatible version which doesn't have the problem the 2.3 compatible version has and that would be a better fit for the majority of the users (except for the ones stuck with python 2.3 like CentOS 4 users and that have other problems getting ganglia running as well, like the lack of an APR1 official package they could use as a dependency), but then that version doesn't exist yet (even if it will be easy to come up with as you explained by rolling back the 2.3 compatibility patches) and hasn't been tested probably as much as the buggy one. It still works reliably, it just has a wait timeout issue that is really only noticeable when using the -m parameter. but that would result in some metric samples failing silently and therefore in some wholes in the RRD values that could then result in mysterious drops in the graphs or flat lines. No and the reason why is because the actual gathering of the data is threaded. tcpconn.py spins up its own gathering thread that periodically exec's netstat and updates an internal array of metrics. When the gmond main thread requests the metrics, all it does is read the internal array and return whatever the last gathered value was. Ok, but then that spawning netstat thread will randomly fail, an so depending on the frequency it fails compared with the polling gmond does you will get flat lines. There is no delay to gmond at all. At worst, the tcpconn gathering thread might delay occasionally which has no effect on anything else. It was written this way on purpose so that gmond would never be at the mercy of the python exec code, netstat delays in execution or OS delays. Good to know, and definitely a sound architectural design. The delay only shows up for gmond when the tcpconn metric_clean() function is called and the main gmond process has to wait for the tcpconn gathering thread to shutdown. That's why you see the delay in with the -m parameter and no where else. Well, as you explained you also see it at shutdown. The gmond -m option causes the metric_init(), which starts the gathering thread and the metric_cleanup() which shuts down the gathering thread, to happen one immediately after the other. Gmond has to delay waiting for the thread cleanup. And this is IMHO a bug, but a fix for it is not something that will be ready to release anytime soon as spelled in the STATUS file. It would be better if the metric_init() doesn't initialize the spawning netstat thread but leave that to the collection method that is scheduled by gmond and who would just need to do the first sample and initialize that thread the first time it is called. This way the metric_cleanup() method won't need to wait either for the `gmond -m` case which shouldn't execute any metric collection code in principle. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list
Re: [Ganglia-developers] Ganglia 3.1.x release plan
On Fri, Jul 11, 2008 at 10:07:08AM -0600, Brad Nicholes wrote: On 7/10/2008 at 12:37 PM, in message [EMAIL PROTECTED], Bernard Li [EMAIL PROTECTED] wrote: Dear all: Here's the release plan for the upcoming 3.1 release. I believe all the important, show-stopping backport proposals in the STATUS file for 3.1 branch have already been processed and voted on. So all the remaining backport proposals will be moved from BACKPORT PROPOSALS to BACKPORT PROPOSALS NEXT VERSION except for documentation patches. As for... * gmond: avoid latency and timeouts when using the tcpconn python module If this causes issues, we could just turn it off by default and put in documentation about its potential pitfalls on certain platforms. Let's get this done by Friday and roll out a beta. We'll test this for a week, and roll out RC1, RC2, etc. etc. I would just like to make a comment about version numbers as we are about to generate our first release of 3.1. I noted this on the wiki several months ago under the section Generating a Release Candidate and GA Release (http://ganglia.wiki.sourceforge.net/ganglia_works) which describes the same release versioning process that the Apache project uses. This also goes back to our discussions about 3.1.0 vs. 3.1.1 version number. right, the first beta that Bernard is going to generate sometime today will be either called 3.1.0 or 3.1.1 (depending on what he decides to do, and which will be most likely 3.1.0 since there shouldn't be any technical reason not to anyway and he expressed several times that is what he wanted to do) since we had been testing snapshots for more than a year, I am pretty sure is going to be rock solid (except of course for the platforms that will have no support and that we are most likely going to have to defer to the next release but will be interesting to test as well, even if that means will need to have unofficial patches applied to them to work for 3.1.0) The Apache project does not use the labels Alpha, Beta, RCx for any of the actual tarball file names or internal version numbers in the source code itself. The only time these labels are used are in the mailing list announcements during the testing period. The reason why these labels are not used in the file name or in the source code is so that a tarball only has to be rolled once and if determined during the testing period to be releasable, no alterations to the actual tarball are made. It is simply released officially. This could be a little confusing, but we agreed to it so be it, hopefully again, since we had been testing this for a long time, the beta won't need to be thrown away but used AS-IS all the way through the RCs and we would make a 3.1.0 official release instead of having to resort into a 3.1.25 like Apache 2.0 did. any one willing to take some bets? If we did, then our first official release would be 3.2.0 rather than 3.1.whatever. My preference would be to stick to the 3.1.x scheme as described in the wiki and the paragraph above. Agree, we could reconsider it when 3.2.0 gets released. Carlo - Sponsored by: SourceForge.net Community Choice Awards: VOTE NOW! Studies have shown that voting for your favorite open source project, along with a healthy diet, reduces your potential for chronic lameness and boredom. Vote Now at http://www.sourceforge.net/community/cca08 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
