Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Hi Bernard, as I said, if nobody is hurt :-) Being curious I did a quick check on my favourite distribution (FC4, yum-uptodate): - apr/apr-devel is 0.9.6-3.5, which is older that what we ship (0.9.7) - expat/expat-devel is 1.95.8-6, which is better thatn what we ship (1.95.1) - libconfuse/libconfuse-devel does not seem to exist. Only RPMs I found are for Mandrake/Mandriva and are version 2.1, which is way older than what we ship (2.5) Seems we are safe for expat, need to check apr and are kind of lost for libconfuse. Cheers Martin --- Bernard Li [EMAIL PROTECTED] wrote: I think most people like this idea (Martin, Matt??) - so I think we can start thinking about how to modify the configure/make system. Jarod, have you already done some work on this front? Thanks, Bernard -Original Message- From: [EMAIL PROTECTED] on behalf of Jarod Wilson Sent: Mon 14/08/2006 10:42 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On Sat, 2006-08-12 at 15:41 -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Just to confirm, that works nicely for Red Hat and Fedora too. -- Jarod Wilson [EMAIL PROTECTED] - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Wed, 2006-08-16 at 06:13 -0700, Martin Knoblauch wrote: Hi Bernard, as I said, if nobody is hurt :-) Being curious I did a quick check on my favourite distribution (FC4, yum-uptodate): - apr/apr-devel is 0.9.6-3.5, which is older that what we ship (0.9.7) - expat/expat-devel is 1.95.8-6, which is better thatn what we ship (1.95.1) - libconfuse/libconfuse-devel does not seem to exist. Only RPMs I found are for Mandrake/Mandriva and are version 2.1, which is way older than what we ship (2.5) Seems we are safe for expat, need to check apr and are kind of lost for libconfuse. I can see about packaging the latest libconfuse for Fedora. However, without a special exception, it won't be built for FC4, as FC4 has entered 'maintenance mode' (meaning generally nothing new gets built, only fixes for existing packages allowed). [...time passes...] Almost there with a package that I think will pass Fedora Extras muster... -- Jarod Wilson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
--- Jarod Wilson [EMAIL PROTECTED] wrote: On Wed, 2006-08-16 at 06:13 -0700, Martin Knoblauch wrote: Hi Bernard, as I said, if nobody is hurt :-) Being curious I did a quick check on my favourite distribution (FC4, yum-uptodate): - apr/apr-devel is 0.9.6-3.5, which is older that what we ship (0.9.7) - expat/expat-devel is 1.95.8-6, which is better thatn what we ship (1.95.1) - libconfuse/libconfuse-devel does not seem to exist. Only RPMs I found are for Mandrake/Mandriva and are version 2.1, which is way older than what we ship (2.5) Seems we are safe for expat, need to check apr and are kind of lost for libconfuse. I can see about packaging the latest libconfuse for Fedora. However, without a special exception, it won't be built for FC4, as FC4 has entered 'maintenance mode' (meaning generally nothing new gets built, only fixes for existing packages allowed). [...time passes...] Almost there with a package that I think will pass Fedora Extras muster... cool :-) But given the fact that libconfuse RPMs are really really rare, it was not was I were asking for. -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Wed, 2006-08-16 at 08:34 -0700, Martin Knoblauch wrote: --- Jarod Wilson [EMAIL PROTECTED] wrote: Seems we are safe for expat, need to check apr and are kind of lost for libconfuse. I can see about packaging the latest libconfuse for Fedora. However, without a special exception, it won't be built for FC4, as FC4 has entered 'maintenance mode' (meaning generally nothing new gets built, only fixes for existing packages allowed). [...time passes...] Almost there with a package that I think will pass Fedora Extras muster... cool :-) But given the fact that libconfuse RPMs are really really rare, it was not was I were asking for. How about just build against distro-provided libconfuse if it exists, use included version if it doesn't? More or less what Marcus said: i can tell you that there should be at least an option to say --with-$lib=/usr so it will use the system wide copy. -- Jarod Wilson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Wed, 2006-08-16 at 10:16 -0400, Jarod Wilson wrote: I can see about packaging the latest libconfuse for Fedora. However, without a special exception, it won't be built for FC4, as FC4 has entered 'maintenance mode' (meaning generally nothing new gets built, only fixes for existing packages allowed). [...time passes...] Almost there with a package that I think will pass Fedora Extras muster... Just submitted it for Fedora Extras review: https://bugzilla.redhat.com/202820/ -- Jarod Wilson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Says 404 for me... BTW, I don't suppose there'll be any chance for it to get into RHEL5...? Cheers, Bernard -Original Message- From: [EMAIL PROTECTED] on behalf of Jarod Wilson Sent: Wed 16/08/2006 09:28 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shippedwith ganglia On Wed, 2006-08-16 at 10:16 -0400, Jarod Wilson wrote: I can see about packaging the latest libconfuse for Fedora. However, without a special exception, it won't be built for FC4, as FC4 has entered 'maintenance mode' (meaning generally nothing new gets built, only fixes for existing packages allowed). [...time passes...] Almost there with a package that I think will pass Fedora Extras muster... Just submitted it for Fedora Extras review: https://bugzilla.redhat.com/202820/ -- Jarod Wilson [EMAIL PROTECTED]
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Sat, 2006-08-12 at 15:41 -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Just to confirm, that works nicely for Red Hat and Fedora too. -- Jarod Wilson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
I think most people like this idea (Martin, Matt??) - so I think we can start thinking about how to modify the configure/make system. Jarod, have you already done some work on this front? Thanks, Bernard -Original Message- From: [EMAIL PROTECTED] on behalf of Jarod Wilson Sent: Mon 14/08/2006 10:42 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On Sat, 2006-08-12 at 15:41 -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Just to confirm, that works nicely for Red Hat and Fedora too. -- Jarod Wilson [EMAIL PROTECTED]
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Mon, 2006-08-14 at 11:04 -0700, Bernard Li wrote: I think most people like this idea (Martin, Matt??) - so I think we can start thinking about how to modify the configure/make system. Jarod, have you already done some work on this front? No, I haven't really looked at it yet, currently buried in xen, kexec and ext4 fun... I just know using distro-provided libs is definitely preferred in Red Hat land. :) -Original Message- From: [EMAIL PROTECTED] on behalf of Jarod Wilson Sent: Mon 14/08/2006 10:42 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On Sat, 2006-08-12 at 15:41 -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Just to confirm, that works nicely for Red Hat and Fedora too. -- Jarod Wilson [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Sat, Aug 12, 2006 at 03:41:01PM -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. Sounds excellent. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Indeed, it'll vastly reduce the source package size and reduce the amount on autoconf diving I have to do for new releases :). Stu -- From the prompt of Stu Teasdale The clash of ideas is the sound of freedom.
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Thanks! Bernard From: Martin Knoblauch [mailto:[EMAIL PROTECTED] Sent: Fri 21/07/2006 02:21 To: Bernard Li; Stu Teasdale; ganglia-developers@lists.sourceforge.net Subject: RE: [Ganglia-developers] apr, expat, confuse as shipped with ganglia Hi Bernard, sorry for note commenting earlier. If we were only talking about Linux, I would say we should just throw away the 3rd party stuff and require the proper libraries being installed. Unfortunatelly this may cause trouble for those building on AIX, HP-UX, Solaris, *BSD, ... Martin --- Bernard Li [EMAIL PROTECTED] wrote: Guys: I'd like to re-visit this old discussion thread. Would it be possible now to dynamically link against distro supplied apr, expact and confuse? It sounds like people have already been doing that without huge issues so perhaps we can make the official change in the code repository? Thanks, Bernard From: [EMAIL PROTECTED] on behalf of Stu Teasdale Sent: Thu 23/02/2006 08:53 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Throwing away all the external junk would not cause problems for any BSDs since we have perfectly good package systems. IMO, the lack of usable package systems on some platforms should not be our problem. -- Brooks On Sat, Aug 12, 2006 at 03:41:01PM -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Thanks! Bernard From: Martin Knoblauch [mailto:[EMAIL PROTECTED] Sent: Fri 21/07/2006 02:21 To: Bernard Li; Stu Teasdale; ganglia-developers@lists.sourceforge.net Subject: RE: [Ganglia-developers] apr, expat, confuse as shipped with ganglia Hi Bernard, sorry for note commenting earlier. If we were only talking about Linux, I would say we should just throw away the 3rd party stuff and require the proper libraries being installed. Unfortunatelly this may cause trouble for those building on AIX, HP-UX, Solaris, *BSD, ... Martin --- Bernard Li [EMAIL PROTECTED] wrote: Guys: I'd like to re-visit this old discussion thread. Would it be possible now to dynamically link against distro supplied apr, expact and confuse? It sounds like people have already been doing that without huge issues so perhaps we can make the official change in the code repository? Thanks, Bernard From: [EMAIL PROTECTED] on behalf of Stu Teasdale Sent: Thu 23/02/2006 08:53 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers pgpH21Nbjvmqn.pgp Description: PGP signature
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Perhaps we should expect users to install the prereqs separately on their own system. This way, it will offload the necessity for us to include these external libs that are not really part of Ganglia. Anyway, from Martin's list, we can cross out BSD... do we have the Solaris package maintainer (do we have one?) here? Thanks, Bernard From: Brooks Davis [mailto:[EMAIL PROTECTED] Sent: Sat 12/08/2006 16:28 To: Bernard Li Cc: [EMAIL PROTECTED]; Stu Teasdale; ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia Throwing away all the external junk would not cause problems for any BSDs since we have perfectly good package systems. IMO, the lack of usable package systems on some platforms should not be our problem. -- Brooks On Sat, Aug 12, 2006 at 03:41:01PM -0700, Bernard Li wrote: Hi all: In discussion with Marcus Rueckert (SUSE Linux packager), he suggested a solution which the subversion folks are doing in their latest 1.4.0 RC - to ship dependencies in a separate tarball. This way, for Linux packagers, we only need the core tarball and will not include the dependencies, then we can simply link against whatever the distro provides. I think this is a good solution and will eliminate headaches for package maintainers of the various distroes. Thanks! Bernard From: Martin Knoblauch [mailto:[EMAIL PROTECTED] Sent: Fri 21/07/2006 02:21 To: Bernard Li; Stu Teasdale; ganglia-developers@lists.sourceforge.net Subject: RE: [Ganglia-developers] apr, expat, confuse as shipped with ganglia Hi Bernard, sorry for note commenting earlier. If we were only talking about Linux, I would say we should just throw away the 3rd party stuff and require the proper libraries being installed. Unfortunatelly this may cause trouble for those building on AIX, HP-UX, Solaris, *BSD, ... Martin --- Bernard Li [EMAIL PROTECTED] wrote: Guys: I'd like to re-visit this old discussion thread. Would it be possible now to dynamically link against distro supplied apr, expact and confuse? It sounds like people have already been doing that without huge issues so perhaps we can make the official change in the code repository? Thanks, Bernard From: [EMAIL PROTECTED] on behalf of Stu Teasdale Sent: Thu 23/02/2006 08:53 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de - Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnkkid=120709bid=263057dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Hi Bernard, sorry for note commenting earlier. If we were only talking about Linux, I would say we should just throw away the 3rd party stuff and require the proper libraries being installed. Unfortunatelly this may cause trouble for those building on AIX, HP-UX, Solaris, *BSD, ... Martin --- Bernard Li [EMAIL PROTECTED] wrote: Guys: I'd like to re-visit this old discussion thread. Would it be possible now to dynamically link against distro supplied apr, expact and confuse? It sounds like people have already been doing that without huge issues so perhaps we can make the official change in the code repository? Thanks, Bernard From: [EMAIL PROTECTED] on behalf of Stu Teasdale Sent: Thu 23/02/2006 08:53 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
RE: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Guys: I'd like to re-visit this old discussion thread. Would it be possible now to dynamically link against distro supplied apr, expact and confuse? It sounds like people have already been doing that without huge issues so perhaps we can make the official change in the code repository? Thanks, Bernard From: [EMAIL PROTECTED] on behalf of Stu Teasdale Sent: Thu 23/02/2006 08:53 To: ganglia-developers@lists.sourceforge.net Subject: Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu --- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnkkid=110944bid=241720dat=121642 ___ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Ole, Thanks for this level of information. I think I also share concerns about load scripts on the hosts and have a slightly more complex structure but close enough so it is good to hear that other people are having good success in this type of deployment. If you are ever allowed, I would welcome getting a look at your db+web structure. This is the one that I have most concerns about i.e. performance, amount of data, etc. Again thanks for the info and I hope to share my deployment scheme in the future (once I get there!). Chris On Fri, 2006-02-24 at 11:41, Ole Turvoll wrote: Chris, all, I'm in the same legal position as Alex (In addition I'm not allowed to use my work email address and rely on my ISP email service is only up intermittently - but webmail is blocked while I'm at work). However I'd like to share my experience. Our hierarchy (by geography) is as follows: Global gmetad | | Regional gmetad (collecting every 60 seconds) | | send_receive gmonds (from 10 ~ 400 nodes) | | nodes (~10k) sending udp unicast I agree with Alex's notions though we do not use gmetrics functionality, for various reasons, mainly around the impact of loading scripts and the manageability around using this solution. What we've implemented is a global gmond web server configuration engine. The architecture of this is an oracle database with a web front end which controls our ganglia architecture. A walk through of functionality: On the nodes (gmond package) With each gmond package I include a perl script which sends variables taken from the local host (fqdn, interface) via HTTP to the web server. On the server (gmetad package) The PHP cgi script sitting on the web server will return to the node a gmond.conf from which specifics of which gmonds it will report to are included. Depending on the fqdn the PHP cgi script gets it will either (1) enter the host into a default gmond (updating the database) or (2) send its configuration file back with the predesigned gmonds (taken from the database) it will report to. Finally on the node end it starts the gmond (last phase of a package install) with the newly acquired gmond.conf. Some other points about the architecture. - It uses the TemplatePower engine. - A cronjob checks that the gmond.conf is up to date every day at 12 pm local time (there is no DoS since we've included a timeout on the node side) - The database tables are very simple. - Anyone can bulk update the database through a simple DBD perl script - A web front end for the database table allows us to easily view the send_receive gmonds and the send gmonds, which enables us to understand and manage our environment with very low overhead. That's all I can think of for now Any questions/queries are welcome. Unfortunately I'm in the same position as Alex, would like to share this but am not aware if I can at this time. Thanks, Ole Chris Croswhite wrote: Alex, Thanks for the great information. I'll check out the Jan email thread and then follow up with more questions. BTW, the script statement did come across rather badly, sorry about that (and after all that PC training I was required to take!) Thanks Chris On Thu, 2006-02-23 at 10:36, Alex Balk wrote: Chris Croswhite wrote: Alex, Yeah, I already have a ton of questions and need some pointers in large scale deploys (best practices, do's, dont's, etc,). Till I get the legal issues out of the way, I can't share the scripts... What I can do, however, is share the ideas I've implemented, as those were developed outside the customer environment and were just spin-offs of common concepts like orchestration, federation, etc. Here's a few things: * When unicasting a tree hierarchy of nodes could provide useful drill-down capabilities. * Most organization already have some form of logical grouping for cluster nodes. For example: faculty, course, devel-group, etc. Within those groups one might find additional logical partitioning. For example: platform, project, developer, etc. Using these as the basis for constructing your logical hierarchy provides real-world basis for information analysis, saves you the trouble of deciding how to construct your grid tree and prevents gmond aggregators from handling too many nodes (though I've found that a single gmond can store information for 1k nodes without noticeable impact on performance). * Nodes will sometimes move between logical clusters. Hence, whatever mechanism you have in place has to detect this and regenerate its gmond.conf. * Using a central map which stores cluster_name gmond_aggregator gmetad_aggregator will save you the headache of figuring out who reports where, who pulls info from where, etc.
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
--- Brooks Davis [EMAIL PROTECTED] wrote: On Wed, Feb 22, 2006 at 05:07:08PM -0800, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? I believe apr was modified to include multicast functionality, I'm not sure about the rest. actually, the UDP stuff seems to be added elsewhere. The whole apr tree was checked in before that and never changed since. Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On 23 Feb 2006, at 01:07, Martin Knoblauch wrote: Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? From a packaging POV it's generally nicer to use the shipped versions of these libs dynamically linked, for a variety of reasons. To this end where possible I've changed the build in the debian packages to dynamically link against the distribution's libraries. To this end my 3.0.1 gmetad packages dynamically link agains the distro's apr and expat (but not confuse AFAICS, I suspect that's a bug). It all seems to work ok, and there are a few people out there using my packages (it can't go into debian till the glibc code problem is resolved). Stu
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Hi Chris, great :-) - HP-UX is highly needed (HPPA and IA64) - I can do Solaris 8 with gcc, but 7/9/10 would be great. Especially with the Sun compiler - Solaris 10 on AMD-64 is needed - I can do AIX 5.3 with xlc and with you we have two other victims - Linux is not a big deal Thanks Martin --- Chris Croswhite [EMAIL PROTECTED] wrote: This raises another issue, which I believe is significant to the development process of Ganglia. At the moment we don't seem to have (correct me if I'm wrong) official testers for various platforms. Maybe we could have some people volunteer to be official beta testers? We wouldn't have to have a release out the door without properly testing it under most OS/archs. The company I work for is looking to deploy ganglia across all compute farms, some ~10k systems. I could help with beta testing on these platforms: HP-UX 11+11i AIX51+53 slowlaris7-10 solaris10 x64 linux32/64 (SuSE and RH) Just let me know when you have a new candidate and I can push the client onto some test systems. Chris -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Chris, Cool! Thanks! If you need any pointers on large-scale deployments, beyond the excellent thread that was discussed here last month, drop us a line. I'm managing Ganglia on a cluster of about the same size as yours, spanning multiple sites. I've developed a framework for automating the deployment of Ganglia in a federated mode (we use unicast). I'm currently negotiating the possibility of releasing this framework to the Ganglia community. It's not the prettiest piece of code, as it's written in bash and spans a few thousands lines of code (I didn't expect it to grow into something like that), but it provides some nice functionality like map-based logical clusters, automatic node migration between clusters, map-based gmetrics, and some other candies. If negotiations fail I'll consider rewriting it from scratch in perl on my own free time. btw, I think Martin was looking for a build on HP-UX 11... Cheers, Alex Chris Croswhite wrote: This raises another issue, which I believe is significant to the development process of Ganglia. At the moment we don't seem to have (correct me if I'm wrong) official testers for various platforms. Maybe we could have some people volunteer to be official beta testers? We wouldn't have to have a release out the door without properly testing it under most OS/archs. The company I work for is looking to deploy ganglia across all compute farms, some ~10k systems. I could help with beta testing on these platforms: HP-UX 11+11i AIX51+53 slowlaris7-10 solaris10 x64 linux32/64 (SuSE and RH) Just let me know when you have a new candidate and I can push the client onto some test systems. Chris
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
--- Alex Balk [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Knoblauch wrote: I believe apr was modified to include multicast functionality, I'm not sure about the rest. Hi Brooks, good point. Now I remember the same. Time for a session with diff :-) Martin Matt, diff does not make me happy. It seems our version of apr-0.9.5 was not a final release. The apr-0.9.5 sources from apache show pretty big differences to our stuff. Martin I went over the diffs between the version of apr provided w/ Ganglia 3.0.2 and apr-0.9.5. While I didn't dig too heavily into the code, I've made the following observations: 1. Most of the changes relate to the build process (makefiles mostly). 2. There are changes to how threads are handled, but they are all internal to apr function implementations. 3. There are a few changes to how locks are handled - again, inside apr functions. 4. Added support for various platforms. 5. There seem to be _no changes to function parameters_. 6. Only a handful of functions were added, apparently all of them used internally by apr. 7. There's no mention of multicast support in Ganglia's version of apr. 8. A few files are now shipped under version 2 of the Apache license rather than v1.1. yeah, the licenses and the build stuff seemed the biggest changes to me to. I believe we can run a test of Ganglia using apr-0.9.5 on several platforms to see how we're doing. If this comes out okay, we could start looking at newer versions. The reason I'm suggesting we look at 0.9.5 first is that changes against our current version don't seem significant (0.9.7 doesn't seem like a major bump either but I haven't gone over the diffs). I personally think we should go directly to 0.9.7. It seems to fix real problems for some people (bugzilla #87). I already did a quick compile test and everything seems OK from that point. This raises another issue, which I believe is significant to the development process of Ganglia. At the moment we don't seem to have (correct me if I'm wrong) official testers for various platforms. Maybe we could have some people volunteer to be official beta testers? Another reason why I would go directly to 0.9.7. Not enough testers on to few platforms. We wouldn't have to have a release out the door without properly testing it under most OS/archs. We did last time :-) And it works pretty well. Doing a 3.0.3 with apr-0.9.7 does not sound a big risk to me. Cheers Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
On Feb 22, 2006, at 5:46 PM, Martin Knoblauch wrote: I believe apr was modified to include multicast functionality, I'm not sure about the rest. Hi Brooks, good point. Now I remember the same. Time for a session with diff :-) Martin Matt, diff does not make me happy. It seems our version of apr-0.9.5 was not a final release. The apr-0.9.5 sources from apache show pretty big differences to our stuff. i was contacted a long while back by an apache developer who was adding multicast support to apr. he took our code to use as a starting point and when i last looked the newer versions of apr support multicast. we _should_ be able to use that code. a simple grep apr_* gmond.c will show what we are using apr for. apr_strings.h apr_hash.h apr_time.h apr_pools.h apr_poll.h apr_network_io.h apr_signal.h apr_thread_proc.h apr_tables.h apr_net.h --- used for mashing multicast into apr we can check the newer version of apr to see if the portions we use are compatible. i don't think it would be too painful but that's easy to save before you start slinging code. -- [EMAIL PROTECTED] http://massie.us
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Chris Croswhite wrote: Alex, Yeah, I already have a ton of questions and need some pointers in large scale deploys (best practices, do's, dont's, etc,). Till I get the legal issues out of the way, I can't share the scripts... What I can do, however, is share the ideas I've implemented, as those were developed outside the customer environment and were just spin-offs of common concepts like orchestration, federation, etc. Here's a few things: * When unicasting a tree hierarchy of nodes could provide useful drill-down capabilities. * Most organization already have some form of logical grouping for cluster nodes. For example: faculty, course, devel-group, etc. Within those groups one might find additional logical partitioning. For example: platform, project, developer, etc. Using these as the basis for constructing your logical hierarchy provides real-world basis for information analysis, saves you the trouble of deciding how to construct your grid tree and prevents gmond aggregators from handling too many nodes (though I've found that a single gmond can store information for 1k nodes without noticeable impact on performance). * Nodes will sometimes move between logical clusters. Hence, whatever mechanism you have in place has to detect this and regenerate its gmond.conf. * Using a central map which stores cluster_name gmond_aggregator gmetad_aggregator will save you the headache of figuring out who reports where, who pulls info from where, etc. If you take this approach be sure to cache this file locally or put it on your parallel FS (if you use one). You wouldn't want 10k hosts trying to retrieve it from a single filer. * The same map file approach can be used for gmetrics. This allows anyone in your IT group to add custom metrics without having to be familiar with gmetric and without having to handle crontabs. A mechanism which reads (the cached version of) this file could handle inserting/removing crontabs as needed. Also, check out the ganglia-general thread from January 2006 called Pointers on architecting a large scale ganglia setup. I would love to get my hands on your shell scripts to figure out what you are doing (the unicast idea is pretty good). Okay, that sounds almost obscene ;-) Cheers, Alex Chris On Thu, 2006-02-23 at 09:35, Alex Balk wrote: Chris, Cool! Thanks! If you need any pointers on large-scale deployments, beyond the excellent thread that was discussed here last month, drop us a line. I'm managing Ganglia on a cluster of about the same size as yours, spanning multiple sites. I've developed a framework for automating the deployment of Ganglia in a federated mode (we use unicast). I'm currently negotiating the possibility of releasing this framework to the Ganglia community. It's not the prettiest piece of code, as it's written in bash and spans a few thousands lines of code (I didn't expect it to grow into something like that), but it provides some nice functionality like map-based logical clusters, automatic node migration between clusters, map-based gmetrics, and some other candies. If negotiations fail I'll consider rewriting it from scratch in perl on my own free time. btw, I think Martin was looking for a build on HP-UX 11... Cheers, Alex Chris Croswhite wrote: This raises another issue, which I believe is significant to the development process of Ganglia. At the moment we don't seem to have (correct me if I'm wrong) official testers for various platforms. Maybe we could have some people volunteer to be official beta testers? We wouldn't have to have a release out the door without properly testing it under most OS/archs. The company I work for is looking to deploy ganglia across all compute farms, some ~10k systems. I could help with beta testing on these platforms: HP-UX 11+11i AIX51+53 slowlaris7-10 solaris10 x64 linux32/64 (SuSE and RH) Just let me know when you have a new candidate and I can push the client onto some test systems. Chris
Re: [Ganglia-developers] apr, expat, confuse as shipped with ganglia
Martin Knoblauch wrote: [snip] We wouldn't have to have a release out the door without properly testing it under most OS/archs. We did last time :-) And it works pretty well. Doing a 3.0.3 with apr-0.9.7 does not sound a big risk to me. Not to nitpick, but 3.0.2 was a minor bugfix version. I hope your hunch about apr-0.9.7 is right, and peeking at 0.9.5 seems to support that... But someone has to play devil's advocate here. Better safe than sorry - a beta 3.0.3 tested on major platforms could provide us with that warm fuzzy go ahead feeling. Cheers, Alex Cheers Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
[Ganglia-developers] apr, expat, confuse as shipped with ganglia
Hi Matt, you may have seen the recent report about problems in apr code which are solved in later versions. This opens the question how to handle this for Ganglia. Are the versions of apr, expat and confuse shipped with the current code just unmodified copies of the stuff, or are there ganglia specific changes in there? What are your thoughts on upgrading the stuff to newer versions? Did you have a strategy/policy for that in the past? Thanks Martin -- Martin Knoblauch email: k n o b i AT knobisoft DOT de www: http://www.knobisoft.de
