>>> On 11/6/2008 at 5:07 AM, in message <[EMAIL PROTECTED]>, Carlo Marcelo Arenas Belon <[EMAIL PROTECTED]> wrote: > On Tue, Nov 04, 2008 at 06:13:47PM -0700, Brad Nicholes wrote: >> Carlo, >> In the STATUS file you commented that the spoofing patch needs more > work. > > my comment was about the spoofing "feature" needing more work in 3.1 as you > pointed out below. > > the patch just makes the problem bigger by adding several changes (more than > 500 lines) on top of an implementation that has open regressions and that > will need to be cleaned up first IMHO. >
You really have me confused here. What problem are you talking about that the spoofing patch makes worse? AFAIK, the spoofing patch is independent of any other problem. The spoofing patch just adds spoofing functionality to the modular interface. Are you saying that there is something wrong with the modular interface. If so then that is a different issue that should be dealt with separately. >> Can you explain what work needs to be done. Other than supporting the >> short commandline for spoofing a heartbeat, AFAICT everything is working >> as it should. > > current 3.1, without the additional patch is missing the heartbeat spoofing > support as you pointed out, and is also exporting the spoofed data in the > XML which even if mostly harmless is a change of behaviour and should be > cleaned up as well (as it really adds no value and is inconsistent) > It don't see what the problem is here. The point of the spoofing patch is to add spoofing functionality to the modular interface, including heartbeat spoofing. The only thing missing in the modular spoofing patch is the shortened commandline for gmetric. That issue has already been dealt with in a separate proposal. Both proposals can be backported and done at the same time if needed. They both just need to be reviewed and voted on. I just wanted to get one thing done at a time. So rather than making the modular spoofing patch more complex by adding the shortened command line to it, I chose to deal with that patch separately so that they are easier to review. For the XML data that is being returned, I explained that in my last email. With the refactoring of the XDR data and the addition of the metadata packet and extra data, any extra data tag added to a metric definition will show up in the XML output for the metric. This is by design. Since spoofing is simply an extra attribute on a metric definition, the spoofing tag shows up as extra data in the XML. Gmond should not have to worry about individual extra data tags and have to selectively include or remove them from the XML output. The heartbeat spoofing tag doesn't show up in the XML output because the heartbeat metric itself is completely stripped before the XML is sent. This is how the heartbeat metric has always been handled. Therefore not seeing a heartbeat spoofing tag in the XML is just a side effect of legacy functionality. > as for 3.1 with the patch added, there are the following problems : > > * the added calls for toupper() in libgmond could result in undefined > behaviour in platforms where char is signed and toupper is implemented > using an array lookup (NetBSD, and probably HPUX and AIX) > No problem. I can switch these calls to apr_toupper(). > * the proposed patch has some additional patches that will need to be > added on top of it (some of them already proposed and approved like > the modpython fixes, but some others still not even proposed) > I don't see this as a reason for holding up the proposed patch. The question is, does the modular spoofing patch as proposed, work and is it stable enough to be backported? If there are other enhancements that can be made later, then these patches can be added separately. The modular spoofing patch is already complicated enough. Let's not make it more complex by adding more features. The modpython fix (if we are talking about the same thing) is a separate issue. It is a patch that fixes a python incompatibility. It has nothing to do with the spoofing patch other than it was introduced in trunk after the spoofing patch and therefore has some dependancies due to code changes in trunk. This shouldn't have anything to do with accepting the spoofing patch. It is just a side effect of developing in trunk. >> I have committed a spoofing example python module in trunk that spoofs >> cpu_util, boottime, heartbeat and osname for three imaginary machines. >> This spoof example module runs under both trunk and the patched 3.1.x. > > Sadly I haven't been able to make it work, with `gmond -m` showing : > > # gmond -m > (module python_module) > load_one One minute load average (module load_module) > ... > I'm not sure what you are talking about here. The spoofing example module has nothing to do with load_one. I didn't spoof the load_one metric in the example module. What you should see after you have loaded the spoofing example module, are the following additions to the output of gmond -m spf_boottime:spf1.myhost.org Spoofed Boot Time (module python_module) spf_heartbeat:spf1.myhost.org Spoofed Heartbeat (module python_module) spf_osname:spf3.myhost.org Spoofed Operating System Name (module python_module) spf_location:spf3.myhost.org Spoofed Location (module python_module) spf_osname:spf2.myhost.org Spoofed Operating System Name (module python_module) spf_boottime:spf3.myhost.org Spoofed Boot Time (module python_module) spf_random_cpu_util:spf2.myhost.org Spoofed CPU Utilization (module python_module) spf_heartbeat:spf3.myhost.org Spoofed Heartbeat (module python_module) spf_location:spf2.myhost.org Spoofed Location (module python_module) spf_osname:spf1.myhost.org Spoofed Operating System Name (module python_module) spf_boottime:spf2.myhost.org Spoofed Boot Time (module python_module) spf_random_cpu_util:spf1.myhost.org Spoofed CPU Utilization (module python_module) spf_heartbeat:spf2.myhost.org Spoofed Heartbeat (module python_module) spf_location:spf1.myhost.org Spoofed Location (module python_module) Basically a spoof metric for boottime, heartbeat, osname, location and random_cpu_util for 3 imaginary machines. If you are talking about a spoofing module that you created, then that is a different matter. > and python modules in general not working anymore (linked against amd64's > python 2.5.2 in Linux) > What doesn't work? Also, whether or not the python modules are not working in a specific environment is a separate issue from the modular spoofing proposal. This issue should be dealt with as a separate bug and backport proposal if required. > the "C" interface seems to work fine at least in 3.1 (in trunk it messes > GMOND_STARTED as explained before) > I responded to this issue in a separate thread. I have only been able to find one instance where GMOND_STARTED was being updated or changed. It turned out to be a configuration error on my part and had nothing to do with the spoofing patch. The problem was that I had commented out the bind directive for the multicast recv channel. Since I was running two different versions of gmond on my system during testing and also sending spoofed heartbeat packets with gmetric, the mis-configuration of the udp channel caused gmond to ignore the IP addresses of the different channels and read all UDP traffic on port 8649. That meant that gmond was getting multiple heartbeat packets for the same host which caused the GMOND_STARTED value to flip-flop between the various heartbeat values. Once I uncommented the bind directives, everything stabilized as it should including the spoofed heartbeats for the imaginary machines that I expected. Also if you had used gmetric to send a non-spoofed heartbeat, this will cause the GMOND_STARTED value to update. The reason why is because the gmetric heartbeat for your host will override the gmond heartbeat until the next time that gmond sends it's own heartbeat for the same host. >> I have also created a patch for sending a heartbeat through the shortened >> commandline which has been proposed as a follow-on backport. > > saw that, not sure about the needed dependency in the modular spoofing > support as IMHO a change to gmetric should be independent of that. > It is, but due to the fact that the shortened commandline patch was introduced in trunk after the modular spoofing, it just created an ordering dependancy. Again, this is only due to the way that we develop in trunk. As I mentioned before, I chose to treat it as a separate patch because they are really separate issues and there is no need to make the modular spoofing patch more complicated than it needs to be for review. If I could break the modular spoofing patch down into multiple functional patches, I would do that also so that patch reviews would be easier. Unfortunately, I can't. >> If there is nothing else missing, can we get this one backported? > > with the availability of a proposed backport patch that fixes the conflicts > and an example python spoofing module it should be easier to do so but as I > pointed before not yet sure enough to stamp a vote on it, but I agree we > should get this patch/feature released with the next release. > The only thing really holding up the next release, at least from my perspective, is backporting this patch. I would like to get this backported along with some of the other proposals that are sitting in the STATUS file and then move forward with a 3.1.2 release. The modular metric spoofing functionality opens up a whole new set of capability for Ganglia in that it will allow a gmond daemon running on a VM host to spoof metrics that have been implemented as modules for each hosted VM without having to put gmond on each hosted VM. > Carlo > > PS. can the full list of patches needed from backport be added to the list, > I > suspect r1615 is missing as that should be required for r1622 which I added > and is included in your consolidated patch The full list of patches needed for the modular spoofing support is already listed. Both r1615 and r1622 were already backported to the 3.1.x branch on Aug. 5 (r1627). Neither r1615 nor r1622 were required for the modular spoofing support proposal. The problem with allowing backport proposals to sit in the STATUS file for so long is because over time they start to degrade as other later patches get backported ahead of it. Since r1615 backported ahead of the modular spoofing proposal, it caused a conflict with r1389. This is one of the reasons why I created the consolidated patch. The consolidated patch resolves the conflict so that the proposed spoofing patch can be more easily applied to the 3.1.x branch and reviewed. Since r1622 has ready been backported, it shouldn't have been added to the modular spoofing proposal. Brad ------------------------------------------------------------------------- This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ _______________________________________________ Ganglia-developers mailing list Ganglia-developers@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-developers
