Hi everyone, any news on this? Another symptom is that this happens quite as often as the cluster changes, meaning that the more activity there is in the cluster (delete machines, create...) the more this issue happens. Could it be related with the deletion of old hosts by gmond causing gmetad to try to access files that are already gone?
Cumprimentos / Best regards, Cristóvão José Domingues Cordeiro ________________________________________ From: Cristovao Cordeiro [cristovao.corde...@cern.ch] Sent: 09 November 2015 13:40 To: Devon H. O'Dell Cc: Ganglia-general@lists.sourceforge.net Subject: Re: [Ganglia-general] gmetad segmentation fault Hi Devon, thanks! * I don't think there was a core dump. At least that is not stated in /var/log/messages and I don't find anything relevant in /var/spool/abrt/ * I am running 3.7.1 * The addr2line returns ??:0. Also with gdb: > gdb /usr/lib64/libganglia.so.0.0.0 ... Reading symbols from /usr/lib64/libganglia.so.0.0.0...(no debugging symbols found)...done. Some more information about my setup: - I am running several gmonds in the same machine, so all my data_sources are to localhost. Cumprimentos / Best regards, Cristóvão José Domingues Cordeiro ________________________________________ From: Devon H. O'Dell [devon.od...@gmail.com] Sent: 09 November 2015 13:12 To: Cristovao Cordeiro Cc: Ganglia-general@lists.sourceforge.net Subject: Re: [Ganglia-general] gmetad segmentation fault Hi! I have a couple of initial questions that might help figure out the problem: * Did you get a core dump? * What version of ganglia are you running? * This crash happened within libganglia.so at offset 0xb7b0. Can you run: $ addr2line -e /path/to/libganglia.so.0.0.0 0xb7b0 and paste the output? If that does not work, there are a couple other things we can try to get information about the fault, but hopefully we can just work from there. Kind regards, Devon H. O'Dell 2015-11-09 0:13 GMT-08:00 Cristovao Cordeiro <cristovao.corde...@cern.ch>: > Dear all, > > I have several Ganglia monitors running with similar configurations in > different machines (VMs) and for a long time now I have been experiencing > segmentation faults at random times. It seems to happen more on gmetads that > are monitoring larger number of nodes. > > In /var/log/messages I see: > > kernel: gmetad[3948]: segfault at 0 ip 0000003630c0b7b0 sp 00007f0ecbffebc0 > error 4 in libganglia.so.0.0.0[3630c00000+15000] > > > and in the console output there's only this: > > /bin/bash: line 1: 30375 Terminated /usr/sbin/gmetad > > [FAILED] > > > gmetad does not have any special configuration besides the RRD location > which in on a 4Gb ramdisk. > > > Cumprimentos / Best regards, > Cristóvão José Domingues Cordeiro > > > ------------------------------------------------------------------------------ > Presto, an open source distributed SQL query engine for big data, initially > developed by Facebook, enables you to easily query your data on Hadoop in a > more interactive manner. Teradata is also now providing full enterprise > support for Presto. Download a free open source copy now. > http://pubads.g.doubleclick.net/gampad/clk?id=250295911&iu=/4140 > _______________________________________________ > Ganglia-general mailing list > Ganglia-general@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/ganglia-general > ------------------------------------------------------------------------------ Presto, an open source distributed SQL query engine for big data, initially developed by Facebook, enables you to easily query your data on Hadoop in a more interactive manner. Teradata is also now providing full enterprise support for Presto. Download a free open source copy now. http://pubads.g.doubleclick.net/gampad/clk?id=250295911&iu=/4140 _______________________________________________ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general ------------------------------------------------------------------------------ _______________________________________________ Ganglia-general mailing list Ganglia-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/ganglia-general
