It started for me after upgrade from v4.2.x.x to 5.0.0.1 with RHEL7.4. Strangely not immediately, but 2 days after the upgrade (wednesday evening CET). Also I have some doubts that mount -o tcp will help, since TCP should already be the default transport. Have asked for if we can rather block this serverside using iptables.
But, I expect we should get a fix soon, and we’ll stick with v2.3.2 until that. -jf man. 23. apr. 2018 kl. 01:23 skrev Ray Coetzee <[email protected]>: > Hi Jan-Frode > We've been told the same regarding mounts using UDP. > Our exports are already explicitly configured for TCP and the client's > fstab's set to use TCP. > It would be infuriating if the clients are trying UDP first irrespective > of the mount options configured. > > Why the problem started specifically last week for both of us is > interesting. > > Kind regards > > Ray Coetzee > Mob: +44 759 704 7060 > > Skype: ray.coetzee > > Email: [email protected] > > > On Mon, Apr 23, 2018 at 12:02 AM, Jan-Frode Myklebust <[email protected]> > wrote: > >> >> Yes, I've been struggelig with something similiar this week. Ganesha >> dying with SIGABRT -- nothing else logged. After catching a few coredumps, >> it has been identified as a problem with some udp-communication during >> mounts from solaris clients. Disabling udp as transport on the shares >> serverside didn't help. It was suggested to use "mount -o tcp" or whatever >> the solaris version of this is -- but we haven't tested this. So far the >> downgrade to v2.3.2 has been our workaround. >> >> PMR: 48669,080,678 >> >> >> -jf >> >> >> On Mon, Apr 23, 2018 at 12:38 AM, Ray Coetzee <[email protected]> >> wrote: >> >>> Good evening all >>> >>> I'm working with IBM on a PMR where ganesha is segfaulting or causing >>> kernel panics on one group of CES nodes. >>> >>> We have 12 identical CES nodes split into two groups of 6 nodes each & >>> have been running with RHEL 7.3 & GPFS 5.0.0-1 since 5.0.0-1 was >>> released. >>> >>> Only one group started having issues Monday morning where ganesha would >>> segfault and the mounts would move over to the remaining nodes. >>> The remaining nodes then start to fall over like dominos within minutes >>> or hours to the point that all CES nodes are "failed" according to >>> "mmces node list" and the VIP's are unassigned. >>> >>> Recovering the nodes are extremely finicky and works for a few minutes >>> or hours before segfaulting again. >>> Most times a complete stop of Ganesha on all nodes & then only starting >>> it on two random nodes allow mounts to recover for a while. >>> >>> None of the following has helped: >>> A reboot of all nodes. >>> Refresh CCR config file with mmsdrrestore >>> Remove/add CES from nodes. >>> Reinstall GPFS & protocol rpms >>> Update to 5.0.0-2 >>> Fresh reinstall of a node >>> Network checks out with no dropped packets on either data or export >>> networks. >>> >>> The only temporary fix so far has been to downrev ganesha to 2.3.2 from >>> 2.5.3 on the affected nodes. >>> >>> While waiting for IBM development, has anyone seen something similar >>> maybe? >>> >>> Kind regards >>> >>> Ray Coetzee >>> >>> >>> >>> On Sat, Apr 21, 2018 at 12:00 PM, < >>> [email protected]> wrote: >>> >>>> Send gpfsug-discuss mailing list submissions to >>>> [email protected] >>>> >>>> To subscribe or unsubscribe via the World Wide Web, visit >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>> or, via email, send a message with subject or body 'help' to >>>> [email protected] >>>> >>>> You can reach the person managing the list at >>>> [email protected] >>>> >>>> When replying, please edit your Subject line so it is more specific >>>> than "Re: Contents of gpfsug-discuss digest..." >>>> >>>> >>>> Today's Topics: >>>> >>>> 1. Re: UK Meeting - tooling Spectrum Scale (Grunenberg, Renar) >>>> 2. Re: UK Meeting - tooling Spectrum Scale >>>> (Simon Thompson (IT Research Support)) >>>> >>>> >>>> ---------------------------------------------------------------------- >>>> >>>> Message: 1 >>>> Date: Fri, 20 Apr 2018 14:01:55 +0000 >>>> From: "Grunenberg, Renar" <[email protected]> >>>> To: "'[email protected]'" >>>> <[email protected]> >>>> Subject: Re: [gpfsug-discuss] UK Meeting - tooling Spectrum Scale >>>> Message-ID: <[email protected]> >>>> Content-Type: text/plain; charset="utf-8" >>>> >>>> Hallo Simon, >>>> are there any reason why the link of the presentation from Yong ZY >>>> Zheng(Cognitive, ML, Hortonworks) is not linked. >>>> >>>> Renar Grunenberg >>>> Abteilung Informatik ? Betrieb >>>> >>>> HUK-COBURG >>>> Bahnhofsplatz >>>> 96444 Coburg >>>> Telefon: 09561 96-44110 >>>> Telefax: 09561 96-44104 >>>> E-Mail: [email protected] >>>> Internet: www.huk.de >>>> ________________________________ >>>> HUK-COBURG Haftpflicht-Unterst?tzungs-Kasse kraftfahrender Beamter >>>> Deutschlands a. G. in Coburg >>>> Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021 >>>> Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg >>>> Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin. >>>> Vorstand: Klaus-J?rgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans >>>> Olav Her?y, Dr. J?rg Rheinl?nder (stv.), Sarah R?ssler, Daniel Thomas. >>>> ________________________________ >>>> Diese Nachricht enth?lt vertrauliche und/oder rechtlich gesch?tzte >>>> Informationen. >>>> Wenn Sie nicht der richtige Adressat sind oder diese Nachricht >>>> irrt?mlich erhalten haben, >>>> informieren Sie bitte sofort den Absender und vernichten Sie diese >>>> Nachricht. >>>> Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht >>>> ist nicht gestattet. >>>> >>>> This information may contain confidential and/or privileged information. >>>> If you are not the intended recipient (or have received this >>>> information in error) please notify the >>>> sender immediately and destroy this information. >>>> Any unauthorized copying, disclosure or distribution of the material in >>>> this information is strictly forbidden. >>>> ________________________________ >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: < >>>> http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180420/91e3d84d/attachment-0001.html >>>> > >>>> >>>> ------------------------------ >>>> >>>> Message: 2 >>>> Date: Fri, 20 Apr 2018 14:12:11 +0000 >>>> From: "Simon Thompson (IT Research Support)" <[email protected]> >>>> To: gpfsug main discussion list <[email protected]> >>>> Subject: Re: [gpfsug-discuss] UK Meeting - tooling Spectrum Scale >>>> Message-ID: <[email protected]> >>>> Content-Type: text/plain; charset="utf-8" >>>> >>>> Sorry, it was a typo from my side. >>>> >>>> The talks that are missing we are chasing for copies of the slides that >>>> we can release. >>>> >>>> Simon >>>> >>>> From: <[email protected]> on behalf of " >>>> [email protected]" <[email protected]> >>>> Reply-To: "[email protected]" < >>>> [email protected]> >>>> Date: Friday, 20 April 2018 at 15:02 >>>> To: "[email protected]" < >>>> [email protected]> >>>> Subject: Re: [gpfsug-discuss] UK Meeting - tooling Spectrum Scale >>>> >>>> Hallo Simon, >>>> are there any reason why the link of the presentation from Yong ZY >>>> Zheng(Cognitive, ML, Hortonworks) is not linked. >>>> >>>> Renar Grunenberg >>>> Abteilung Informatik ? Betrieb >>>> >>>> HUK-COBURG >>>> Bahnhofsplatz >>>> 96444 Coburg >>>> Telefon: >>>> >>>> 09561 96-44110 >>>> >>>> Telefax: >>>> >>>> 09561 96-44104 >>>> >>>> E-Mail: >>>> >>>> [email protected] >>>> >>>> Internet: >>>> >>>> www.huk.de >>>> >>>> ________________________________ >>>> HUK-COBURG Haftpflicht-Unterst?tzungs-Kasse kraftfahrender Beamter >>>> Deutschlands a. G. in Coburg >>>> Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021 >>>> Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg >>>> Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin. >>>> Vorstand: Klaus-J?rgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans >>>> Olav Her?y, Dr. J?rg Rheinl?nder (stv.), Sarah R?ssler, Daniel Thomas. >>>> ________________________________ >>>> Diese Nachricht enth?lt vertrauliche und/oder rechtlich gesch?tzte >>>> Informationen. >>>> Wenn Sie nicht der richtige Adressat sind oder diese Nachricht >>>> irrt?mlich erhalten haben, >>>> informieren Sie bitte sofort den Absender und vernichten Sie diese >>>> Nachricht. >>>> Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht >>>> ist nicht gestattet. >>>> >>>> This information may contain confidential and/or privileged information. >>>> If you are not the intended recipient (or have received this >>>> information in error) please notify the >>>> sender immediately and destroy this information. >>>> Any unauthorized copying, disclosure or distribution of the material in >>>> this information is strictly forbidden. >>>> ________________________________ >>>> -------------- next part -------------- >>>> An HTML attachment was scrubbed... >>>> URL: < >>>> http://gpfsug.org/pipermail/gpfsug-discuss/attachments/20180420/0b8e9ffa/attachment-0001.html >>>> > >>>> >>>> ------------------------------ >>>> >>>> _______________________________________________ >>>> gpfsug-discuss mailing list >>>> gpfsug-discuss at spectrumscale.org >>>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>>> >>>> >>>> End of gpfsug-discuss Digest, Vol 75, Issue 34 >>>> ********************************************** >>>> >>> >>> >>> _______________________________________________ >>> gpfsug-discuss mailing list >>> gpfsug-discuss at spectrumscale.org >>> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >>> >>> >> >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
