Thanks for the excellent description… I have my PMR open for the e-fix, -Bryan
From: [email protected] [mailto:[email protected]] On Behalf Of Buterbaugh, Kevin L Sent: Wednesday, August 30, 2017 9:28 AM To: gpfsug main discussion list <[email protected]> Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question Note: External Email ________________________________ Hi Bryan, NO - it has the fix for the mmrestripefs data loss bug, but you need the efix on top of 4.2.3-4 for the mmadddisk / mmdeldisk issue. Let me take this opportunity to also explain a workaround that has worked for us so far for that issue … the basic problem is two-fold (on our cluster, at least). First, the /var/mmfs/gen/mmsdrfs file isn’t making it out to all nodes all the time. That is simple enough to fix (mmrefresh -fa) and verify that it’s fixed (md5sum /var/mmfs/gen/mmsdrfs). Second, however - and this is the real problem … some nodes are never actually rereading that file and therefore have incorrect information *in memory*. This has been especially problematic for us as we are replacing a batch of 80 8 TB drives with bad firmware. I am therefore deleting and subsequently recreating NSDs *with the same name*. If a client node still has the “old” information in memory then it unmounts the filesystem when I try to mmadddisk the new NSD. The workaround is to identify those nodes (mmfsadm dump nsd and grep for the identifier of the NSD(s) in question) and force them to reread the info (tsctl rereadnsd). HTH… Kevin On Aug 30, 2017, at 9:21 AM, Bryan Banister <[email protected]<mailto:[email protected]>> wrote: Ok, I’m completely confused… You’re saying 4.2.3-4 *has* the fix for adding/deleting NSDs? -Bryan From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Sobey, Richard A Sent: Wednesday, August 30, 2017 9:13 AM To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question Note: External Email ________________________________ Aha, I’ve just realised what you actually said, having seen Simon’s response and twigged. The defect 1020461 matches what IBM has told me in my PMR about adding/deleting NSDs. I’m not sure why the description mentions networking though! Richard From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Sobey, Richard A Sent: 30 August 2017 14:56 To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question No worries, I’ve got it sorted and hopefully about to grab the 4.2.3-4 efix2. Cheers for your help! Richard From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Buterbaugh, Kevin L Sent: 30 August 2017 14:55 To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question Hi Richard, Well, I’m not sure, which is why it’s taken me a while to respond. In the README that comes with the efix it lists: Defect APAR Description 1032655 None AFM: Fix Truncate filtering Write incorrectly 1020461 None FS can't be mounted after weird networking error That 1st one is obviously not it and that 2nd one doesn’t reference mmadddisk / mmdeldisk. Plus neither show an APAR number. Sorry I can’t be of more help… Kevin On Aug 29, 2017, at 12:52 PM, Sobey, Richard A <[email protected]<mailto:[email protected]>> wrote: Thanks Kevin, that's good to know. Is there an apar I need to quote in my pmr? Get Outlook for Android<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Faka.ms%2Fghei36&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=op1veAuespuhwL9zsBMZe%2FwVAn6NC%2FkL0U4IKtAmVT4%3D&reserved=0> ________________________________ From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> on behalf of Buterbaugh, Kevin L <[email protected]<mailto:[email protected]>> Sent: Tuesday, August 29, 2017 4:53:51 PM To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question Hi Richard, Since I upgraded my cluster to GPFS 4.2.3.4 over the weekend IBM created an efix for it for the NSD deletion / creation fix. I’m sure they’ll give it to you, too… ;-) Kevin On Aug 29, 2017, at 9:30 AM, Sobey, Richard A <[email protected]<mailto:[email protected]>> wrote: So I can upgrade to 4.2.3-4 to get the mmrestripe fix, or 4.2.3-3 efix3 to get the NSD deletion and creation fix? Not great when on Monday I’m doing a load of all this. What’s the recommendation? Is there a one size fits all patch? From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Frederick Stock Sent: 27 August 2017 01:35 To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] GPFS 4.2.3.4 question The only change missing is the change delivered in 4.2.3 PTF3 efix3 which was provided on August 22. The problem had to do with NSD deletion and creation. Fred __________________________________________________ Fred Stock | IBM Pittsburgh Lab | 720-430-8821 [email protected]<mailto:[email protected]> From: "Buterbaugh, Kevin L" <[email protected]<mailto:[email protected]>> To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Date: 08/26/2017 03:40 PM Subject: [gpfsug-discuss] GPFS 4.2.3.4 question Sent by: [email protected]<mailto:[email protected]> ________________________________ Hi All, Does anybody know if GPFS 4.2.3.4, which came out today, contains all the patches that are in GPFS 4.2.3.3 efix3? If anybody does, and can respond, I’d greatly appreciate it. Our cluster is in a very, very bad state right now and we may need to just take it down and bring it back up. I was already planning on rolling out GPFS 4.2.3.3 efix 3 over the next few weeks anyway, so if I can just go to 4.2.3.4 that would be great… Thanks! — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]>- (615)875-9633 _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org%2F&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=7aIpvwHm9jiSsj0kOAXLwEO1EvXb%2FH6ntKysDCh0WuY%3D&reserved=0> https://urldefense.proofpoint.com/v2/url?u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx-siA1ZOg&r=p_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw&m=7r9GsD1C2HiY4j21vPYIoQPHXePHxeMhzQeaw_ne4lM&s=-SFnqoJw--FN3wqClEEBGa9-XSLljgSseIU_SxGoWy0&e=<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss%26d%3DDwICAg%26c%3Djf_iaSHvJObTbx-siA1ZOg%26r%3Dp_1XEUyoJ7-VJxF_w8h9gJh8_Wj0Pey73LCLLoxodpw%26m%3D7r9GsD1C2HiY4j21vPYIoQPHXePHxeMhzQeaw_ne4lM%26s%3D-SFnqoJw--FN3wqClEEBGa9-XSLljgSseIU_SxGoWy0%26e%3D&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=fBBdaghWvQ9%2By%2B8eIM1%2FRJ9PlxJ63MjNwr7UJ50AeNM%3D&reserved=0> _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org%2F&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=7aIpvwHm9jiSsj0kOAXLwEO1EvXb%2FH6ntKysDCh0WuY%3D&reserved=0> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=qYxCMMg9O31LzFg%2FQkCdQg8vV%2FgL2AuRk%2B6V2j76c7Y%3D&reserved=0> _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fspectrumscale.org&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=%2F2nCbkq2CJh3FX4UzyT3rWiImQE2Q%2BphLeFqaD9fhMg%3D&reserved=0> http://gpfsug.org/mailman/listinfo/gpfsug-discuss<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=qYxCMMg9O31LzFg%2FQkCdQg8vV%2FgL2AuRk%2B6V2j76c7Y%3D&reserved=0> — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]> - (615)875-9633 ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org<http://spectrumscale.org/> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgpfsug.org%2Fmailman%2Flistinfo%2Fgpfsug-discuss&data=02%7C01%7CKevin.Buterbaugh%40vanderbilt.edu%7C493f1f9e41e343324f1508d4efb25f4f%7Cba5a7f39e3be4ab3b45067fa80faecad%7C0%7C0%7C636396996783027614&sdata=qYxCMMg9O31LzFg%2FQkCdQg8vV%2FgL2AuRk%2B6V2j76c7Y%3D&reserved=0 — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]> - (615)875-9633 ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
