Here is the latest I got from IBM: The fix only needs to be installed on the file system manager nodes.
About how to know if your cluster is affected already, you can check if there was any MMFS_FSSTRUCT error in the system logs. If you encounter any lookup failure, funny ls cmd outputs. Or if any cmd would give some replica mismatch error or warning. If you encountered the following kind of Assertion failure you hit the bug. Thu Jul 21 03:26:32.373 2016: [X] *** Assert exp(prevIndEntryP->nextP->dataBlockNum > dataBlockNum) in line 4552 of file /project/sprelbmd/build/rbmd1629a/src/avs/fs/mmfs/ts/log/repUpdate.C Thu Jul 21 03:26:32.374 2016: [E] *** Traceback: Thu Jul 21 03:26:32.375 2016: [E] 2:0x7FE6E141AB36 logAssertFailed + 0x2D6 at Logger.C:546 Thu Jul 21 03:26:32.376 2016: [E] 3:0x7FE6E13FCD25 InodeRecoveryList::addInodeAndIndBlock(long long, unsigned int, RepDiskAddr const&, InodeRecoveryList::FlagsToSet, long long, RepDiskAddr const&) + 0x355 at repUpdate.C:4552 Thu Jul 21 03:26:32.377 2016: [E] 4:0x7FE6E1066879 RecoverDirEntry(StripeGroup*, LogRecovery*, LogFile*, LogRecordType, long long, int, unsigned int*, char*, int*, RepDiskAddr) + 0x1089 at direct.C:2312 Thu Jul 21 03:26:32.378 2016: [E] 5:0x7FE6E13F8741 LogRecovery::recoverOneObject(long long) + 0x1E1 at recoverlog.C:362 Thu Jul 21 03:26:32.379 2016: [E] 6:0x7FE6E0F29B25 MultiThreadWork::doNextStep() + 0xC5 at workthread.C:533 Thu Jul 21 03:26:32.380 2016: [E] 7:0x7FE6E0F29FBB MultiThreadWork::helperThreadBody(void*) + 0xCB at workthread.C:455 Thu Jul 21 03:26:32.381 2016: [E] 8:0x7FE6E0F5FB26 Thread::callBody(Thread*) + 0x46 at thread.C:393 Thu Jul 21 03:26:32.382 2016: [E] 9:0x7FE6E0F4DD12 Thread::callBodyWrapper(Thread*) + 0xA2 at mastdep.C:1077 Thu Jul 21 03:26:32.383 2016: [E] 10:0x7FE6E0667851 start_thread + 0xD1 at mastdep.C:1077 Thu Jul 21 03:26:32.384 2016: [E] 11:0x7FE6DF7BE90D clone + 0x6D at mastdep.C:1077 Hope that helps, -Bryan From: [email protected] [mailto:[email protected]] On Behalf Of Fosburgh,Jonathan Sent: Friday, February 24, 2017 9:30 AM To: [email protected] Subject: Re: [gpfsug-discuss] Fw: Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption FWIW, my contact said to do everything, even client only clusters. -- Jonathan Fosburgh Principal Application Systems Analyst Storage Team IT Operations [email protected]<mailto:[email protected]> (713) 745-9346 -----Original Message----- Date: Fri, 24 Feb 2017 15:25:14 +0000 Subject: Re: [gpfsug-discuss] Fw: Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption To: gpfsug main discussion list <[email protected]<mailto:gpfsug%20main%20discussion%20list%20%[email protected]%3e>> Reply-to: gpfsug main discussion list <[email protected]<mailto:[email protected]>> From: Bryan Banister <[email protected]<mailto:bryan%20banister%20%[email protected]%3e>> I just got word that you only need to update the active file system manager node… I’ll let you know if I hear differently, -Bryan From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Sanchez, Paul Sent: Friday, February 24, 2017 9:16 AM To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: Re: [gpfsug-discuss] Fw: Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption Can anyone from IBM confirm whether this only affects manager nodes or if parallel log recovery is expected to happen on any other nodes? Thx Paul From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Bryan Banister Sent: Friday, February 24, 2017 9:08 AM To: gpfsug main discussion list Subject: Re: [gpfsug-discuss] Fw: Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption Has anyone been hit by this data corruption issue and if so how did you determine the file system had corruption? Thanks! -Bryan From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Oesterlin, Robert Sent: Thursday, February 23, 2017 9:46 AM To: gpfsug main discussion list <[email protected]<mailto:[email protected]>> Subject: [gpfsug-discuss] Fw: Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption For those not subscribed, see below. Bob Oesterlin Sr Principal Storage Engineer, Nuance From: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Thursday, February 23, 2017 at 9:42 AM Subject: [EXTERNAL] [Forums] '[email protected]' replied to the 'IBM Spectrum Scale V4.2.2 announcements' topic thread in the 'General Parallel File System - Announce (GPFS - Announce)' forum. [cid:[email protected]] [email protected]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ibm.com_developerworks_community_profiles_html_profileView.do-3Fuserid-3D060000T9GF&d=DgMFaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=LPDewt1Z4o9eKc86MXmhqX-45Cz1yz1ylYELF9olLKU&m=VnicQ7hM5kAbUhEZZxbyL6g9pAlwAYXG0f12gJWrEew&s=TdzECmVhE3t4jdrNWogfBMSo8vgP_met3YTFRRdJARc&e=> replied to the IBM Spectrum Scale V4.2.2 announcements<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ibm.com_developerworks_community_forums_html_topic-3Fid-3Da1939921-2D633b-2D4a45-2D8f0f-2D8f9181fc2bc5&d=DgMFaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=LPDewt1Z4o9eKc86MXmhqX-45Cz1yz1ylYELF9olLKU&m=VnicQ7hM5kAbUhEZZxbyL6g9pAlwAYXG0f12gJWrEew&s=DPv9xlNyJnRXkqQPzn7f9lieOuJi5GHz3HRuN7MhFXA&e=> topic thread in the General Parallel File System - Announce (GPFS - Announce)<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.ibm.com_developerworks_community_forums_html_forum-3Fid-3D11111111-2D0000-2D0000-2D0000-2D000000001606&d=DgMFaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=LPDewt1Z4o9eKc86MXmhqX-45Cz1yz1ylYELF9olLKU&m=VnicQ7hM5kAbUhEZZxbyL6g9pAlwAYXG0f12gJWrEew&s=NYTL-LzWTlb2LrZOkD1DwKonf9YHX5ujTpJYQmfQYiE&e=> forum. Flash (Alert) IBM Spectrum Scale V4.2.1/4.2.2 parallel log recovery function may result in undetected data corruption Abstract IBM has identified a problem with the IBM Spectrum Scale parallel log recovery function in V4.2.1/V4.2.2, which may result in undetected data corruption during the course of a file system recovery. See the complete Flash at http://www-01.ibm.com/support/docview.wss?uid=ssg1S1009965<https://urldefense.proofpoint.com/v2/url?u=http-3A__www-2D01.ibm.com_support_docview.wss-3Fuid-3Dssg1S1009965&d=DgMFaQ&c=djjh8EKwHtOepW4Bjau0lKhLlu-DxM1dlgP0rrLsOzY&r=LPDewt1Z4o9eKc86MXmhqX-45Cz1yz1ylYELF9olLKU&m=VnicQ7hM5kAbUhEZZxbyL6g9pAlwAYXG0f12gJWrEew&s=GZ88LDgDtQk90LnHRUEAuf08BL83oSKb0NFcUH-j5mU&e=> ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product. ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss The information contained in this e-mail message may be privileged, confidential, and/or protected from disclosure. This e-mail message may contain protected health information (PHI); dissemination of PHI should comply with applicable federal and state laws. If you are not the intended recipient, or an authorized representative of the intended recipient, any further review, disclosure, use, dissemination, distribution, or copying of this message or any attachment (or the information contained therein) is strictly prohibited. If you think that you have received this e-mail message in error, please notify the sender by return e-mail and delete all references to it and its contents from your systems. ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product.
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
