Re: [gpfsug-discuss] Disk in unrecovered state
Thanks a lot!! Guys, this do the trick Now, whole disks are up again and the FS has been mounted without troubles, Cheers, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Hi Iban, given that you have physical access to the disks and they are readable ( i see you checked that via dd command) you should mmchdisk start them. Note: as you have down disks in more than one FG, you will need to be able to at least get one good copy of the metadata readable .. in order to be able to mmchdisk start a disk. In that case i would run : mmchdiskstart -a (so gpfs can get data from all readable disks) Mit freundlichen Grüßen / Kind regards Achim Rehor Remote Technical Support Engineer Storage IBM Systems Storage Support - EMEA Storage Competence Center (ESCC) Spectrum Scale / Elastic Storage Server --- IBM Deutschland Am Weiher 24 65451 Kelsterbach Phone: +49-170-4521194 E-Mail: achim.re...@de.ibm.com --- IBM Deutschland GmbH / Vorsitzender des Aufsichtsrats: Sebastian Krause Geschäftsführung: Gregor Pillen (Vorsitzender), Agnes Heftberger, Norbert Janzen, Markus Koerner, Christian Noll, Nicole Reimer Sitz der Gesellschaft: Ehningen / Registergericht: Amtsgericht Stuttgart, HRB 14562 / WEEE-Reg.-Nr. DE 99369940 gpfsug-discuss-boun...@spectrumscale.org wrote on 12/01/2021 16:59:03: > From: Iban Cabrillo > To: gpfsug-discuss > Date: 12/01/2021 16:59 > Subject: [EXTERNAL] Re: [gpfsug-discuss] Disk in unrecovered state > Sent by: gpfsug-discuss-boun...@spectrumscale.org > > Hi Renar, >The version we are installed is 5.0.4-3, and the paths to these > wrong disks seems to be fine: > >[root@gpfs06 ~]# mmlsnsd -m| grep nsd18jbod1 > nsd18jbod1 0A0A00675EE76CF5 /dev/sdsgpfs05.ifca.es > server node > nsd18jbod1 0A0A00675EE76CF5 /dev/sdby gpfs06.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod1 > nsd19jbod1 0A0A00665EE76CF6 /dev/sdtgpfs05.ifca.es > server node > nsd19jbod1 0A0A00665EE76CF6 /dev/sdaa gpfs06.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod2 > nsd19jbod2 0A0A00695EE79A12 /dev/sdtgpfs07.ifca.es > server node > nsd19jbod2 0A0A00695EE79A12 /dev/sdat gpfs08.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd24jbod2 > nsd24jbod2 0A0A00685EE79749 /dev/sdbn gpfs07.ifca.es > server node > nsd24jbod2 0A0A00685EE79749 /dev/sdcg gpfs08.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd57jbod1 > nsd57jbod1 0A0A00665F243CE1 /dev/sdbg gpfs05.ifca.es > server node > nsd57jbod1 0A0A00665F243CE1 /dev/sdbx gpfs06.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd61jbod1 > nsd61jbod1 0A0A00665F243CFA /dev/sdbk gpfs05.ifca.es > server node > nsd61jbod1 0A0A00665F243CFA /dev/sdygpfs06.ifca.es > server node > [root@gpfs06 ~]# mmlsnsd -m| grep nsd71jbod1 > nsd71jbod1 0A0A00665F243D38 /dev/sdbu gpfs05.ifca.es > server node > nsd71jbod1 0A0A00665F243D38 /dev/sdbv gpfs06.ifca.es > server node > > trying to start 19jbod1 again: > [root@gpfs06 ~]# mmchdisk gpfs2 start -d nsd19jbod1 > mmnsddiscover: Attempting to rediscover the disks. This may take awhile ... > mmnsddiscover: Finished. > gpfs06.ifca.es: Rediscovered nsd server access to nsd19jbod1. > gpfs05.ifca.es: Rediscovered nsd server access to nsd19jbod1. > Failed to open gpfs2. > Log recovery failed. > Input/output error > Initial disk state was updated successfully, but another error may > have changed the state again. > mmchdisk: Command failed. Examine previous error messages to determine cause. > > Regards, I > > ___ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > INVALID URI REMOVED > u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss=DwICAg=jf_iaSHvJObTbx- > siA1ZOg=RGTETs2tk0Kz_VOpznDVDkqChhnfLapOTkxLvgmR2- > M=f4oAWXtPlhIm5cEShA0Amlf1ZUG3PyXvVbzB9e- > I3hk=SA1wXw8XXPjvMbSU6TILc2vnC4KxkfoboM8RolqBmuc= > ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Hi Guys, Devices seems to be accesible from both server primary and secondary, and thr harware state is "Optimal" [root@gpfs05 ~]# mmlsnsd -m| grep nsd18jbod1 nsd18jbod1 0A0A00675EE76CF5 /dev/sds gpfs05.ifca.es server node nsd18jbod1 0A0A00675EE76CF5 /dev/sdby gpfs06.ifca.es server node [root@gpfs05 ~]# #dd if=/dev/sds [root@gpfs05 ~]# man od [root@gpfs05 ~]# dd if=/dev/sds bs=4k count=2 | od -c 2+0 records in 2+0 records out 8192 bytes (8.2 kB) copied, 0.000249162 s, 32.9 MB/s 000 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 700 001 \0 356 376 377 377 001 \0 \0 \0 377 377 377 377 \0 \0 720 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 760 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 U 252 0001000 E F I P A R T \0 \0 001 \0 \ \0 \0 \0 0001020 \r 0 267 u \0 \0 \0 \0 001 \0 \0 \0 \0 \0 \0 \0 0001040 257 * 201 243 003 \0 \0 \0 " \0 \0 \0 \0 \0 \0 \0 0001060 216 * 201 243 003 \0 \0 \0 240 ! 302 3 . R \f M 0001100 200 241 323 024 245 h | G 002 \0 \0 \0 \0 \0 \0 \0 0001120 200 \0 \0 \0 200 \0 \0 \0 p b 203 F \0 \0 \0 \0 0001140 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 0002000 220 374 257 7 } 357 226 N 221 303 - z 340 U 261 t 0002020 316 343 324 ' } 033 K C 203 a 314 = 220 k 336 023 0002040 0 \0 \0 \0 \0 \0 \0 \0 177 * 201 243 003 \0 \0 \0 0002060 001 \0 \0 \0 \0 \0 \0 @ G \0 P \0 F \0 S \0 0002100 : \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 0002120 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 \0 * 002 Regards, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Definitely recommend getting a IBM Case in and ask someone for direct assistance (Zoom even). Then also check that you can access all of the underlying storage with READ ONLY operations from all defined NSD Servers in the NSD ServerList for nsd18jbod1 and nsd19jbod1. Given the name of the NSDs, sound like there is not any RAID protection on theses disks. If so then you would have serious data loss issues with one of the drives corrupted. Hope that helps, -Bryan From: gpfsug-discuss-boun...@spectrumscale.org On Behalf Of Iban Cabrillo Sent: Tuesday, January 12, 2021 8:32 AM To: gpfsug-discuss Subject: [gpfsug-discuss] Disk in unrecovered state [EXTERNAL EMAIL] Dear, Since this moning I have a couple of disk (7) in down state, I have tried to start them again but after that they change to unrecovered. These "failed" disk are only DATA. Both pool Data and Metadata has two failures groups, and set replica to 2. The Metadata disks are in two different enclosures one for each filure group. The filesystem has been unmounted , but when i have tried to run the mmfsck told me the I should remove the down disk [root@gpfs06 ~]# mmlsdisk gpfs2 -L | grep -v up disk driver sector failure holdsholds storage -- --- - - --- - . nsd18jbod1 nsd 512 2 No Yes to be emptied unrecovered 26 data nsd19jbod1 nsd 512 2 No Yes ready unrecovered 27 data nsd19jbod2 nsd 512 3 No Yes ready down 46 data nsd24jbod2 nsd 512 3 No Yes ready down 51 data nsd57jbod1 nsd 512 2 No Yes ready down 109 data nsd61jbod1 nsd 512 2 No Yes ready down 113 data nsd71jbod1 nsd 512 2 No Yes ready down 123 data . Any help is welcomed. Regards, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
I agree, this sounds like hardware. If you haven't already, checking 'dmesg' or /var/log/messages (or syslog) on the nsd19jbod1 server may highlight any disk/adapter issues. Also, as Olaf mentioned, you may want to engage support via a PMR. Good Luck, Nikhil - Original message -From: "Olaf Weiser" Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug-discuss@spectrumscale.orgCc: gpfsug-discuss@spectrumscale.orgSubject: [EXTERNAL] Re: [gpfsug-discuss] Disk in unrecovered stateDate: Tue, Jan 12, 2021 8:30 AM Hallo Iban, this seems to be a hardware issue Input/output error just try /make sure.. that you really can read from the disk .. all NSDs from all of its NSD servers so to say.. its most important, that the NSD is accessible on the primary NSD server.. as long this primary NSD server is up n running either use dd (be carfully ;-) .. only try to READ ) e.g. dd if=/dev/$yourblockdevice bs=4k count=2 | od -xc or mmfsadm test ...(if you are familiar with that - if not.. feel free, open a problem record.. and we'll guide your through ..) cheers olaf - Original message -From: Iban Cabrillo Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug-discuss Cc:Subject: [EXTERNAL] Re: [gpfsug-discuss] Disk in unrecovered stateDate: Tue, Jan 12, 2021 4:59 PM Hi Renar, The version we are installed is 5.0.4-3, and the paths to these wrong disks seems to be fine: [root@gpfs06 ~]# mmlsnsd -m| grep nsd18jbod1 nsd18jbod1 0A0A00675EE76CF5 /dev/sds gpfs05.ifca.es server node nsd18jbod1 0A0A00675EE76CF5 /dev/sdby gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod1 nsd19jbod1 0A0A00665EE76CF6 /dev/sdt gpfs05.ifca.es server node nsd19jbod1 0A0A00665EE76CF6 /dev/sdaa gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod2 nsd19jbod2 0A0A00695EE79A12 /dev/sdt gpfs07.ifca.es server node nsd19jbod2 0A0A00695EE79A12 /dev/sdat gpfs08.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd24jbod2 nsd24jbod2 0A0A00685EE79749 /dev/sdbn gpfs07.ifca.es server node nsd24jbod2 0A0A00685EE79749 /dev/sdcg gpfs08.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd57jbod1 nsd57jbod1 0A0A00665F243CE1 /dev/sdbg gpfs05.ifca.es server node nsd57jbod1 0A0A00665F243CE1 /dev/sdbx gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd61jbod1 nsd61jbod1 0A0A00665F243CFA /dev/sdbk gpfs05.ifca.es server node nsd61jbod1 0A0A00665F243CFA /dev/sdy gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd71jbod1 nsd71jbod1 0A0A00665F243D38 /dev/sdbu gpfs05.ifca.es server node nsd71jbod1 0A0A00665F243D38 /dev/sdbv gpfs06.ifca.es server nodetrying to start 19jbod1 again:[root@gpfs06 ~]# mmchdisk gpfs2 start -d nsd19jbod1mmnsddiscover: Attempting to rediscover the disks. This may take a while ...mmnsddiscover: Finished.gpfs06.ifca.es: Rediscovered nsd server access to nsd19jbod1.gpfs05.ifca.es: Rediscovered nsd server access to nsd19jbod1.Failed to open gpfs2.Log recovery failed.Input/output errorInitial disk state was updated successfully, but another error may have changed the state again.mmchdisk: Command failed. Examine previous error messages to determine cause.Regards, I___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Hallo Iban, this seems to be a hardware issue Input/output error just try /make sure.. that you really can read from the disk .. all NSDs from all of its NSD servers so to say.. its most important, that the NSD is accessible on the primary NSD server.. as long this primary NSD server is up n running either use dd (be carfully ;-) .. only try to READ ) e.g. dd if=/dev/$yourblockdevice bs=4k count=2 | od -xc or mmfsadm test ...(if you are familiar with that - if not.. feel free, open a problem record.. and we'll guide your through ..) cheers olaf - Original message -From: Iban Cabrillo Sent by: gpfsug-discuss-boun...@spectrumscale.orgTo: gpfsug-discuss Cc:Subject: [EXTERNAL] Re: [gpfsug-discuss] Disk in unrecovered stateDate: Tue, Jan 12, 2021 4:59 PM Hi Renar, The version we are installed is 5.0.4-3, and the paths to these wrong disks seems to be fine: [root@gpfs06 ~]# mmlsnsd -m| grep nsd18jbod1 nsd18jbod1 0A0A00675EE76CF5 /dev/sds gpfs05.ifca.es server node nsd18jbod1 0A0A00675EE76CF5 /dev/sdby gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod1 nsd19jbod1 0A0A00665EE76CF6 /dev/sdt gpfs05.ifca.es server node nsd19jbod1 0A0A00665EE76CF6 /dev/sdaa gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod2 nsd19jbod2 0A0A00695EE79A12 /dev/sdt gpfs07.ifca.es server node nsd19jbod2 0A0A00695EE79A12 /dev/sdat gpfs08.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd24jbod2 nsd24jbod2 0A0A00685EE79749 /dev/sdbn gpfs07.ifca.es server node nsd24jbod2 0A0A00685EE79749 /dev/sdcg gpfs08.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd57jbod1 nsd57jbod1 0A0A00665F243CE1 /dev/sdbg gpfs05.ifca.es server node nsd57jbod1 0A0A00665F243CE1 /dev/sdbx gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd61jbod1 nsd61jbod1 0A0A00665F243CFA /dev/sdbk gpfs05.ifca.es server node nsd61jbod1 0A0A00665F243CFA /dev/sdy gpfs06.ifca.es server node[root@gpfs06 ~]# mmlsnsd -m| grep nsd71jbod1 nsd71jbod1 0A0A00665F243D38 /dev/sdbu gpfs05.ifca.es server node nsd71jbod1 0A0A00665F243D38 /dev/sdbv gpfs06.ifca.es server nodetrying to start 19jbod1 again:[root@gpfs06 ~]# mmchdisk gpfs2 start -d nsd19jbod1mmnsddiscover: Attempting to rediscover the disks. This may take a while ...mmnsddiscover: Finished.gpfs06.ifca.es: Rediscovered nsd server access to nsd19jbod1.gpfs05.ifca.es: Rediscovered nsd server access to nsd19jbod1.Failed to open gpfs2.Log recovery failed.Input/output errorInitial disk state was updated successfully, but another error may have changed the state again.mmchdisk: Command failed. Examine previous error messages to determine cause.Regards, I___gpfsug-discuss mailing listgpfsug-discuss at spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Hi Renar, The version we are installed is 5.0.4-3, and the paths to these wrong disks seems to be fine: [root@gpfs06 ~]# mmlsnsd -m| grep nsd18jbod1 nsd18jbod1 0A0A00675EE76CF5 /dev/sdsgpfs05.ifca.es server node nsd18jbod1 0A0A00675EE76CF5 /dev/sdby gpfs06.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod1 nsd19jbod1 0A0A00665EE76CF6 /dev/sdtgpfs05.ifca.es server node nsd19jbod1 0A0A00665EE76CF6 /dev/sdaa gpfs06.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd19jbod2 nsd19jbod2 0A0A00695EE79A12 /dev/sdtgpfs07.ifca.es server node nsd19jbod2 0A0A00695EE79A12 /dev/sdat gpfs08.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd24jbod2 nsd24jbod2 0A0A00685EE79749 /dev/sdbn gpfs07.ifca.es server node nsd24jbod2 0A0A00685EE79749 /dev/sdcg gpfs08.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd57jbod1 nsd57jbod1 0A0A00665F243CE1 /dev/sdbg gpfs05.ifca.es server node nsd57jbod1 0A0A00665F243CE1 /dev/sdbx gpfs06.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd61jbod1 nsd61jbod1 0A0A00665F243CFA /dev/sdbk gpfs05.ifca.es server node nsd61jbod1 0A0A00665F243CFA /dev/sdygpfs06.ifca.es server node [root@gpfs06 ~]# mmlsnsd -m| grep nsd71jbod1 nsd71jbod1 0A0A00665F243D38 /dev/sdbu gpfs05.ifca.es server node nsd71jbod1 0A0A00665F243D38 /dev/sdbv gpfs06.ifca.es server node trying to start 19jbod1 again: [root@gpfs06 ~]# mmchdisk gpfs2 start -d nsd19jbod1 mmnsddiscover: Attempting to rediscover the disks. This may take a while ... mmnsddiscover: Finished. gpfs06.ifca.es: Rediscovered nsd server access to nsd19jbod1. gpfs05.ifca.es: Rediscovered nsd server access to nsd19jbod1. Failed to open gpfs2. Log recovery failed. Input/output error Initial disk state was updated successfully, but another error may have changed the state again. mmchdisk: Command failed. Examine previous error messages to determine cause. Regards, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
Re: [gpfsug-discuss] Disk in unrecovered state
Hallo Iban, first you should check the path to the disk. (mmlsnsd -m) It seems to be broken from the OS view. This should fixed first. If you see no dev entry you have a HW problem. If this is fixed then you can start each disk individuell to see there are something start here. On wich scale version do you are? Renar Grunenberg Abteilung Informatik - Betrieb HUK-COBURG Bahnhofsplatz 96444 Coburg Telefon:09561 96-44110 Telefax:09561 96-44104 E-Mail: renar.grunenb...@huk-coburg.de Internet: www.huk.de HUK-COBURG Haftpflicht-Unterstützungs-Kasse kraftfahrender Beamter Deutschlands a. G. in Coburg Reg.-Gericht Coburg HRB 100; St.-Nr. 9212/101/00021 Sitz der Gesellschaft: Bahnhofsplatz, 96444 Coburg Vorsitzender des Aufsichtsrats: Prof. Dr. Heinrich R. Schradin. Vorstand: Klaus-Jürgen Heitmann (Sprecher), Stefan Gronbach, Dr. Hans Olav Herøy, Dr. Jörg Rheinländer, Sarah Rössler, Thomas Sehn, Daniel Thomas. Diese Nachricht enthält vertrauliche und/oder rechtlich geschützte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese Nachricht irrtümlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Nachricht. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser Nachricht ist nicht gestattet. This information may contain confidential and/or privileged information. If you are not the intended recipient (or have received this information in error) please notify the sender immediately and destroy this information. Any unauthorized copying, disclosure or distribution of the material in this information is strictly forbidden. Von: gpfsug-discuss-boun...@spectrumscale.org Im Auftrag von Iban Cabrillo Gesendet: Dienstag, 12. Januar 2021 15:32 An: gpfsug-discuss Betreff: [gpfsug-discuss] Disk in unrecovered state Dear, Since this moning I have a couple of disk (7) in down state, I have tried to start them again but after that they change to unrecovered. These "failed" disk are only DATA. Both pool Data and Metadata has two failures groups, and set replica to 2. The Metadata disks are in two different enclosures one for each filure group. The filesystem has been unmounted , but when i have tried to run the mmfsck told me the I should remove the down disk [root@gpfs06 ~]# mmlsdisk gpfs2 -L | grep -v up disk driver sector failure holdsholds storage -- --- - - --- - . nsd18jbod1 nsd 512 2 No Yes to be emptied unrecovered 26 data nsd19jbod1 nsd 512 2 No Yes ready unrecovered 27 data nsd19jbod2 nsd 512 3 No Yes ready down 46 data nsd24jbod2 nsd 512 3 No Yes ready down 51 data nsd57jbod1 nsd 512 2 No Yes ready down 109 data nsd61jbod1 nsd 512 2 No Yes ready down 113 data nsd71jbod1 nsd 512 2 No Yes ready down 123 data . Any help is welcomed. Regards, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
[gpfsug-discuss] Disk in unrecovered state
Dear, Since this moning I have a couple of disk (7) in down state, I have tried to start them again but after that they change to unrecovered. These "failed" disk are only DATA. Both pool Data and Metadata has two failures groups, and set replica to 2. The Metadata disks are in two different enclosures one for each filure group. The filesystem has been unmounted , but when i have tried to run the mmfsck told me the I should remove the down disk [root@gpfs06 ~]# mmlsdisk gpfs2 -L | grep -v up disk driver sector failure holds holds storage -- --- - - --- - . nsd18jbod1 nsd 512 2 No Yes to be emptied unrecovered 26 data nsd19jbod1 nsd 512 2 No Yes ready unrecovered 27 data nsd19jbod2 nsd 512 3 No Yes ready down 46 data nsd24jbod2 nsd 512 3 No Yes ready down 51 data nsd57jbod1 nsd 512 2 No Yes ready down 109 data nsd61jbod1 nsd 512 2 No Yes ready down 113 data nsd71jbod1 nsd 512 2 No Yes ready down 123 data . Any help is welcomed. Regards, I ___ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
