What is your unmountOnDiskFail configuration setting on the cluster? You need to set unmountOnDiskFail to meta if you only have metadata replication.
Steve Y. Xiao > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 13 Apr 2018 20:05:53 +0000 > From: "Simon Thompson (IT Research Support)" <s.j.thomp...@bham.ac.uk> > To: "gpfsug-discuss@spectrumscale.org" > <gpfsug-discuss@spectrumscale.org> > Subject: [gpfsug-discuss] Replicated and non replicated data > Message-ID: <98f781f7-7063-4293-a5bc-1e8f5a0c9...@bham.ac.uk> > Content-Type: text/plain; charset="utf-8" > > I have a question about file-systems with replicated an non replicated data. > > We have a file-system where metadata is set to copies=2 and data > copies=2, we then use a placement policy to selectively replicate > some data only once based on file-set. We also place the non- > replicated data into a specific pool (6tnlsas) to ensure we know > where it is placed. > > My understanding was that in doing this, if we took the disks with > the non replicated data offline, we?d still have the FS available > for users as the metadata is replicated. Sure accessing a non- > replicated data file would give an IO error, but the rest of the FS > should be up. > > We had a situation today where we wanted to take stg01 offline > today, so tried using mmchdisk stop -d ?. Once we got to about disk > stg01-01_12_12, GPFS would refuse to stop any more disks and > complain about too many disks, similarly if we shutdown the NSD > servers hosting the disks, the filesystem would have an SGPanic and > force unmount. > > First, am I correct in thinking that a FS with non-replicated data, > but replicated metadata should still be accessible (not the non- > replicated data) when the LUNS hosting it are down? > > If so, any suggestions why my FS is panic-ing when we take down the > one set of disks? > > I thought at first we had some non-replicated metadata, tried a > mmrestripefs -R ?metadata-only to force it to ensure 2 replicas, but > this didn?t help. > > Running 5.0.0.2 on the NSD server nodes. > > (First time we went round this we didn?t have a FS descriptor disk, > but you can see below that we added this) > > Thanks > > Simon > > [root@nsd01 ~]# mmlsdisk castles -L > disk driver sector failure holds holds storage > name type size group metadata data status > availability disk id pool remarks > ------------ -------- ------ ----------- -------- ----- > ------------- ------------ ------- ------------ --------- > CASTLES_GPFS_DESCONLY01 nsd 512 310 no no > ready up 1 system desc > stg01-01_3_3 nsd 4096 210 no yes ready > down 4 6tnlsas > stg01-01_4_4 nsd 4096 210 no yes ready > down 5 6tnlsas > stg01-01_5_5 nsd 4096 210 no yes ready > down 6 6tnlsas > stg01-01_6_6 nsd 4096 210 no yes ready > down 7 6tnlsas > stg01-01_7_7 nsd 4096 210 no yes ready > down 8 6tnlsas > stg01-01_8_8 nsd 4096 210 no yes ready > down 9 6tnlsas > stg01-01_9_9 nsd 4096 210 no yes ready > down 10 6tnlsas > stg01-01_10_10 nsd 4096 210 no yes ready > down 11 6tnlsas > stg01-01_11_11 nsd 4096 210 no yes ready > down 12 6tnlsas > stg01-01_12_12 nsd 4096 210 no yes ready > down 13 6tnlsas > stg01-01_13_13 nsd 4096 210 no yes ready > down 14 6tnlsas > stg01-01_14_14 nsd 4096 210 no yes ready > down 15 6tnlsas > stg01-01_15_15 nsd 4096 210 no yes ready > down 16 6tnlsas > stg01-01_16_16 nsd 4096 210 no yes ready > down 17 6tnlsas > stg01-01_17_17 nsd 4096 210 no yes ready > down 18 6tnlsas > stg01-01_18_18 nsd 4096 210 no yes ready > down 19 6tnlsas > stg01-01_19_19 nsd 4096 210 no yes ready > down 20 6tnlsas > stg01-01_20_20 nsd 4096 210 no yes ready > down 21 6tnlsas > stg01-01_21_21 nsd 4096 210 no yes ready > down 22 6tnlsas > stg01-01_ssd_54_54 nsd 4096 210 yes no ready > down 23 system > stg01-01_ssd_56_56 nsd 4096 210 yes no ready > down 24 system > stg02-01_0_0 nsd 4096 110 no yes ready > up 25 6tnlsas > stg02-01_1_1 nsd 4096 110 no yes ready > up 26 6tnlsas > stg02-01_2_2 nsd 4096 110 no yes ready > up 27 6tnlsas > stg02-01_3_3 nsd 4096 110 no yes ready > up 28 6tnlsas > stg02-01_4_4 nsd 4096 110 no yes ready > up 29 6tnlsas > stg02-01_5_5 nsd 4096 110 no yes ready > up 30 6tnlsas > stg02-01_6_6 nsd 4096 110 no yes ready > up 31 6tnlsas > stg02-01_7_7 nsd 4096 110 no yes ready > up 32 6tnlsas > stg02-01_8_8 nsd 4096 110 no yes ready > up 33 6tnlsas > stg02-01_9_9 nsd 4096 110 no yes ready > up 34 6tnlsas > stg02-01_10_10 nsd 4096 110 no yes ready > up 35 6tnlsas > stg02-01_11_11 nsd 4096 110 no yes ready > up 36 6tnlsas > stg02-01_12_12 nsd 4096 110 no yes ready > up 37 6tnlsas > stg02-01_13_13 nsd 4096 110 no yes ready > up 38 6tnlsas > stg02-01_14_14 nsd 4096 110 no yes ready > up 39 6tnlsas > stg02-01_15_15 nsd 4096 110 no yes ready > up 40 6tnlsas > stg02-01_16_16 nsd 4096 110 no yes ready > up 41 6tnlsas > stg02-01_17_17 nsd 4096 110 no yes ready > up 42 6tnlsas > stg02-01_18_18 nsd 4096 110 no yes ready > up 43 6tnlsas > stg02-01_19_19 nsd 4096 110 no yes ready > up 44 6tnlsas > stg02-01_20_20 nsd 4096 110 no yes ready > up 45 6tnlsas > stg02-01_21_21 nsd 4096 110 no yes ready > up 46 6tnlsas > stg02-01_ssd_22_22 nsd 4096 110 yes no ready > up 47 system desc > stg02-01_ssd_23_23 nsd 4096 110 yes no ready > up 48 system > stg02-01_ssd_24_24 nsd 4096 110 yes no ready > up 49 system > stg02-01_ssd_25_25 nsd 4096 110 yes no ready > up 50 system > stg01-01_22_22 nsd 4096 210 no yes ready > up 51 6tnlsasnonrepl desc > stg01-01_23_23 nsd 4096 210 no yes ready > up 52 6tnlsasnonrepl > stg01-01_24_24 nsd 4096 210 no yes ready > up 53 6tnlsasnonrepl > stg01-01_25_25 nsd 4096 210 no yes ready > up 54 6tnlsasnonrepl > stg01-01_26_26 nsd 4096 210 no yes ready > up 55 6tnlsasnonrepl > stg01-01_27_27 nsd 4096 210 no yes ready > up 56 6tnlsasnonrepl > stg01-01_31_31 nsd 4096 210 no yes ready > up 58 6tnlsasnonrepl > stg01-01_32_32 nsd 4096 210 no yes ready > up 59 6tnlsasnonrepl > stg01-01_33_33 nsd 4096 210 no yes ready > up 60 6tnlsasnonrepl > stg01-01_34_34 nsd 4096 210 no yes ready > up 61 6tnlsasnonrepl > stg01-01_35_35 nsd 4096 210 no yes ready > up 62 6tnlsasnonrepl > stg01-01_36_36 nsd 4096 210 no yes ready > up 63 6tnlsasnonrepl > stg01-01_37_37 nsd 4096 210 no yes ready > up 64 6tnlsasnonrepl > stg01-01_38_38 nsd 4096 210 no yes ready > up 65 6tnlsasnonrepl > stg01-01_39_39 nsd 4096 210 no yes ready > up 66 6tnlsasnonrepl > stg01-01_40_40 nsd 4096 210 no yes ready > up 67 6tnlsasnonrepl > stg01-01_41_41 nsd 4096 210 no yes ready > up 68 6tnlsasnonrepl > stg01-01_42_42 nsd 4096 210 no yes ready > up 69 6tnlsasnonrepl > stg01-01_43_43 nsd 4096 210 no yes ready > up 70 6tnlsasnonrepl > stg01-01_44_44 nsd 4096 210 no yes ready > up 71 6tnlsasnonrepl > stg01-01_45_45 nsd 4096 210 no yes ready > up 72 6tnlsasnonrepl > stg01-01_46_46 nsd 4096 210 no yes ready > up 73 6tnlsasnonrepl > stg01-01_47_47 nsd 4096 210 no yes ready > up 74 6tnlsasnonrepl > stg01-01_48_48 nsd 4096 210 no yes ready > up 75 6tnlsasnonrepl > stg01-01_49_49 nsd 4096 210 no yes ready > up 76 6tnlsasnonrepl > stg01-01_50_50 nsd 4096 210 no yes ready > up 77 6tnlsasnonrepl > stg01-01_51_51 nsd 4096 210 no yes ready > up 78 6tnlsasnonrepl > Number of quorum disks: 3 > Read quorum value: 2 > Write quorum value: 2 > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: <https://urldefense.proofpoint.com/v2/url? > u=http-3A__gpfsug.org_pipermail_gpfsug-2Ddiscuss_attachments_20180413_c22c8133_attachment.html&d=DwICAg&c=jf_iaSHvJObTbx- > siA1ZOg&r=ck4PYlaRFvCcNKlfHMPhoA&m=BX4uqSaNFY5Jl4ZNPLYjML8nanjAa57Nuz_7J2jSqMs&s=2P7GHehsFTuGZ39pBTBsUzcdwo9jkidie2etD8_llas&e= > > > > ------------------------------ > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > https://urldefense.proofpoint.com/v2/url? > u=http-3A__gpfsug.org_mailman_listinfo_gpfsug-2Ddiscuss&d=DwICAg&c=jf_iaSHvJObTbx- > siA1ZOg&r=ck4PYlaRFvCcNKlfHMPhoA&m=BX4uqSaNFY5Jl4ZNPLYjML8nanjAa57Nuz_7J2jSqMs&s=Q5EVJvSbunfieiHUrDHMpC3WAhP1fX2sQFwLLgLFb8Y&e= > > > End of gpfsug-discuss Digest, Vol 75, Issue 23 > ********************************************** >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss