So is there a way to change it if it’s set incorrectly? jeff
From: [email protected] [mailto:[email protected]] On Behalf Of Jim Doherty Sent: Wednesday, January 17, 2018 6:28 PM To: gpfsug main discussion list <[email protected]> Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, nsddevices file Run a mmlsnsd -X I suspect you will see that GPFS is using one of the /dev/sd* "generic" paths to the LUN, not the /dev/mapper/ path. In our case the device is setup as dmm [root@service5 ~]# mmlsnsd -X Disk name NSD volume ID Device Devtype Node name Remarks --------------------------------------------------------------------------------------------------- volume1 0972B6CD587CD8E0 /dev/dm-0 dmm service5.pok.stglabs.ibm.com server node volume1 0972B6CD587CD8E0 /dev/dm-0 dmm service6.pok.stglabs.ibm.com server node volume2 0972B6CE587CD8E4 /dev/dm-4 dmm service5.pok.stglabs.ibm.com server node volume2 0972B6CE587CD8E4 /dev/dm-3 dmm service6.pok.stglabs.ibm.com server node volume3 0972B6CD587CD8E7 /dev/dm-1 dmm service5.pok.stglabs.ibm.com server node volume3 0972B6CD587CD8E7 /dev/dm-2 dmm service6.pok.stglabs.ibm.com server node volume4 0972B6CE587CF625 /dev/dm-3 dmm service5.pok.stglabs.ibm.com server node volume4 0972B6CE587CF625 /dev/dm-4 dmm service6.pok.stglabs.ibm.com server node [root@service5 ~]# grep volume1 /var/mmfs/gen/mmsdrfs | grep SG_DISK %%home%%:60_SG_DISKS:gpfs5:1:volume1:0:5001:dataAndMetadata:0972B6CD587CD8E0:nsd:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::other::dmm:user:::quorumDisk:ready::system:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::::: [root@service5 ~]# If you run an tspreparedisk -s it will show you all of the paths. [root@service5 ~]# tspreparedisk -s | grep 0972B6CD587CD8E0 0972B6CD587CD8E0 /dev/sda generic 0972B6CD587CD8E0 /dev/sdk generic 0972B6CD587CD8E0 /dev/sdu generic 0972B6CD587CD8E0 /dev/sdah generic 0972B6CD587CD8E0 /dev/dm-0 dmm [root@service5 ~]# Jim Jim On Wednesday, January 17, 2018, 5:12:10 PM EST, Bryan Banister <[email protected]<mailto:[email protected]>> wrote: Hi all, We are reviewing some of our configurations and were not sure what to make of the NSD Device Types that GPFS uses and what, if anything, do they change about how GPFS accesses/recovers/manages/etc the underlying storage based on this setting. The documentation doesn’t say much about it other than to consult the /usr/lpp/mmfs/bin/mmdevdiscover command (no man page), which has this section: # Known disk types currently are: # # powerdisk - EMC power path disk # vpath - IBM virtual path disk # dmm - Device-Mapper Multipath (DMM) # dlmfdrv - Hitachi dlm # hdisk - AIX hard disk # lv - AIX logical volume. Historical usage only. # Not allowed as a new device to mmcrnsd. # gpt - GPFS partition on Windows disk # generic - Device having no unique failover or multipathing # characteristic (predominantly Linux devices). # dasd - DASD device (for Linux on z Systems) We have our storage under Linux Device-Mapper Multipath control (two device paths to all storage, active/passive) and are accessible under /dev/mapper, but the NSD types are current set to ‘generic’ not ‘dmm’. This is configured in the /var/mmfs/etc/nsddevices file: if [[ $osName = Linux ]] then : # Add function to discover disks in the Linux environment. ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}' ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 " generic"}' ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}' fi Can somebody from IBM explain what the correct setting should be and what differences GPFS does with ‘generic’ vs. ‘dmm’ vs. others? Thanks in advance! -Bryan ________________________________ Note: This email is for the confidential use of the named addressee(s) only and may contain proprietary, confidential or privileged information. If you are not the intended recipient, you are hereby notified that any review, dissemination or copying of this email is strictly prohibited, and to please notify the sender immediately and destroy this email and any attachments. Email transmission cannot be guaranteed to be secure or error-free. The Company, therefore, does not make any guarantees as to the completeness or accuracy of this email or any attachments. This email is for informational purposes only and does not constitute a recommendation, offer, request or solicitation of any kind to buy, sell, subscribe, redeem or perform any type of transaction of a financial product. _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
