Run a mmlsnsd -X I suspect you will see that GPFS is using one of the
/dev/sd* "generic" paths to the LUN, not the /dev/mapper/ path. In our case
the device is setup as dmm
[root@service5 ~]# mmlsnsd -X
Disk name NSD volume ID Device Devtype Node name
Remarks
---------------------------------------------------------------------------------------------------
volume1 0972B6CD587CD8E0 /dev/dm-0 dmm
service5.pok.stglabs.ibm.com server node
volume1 0972B6CD587CD8E0 /dev/dm-0 dmm
service6.pok.stglabs.ibm.com server node
volume2 0972B6CE587CD8E4 /dev/dm-4 dmm
service5.pok.stglabs.ibm.com server node
volume2 0972B6CE587CD8E4 /dev/dm-3 dmm
service6.pok.stglabs.ibm.com server node
volume3 0972B6CD587CD8E7 /dev/dm-1 dmm
service5.pok.stglabs.ibm.com server node
volume3 0972B6CD587CD8E7 /dev/dm-2 dmm
service6.pok.stglabs.ibm.com server node
volume4 0972B6CE587CF625 /dev/dm-3 dmm
service5.pok.stglabs.ibm.com server node
volume4 0972B6CE587CF625 /dev/dm-4 dmm
service6.pok.stglabs.ibm.com server node
[root@service5 ~]# grep volume1 /var/mmfs/gen/mmsdrfs | grep SG_DISK
%%home%%:60_SG_DISKS:gpfs5:1:volume1:0:5001:dataAndMetadata:0972B6CD587CD8E0:nsd:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::other::dmm:user:::quorumDisk:ready::system:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com:::::
[root@service5 ~]#
If you run an tspreparedisk -s it will show you all of the paths.
[root@service5 ~]# tspreparedisk -s | grep 0972B6CD587CD8E0
0972B6CD587CD8E0 /dev/sda generic
0972B6CD587CD8E0 /dev/sdk generic
0972B6CD587CD8E0 /dev/sdu generic
0972B6CD587CD8E0 /dev/sdah generic
0972B6CD587CD8E0 /dev/dm-0 dmm
[root@service5 ~]#
Jim
Jim
On Wednesday, January 17, 2018, 5:12:10 PM EST, Bryan Banister
<[email protected]> wrote:
<!--#yiv9990787954 _filtered #yiv9990787954 {font-family:"Cambria
Math";panose-1:2 4 5 3 5 4 6 3 2 4;} _filtered #yiv9990787954
{font-family:Calibri;panose-1:2 15 5 2 2 2 4 3 2 4;}#yiv9990787954
#yiv9990787954 p.yiv9990787954MsoNormal, #yiv9990787954
li.yiv9990787954MsoNormal, #yiv9990787954 div.yiv9990787954MsoNormal
{margin:0in;margin-bottom:.0001pt;font-size:11.0pt;font-family:"Calibri",
sans-serif;}#yiv9990787954 a:link, #yiv9990787954
span.yiv9990787954MsoHyperlink
{color:#0563C1;text-decoration:underline;}#yiv9990787954 a:visited,
#yiv9990787954 span.yiv9990787954MsoHyperlinkFollowed
{color:#954F72;text-decoration:underline;}#yiv9990787954
span.yiv9990787954EmailStyle17 {font-family:"Calibri",
sans-serif;color:windowtext;}#yiv9990787954 .yiv9990787954MsoChpDefault
{font-family:"Calibri", sans-serif;} _filtered #yiv9990787954 {margin:1.0in
1.0in 1.0in 1.0in;}#yiv9990787954 div.yiv9990787954WordSection1 {}-->
Hi all,
We are reviewing some of our configurations and were not sure what to make of
the NSD Device Types that GPFS uses and what, if anything, do they change about
how GPFS accesses/recovers/manages/etc the underlying storage based on this
setting.
The documentation doesn’t say much about it other than to consult the
/usr/lpp/mmfs/bin/mmdevdiscover command (no man page), which has this section:
# Known disk types currently are:
#
# powerdisk - EMC power path disk
# vpath - IBM virtual path disk
# dmm - Device-Mapper Multipath (DMM)
# dlmfdrv - Hitachi dlm
# hdisk - AIX hard disk
# lv - AIX logical volume. Historical usage only.
# Not allowed as a new device to mmcrnsd.
# gpt - GPFS partition on Windows disk
# generic - Device having no unique failover or multipathing
# characteristic (predominantly Linux devices).
# dasd - DASD device (for Linux on z Systems)
We have our storage under Linux Device-Mapper Multipath control (two device
paths to all storage, active/passive) and are accessible under /dev/mapper, but
the NSD types are current set to ‘generic’ not ‘dmm’. This is configured in
the /var/mmfs/etc/nsddevices file:
if [[ $osName = Linux ]]
then
: # Add function to discover disks in the Linux environment.
ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}'
ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 "generic"}'
ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}'
fi
Can somebody from IBM explain what the correct setting should be and what
differences GPFS does with ‘generic’ vs. ‘dmm’ vs. others?
Thanks in advance!
-Bryan
Note: This email is for the confidential use of the named addressee(s) only and
may contain proprietary, confidential or privileged information. If you are not
the intended recipient, you are hereby notified that any review, dissemination
or copying of this email is strictly prohibited, and to please notify the
sender immediately and destroy this email and any attachments. Email
transmission cannot be guaranteed to be secure or error-free. The Company,
therefore, does not make any guarantees as to the completeness or accuracy of
this email or any attachments. This email is for informational purposes only
and does not constitute a recommendation, offer, request or solicitation of any
kind to buy, sell, subscribe, redeem or perform any type of transaction of a
financial product.
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at spectrumscale.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
