I would say that's what it *appears* to be doing. The sys admin in me would still want some "official" word from IBM, as I wouldn't my assumption to be the cause of a meltdown at 16.50 on a Friday when everyone is heading out for the weekend ;)
Cheers, -Eric On Thu, Jan 18, 2018 at 2:09 PM, Bryan Banister <[email protected]> wrote: > Great! So this is just for selecting devices and according to my `mmlsnsd > -X` output it's using the correct ones, so I can probably return to ignoring > this parameter! > -Bryan > > -----Original Message----- > From: [email protected] > [mailto:[email protected]] On Behalf Of Eric Ross > Sent: Thursday, January 18, 2018 2:01 PM > To: gpfsug main discussion list <[email protected]> > Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, > nsddevices file > > Note: External Email > ------------------------------------------------- > > Bryan, > > While waiting on the "official" word as to what each setting does > differently, I remembered there was a brief explanation of the > differences (at least of dmm vs generic) in the > /var/mmfs/etc/nsddevices included with the GPFS-goodies toolkit > (https://github.com/finley/GPFS-Goodies) I use to use when I was at > IBM. > > //snip > > dmm vs. generic is used by GPFS to prioritize internal order of > searching through available disks, then later GPFS discards other > disk device names that it finds that match as the same NSD device > by a different path. For this reason, dmm vs. generic is an > important distinction if you are not explicitly producing the > entire and exclusive set of disks that GPFS should use, as output > from this script (nsddevices) _and_ exiting this script with a > "return 0". -Brian Finley > > //end snip > > If I read that correctly, it would indicate GPFS uses those additional > labels (at least dmm vs generic) as a mechanism to determine which > device files to prefer when scanning a system and finding the same NSD > available via different devices (i.e. /dev/mapper/foo vs > /dev/sdq,/dev/sdx). By associating dmm with the dm-multipathed > device, I guess it would just ignore the /dev/sd${foo} devices when it > scans them. Also, the difference only seems to matter if you're not > explicitly creating a list a Brian F. indicates. If you simply > generate the list and exit (via return 0), GPFS wouldn't continue > scanning, and then find the associated /dev/sd${foo} devices to begin > with, and therefore wouldn't need a label like dmm to prioritize it > over say a generic device. > > > - Eric > > On Thu, Jan 18, 2018 at 10:59 AM, Bryan Banister > <[email protected]> wrote: >> Yes, just change the /var/mmfs/etc/nsddevices file so that it sets the >> Devtype to the “correct” setting, for example: >> >> >> >> if [[ $osName = Linux ]] >> >> then >> >> : # Add function to discover disks in the Linux environment. >> >> ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}' >> >> ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 " dmm"}' >> >> ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}' >> >> fi >> >> >> >> My question is what is the correct setting? >> >> >> >> And what does GPFS do differently with each setting? >> >> >> >> Thanks, >> >> -Bryan >> >> >> >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Jeffrey R. >> Lang >> Sent: Thursday, January 18, 2018 9:20 AM >> To: gpfsug main discussion list <[email protected]> >> Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, >> nsddevices file >> >> >> >> Note: External Email >> >> ________________________________ >> >> So is there a way to change it if it’s set incorrectly? >> >> >> >> jeff >> >> >> >> From: [email protected] >> [mailto:[email protected]] On Behalf Of Jim Doherty >> Sent: Wednesday, January 17, 2018 6:28 PM >> To: gpfsug main discussion list <[email protected]> >> Subject: Re: [gpfsug-discuss] Question about NSD "Devtype" setting, >> nsddevices file >> >> >> >> >> >> Run a mmlsnsd -X I suspect you will see that GPFS is using one of the >> /dev/sd* "generic" paths to the LUN, not the /dev/mapper/ path. In our >> case the device is setup as dmm >> >> >> >> [root@service5 ~]# mmlsnsd -X >> >> Disk name NSD volume ID Device Devtype Node name >> Remarks >> --------------------------------------------------------------------------------------------------- >> volume1 0972B6CD587CD8E0 /dev/dm-0 dmm >> service5.pok.stglabs.ibm.com server node >> volume1 0972B6CD587CD8E0 /dev/dm-0 dmm >> service6.pok.stglabs.ibm.com server node >> volume2 0972B6CE587CD8E4 /dev/dm-4 dmm >> service5.pok.stglabs.ibm.com server node >> volume2 0972B6CE587CD8E4 /dev/dm-3 dmm >> service6.pok.stglabs.ibm.com server node >> volume3 0972B6CD587CD8E7 /dev/dm-1 dmm >> service5.pok.stglabs.ibm.com server node >> volume3 0972B6CD587CD8E7 /dev/dm-2 dmm >> service6.pok.stglabs.ibm.com server node >> volume4 0972B6CE587CF625 /dev/dm-3 dmm >> service5.pok.stglabs.ibm.com server node >> volume4 0972B6CE587CF625 /dev/dm-4 dmm >> service6.pok.stglabs.ibm.com server node >> >> [root@service5 ~]# grep volume1 /var/mmfs/gen/mmsdrfs | grep SG_DISK >> %%home%%:60_SG_DISKS:gpfs5:1:volume1:0:5001:dataAndMetadata:0972B6CD587CD8E0:nsd:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::other::dmm:user:::quorumDisk:ready::system:service5.pok.stglabs.ibm.com,service6.pok.stglabs.ibm.com::::: >> [root@service5 ~]# >> >> >> >> If you run an tspreparedisk -s it will show you all of the paths. >> >> >> >> [root@service5 ~]# tspreparedisk -s | grep 0972B6CD587CD8E0 >> 0972B6CD587CD8E0 /dev/sda generic >> 0972B6CD587CD8E0 /dev/sdk generic >> 0972B6CD587CD8E0 /dev/sdu generic >> 0972B6CD587CD8E0 /dev/sdah generic >> 0972B6CD587CD8E0 /dev/dm-0 dmm >> [root@service5 ~]# >> >> >> >> Jim >> >> >> >> >> >> >> >> Jim >> >> On Wednesday, January 17, 2018, 5:12:10 PM EST, Bryan Banister >> <[email protected]> wrote: >> >> >> >> >> >> Hi all, >> >> >> >> We are reviewing some of our configurations and were not sure what to make >> of the NSD Device Types that GPFS uses and what, if anything, do they change >> about how GPFS accesses/recovers/manages/etc the underlying storage based on >> this setting. >> >> >> >> The documentation doesn’t say much about it other than to consult the >> /usr/lpp/mmfs/bin/mmdevdiscover command (no man page), which has this >> section: >> >> >> >> # Known disk types currently are: >> >> # >> >> # powerdisk - EMC power path disk >> >> # vpath - IBM virtual path disk >> >> # dmm - Device-Mapper Multipath (DMM) >> >> # dlmfdrv - Hitachi dlm >> >> # hdisk - AIX hard disk >> >> # lv - AIX logical volume. Historical usage only. >> >> # Not allowed as a new device to mmcrnsd. >> >> # gpt - GPFS partition on Windows disk >> >> # generic - Device having no unique failover or multipathing >> >> # characteristic (predominantly Linux devices). >> >> # dasd - DASD device (for Linux on z Systems) >> >> >> >> We have our storage under Linux Device-Mapper Multipath control (two device >> paths to all storage, active/passive) and are accessible under /dev/mapper, >> but the NSD types are current set to ‘generic’ not ‘dmm’. This is >> configured in the /var/mmfs/etc/nsddevices file: >> >> >> >> if [[ $osName = Linux ]] >> >> then >> >> : # Add function to discover disks in the Linux environment. >> >> ls -l /dev/mpath/ 2>/dev/null | awk '{print "mpath/"$9 " generic"}' >> >> ls -l /dev/mapper/ 2>/dev/null | awk '{print "mapper/"$9 " generic"}' >> >> ls -1 /dev/vd* 2>/dev/null | awk -F '/' '{print ""$3 " generic"}' >> >> fi >> >> >> >> Can somebody from IBM explain what the correct setting should be and what >> differences GPFS does with ‘generic’ vs. ‘dmm’ vs. others? >> >> >> >> Thanks in advance! >> >> -Bryan >> >> >> >> ________________________________ >> >> >> Note: This email is for the confidential use of the named addressee(s) only >> and may contain proprietary, confidential or privileged information. If you >> are not the intended recipient, you are hereby notified that any review, >> dissemination or copying of this email is strictly prohibited, and to please >> notify the sender immediately and destroy this email and any attachments. >> Email transmission cannot be guaranteed to be secure or error-free. The >> Company, therefore, does not make any guarantees as to the completeness or >> accuracy of this email or any attachments. This email is for informational >> purposes only and does not constitute a recommendation, offer, request or >> solicitation of any kind to buy, sell, subscribe, redeem or perform any type >> of transaction of a financial product. >> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at spectrumscale.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >> >> >> ________________________________ >> >> Note: This email is for the confidential use of the named addressee(s) only >> and may contain proprietary, confidential or privileged information. If you >> are not the intended recipient, you are hereby notified that any review, >> dissemination or copying of this email is strictly prohibited, and to please >> notify the sender immediately and destroy this email and any attachments. >> Email transmission cannot be guaranteed to be secure or error-free. The >> Company, therefore, does not make any guarantees as to the completeness or >> accuracy of this email or any attachments. This email is for informational >> purposes only and does not constitute a recommendation, offer, request or >> solicitation of any kind to buy, sell, subscribe, redeem or perform any type >> of transaction of a financial product. >> >> _______________________________________________ >> gpfsug-discuss mailing list >> gpfsug-discuss at spectrumscale.org >> http://gpfsug.org/mailman/listinfo/gpfsug-discuss >> > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss > > ________________________________ > > Note: This email is for the confidential use of the named addressee(s) only > and may contain proprietary, confidential or privileged information. If you > are not the intended recipient, you are hereby notified that any review, > dissemination or copying of this email is strictly prohibited, and to please > notify the sender immediately and destroy this email and any attachments. > Email transmission cannot be guaranteed to be secure or error-free. The > Company, therefore, does not make any guarantees as to the completeness or > accuracy of this email or any attachments. This email is for informational > purposes only and does not constitute a recommendation, offer, request or > solicitation of any kind to buy, sell, subscribe, redeem or perform any type > of transaction of a financial product. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
