i am not sure i understand your comment with 'persistent' do you mean when you create a nsddevice on a nvme device it won't get recognized after a restart ? if thats what you mean there are 2 answers , short term you need to add a /var/mmfs/etc/nsddevices script to your node that simply adds an echo for the nvme device like :
echo nvme0n1 generic this will tell the daemon to include that device on top of all other discovered devices that we include by default (like dm-* , sd*, etc) the longer term answer is that we have a tracking item to ad nvme* to the automatically discovered devices. on your second question, given that GPFS does workload balancing across devices you don't want to add extra complexity and path length to anything , so stick with raw devices . sven On Tue, Dec 6, 2016 at 8:40 AM Matt Weil <[email protected]> wrote: > Hello all, > > Thanks for sharing that. I am setting this up on our CES nodes. In this > example the nvme devices are not persistent. RHEL's default udev rules put > them in /dev/disk/by-id/ persistently by serial number so I modified > mmdevdiscover to look for them there. What are others doing? custom udev > rules for the nvme devices? > > Also I have used LVM in the past to stitch multiple nvme together for > better performance. I am wondering in the use case with GPFS that it may > hurt performance by hindering the ability of GPFS to do direct IO or > directly accessing memory. Any opinions there? > > Thanks > > Matt > On 12/5/16 10:33 AM, Ulf Troppens wrote: > > FYI ... in case not seen .... benchmark for LROC with NVMe > > http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/performance-gains-ibm-spectrum-scale.pdf > > > -- > IBM Spectrum Scale Development - Client Engagements & Solutions Delivery > Consulting IT Specialist > Author "Storage Networks Explained" > > IBM Deutschland Research & Development GmbH > Vorsitzende des Aufsichtsrats: Martina Koederitz > Geschäftsführung: Dirk Wittkopp > Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, > HRB 243294 > > > > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at > spectrumscale.orghttp://gpfsug.org/mailman/listinfo/gpfsug-discuss > > > > ------------------------------ > > The materials in this message are private and may contain Protected > Healthcare Information or other information of a sensitive nature. If you > are not the intended recipient, be advised that any unauthorized use, > disclosure, copying or the taking of any action in reliance on the contents > of this information is strictly prohibited. If you have received this email > in error, please immediately notify the sender via telephone or return mail. > _______________________________________________ > gpfsug-discuss mailing list > gpfsug-discuss at spectrumscale.org > http://gpfsug.org/mailman/listinfo/gpfsug-discuss >
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
