Hi All, Just wanted to follow up with the results of my survey … I received a grand total of two responses (Thanks Alex and John). In their case, they’re using SSDs with a 10 DWPD rating.
The motivation behind my asking this question was … money! ;-). Seriously, 10 DWPD drives are still very expensive, while 3 DWPD drives are significantly less expensive and 1 DWPD drives are even cheaper still. While we would NOT feel comfortable using anything less than 10 DWPD drives for metadata, we’re wondering about using less expensive drives for data. For example, let’s just say that you’re getting ready to set up a brand new GPFS 5 formatted filesystem of 1-2 PB in size. You’re considering having 3 pools: 1) a metadata only system pool of 10 DWPD SSDs. 4K inodes, and a ton of small files that’ll fit in the inode. 2) a data only “hot” pool (i.e. the default pool for writes) of SSDs. 3) a data only “capacity” pool of 12 TB spinning disks. And let’s just say that you have looked back at the historical data you’ve collected and you see that over the last 6 months or so you’ve been averaging 10-12 TB of data being written into your existing filesystem per day. You want to do migrations between pools only on the weekends if at all possible. 12 * 7 = 84 TB. So if you had somewhere between 125 - 150 TB of SSDs ... 1 DWPD SSDs … then in theory you should easily be able to handle your anticipated workload without coming close to exceeding the 1 DWPD rating of the SSDs. However, as the saying goes, while in theory there’s no difference between theory and practice, in practice there is ... so am I overlooking anything here from a GPFS perspective??? If anybody still wants to respond on the DWPD rating of the SSDs they use for data, I’m still listening. Thanks… Kevin P.S. I still have a couple of “outstanding issues” to respond to that I’ve posted to the list about previously: 1) the long I/O’s we see occasionally in the output of “mmdiag —iohist” on our NSD servers. We’re still trying to track that down … it seems to happen only with a subset of our hardware - most of the time at least - but we’re still working to track down what triggers it … i.e. at this point I can’t say whether it’s really the hardware or a user abusing the hardware. 2) I promised to post benchmark results of 3 different metadata configs: a) RAID 1 mirrors, b) a RAID 5 stripe, c) no RAID, but GPFS metadata replication of 3. That benchmarking has been put on hold for reasons I can’t really discuss on this mailing list at this time … but hopefully soon. I haven’t forgotten the above and will respond back on the list when it’s appropriate. Thanks... On Mar 8, 2019, at 10:24 AM, Buterbaugh, Kevin L <[email protected]<mailto:[email protected]>> wrote: Hi All, This is kind of a survey if you will, so for this one it might be best if you responded directly to me and I’ll summarize the results next week. Question 1 - do you use SSDs for data? If not - i.e. if you only use SSDs for metadata (as we currently do) - thanks, that’s all! If, however, you do use SSDs for data, please see Question 2. Question 2 - what is the DWPD (daily writes per day) of the SSDs that you use for data? Question 3 - is that different than the DWPD of the SSDs for metadata? Question 4 - any pertinent information in regards to your answers above (i.e. if you’ve got a filesystem that data is uploaded to only once and never modified after that then that’s useful to know!)? Thanks… Kevin — Kevin Buterbaugh - Senior System Administrator Vanderbilt University - Advanced Computing Center for Research and Education [email protected]<mailto:[email protected]> - (615)875-9633
_______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
