On 04/19/2017 12:53 PM, Buterbaugh, Kevin L wrote:
So you’re considering the purchase of a dual-controller FC storage array with 12 or so 1.8 TB SSD’s in it, with the idea being that that storage would be in its’ own storage pool and that pool would be the default location for I/O for your main filesystem … at least for smaller files. You intend to use mmapplypolicy nightly to move data to / from this pool and the spinning disk pools.
We did this and failed in interesting (but in retrospect obvious) ways. You will want to ensure that your users cannot fill your write target pool within a day. The faster the storage, the more likely that is to happen. Or else your users will get ENOSPC. You will want to ensure that your pools can handle the additional I/O from the migration in aggregate with all the user I/O. Or else your users will see worse performance from the fast pool than the slow pool while the migration is running. You will want to make sure that the write throughput of your slow pool is faster than the read throughput of your fast pool.
In our case, the fast pool was undersized in capacity, and oversized in terms of performance. And overall the filesystem was oversubscribed (~100 10GbE clients, 8 x 10GbE NSD servers) So the fast pool would fill very quickly. Then I would switch the placement policy to the big slow pool and performance would drop dramatically, and then if I ran a migration it would either (depending on parameters) take up all the I/O to the slow pool (leaving none for the users), or else take forever (weeks) because the user I/O was maxing out the slow pool.
Things should better today with QoS stuff, but your relative pool capacities (in our case it was like 1% fast, 99% slow) and your relative pool performance (in our case, slow pool had fewer IOPS than fast pool) are still going to matter a lot.
-- Alex Chekholko [email protected] _______________________________________________ gpfsug-discuss mailing list gpfsug-discuss at spectrumscale.org http://gpfsug.org/mailman/listinfo/gpfsug-discuss
