hi bob,
- One thing you might consider (depending on your pattern of read/write
traffic), is selectively suspending one or more of the existing NSDs,
forcing GPFS to write new blocks to the new NSDs. That way at least some of
the new data is being written to the new storage by default, rather than
using up blocks on the existing NSDs. You can suspend/resume disks at any
time.
is the gpfs placment weighted with the avalaible volume? i'd rather not
make this a manual operation.
- You can pick a subset of nodes to perform the restripe with "mmrestripefs
-N node1,node2,..." Keep in mind you'll get much better performance and
less impact to the filesystem if you choose NSD servers with direct access
to the disk.
yes and i no i guess, our nsds see all disks, but the problem with nsds
is that they don't honour any roles (our primary nsds have the preferred
path to the controller and lun, meaning all access from non-primary nsd
to that disk is suboptimal).
- Resume of restripe: Yes, you can do this, no harm, done it many times.
You can track the balance of the disks using "mmdf <filesystem>". This is a
pretty intensive command, so I wouldn't run in frequently. Check it a few
times each day, see if the data balance is improving by itself. When you
thanks for the tip to monitor it with mmdf!
stop/restart it, the restripe doesn't pick up exactly where it left off,
it's going to scan the entire file system again.
yeah, i realised that this is a flaw in my "one-hour a day" restripe idea ;)
- You can also restripe single files if the are large and get a heavy I/O
(mmrestripefile)
excellent tip! forgot about that one. if the rebalnce is to slow, i can
run this on the static data.
thanks a lot for the feedback
stijn
Bob Oesterlin
Nuance Communications
On Mon, Nov 24, 2014 at 3:22 PM, Stijn De Weirdt <[email protected]>
wrote:
hi all,
we are going to expand an existing filestytem with approx 50% capacity.
the current filesystem is 75% full.
we are in downtime (for more then just this reason), so we can take the IO
rebalance hit for a while (say max 48hours).
my questions:
a. do we really need to rebalance? the mmadddisk page suggest normally
it's going to be ok, but i never understood that. new data will end up
mainly on new disks, so wrt to performance, this can't really work out, can
it?
b. can we change the priority of rebalancing somehow (fewer nodes taking
part in the rebalance?)
c. once we start the rebalance, how save is it to stop with kill or ctrl-c
(or can we say eg. rebalance 25% now, rest later?)
(and how often can we do this? eg a daily cron job to restripe at max one
hour per day, would this cause issue in the long term
many thanks,
stijn
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
_______________________________________________
gpfsug-discuss mailing list
gpfsug-discuss at gpfsug.org
http://gpfsug.org/mailman/listinfo/gpfsug-discuss
