On Mar 25, 2008, at 11:22 AM, Chris Worley wrote: > On Tue, Mar 25, 2008 at 10:39 AM, Andreas Dilger <[EMAIL PROTECTED]> > wrote: >> On Mar 25, 2008 08:53 -0600, Chris Worley wrote: >>> On Tue, Mar 25, 2008 at 2:13 AM, Andreas Dilger <[EMAIL PROTECTED]> >>> wrote: >>>> On Mar 25, 2008 01:28 -0600, Chris Worley wrote: >>>>> I do an "lctl dl" and it shows "UP" in the first column for all >>>>> OST's... even though I've deactivated many disks. "iostat" >>>>> shows the >>>>> disks are still in use too. >>>> >>>> What does it mean when you say "deactivated many disks"? >>> >>> To deactivate the disk, I use an incantation like: >>> >>> lctl --device ddnlfs-OST001f-osc deactivate >> >> Note that "deactivate" only affects the node on which it is run. >> The normal place to do this is on the MDS. > > That's what I do.
One thing Andreas did not mention is that when you deactivate a device (OST) on the MDS, you disable NEW file creation. You can still append to existing files. The device is not read-only. So you may still see traffic to those disks as users modify existing files. >> Note that you also >> mount the client filesystem on the MDS node you need to deactivate >> the MDS OSC connection, and not the client filesystem one: > > I'm not sure I understand the above? > > I think you're saying that when deactivating, use the block device > label with "-osc" appended. That I do too. > >> >> >>> ...but new files are still going there, and, if I'm reading it >>> right, >>> the disk is still "up" in Lustre: >>> >>> # lctl dl | grep 1f >>> 36 UP osc ddnlfs-OST001f-osc ddnlfs-mdtlov_UUID 5 >> >> This does look like you have the right device. Using "device_list" >> only shows which devices are configured. A deactivated device is >> still configured... The "UP" status is related to the >> configuration >> status and not the current connection state. Have a look at the >> file >> /proc/fs/lustre/ddnlfs-mdtlov/target_obd to see the device status. >> > > Ahh, that verifies what's active/inactive: > > # cat /proc/fs/lustre/lov/ddnlfs-mdtlov/target_obd | grep " ACTIVE" > | wc -l > 48 > # cat /proc/fs/lustre/lov/ddnlfs-mdtlov/target_obd | grep " > INACTIVE" | wc -l > 22 > >> # lfs df > > This command returns nothing? lfs df should be run on a client, not the MDS (unless the MDS is a client as well). >> UUID 1K-blocks Used Available Use% Mounted on >> mds-myth-0_UUID 9174328 678000 8496328 7% /myth >> [MDT:0] >> ost-myth-0_UUID 292223856 286837752 5386104 98% /myth >> [OST:0] >> ost-myth-1_UUID 94442984 92833972 1609012 98% /myth >> [OST:1] >> ost-myth-2_UUID 487388376 474792788 12595588 97% /myth >> [OST:2] >> ost-myth-3_UUID 487865304 472221312 15643992 96% /myth >> [OST:3] >> >> filesystem summary: 1361920520 1326685824 35234696 97% /myth >> >> # lctl --device %myth-OST0001-osc deactivate >> # cat /proc/fs/lustre/lov/myth-mdtlov/target_obd >> 0: ost-myth-0_UUID ACTIVE >> 1: ost-myth-1_UUID INACTIVE >> 2: ost-myth-2_UUID ACTIVE >> 3: ost-myth-3_UUID ACTIVE >> >> # lctl --device %myth-OST0001-osc recover >> >> >>>>> I'm trying to get rid of slow disks... what's the right way to >>>>> tell >>>>> Lustre to quit using a disk? >>>> >>>> If you deactivate an OST on the MDS node it will stop >>>> allocating new >>>> files there >>> >>> For now, that's all I want to do... but new files are still going >>> there. >>> >>> ... both a way to deactivate the disk and a way to know which are >>> deactivated would be nice. >> >> It was confusing when you say "deactivate the disk" because that >> could >> mean a great many things like deactivating a disk from a RAID >> set, or >> similar. An OST may reside on many disks (via hardware/software >> RAID, >> LVM, etc). >> >> What you are trying to do is the right process. >> > > Thanks for all the help! I think I've got it now. > > Chris >> >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Sr. Staff Engineer, Lustre Group >> Sun Microsystems of Canada, Inc. >> >> > _______________________________________________ > Lustre-discuss mailing list > [email protected] > http://lists.lustre.org/mailman/listinfo/lustre-discuss -Marc ---- D. Marc Stearman LC Lustre Administration Lead [EMAIL PROTECTED] 925.423.9670 Pager: 1.888.203.0641 _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
