On May 7, 2017, at 14:27, E.S. Rosenberg <[email protected]> wrote: > > Since we were pressured for time (the migrate was to empty a disk enclosure > that was functioning too close to failure for comfort) I mailed the affected > user a list of affected files and how they could take care of things. If I > do have time in the future I may write this script.
It looks like there is already a patch that is handling the hard links in lfs_migrate (https://review.whamcloud.com/25851 that I even worked on, but apparently forgot about). This needs to be updated and get some test cases in order to land, but at least provides a good starting point for this work. > Now that we again have a working enclosure, should I be taking action myself > to re-balance the files (with migrate) or should I just let time & lustre do > its' thing? If you aren't close to running out of space on the older OSTs, and your data has a "limited lifetime" (e.g. created and removed over time) then there is no requirement to manually rebalance the storage, it will prefer allocating to the empty OST(s) and will even out over time. The main reason you might want to manually rebalance the OSTs is because the MDS _does_ prefer the empty OSTs for allocations, and if you have large amounts of parallel IO it can cause increased load on that OST and potentially slow down the application(s) as it is doing more work than other OSTs. Cheers, Andreas > On Tue, May 2, 2017 at 3:51 AM, Dilger, Andreas <[email protected]> > wrote: >> If your filesystem was created with Lustre 2.1 or later then you can use: >> >> FID=$(lfs path2fid "/path/to/file") >> lfs fid2path "$FID" >> >> to find all the pathnames that are hard links to that file. There is a patch >> to add a "lfs path2links" option that does this in a single step, but it is >> not in any release yet. >> >> The number of pathnames should match the hard link count returned by "stat >> -c%h" if the files don't have too many hard links (i.e. below 140 or so) and >> then you can manually migrate the file and re-link the other pathnames to >> the new file with "ln -f". >> >> That is something that has been on the todo list for lfs_migrate for a >> while, so if you wanted to implement that in the script and submit a patch >> to Gerrit it would be appreciated. >> >> Cheers, Andreas >> >> On May 1, 2017, at 06:59, E.S. Rosenberg <[email protected]> wrote: >> >>> Now that we have reached close to the end of the migration process we have >>> a lot of files that are being skipped due to "multiple hard links", I am >>> not sure what my strategy should be concerning such files. Is there any >>> migration automation possible on these? Or is my only route contacting the >>> owners (who may just not have known how to use 'ln -s')? >>> >>> Any advice would be very welcome. >>> Thanks, >>> Eliyahu - אליהו >>> >>> On Wed, Apr 12, 2017 at 6:55 PM, Todd, Allen <[email protected]> wrote: >>> Thanks Andreas -- good to know there is yet another reason to upgrade. We >>> are on 2.7.0. I was trying to hold out for progressive file layout to land. >>> >>> Allen >>> >>> -----Original Message----- >>> From: lustre-discuss [mailto:[email protected]] On >>> Behalf Of Dilger, Andreas >>> Sent: Wednesday, April 12, 2017 8:19 AM >>> To: Todd, Allen <[email protected]> >>> Cc: E.S. Rosenberg <[email protected]>; [email protected] >>> Subject: Re: [lustre-discuss] lfs_migrate >>> >>> On Apr 10, 2017, at 14:53, Todd, Allen <[email protected]> wrote: >>> > >>> > While everyone is talking about lfs migrate, I would like to point out >>> > that it appears >>> > to be missing an option to preserve file modification and access times, >>> > which makes it >>> > less useful for behind the scenes data management tasks. >>> >>> This should actually be the default, though there was a bug in older >>> versions of Lustre that didn't preserve the timestamps. That was fixed in >>> Lustre 2.8. >>> >>> Cheers, Andreas >>> >>> > Allen >>> > >>> > -----Original Message----- >>> > From: lustre-discuss [mailto:[email protected]] >>> > On Behalf Of Henri Doreau >>> > Sent: Tuesday, April 04, 2017 3:18 AM >>> > To: E.S. Rosenberg <[email protected]> >>> > Cc: [email protected] >>> > Subject: Re: [lustre-discuss] lfs_migrate >>> > >>> > Hello, >>> > >>> > the manpage for lfs(1) lists the available options in 2.8: >>> > """ >>> > lfs migrate -m <mdt_index> directory >>> > lfs migrate [-c | --stripe-count <stripe_count>] >>> > [-i | --stripe-index <start_ost_idx>] >>> > [-S | --stripe-size <stripe_size>] >>> > [-p | --pool <pool_name>] >>> > [-o | --ost-list <ost_indices>] >>> > [-b | --block] >>> > [-n | --non-block] file|directory """ >>> > >>> > Although this is certainly terse, I guess that most parameters are >>> > intuitive. >>> > >>> > The command will open the file to restripe (blocking concurrent accesses >>> > or not, depending on -b/-n), create a special "volatile" (=unlinked) one >>> > with the requested striping parameters and copy the source into the >>> > destination. >>> > >>> > If the copy succeeds, the two files are atomically swapped and concurrent >>> > access protection is released. >>> > >>> > In non-blocking mode, the process will detect if the source file was >>> > already opened or if there's an open during the copy process and abort >>> > safely. It is then up to the admin to reschedule the migration later, >>> > maybe with -b. >>> > >>> > HTH >>> > >>> > Henri >>> > >>> > On 02/avril - 14:43 E.S. Rosenberg wrote: >>> >> Thanks for all the great replies! >>> >> >>> >> I may be wrong on this but 'lfs migrate' does not seem to be >>> >> documented in the manpage (my local one is 2.8 so I expect that but >>> >> even manpages that I find online). >>> >> >>> >> Any pointers would be very welcome. >>> >> >>> >> On Thu, Mar 23, 2017 at 12:31 PM, Henri Doreau <[email protected]> >>> >> wrote: >>> >> >>> >>> On 20/mars - 22:50 E.S. Rosenberg wrote: >>> >>>> On Mon, Mar 20, 2017 at 10:19 PM, Dilger, Andreas < >>> >>> [email protected]> >>> >>>> wrote: >>> >>>> >>> >>>>> The underlying "lfs migrate" command (not the "lfs_migrate" script) >>> >>>>> in newer Lustre versions (2.9) is capable of migrating files that are >>> >>>>> in use by using the "--block" option, which prevents other processes >>> >>>>> from accessing or modifying the file during migration. >>> >>>>> >>> >>>>> Unfortunately, "lfs_migrate" doesn't pass that argument on, though >>> >>>>> it wouldn't be hard to change the script. Ideally, the "lfs_migrate" >>> >>>>> script would pass all unknown options to "lfs migrate". >>> >>>>> >>> >>>>> >>> >>>>> The other item of note is that setting the OST inactive on the MDS >>> >>>>> will prevent the MDS from deleting objects on the OST (see >>> >>>>> https://jira.hpdd.intel.com/browse/LU-4825 for details). In >>> >>>>> Lustre 2.9 and later it is possible to set on the MDS: >>> >>>>> >>> >>>>> mds# lctl set_param osp.<OST>.create_count=0 >>> >>>>> >>> >>>>> to stop MDS allocation of new objects on that OST. On older >>> >>>>> versions it is possible to set on the OSS: >>> >>>>> >>> >>>>> oss# lctl set_param obdfilter.<OST>.degraded=1 >>> >>>>> >>> >>>>> so that it tells the MDS to avoid it if possible, but this isn't a >>> >>>>> hard exclusion. >>> >>>>> >>> >>>>> It is also possible to use a testing hack to mark an OST as out of >>> >>>>> inodes, but that only works for one OST per OSS and it sounds like >>> >>>>> that >>> >>>>> won't be useful in this case. >>> >>>>> >>> >>>>> Cheers, Andreas >>> >>>>> >>> >>>> You're making me want Lustre 2.9 more :) but for now I'm still >>> >>>> stuck on 2.8 and because this is very much production these days >>> >>>> I'm more careful with the update (hoping to finally get hw allocated >>> >>>> for a test env soon to test the update). >>> >>>> Thanks, >>> >>>> Eli >>> >>>> >>> >>> >>> >>> Hello, >>> >>> >>> >>> this safer version of `lfs migrate' (LU-4840) is actually available >>> >>> in 2.8. >>> >>> >>> >>> When used with --non-block flag, a concurrent open of the file being >>> >>> migrated will cause the migration to fail. With --block (or nothing, >>> >>> it's the default behavior) and as Andreas said, concurrent opens >>> >>> will block until the migration completes. >>> >>> >>> >>> Regards >>> >>> >>> >>> -- >>> >>> Henri Doreau >>> >>> >>> > Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
