On Apr 10, 2017, at 14:53, Todd, Allen <[email protected]> wrote: > > While everyone is talking about lfs migrate, I would like to point out that > it appears to be missing an option to preserve file modification and access > times, which makes it less useful for behind the scenes data management tasks.
This should actually be the default, though there was a bug in older versions of Lustre that didn't preserve the timestamps. That was fixed in Lustre 2.8. Cheers, Andreas > Allen > > -----Original Message----- > From: lustre-discuss [mailto:[email protected]] On > Behalf Of Henri Doreau > Sent: Tuesday, April 04, 2017 3:18 AM > To: E.S. Rosenberg <[email protected]> > Cc: [email protected] > Subject: Re: [lustre-discuss] lfs_migrate > > Hello, > > the manpage for lfs(1) lists the available options in 2.8: > """ > lfs migrate -m <mdt_index> directory > lfs migrate [-c | --stripe-count <stripe_count>] > [-i | --stripe-index <start_ost_idx>] > [-S | --stripe-size <stripe_size>] > [-p | --pool <pool_name>] > [-o | --ost-list <ost_indices>] > [-b | --block] > [-n | --non-block] file|directory """ > > Although this is certainly terse, I guess that most parameters are intuitive. > > The command will open the file to restripe (blocking concurrent accesses or > not, depending on -b/-n), create a special "volatile" (=unlinked) one with > the requested striping parameters and copy the source into the destination. > > If the copy succeeds, the two files are atomically swapped and concurrent > access protection is released. > > In non-blocking mode, the process will detect if the source file was already > opened or if there's an open during the copy process and abort safely. It is > then up to the admin to reschedule the migration later, maybe with -b. > > HTH > > Henri > > On 02/avril - 14:43 E.S. Rosenberg wrote: >> Thanks for all the great replies! >> >> I may be wrong on this but 'lfs migrate' does not seem to be >> documented in the manpage (my local one is 2.8 so I expect that but >> even manpages that I find online). >> >> Any pointers would be very welcome. >> >> On Thu, Mar 23, 2017 at 12:31 PM, Henri Doreau <[email protected]> wrote: >> >>> On 20/mars - 22:50 E.S. Rosenberg wrote: >>>> On Mon, Mar 20, 2017 at 10:19 PM, Dilger, Andreas < >>> [email protected]> >>>> wrote: >>>> >>>>> The underlying "lfs migrate" command (not the "lfs_migrate" >>>>> script) in newer Lustre versions (2.9) is capable of migrating >>>>> files that are in >>> use >>>>> by using the "--block" option, which prevents other processes >>>>> from accessing or modifying the file during migration. >>>>> >>>>> Unfortunately, "lfs_migrate" doesn't pass that argument on, >>>>> though it wouldn't be hard to change the script. Ideally, the >>>>> "lfs_migrate" >>> script >>>>> would pass all unknown options to "lfs migrate". >>>>> >>>>> >>>>> The other item of note is that setting the OST inactive on the >>>>> MDS will prevent the MDS from deleting objects on the OST (see >>>>> https://jira.hpdd.intel.com/browse/LU-4825 for details). In >>>>> Lustre >>> 2.9 >>>>> and later it is possible to set on the MDS: >>>>> >>>>> mds# lctl set_param osp.<OST>.create_count=0 >>>>> >>>>> to stop MDS allocation of new objects on that OST. On older >>>>> versions >>> it is >>>>> possible to set on the OSS: >>>>> >>>>> oss# lctl set_param obdfilter.<OST>.degraded=1 >>>>> >>>>> so that it tells the MDS to avoid it if possible, but this isn't >>>>> a hard exclusion. >>>>> >>>>> It is also possible to use a testing hack to mark an OST as out >>>>> of >>> inodes, >>>>> but that only works for one OST per OSS and it sounds like that >>>>> won't >>> be >>>>> useful in this case. >>>>> >>>>> Cheers, Andreas >>>>> >>>> You're making me want Lustre 2.9 more :) but for now I'm still >>>> stuck on >>> 2.8 >>>> and because this is very much production these days I'm more >>>> careful with the update (hoping to finally get hw allocated for a >>>> test env soon to >>> test >>>> the update). >>>> Thanks, >>>> Eli >>>> >>> >>> Hello, >>> >>> this safer version of `lfs migrate' (LU-4840) is actually available >>> in 2.8. >>> >>> When used with --non-block flag, a concurrent open of the file being >>> migrated will cause the migration to fail. With --block (or nothing, >>> it's the default behavior) and as Andreas said, concurrent opens >>> will block until the migration completes. >>> >>> Regards >>> >>> -- >>> Henri Doreau >>> > > _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org > > ________________________________ > > IMPORTANT: The information contained in this email and/or its attachments is > confidential. If you are not the intended recipient, please notify the sender > immediately by reply and immediately delete this message and all its > attachments. Any review, use, reproduction, disclosure or dissemination of > this message or any attachment by an unintended recipient is strictly > prohibited. Neither this message nor any attachment is intended as or should > be construed as an offer, solicitation or recommendation to buy or sell any > security or other financial instrument. Neither the sender, his or her > employer nor any of their respective affiliates makes any warranties as to > the completeness or accuracy of any of the information contained herein or > that this message or any of its attachments is free of viruses. > _______________________________________________ > lustre-discuss mailing list > [email protected] > http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org Cheers, Andreas -- Andreas Dilger Lustre Principal Architect Intel Corporation _______________________________________________ lustre-discuss mailing list [email protected] http://lists.lustre.org/listinfo.cgi/lustre-discuss-lustre.org
