On Jun 26, 2008 10:57 -0700, Stew Paddaso wrote: > We are considering using Lustre as our backend file platform. The > specific application involves storing a high-volume of sequential data > writes, with a moderate amount of reads (mostly sequencial, with some > random seeks). Our concern is with reclaiming space. As the file > system fills, we need to be able to quickly delete the oldest files. > > Does Lustre have an efficient file delete? I'm not expecting specific > metrics (though they would be nice if available), just some general > info about the Lustre delete process (Does it immediately reclaim the > space, or do it 'lazily' in the background? etc, etc.).
Well, there is some TINY amount of "laziness" involved, in the sense that the RPC to reclaim the space is sent immediately after getting the approval of the MDS, but it doesn't wait for any reply. A single client can have up to 8 (by default) destroy RPCs in flight per OST, which we throttle to avoid overloading the server if this is happening on many clients at the same time. There are distributed transactions that ensure the space is freed even in the case of a crash. I don't think there is any performance problem with Lustre file deletes, in fact a few incidences of "runaway cleanup script" ended up deleting files much more quickly than the site wanted... Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc. _______________________________________________ Lustre-discuss mailing list [email protected] http://lists.lustre.org/mailman/listinfo/lustre-discuss
