Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-12 Thread Arden Wiebe
transfer using Lustre is interesting. Arden --- On Mon, 8/10/09, Jim McCusker james.mccus...@yale.edu wrote: From: Jim McCusker james.mccus...@yale.edu Subject: Re: [Lustre-discuss] Large scale delete results in lag on clients To: Oleg Drokin oleg.dro...@sun.com Cc: lustre-discuss lustre

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-10 Thread Jim McCusker
Just a quick update: the lag is back. It seems to hold off for about 24 hours, then things get so slow that our web applications time out. Is there a delay before freeing inodes commences after a reboot, or is this officially now Something Else? Should I just tough it out for now and see if it

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-10 Thread Jim McCusker
On Monday, August 10, 2009, Oleg Drokin oleg.dro...@sun.com wrote: What lustre version is it now? We used to have uncontrolled unlinking where OSTs might get swamped with unlink requests. Now we limit to 8 unlinks to OST at any one time. This slows down the deletion process, but at least

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-10 Thread Oleg Drokin
Hello! On Aug 10, 2009, at 11:03 PM, Jim McCusker wrote: On Monday, August 10, 2009, Oleg Drokin oleg.dro...@sun.com wrote: What lustre version is it now? We used to have uncontrolled unlinking where OSTs might get swamped with unlink requests. Now we limit to 8 unlinks to OST at any

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-07 Thread Arden Wiebe
--- On Thu, 8/6/09, Andreas Dilger adil...@sun.com wrote: From: Andreas Dilger adil...@sun.com Subject: Re: [Lustre-discuss] Large scale delete results in lag on clients To: Jim McCusker james.mccus...@yale.edu Cc: lustre-discuss lustre-discuss@lists.lustre.org Date: Thursday, August 6

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-07 Thread Arden Wiebe
--- On Fri, 8/7/09, Jim McCusker james.mccus...@yale.edu wrote: From: Jim McCusker james.mccus...@yale.edu Subject: Re: [Lustre-discuss] Large scale delete results in lag on clients To: lustre-discuss lustre-discuss@lists.lustre.org Date: Friday, August 7, 2009, 5:25 AM On Fri, Aug 7, 2009

[Lustre-discuss] Large scale delete results in lag on clients

2009-08-06 Thread Jim McCusker
We have a 15 TB luster volume across 4 OSTs and we recently deleted over 4 million files from it in order to free up the 80 GB MDT/MDS (going from 100% capacity on it to 81%. As a result, after the rm completed, there is significant lag on most file system operations (but fast access once it

Re: [Lustre-discuss] Large scale delete results in lag on clients

2009-08-06 Thread Andreas Dilger
On Aug 06, 2009 15:08 -0400, Jim McCusker wrote: We have a 15 TB luster volume across 4 OSTs and we recently deleted over 4 million files from it in order to free up the 80 GB MDT/MDS (going from 100% capacity on it to 81%. As a result, after the rm completed, there is significant lag on most