Re: [Lustre-discuss] Client directory entry caching

2010-08-05 Thread Alexey Lyashkov
That very trivial patch. 2.0 use CR mode to avoid extra lock cancel with xattr upate, 1.8 use EX mode to any xattr updates. but real world must use both modes :-) because client got ACL (part of xattr) with any getattr / open RPC, so MDS MUST cancel that lock with any conflicted updates (because

Re: [Lustre-discuss] Client directory entry caching

2010-08-04 Thread Andreas Dilger
On 2010-08-03, at 12:45, Kevin Van Maren wrote: Since Bug 22492 hit a lot of people, it sounds like opencache isn't generally useful unless enabled on every node. Is there an easy way to force files out of the cache (ie, echo 3 /proc/sys/vm/drop_caches)? For Lustre you can do lctl set_param

Re: [Lustre-discuss] Client directory entry caching

2010-08-04 Thread Alexey Lyashkov
22492 is exist because someone from Sun/Oracle is disable dentry caching, instead of fixing xattr code for 1.82.0 interoperable. He is kill my patch in revalidate dentry (instead of fixing xattr code). in that case client always send one more RPC to server. see bug 17545 for some details. On

Re: [Lustre-discuss] Client directory entry caching

2010-08-04 Thread Oleg Drokin
Hello! On Aug 4, 2010, at 3:41 AM, Andreas Dilger wrote: mkdir(/mnt/lustre/blah2/b/c/d/e/f/g, 040755) = 0 +1 RPC lstat(/mnt/lustre/blah2/b/c/d/e/f/g, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 +1 RPC If we do the mkdir(), the client does not cache the entry? No. mkdir cannot return

Re: [Lustre-discuss] Client directory entry caching

2010-08-04 Thread Daire Byrne
Oleg, On Tue, Aug 3, 2010 at 6:50 PM, Oleg Drokin oleg.dro...@oracle.com wrote: Yea that works - cheers. FYI some comparisons with a simple find on a remote client (~33,000 files):  find /mnt/lustre (not cached) = 41 secs  find /mnt/lustre (cached) = 19 secs  find /mnt/lustre (opencache) =

Re: [Lustre-discuss] Client directory entry caching

2010-08-04 Thread Oleg Drokin
Hello! On Aug 4, 2010, at 2:04 PM, Daire Byrne wrote: Hm, initially I was going to say that find is not open-intensive so it should not benefit from opencache at all. But then I realized if you have a lot of dirs, then indeed there would be a positive impact on subsequent reruns. I assume

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Daire Byrne
Oleg, On Tue, Aug 3, 2010 at 5:21 AM, Oleg Drokin oleg.dro...@oracle.com wrote: So even with the metadata going over NFS the opencache in the client seems to make quite a difference (I'm not sure how much the NFS client caches though). As expected I see no mdt activity for the NFS export once

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Oleg Drokin
Hello! On Aug 3, 2010, at 12:49 PM, Daire Byrne wrote: So even with the metadata going over NFS the opencache in the client seems to make quite a difference (I'm not sure how much the NFS client caches though). As expected I see no mdt activity for the NFS export once cached. I think it would

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Kevin Van Maren
Since Bug 22492 hit a lot of people, it sounds like opencache isn't generally useful unless enabled on every node. Is there an easy way to force files out of the cache (ie, echo 3 /proc/sys/vm/drop_caches)? Kevin On Aug 3, 2010, at 11:50 AM, Oleg Drokin oleg.dro...@oracle.com wrote:

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Oleg Drokin
Well, you can drop all locks on a given FS that would in effect drop all metadata caches, but will leave data caches intact. echo clear /proc/fs/lustre/ldlm/namespaces/your_MDC_namespace/lru_size On Aug 3, 2010, at 2:45 PM, Kevin Van Maren wrote: Since Bug 22492 hit a lot of people, it sounds

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Jeremy Filizetti
On Tue, Aug 3, 2010 at 12:49 PM, Daire Byrne daire.by...@gmail.com wrote: Oleg, On Tue, Aug 3, 2010 at 5:21 AM, Oleg Drokin oleg.dro...@oracle.com wrote: So even with the metadata going over NFS the opencache in the client seems to make quite a difference (I'm not sure how much the NFS

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Oleg Drokin
Hello! On Aug 3, 2010, at 10:59 PM, Jeremy Filizetti wrote: Another consideration for WAN performance when creating files is the stripe count. When you start writing to a file the first RPC to each OSC requests the lock rather then requesting the lock from all OSCs when the first lock is

Re: [Lustre-discuss] Client directory entry caching

2010-08-03 Thread Jeremy Filizetti
On Tue, Aug 3, 2010 at 11:14 PM, Oleg Drokin oleg.dro...@oracle.com wrote: Hello! On Aug 3, 2010, at 10:59 PM, Jeremy Filizetti wrote: Another consideration for WAN performance when creating files is the stripe count. When you start writing to a file the first RPC to each OSC requests

Re: [Lustre-discuss] Client directory entry caching

2010-08-02 Thread Oleg Drokin
Hello! On Jul 30, 2010, at 7:20 AM, Daire Byrne wrote: Ah yes... that makes sense. I recall the opencache gave a big boost in performance for NFS exporting but I wasn't sure if it had become the default. I haven't been keeping up to date with Lustre developments. It was default for NFS for

Re: [Lustre-discuss] Client directory entry caching

2010-07-30 Thread Daire Byrne
On Thu, Jul 29, 2010 at 8:54 PM, Andreas Dilger andreas.dil...@oracle.com wrote: On 2010-07-29, at 04:47, Daire Byrne wrote: I was wondering if it is possible to have the client completely cache a recursive listing of a lustre filesystem such that on a second run it doesn't have to talk to the

[Lustre-discuss] Client directory entry caching

2010-07-29 Thread Daire Byrne
Hi, I was wondering if it is possible to have the client completely cache a recursive listing of a lustre filesystem such that on a second run it doesn't have to talk to the MDT again? Taking the simplest case where I only have one client that is browsing a million file tree (say), I would expect

Re: [Lustre-discuss] Client directory entry caching

2010-07-29 Thread Andreas Dilger
On 2010-07-29, at 04:47, Daire Byrne wrote: I was wondering if it is possible to have the client completely cache a recursive listing of a lustre filesystem such that on a second run it doesn't have to talk to the MDT again? Taking the simplest case where I only have one client that is