Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Wendy Cheng
Wendy Cheng wrote: Jos Vos wrote: On Fri, Nov 02, 2007 at 04:12:39PM -0400, Wendy Cheng wrote: Also I read your previous mailing list post with "df" issue - didn't have time to comment. Note that both RHEL 4.6 and RHEL 5.1 will have a "fast_statfs" tunable that is specifically added to speed

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Wendy Cheng
Jos Vos wrote: On Fri, Nov 02, 2007 at 04:12:39PM -0400, Wendy Cheng wrote: Also I read your previous mailing list post with "df" issue - didn't have time to comment. Note that both RHEL 4.6 and RHEL 5.1 will have a "fast_statfs" tunable that is specifically added to speed up the "df" comm

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Jos Vos
On Fri, Nov 02, 2007 at 04:12:39PM -0400, Wendy Cheng wrote: > Also I read your previous mailing list post with "df" issue - didn't > have time to comment. Note that both RHEL 4.6 and RHEL 5.1 will have a > "fast_statfs" tunable that is specifically added to speed up the "df" > command. Give it

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Wendy Cheng
Wendy Cheng wrote: 2. The gfs_scand issue is more to do with the number of glock count. One way to tune this is via purge_glock tunable. There is an old write-up in: http://people.redhat.com/wcheng/Patches/GFS/readme.gfs_glock_trimming.R4 . It is for RHEL4 but should work the same way for RHEL5

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Jos Vos
On Fri, Nov 02, 2007 at 04:00:35PM -0400, Wendy Cheng wrote: > The patch on my people's page is a *RHEL4* patch. I know, but you said "It is for RHEL4 but should work the same way for RHEL5", so I tried that. > Just did a quick check, RHEL5.1 gfs-kmod-0.1.19 should have this patch > and will be

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Wendy Cheng
Jos Vos wrote: On Fri, Oct 26, 2007 at 07:57:18PM -0400, Wendy Cheng wrote: 2. The gfs_scand issue is more to do with the number of glock count. One way to tune this is via purge_glock tunable. There is an old write-up in: http://people.redhat.com/wcheng/Patches/GFS/readme.gfs_glock_trimming

Re: [Linux-cluster] GFS RG size (and tuning)

2007-11-02 Thread Jos Vos
On Fri, Oct 26, 2007 at 07:57:18PM -0400, Wendy Cheng wrote: > 2. The gfs_scand issue is more to do with the number of glock count. One > way to tune this is via purge_glock tunable. There is an old write-up in: > http://people.redhat.com/wcheng/Patches/GFS/readme.gfs_glock_trimming.R4 > . It is

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-29 Thread Jos Vos
On Mon, Oct 29, 2007 at 08:53:10PM -0400, Glen Dosey wrote: > We have several 3.3TB and larger filesystems on GFS1. I found that using > the larger RG helps with performance. Increasing the statfs_slots seemed > to help with df time and similar tasks as well. We have a filesystem > with smaller RG

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-29 Thread Glen Dosey
We have several 3.3TB and larger filesystems on GFS1. I found that using the larger RG helps with performance. Increasing the statfs_slots seemed to help with df time and similar tasks as well. We have a filesystem with smaller RG's and I don't think the difference is worth moving the data around.

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-27 Thread Josh Gray
Yea larger - sorry larger I meant Yes, it is definitely possible that using a bigger resource group may help with performance. Since this is not yet in production, I would urge you to do some testing now while you can and compare results from the default size of 256 versus a setting of around 20

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-27 Thread Mark Hlawatschek
We did some system profiling and measured the influence of the lkbtbl_size and rsbtbl_size values (dlm config parameters) to the cpu usage. We noticed, that with large number of dlm locks, these values have a measurable impact to the system performance. You might want to have a look at http:/

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-27 Thread Jos Vos
On Sat, Oct 27, 2007 at 02:15:33PM -0400, Josh Gray wrote: > Support said i would probably expect significant improvement with > more RG's but we went with the other file format before we tried that. You mean *less* (and larger) RGs I presume? -- --Jos Vos <[EMAIL PROTECTED]> --X/OS E

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-27 Thread Josh Gray
I had the same problems this week on GFS only about 800gig of mostly small files. My application did not require it to be mounted on the whole cluster concurrently so i went ahead and switched to EXT3 with much better performance. Support said i would probably expect significant improvemen

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-27 Thread Jos Vos
On Fri, Oct 26, 2007 at 07:57:18PM -0400, Wendy Cheng wrote: > 1. 3TB is not "average size". Smaller RG can help with "df" command - > but if your system is congested, it won't help much. The df also takes ages on an almost idle system. Also, the system often needs to do rsyncs on large trees a

Re: [Linux-cluster] GFS RG size (and tuning)

2007-10-26 Thread Wendy Cheng
Jos Vos wrote: Hi, The gfs_mkfs manual page (RHEL 5.0) says: If not specified, gfs_mkfs will choose the RG size based on the size of the file system: average size file systems will have 256 MB RGs, and bigger file systems will have bigger RGs for better performance. My 3 TB filesystem

[Linux-cluster] GFS RG size (and tuning)

2007-10-26 Thread Jos Vos
Hi, The gfs_mkfs manual page (RHEL 5.0) says: If not specified, gfs_mkfs will choose the RG size based on the size of the file system: average size file systems will have 256 MB RGs, and bigger file systems will have bigger RGs for better performance. My 3 TB filesystems still seem to