Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-28 Thread Neil Perrin
Robert Milkowski wrote On 06/28/06 15:52,: Hello Neil, Wednesday, June 21, 2006, 8:15:54 PM, you wrote: NP Robert Milkowski wrote On 06/21/06 11:09,: Hello Neil, Why is this option available then? (Yes, that's a loaded question.) NP I wouldn't call it an option, but an internal

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-26 Thread Roch
So if you have a single thread doing open/write/close of 8K files and get 1.25MB/sec, that tells me you have something like a 6ms I/O latency. Which look reasonable also. What does iostat -x svc_t (client side) says ? 400ms seems high for the workload _and_ doesn't match my formula, so I don't

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-24 Thread Joe Little
To clarify what has just been stated. With zil disabled I got 4MB/sec. With zil enabled I get 1.25MB/sec On 6/23/06, Tao Chen [EMAIL PROTECTED] wrote: On 6/23/06, Roch [EMAIL PROTECTED] wrote: On Thu, Jun 22, 2006 at 04:22:22PM -0700, Joe Little wrote: On 6/22/06, Jeff Bonwick [EMAIL

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-23 Thread Joe Little
On 6/23/06, Roch [EMAIL PROTECTED] wrote: Joe Little writes: On 6/22/06, Bill Moore [EMAIL PROTECTED] wrote: Hey Joe. We're working on some ZFS changes in this area, and if you could run an experiment for us, that would be great. Just do this: echo 'zil_disable/W1' | mdb -kw

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-23 Thread Richard Elling
Joe Little wrote: On 6/23/06, Roch [EMAIL PROTECTED] wrote: Joe, you know this but for the benefit of others, I have to highlight that running any NFS server this way, may cause silent data corruption from client's point of view. Whenever a server keeps data in RAM this way and does not

Fwd: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-23 Thread Tao Chen
I should copy this to the list.-- Forwarded message --On 6/23/06, Joe Little [EMAIL PROTECTED] wrote: I can post back to Roch what this latency is. I think the latency is aconstant regardless of the zil or not. all that I do by disabling thezil is that I'm able to submit larger

Re: Fwd: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-23 Thread Tao Chen
On 6/23/06, Richard Elling [EMAIL PROTECTED] wrote: comment on analysis below...Tao Chen wrote: === Top 5 Devices with largest number of I/Os === DEVICEREAD AVG.ms MBWRITE AVG.ms MBIOs SEEK -- -- - -- - sd16 0.340 4948 387.88413 4954 0% sd26 0.250

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Roch
How about the 'deferred' option be on a leased basis with a deadline to revert to normal behavior; at most 24hrs at a time. Console output everytime the option is enabled. -r Torrey McMahon writes: Neil Perrin wrote: Of course we would need to stress the dangers of setting

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Darren J Moffat
Bill Sommerfeld wrote: On Wed, 2006-06-21 at 14:15, Neil Perrin wrote: Of course we would need to stress the dangers of setting 'deferred'. What do you guys think? I can think of a use case for deferred: improving the efficiency of a large mega-transaction/batch job such as a nightly build.

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Dana H. Myers
Darren J Moffat wrote: Bill Sommerfeld wrote: On Wed, 2006-06-21 at 14:15, Neil Perrin wrote: Of course we would need to stress the dangers of setting 'deferred'. What do you guys think? I can think of a use case for deferred: improving the efficiency of a large mega-transaction/batch job

Re[2]: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Robert Milkowski
Hello Roch, Thursday, June 22, 2006, 9:55:41 AM, you wrote: R How about the 'deferred' option be on a leased basis with a R deadline to revert to normal behavior; at most 24hrs at a R time. Console output everytime the option is enabled. I really hate when tools try to be more clever than

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Bill Sommerfeld
On Thu, 2006-06-22 at 03:55, Roch wrote: How about the 'deferred' option be on a leased basis with a deadline to revert to normal behavior; at most 24hrs at a time. why? Console output everytime the option is enabled. in general, no. error messages to the console should be reserved

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Roch
Bill Sommerfeld writes: On Thu, 2006-06-22 at 03:55, Roch wrote: How about the 'deferred' option be on a leased basis with a deadline to revert to normal behavior; at most 24hrs at a time. why? I'll trust your judgement over mine on this, so I won't press. But it was

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Bill Sommerfeld
On Thu, 2006-06-22 at 13:01, Roch wrote: Is there a sync command that targets individual FS ? Yes. lockfs -f - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Darren J Moffat
Bill Sommerfeld wrote: On Thu, 2006-06-22 at 13:01, Roch wrote: Is there a sync command that targets individual FS ? Yes. lockfs -f Does lockfs work with ZFS ? The man page appears to indicate it is very UFS specific. -- Darren J Moffat ___

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Bill Sommerfeld
On Thu, 2006-06-22 at 13:19, Darren J Moffat wrote: Yes. lockfs -f Does lockfs work with ZFS ? The man page appears to indicate it is very UFS specific. all of lockfs does not. but, if truss is to believed, the ioctl used by lockfs -f appears to. or at least, it returns without error.

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Jonathan Adams
On Thu, Jun 22, 2006 at 06:19:20PM +0100, Darren J Moffat wrote: Bill Sommerfeld wrote: On Thu, 2006-06-22 at 13:01, Roch wrote: Is there a sync command that targets individual FS ? Yes. lockfs -f Does lockfs work with ZFS ? The man page appears to indicate it is very UFS specific.

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Roch
As I recall, the zfs sync is, unlike UFS, synchronous. -r ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Prabahar Jeyaram
Yep. ZFS supports the ioctl (_FIOFFS) which 'lockfs -f' issues. -- Prabahar. Darren J Moffat wrote: Bill Sommerfeld wrote: On Thu, 2006-06-22 at 13:01, Roch wrote: Is there a sync command that targets individual FS ? Yes. lockfs -f Does lockfs work with ZFS ? The man page appears to

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Jeff Bonwick
a test against the same iscsi targets using linux and XFS and the NFS server implementation there gave me 1.25MB/sec writes. I was about to throw in the towel and deem ZFS/NFS has unusable until B41 came along and at least gave me 1.25MB/sec. That's still super slow -- is this over a 10Mb

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Joe Little
On 6/22/06, Jeff Bonwick [EMAIL PROTECTED] wrote: a test against the same iscsi targets using linux and XFS and the NFS server implementation there gave me 1.25MB/sec writes. I was about to throw in the towel and deem ZFS/NFS has unusable until B41 came along and at least gave me 1.25MB/sec.

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Nicolas Williams
On Thu, Jun 22, 2006 at 04:22:22PM -0700, Joe Little wrote: Again, the issue is the multiple fsyncs that NFS requires, and likely the serialization of those iscsi requests. Apparently, there is a basic latency in iscsi that one could improve upon with FC, but we are definitely in the all

Re: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-22 Thread Joe Little
On 6/22/06, Bill Moore [EMAIL PROTECTED] wrote: Hey Joe. We're working on some ZFS changes in this area, and if you could run an experiment for us, that would be great. Just do this: echo 'zil_disable/W1' | mdb -kw We're working on some fixes to the ZIL so it won't be a bottleneck when

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Sean Meighan
The vi we were doing was a 2 line file. If you just vi a new file, add one line and exit it would take 15 minutes in fdsynch. On recommendation of a workaround we set set zfs:zil_disable=1 after the reboot the fdsynch is now 0.1 seconds. Now I have no idea if it was this setting or the fact

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Neil Perrin
Well this does look more and more like a duplicate of: 6413510 zfs: writing to ZFS filesystem slows down fsync() on other files in the same FS Neil ___ zfs-discuss mailing list zfs-discuss@opensolaris.org

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Roch
Sean Meighan writes: The vi we were doing was a 2 line file. If you just vi a new file, add one line and exit it would take 15 minutes in fdsynch. On recommendation of a workaround we set set zfs:zil_disable=1 after the reboot the fdsynch is now 0.1 seconds. Now I have no idea

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Torrey McMahon
Roch wrote: Sean Meighan writes: The vi we were doing was a 2 line file. If you just vi a new file, add one line and exit it would take 15 minutes in fdsynch. On recommendation of a workaround we set set zfs:zil_disable=1 after the reboot the fdsynch is now 0.1 seconds. Now I

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Neil Perrin
Torrey McMahon wrote On 06/21/06 10:29,: Roch wrote: Sean Meighan writes: The vi we were doing was a 2 line file. If you just vi a new file, add one line and exit it would take 15 minutes in fdsynch. On recommendation of a workaround we set set zfs:zil_disable=1 after the

Re[2]: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Robert Milkowski
Hello Neil, Wednesday, June 21, 2006, 6:41:50 PM, you wrote: NP Torrey McMahon wrote On 06/21/06 10:29,: Roch wrote: Sean Meighan writes: The vi we were doing was a 2 line file. If you just vi a new file, add one line and exit it would take 15 minutes in fdsynch. On recommendation

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Nicolas Williams
On Wed, Jun 21, 2006 at 10:41:50AM -0600, Neil Perrin wrote: Why is this option available then? (Yes, that's a loaded question.) I wouldn't call it an option, but an internal debugging switch that I originally added to allow progress when initially integrating the ZIL. As Roch says it really

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Torrey McMahon
Nicolas Williams wrote: On Wed, Jun 21, 2006 at 10:41:50AM -0600, Neil Perrin wrote: Why is this option available then? (Yes, that's a loaded question.) I wouldn't call it an option, but an internal debugging switch that I originally added to allow progress when initially integrating

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Neil Perrin
Robert Milkowski wrote On 06/21/06 11:09,: Hello Neil, Why is this option available then? (Yes, that's a loaded question.) NP I wouldn't call it an option, but an internal debugging switch that I NP originally added to allow progress when initially integrating the ZIL. NP As Roch says it

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Bill Sommerfeld
On Wed, 2006-06-21 at 14:15, Neil Perrin wrote: Of course we would need to stress the dangers of setting 'deferred'. What do you guys think? I can think of a use case for deferred: improving the efficiency of a large mega-transaction/batch job such as a nightly build. You create an initially

RE: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Martin, Marcia R
@opensolaris.org; Torrey McMahon Subject: Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved Neil Perrin wrote: Robert Milkowski wrote On 06/21/06 11:09,: Hello Neil, Why is this option available then? (Yes, that's a loaded question.) NP I wouldn't call it an option, but an internal

Re: [zfs-discuss] 15 minute fdsync problem and ZFS: Solved

2006-06-21 Thread Jason Ozolins
Bill Sommerfeld wrote: On Wed, 2006-06-21 at 14:15, Neil Perrin wrote: Of course we would need to stress the dangers of setting 'deferred'. What do you guys think? I can think of a use case for deferred: improving the efficiency of a large mega-transaction/batch job such as a nightly build.