[zfs-discuss] Mount ZFS hangs on boot

2009-03-18 Thread Brent Jones
Hello, I have an X4540 running 2008.11 snv_106 I rebooted it tonight since I had a hung iSCSI connection to the Sun box that wouldn't go away (couldn't delete that particular ZFS filesystem till the initiator drops connection) Upon reboot, the system will hang after printing the license header. I

Re: [zfs-discuss] Public ZFS API ?

2009-03-18 Thread Ian Collins
Cherry Shu wrote: Are any plans for an API that would allow ZFS commands including snapshot/rollback integrated with customer's application? libzfs.h? -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Casper . Dik
Recently there's been discussion [1] in the Linux community about how filesystems should deal with rename(2), particularly in the case of a crash. ext4 was found to truncate files after a crash, that had been written with open(foo.tmp), write(), close() and then rename(foo.tmp, foo). This is

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Joerg Schilling
James Andrewartha jam...@daa.com.au wrote: Recently there's been discussion [1] in the Linux community about how filesystems should deal with rename(2), particularly in the case of a crash. ext4 was found to truncate files after a crash, that had been written with open(foo.tmp), write(),

Re: [zfs-discuss] Public ZFS API ?

2009-03-18 Thread Darren J Moffat
Ian Collins wrote: Cherry Shu wrote: Are any plans for an API that would allow ZFS commands including snapshot/rollback integrated with customer's application? libzfs.h? The API in there is Contracted Consolidation Private. Note that private does not mean hidden it means: Private

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Moore, Joe
Joerg Schilling wrote: James Andrewartha jam...@daa.com.au wrote: Recently there's been discussion [1] in the Linux community about how filesystems should deal with rename(2), particularly in the case of a crash. ext4 was found to truncate files after a crash, that had been written with

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Casper . Dik
AFAIUI, the ZFS transaction group maintains write ordering, at least as far as write()s to the fil e would be in the ZIL ahead of the rename() metadata updates. So I think the atomicity is maintained without requiring the application to call fsync() before cl osing the file. If the TXG is

Re: [zfs-discuss] Public ZFS API ?

2009-03-18 Thread Erast Benson
On Tue, 2009-03-17 at 14:53 -0400, Cherry Shu wrote: Are any plans for an API that would allow ZFS commands including snapshot/rollback integrated with customer's application? Sounds like you are looking for abstraction layering on top of integrated solution such as NexentaStor. Take a look on

Re: [zfs-discuss] Public ZFS API ?

2009-03-18 Thread Richard Elling
Cherry Shu wrote: Are any plans for an API that would allow ZFS commands including snapshot/rollback integrated with customer's application? This is trivially implemented with system(3c). It is somewhat more difficult with libzfs. So it really depends on how much work they want to do. --

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Bob Friesenhahn
On Wed, 18 Mar 2009, Joerg Schilling wrote: The problem in this case is not whether rename() is atomic but whether the file that replaces the old file in an atomic rename() operation is in a stable state on the disk before calling rename(). This topic is quite disturbing to me ... The

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread David Dyer-Bennet
On Wed, March 18, 2009 05:08, Joerg Schilling wrote: The problem in this case is not whether rename() is atomic but whether the file that replaces the old file in an atomic rename() operation is in a stable state on the disk before calling rename(). Good, I was hoping somebody saw it that

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread A Darren Dunham
On Tue, Mar 17, 2009 at 03:51:25PM -0700, Neal Pollack wrote: Step 3, you'll be presented with the disks to be selected as in previous releases. So, for example, to select the boot disks on the Thumper, select both of them: [x] c5t0d0 [x] c4t0d0 Why have the controller

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Richard Elling
Bob Friesenhahn wrote: As it happens, current versions of my own application should be safe from this Linux filesystem bug, but older versions are not. There is even a way to request fsync() on every file close, but that could be quite expensive so it is not the default. Pragmatically, it

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Tim
On Wed, Mar 18, 2009 at 10:59 AM, A Darren Dunham ddun...@taos.com wrote: On Tue, Mar 17, 2009 at 03:51:25PM -0700, Neal Pollack wrote: Step 3, you'll be presented with the disks to be selected as in previous releases. So, for example, to select the boot disks on the Thumper, select both

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Richard Elling
Tim wrote: Just an observation, but it sort of defeats the purpose of buying sun hardware with sun software if you can't even get a this is how your drives will map out of the deal... Sun could fix that, but would you really want a replacement for BIOS? -- richard

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Bryan Allen
+-- | On 2009-03-18 10:14:26, Richard Elling wrote: | | Just an observation, but it sort of defeats the purpose of buying sun | hardware with sun software if you can't even get a this is how your | drives will map out

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Nicolas Williams
On Wed, Mar 18, 2009 at 11:15:48AM -0400, Moore, Joe wrote: Posix doesn't require the OS to sync() the file contents on close for local files like it does for NFS access? How odd. Why should it? If POSIX is agnostic as to system crashes / power failures, then why should it say anything about

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Tim
On Wed, Mar 18, 2009 at 12:14 PM, Richard Elling richard.ell...@gmail.comwrote: Tim wrote: Just an observation, but it sort of defeats the purpose of buying sun hardware with sun software if you can't even get a this is how your drives will map out of the deal... Sun could fix that, but

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Neal Pollack
On 03/18/09 10:43 AM, Tim wrote: On Wed, Mar 18, 2009 at 12:14 PM, Richard Elling richard.ell...@gmail.com mailto:richard.ell...@gmail.com wrote: Tim wrote: Just an observation, but it sort of defeats the purpose of buying sun hardware with sun software if you can't even

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Tim
On Wed, Mar 18, 2009 at 12:49 PM, Neal Pollack neal.poll...@sun.com wrote: On 03/18/09 10:43 AM, Tim wrote: On Wed, Mar 18, 2009 at 12:14 PM, Richard Elling richard.ell...@gmail.com wrote: Tim wrote: Just an observation, but it sort of defeats the purpose of buying sun hardware with

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Carsten Aulbert
Hi Tim, Tim wrote: How does any of that affect an x4500 with onboard controllers that can't ever be moved? Well, consider one box being installed from CD (external USB-CD) and another one which is jumpstarted via the network. The results usually are two different boot device names :( Q: Is

Re: [zfs-discuss] Public ZFS API ?

2009-03-18 Thread Ian Collins
Darren J Moffat wrote: Ian Collins wrote: Cherry Shu wrote: Are any plans for an API that would allow ZFS commands including snapshot/rollback integrated with customer's application? libzfs.h? The API in there is Contracted Consolidation Private. Note that private does not mean hidden

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Nicolas Williams
On Wed, Mar 18, 2009 at 11:43:09AM -0500, Bob Friesenhahn wrote: In summary, I don't agree with you that the misbehavior is correct, but I do agree that copious expensive fsync()s should be assured to work around the problem. fsync() is, indeed, expensive. Lots of calls to fsync() that are

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Bob Friesenhahn
On Wed, 18 Mar 2009, Richard Elling wrote: Bob Friesenhahn wrote: As it happens, current versions of my own application should be safe from this Linux filesystem bug, but older versions are not. There is even a way to request fsync() on every file close, but that could be quite expensive so

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Miles Nordin
ja == James Andrewartha jam...@daa.com.au writes: ja other people are arguing that POSIX says rename(2) is atomic, Their statement is true but it's NOT an argument against T'so who is 100% right: the applications using that calling sequence for crash consistency are not portable under

Re: [zfs-discuss] Mount ZFS hangs on boot

2009-03-18 Thread Brent Jones
On Wed, Mar 18, 2009 at 11:28 AM, Miles Nordin car...@ivy.net wrote: bj == Brent Jones br...@servuhome.net writes:    bj I only have about 50 filesystems, and just a handful of    bj snapshots for each filesystem. there were earlier stories of people who had imports taking hours to complete

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Miles Nordin
c == Miles Nordin car...@ivy.net writes: c fbarrier() on second thought that couldn't help this problem. The goal is to associate writing to the directory (rename) with writing to the file referenced by that inode/handle (write/fsync/``fbarrier''), and in POSIX these two things are pretty

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Casper . Dik
On Wed, Mar 18, 2009 at 11:43:09AM -0500, Bob Friesenhahn wrote: In summary, I don't agree with you that the misbehavior is correct, but I do agree that copious expensive fsync()s should be assured to work around the problem. fsync() is, indeed, expensive. Lots of calls to fsync() that are

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread David Dyer-Bennet
On Wed, March 18, 2009 11:43, Bob Friesenhahn wrote: On Wed, 18 Mar 2009, Joerg Schilling wrote: The problem in this case is not whether rename() is atomic but whether the file that replaces the old file in an atomic rename() operation is in a stable state on the disk before calling

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread David Dyer-Bennet
On Wed, March 18, 2009 11:59, Richard Elling wrote: Bob Friesenhahn wrote: As it happens, current versions of my own application should be safe from this Linux filesystem bug, but older versions are not. There is even a way to request fsync() on every file close, but that could be quite

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread Neal Pollack
On 03/18/09 11:09 AM, Tim wrote: On Wed, Mar 18, 2009 at 12:49 PM, Neal Pollack neal.poll...@sun.com mailto:neal.poll...@sun.com wrote: On 03/18/09 10:43 AM, Tim wrote: On Wed, Mar 18, 2009 at 12:14 PM, Richard Elling richard.ell...@gmail.com mailto:richard.ell...@gmail.com

Re: [zfs-discuss] How do I mirror zfs rpool, x4500?

2009-03-18 Thread A Darren Dunham
On Wed, Mar 18, 2009 at 07:13:41PM +0100, Carsten Aulbert wrote: Well, consider one box being installed from CD (external USB-CD) and another one which is jumpstarted via the network. The results usually are two different boot device names :( Q: Is there an easy way to reset this without

[zfs-discuss] SQLite3 on ZFS (Re: rename(2), atomicity, crashes and fsync())

2009-03-18 Thread Nicolas Williams
On Wed, Mar 18, 2009 at 03:01:30PM -0400, Miles Nordin wrote: IMHO the best reaction to the KDE hysteria would be to make sure SQLite and BerkeleyDB are fast as possible and effortlessly correct on ZFS, and anything that's slow because of too much synchronous writing I tried to do that for

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread David Magda
On Mar 18, 2009, at 12:43, Bob Friesenhahn wrote: POSIX does not care about disks or filesystems. The only correct behavior is for operations to be applied in the order that they are requested of the operating system. This is a core function of any operating system. It is therefore ok

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread James Litchfield
POSIX has a Synchronized I/O Data (and File) Integrity Completion definition (line 115434 of the Issue 7 (POSIX.1-2008) specification). What it says is that writes for a byte range in a file must complete before any pending reads for that byte range are satisfied. It does not say that if you

Re: [zfs-discuss] rename(2), atomicity, crashes and fsync()

2009-03-18 Thread Miles Nordin
dm == David Magda dma...@ee.ryerson.ca writes: dm is this what POSIX actually specifies? i doubt it. If it did, it would basically mandate a log-structured / COW filesystem, which, although not a _bad_ idea, is way too far from a settled debate to be enshrining in a mandatory ``standard''