Re: [zfs-discuss] zfs streams data corruption

2009-02-26 Thread Nicolas Williams
On Wed, Feb 25, 2009 at 07:33:34PM -0500, Miles Nordin wrote: You might also have a look at the, somewhat overcomplicated w.r.t. database-running-snapshot backups, SQLite2 atomic commit URL Toby posted: http://sqlite.org/atomiccommit.html That's for SQLite_3_, 3, not 2. Also, we don't

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Moore, Joe
Miles Nordin wrote: that SQLite2 should be equally as tolerant of snapshot backups as it is of cord-yanking. The special backup features of databases including ``performing a checkpoint'' or whatever, are for systems incapable of snapshots, which is most of them. Snapshots are not

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Toby Thain
On 25-Feb-09, at 9:53 AM, Moore, Joe wrote: Miles Nordin wrote: that SQLite2 should be equally as tolerant of snapshot backups as it is of cord-yanking. The special backup features of databases including ``performing a checkpoint'' or whatever, are for systems incapable of snapshots,

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Miles Nordin
jm == Moore, Joe joe.mo...@siemens.com writes: jm This is correct. The general term for these sorts of jm point-in-time backups is crash consistant. phew, thanks, glad I wasn't talking out my ass again. jm In-flight transactions (ones that have not been committed) at jm the

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Miles Nordin
tt == Toby Thain t...@telegraphics.com.au writes: c so it's ok for snapshots but not cord-yanks if VBox never c bothers to call fsync(). tt Taking good host snapshots may require VB to do that, though. AIUI the contents of a snapshot on the host will be invariant no matter where

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Greg Palmer
Miles Nordin wrote: gp Performing a checkpoint will perform such tasks as making sure gp that all transactions recorded in the log but not yet written gp to the database are written out and that the system is not in gp the middle of a write when you grab the data. great copying

Re: [zfs-discuss] zfs streams data corruption

2009-02-25 Thread Miles Nordin
gp == Greg Palmer gregorylpal...@netscape.net writes: gp Relying on the equivalent of crashing the database to perform gp backups isn't how professionals get the job done. well, nevertheless, it is, and should be, supported by SQLite2. gp Let's take a simple case of a transaction

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of ZFS root filesystem boxes. I have to say that it initially looks to work out cleanly, but of course there are kinks to be worked

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Mattias Pantzare
On Tue, Feb 24, 2009 at 19:18, Nicolas Williams nicolas.willi...@sun.com wrote: On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of ZFS root filesystem boxes.  I have to say that it

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Christopher Mera
-Original Message- From: Mattias Pantzare [mailto:pantz...@gmail.com] Sent: Tuesday, February 24, 2009 1:38 PM To: Nicolas Williams Cc: Christopher Mera; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption On Tue, Feb 24, 2009 at 19:18, Nicolas Williams nicolas.willi

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Brent Jones
] zfs streams data corruption On Tue, Feb 24, 2009 at 19:18, Nicolas Williams nicolas.willi...@sun.com wrote: On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of ZFS root filesystem

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 07:37:39PM +0100, Mattias Pantzare wrote: On Tue, Feb 24, 2009 at 19:18, Nicolas Williams nicolas.willi...@sun.com wrote: When you snapshot a ZFS filesystem you get just that -- a snapshot at the filesystem level.  That does not mean you get a snapshot at the

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Christopher Mera
Thanks for your responses.. Brent: And I'd have to do that for every system that I'd want to clone? There must be a simpler way.. perhaps I'm missing something. Regards, Chris ___ zfs-discuss mailing list zfs-discuss@opensolaris.org

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 10:56:45AM -0800, Brent Jones wrote: If you are writing a script to handle ZFS snapshots/backups, you could issue an SMF command to stop the service before taking the snapshot. Or at the very minimum, perform an SQL dump of the DB so you at least have a consistent full

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Brent Jones
On Tue, Feb 24, 2009 at 11:32 AM, Christopher Mera cm...@reliantsec.net wrote: Thanks for your responses.. Brent: And I'd have to do that for every system that I'd want to clone?  There must be a simpler way.. perhaps I'm missing something. Regards, Chris Well, unless the database

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Miles Nordin
cm == Christopher Mera cm...@reliantsec.net writes: cm it would be ideal to quiesce the system before a snapshot cm anyway, no? It would be more ideal to find the bug in SQLite2 or ZFS. Training everyone, ``you always have to quiesce the system before proceeding, because it's full of

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Miles Nordin
bj == Brent Jones br...@servuhome.net writes: bj tolerating that your database will not be consistent after a bj snapshot and have to replay logs / consistency check it ``not be consistent'' != ``have to replay logs'' pgpLNmP6hsO3I.pgp Description: PGP signature

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Christopher Mera
Mera Cc: Mattias Pantzare; Nicolas Williams; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption On Tue, Feb 24, 2009 at 11:32 AM, Christopher Mera cm...@reliantsec.net wrote: Thanks for your responses.. Brent: And I'd have to do that for every system that I'd

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Lori Alt
; Nicolas Williams; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption On Tue, Feb 24, 2009 at 11:32 AM, Christopher Mera cm...@reliantsec.net wrote: Thanks for your responses.. Brent: And I'd have to do that for every system that I'd want to clone? There must

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Christopher Mera
[mailto:lori@sun.com] Sent: Tuesday, February 24, 2009 3:13 PM To: Christopher Mera Cc: Brent Jones; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption On 02/24/09 12:57, Christopher Mera wrote: How is it that flash archives can avoid these headaches

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: I don't think there's any way to ask svc.config to pause. Well, IIRC that's not quite right. You can pstop svc.startd, gently kill (i.e., not with SIGKILL) svc.configd, take your snapshot, then prun svc.startd. Nico --

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Tim
On Tue, Feb 24, 2009 at 2:15 PM, Nicolas Williams nicolas.willi...@sun.comwrote: On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: I don't think there's any way to ask svc.config to pause. Well, IIRC that's not quite right. You can pstop svc.startd, gently kill (i.e., not

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 02:53:14PM -0500, Miles Nordin wrote: cm == Christopher Mera cm...@reliantsec.net writes: cm it would be ideal to quiesce the system before a snapshot cm anyway, no? It would be more ideal to find the bug in SQLite2 or ZFS. Training everyone, ``you always

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 02:27:18PM -0600, Tim wrote: On Tue, Feb 24, 2009 at 2:15 PM, Nicolas Williams nicolas.willi...@sun.comwrote: On Tue, Feb 24, 2009 at 01:17:47PM -0600, Nicolas Williams wrote: I don't think there's any way to ask svc.config to pause. Well, IIRC that's not quite

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 12:19:22PM -0800, Christopher Mera wrote: There are over 700 boxes deployed using Flash Archive's on an S10 system with a UFS root. We've been working on basing our platform on a ZFS root and took Scott Dickson's suggestions

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Toby Thain
On 24-Feb-09, at 1:37 PM, Mattias Pantzare wrote: On Tue, Feb 24, 2009 at 19:18, Nicolas Williams nicolas.willi...@sun.com wrote: On Mon, Feb 23, 2009 at 10:05:31AM -0800, Christopher Mera wrote: I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Tim
On Tue, Feb 24, 2009 at 2:37 PM, Nicolas Williams nicolas.willi...@sun.comwrote: Hot Backup? # Connect to the database sqlite3 db $dbfile # Lock the database, copy and commit or rollback if {[catch {db transaction immediate {file copy $dbfile ${dbfile}.bak}} res]} {

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 03:25:53PM -0600, Tim wrote: On Tue, Feb 24, 2009 at 2:37 PM, Nicolas Williams nicolas.willi...@sun.comwrote: Hot Backup? # Connect to the database sqlite3 db $dbfile # Lock the database, copy and commit or rollback if {[catch {db

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Mon, Feb 23, 2009 at 02:36:07PM -0800, Christopher Mera wrote: panic[cpu0]/thread=dacac880: BAD TRAP: type=e (#pf Page fault) rp=d9f61850 addr=1048c0d occurred in module zfs due to an illegal access to a user address Can you describe what you're doing with your snapshot? Are you zfs

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Christopher Mera
fine. -Original Message- From: Nicolas Williams [mailto:nicolas.willi...@sun.com] Sent: Tuesday, February 24, 2009 5:43 PM To: Christopher Mera Cc: lori@sun.com; zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption On Mon, Feb 23, 2009 at 02:36:07PM -0800

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Nicolas Williams
On Tue, Feb 24, 2009 at 03:08:18PM -0800, Christopher Mera wrote: It's a zfs snapshot that's then sent to a file.. On the new boxes I'm doing a jumpstart install with the SUNWCreq package, and using the finish script to mount an NFS filesystem that contains the *.zfs dump files. Zfs receive

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Greg Palmer
Miles Nordin wrote: Hope this helps untangle some FUD. Snapshot backups of databases *are* safe, unless the database or application above it is broken in a way that makes cord-yanking unsafe too. Actually Miles, what they were asking for is generally referred to as a checkpoint and they are

Re: [zfs-discuss] zfs streams data corruption

2009-02-24 Thread Miles Nordin
gp == Greg Palmer gregorylpal...@netscape.net writes: gp Performing a checkpoint will perform such tasks as making sure gp that all transactions recorded in the log but not yet written gp to the database are written out and that the system is not in gp the middle of a write when

[zfs-discuss] zfs streams data corruption

2009-02-23 Thread Christopher Mera
Hi folks, I recently read up on Scott Dickson's blog with his solution for jumpstart/flashless cloning of ZFS root filesystem boxes. I have to say that it initially looks to work out cleanly, but of course there are kinks to be worked out that deal with auto mounting filesystems mostly.

Re: [zfs-discuss] zfs streams data corruption

2009-02-23 Thread Lori Alt
I don't know what's causing this, nor have I seen it. Can you send more information about the errors you see when the system crashes and svc.configd fails? Doing the scrub seems like a harmless and possibly useful thing to do. Let us know what you find out from it. Lori On 02/23/09 11:05,

Re: [zfs-discuss] zfs streams data corruption

2009-02-23 Thread Christopher Mera
, February 23, 2009 1:17 PM To: Christopher Mera Cc: zfs-discuss@opensolaris.org Subject: Re: [zfs-discuss] zfs streams data corruption I don't know what's causing this, nor have I seen it. Can you send more information about the errors you see when the system crashes and svc.configd fails