Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
Not quite brave enough to put dedup into prodiction here. Concerned about the issues some folks have had when releasing large numbers of blocks in one go. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Mar 19, 2010, at 1:28 PM, Richard Jahnel wrote: They way we do this here is: zfs snapshot voln...@snapnow [i]#code to break on error and email not shown.[/i] zfs send -i voln...@snapbefore voln...@snapnow | pigz -p4 -1 file [i]#code to break on error and email not shown.[/i] scp /dir/file u...@remote:/dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gzip -t /dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gunzip /dir/file | zfs receive volname It works for me and it sends a minimum amount of data across the wire which is tested to minimize the chance of inflight issues. Excpet on Sundays when we do a full send. NB. deduped streams should further reduce the snapshot size. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Mon, Mar 22, 2010 at 10:26 AM, Richard Elling richard.ell...@gmail.comwrote: NB. deduped streams should further reduce the snapshot size. I haven't seen a lot of discussion on the list regarding send dedup, but I understand it'll use the DDT if you have dedup enabled on your dataset. What's the process and penalty for using it on a dataset that is not already deduped? Does it build a DDT for just the data in the send? -B -- Brandon High : bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On 03/22/10 05:04 PM, Brandon High wrote: On Mon, Mar 22, 2010 at 10:26 AM, Richard Elling richard.ell...@gmail.com mailto:richard.ell...@gmail.com wrote: NB. deduped streams should further reduce the snapshot size. I haven't seen a lot of discussion on the list regarding send dedup, but I understand it'll use the DDT if you have dedup enabled on your dataset. The send code (which is user-level) builds its own DDT no matter what, but it will use existing checksums if on-disk dedup is already in effect. What's the process and penalty for using it on a dataset that is not already deduped? The penalty is the cost of doing the checksums. Does it build a DDT for just the data in the send? Yes, currently limited to 20% of physical memory size. Lori -B -- Brandon High : bh...@freaks.com mailto:bh...@freaks.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Thu, Mar 18, 2010 at 10:38:00PM -0700, Rob wrote: Can a ZFS send stream become corrupt when piped between two hosts across a WAN link using 'ssh'? No. SSHv2 uses HMAC-MD5 and/or HMAC-SHA-1, depending on what gets negotiated, for integrity protection. The chances of random on the wire corruption going undetected by link-layer CRCs, TCP's CRC and SSHv2's MACs is infinitessimally small. I suspect the chances of local bit flips due to cosmic rays and what not are higher. A bigger problem is that SSHv2 connections do not survive corruption on the wire. That is, if corruption is detected then the connection gets aborted. If you were zfs send'ing 1TB across a long, narrow link and corruption hit the wire while sending the last block you'd have to re-send the whole thing (but even then such corruption would still have to get past link-layer and TCP checksums -- I've seen it happen, so it is possible, but it is also unlikely). Nico -- ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Fri, Mar 19, 2010 at 12:38 PM, Rob slewb...@yahoo.com wrote: Can a ZFS send stream become corrupt when piped between two hosts across a WAN link using 'ssh'? unless the end computers are bad (memory problems, etc.), then the answer should be no. ssh has its own error detection method, and the zfs send stream itself is checksummed. -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Fri, March 19, 2010 00:38, Rob wrote: Can a ZFS send stream become corrupt when piped between two hosts across a WAN link using 'ssh'? For example a host in Australia sends a stream to a host in the UK as follows: # zfs send tank/f...@now | ssh host.uk receive tank/bar In general, errors would be detected by TCP (or by lower-level hardware media error-checking), and the packet retransmitted. I'm not sure what error-checking ssh does on top of that (if any). However, these legacy mechanisms aren't guaranteed to give you the less-than-one-wrong-bit-in-10^15 level of accuracy people tend to want for enterprise backups today (or am I off a couple of orders of magnitude there?). They were defined when data rates were much slower and data volumes much lower. In addition, memory errors on the receiving host (after the TCP stack turns the data over to the application), if undetected, could leave you with corrupted data; not sure what the probability is there. Every scheme has SOME weak spots. The well-designed ones at least tell you the bit error rate. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Fri, 19 Mar 2010, David Dyer-Bennet wrote: However, these legacy mechanisms aren't guaranteed to give you the less-than-one-wrong-bit-in-10^15 level of accuracy people tend to want for enterprise backups today (or am I off a couple of orders of magnitude there?). They were defined when data rates were much slower and data volumes much lower. Are you sure? Have you done any research on this? You are saying that NSA+-grade crypto on the stream is insufficient to detect a modification to the data? It seems that the main failure mode would be disconnect by ssh. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Fri, March 19, 2010 09:49, Bob Friesenhahn wrote: On Fri, 19 Mar 2010, David Dyer-Bennet wrote: However, these legacy mechanisms aren't guaranteed to give you the less-than-one-wrong-bit-in-10^15 level of accuracy people tend to want for enterprise backups today (or am I off a couple of orders of magnitude there?). They were defined when data rates were much slower and data volumes much lower. Are you sure? Have you done any research on this? You are saying that NSA+-grade crypto on the stream is insufficient to detect a modification to the data? I was referring to the tcp and hardware-level checksums. I specifically said I didn't know if SSH did anything on top of that (other people have since said that it does, and it might well be plenty good enough; also that ZFS itself has checksums in the send stream). I don't think of stream crypto as inherently including validity checking, though in practice I suppose it would always be a good idea. It seems that the main failure mode would be disconnect by ssh. Sure, can't guarantee against aborted connections at whatever level (actual interruption of IP connectivity). But those are generally detected and reported as an error; one shouldn't be left with the impression the transfer succeeded. -- David Dyer-Bennet, d...@dd-b.net; http://dd-b.net/ Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/ Photos: http://dd-b.net/photography/gallery/ Dragaera: http://dragaera.info ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On Fri, 19 Mar 2010, David Dyer-Bennet wrote: I don't think of stream crypto as inherently including validity checking, though in practice I suppose it would always be a good idea. This is obviously a vital and necessary function of ssh in order to defend against man in the middle attacks. The main requirement is to make sure that the transferred data can not be deciphered or modified by something other than the two end-points. I don't know if ssh includes retry logic to request that modified data be retransmitted. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
They way we do this here is: zfs snapshot voln...@snapnow [i]#code to break on error and email not shown.[/i] zfs send -i voln...@snapbefore voln...@snapnow | pigz -p4 -1 file [i]#code to break on error and email not shown.[/i] scp /dir/file u...@remote:/dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gzip -t /dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gunzip /dir/file | zfs receive volname It works for me and it sends a minimum amount of data across the wire which is tested to minimize the chance of inflight issues. Excpet on Sundays when we do a full send. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
On 03/20/10 09:28 AM, Richard Jahnel wrote: They way we do this here is: zfs snapshot voln...@snapnow [i]#code to break on error and email not shown.[/i] zfs send -i voln...@snapbefore voln...@snapnow | pigz -p4 -1 file [i]#code to break on error and email not shown.[/i] scp /dir/file u...@remote:/dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gzip -t /dir/file [i]#code to break on error and email not shown.[/i] shh u...@remote gunzip /dir/file | zfs receive volname It works for me and it sends a minimum amount of data across the wire which is tested to minimize the chance of inflight issues. Excpet on Sundays when we do a full send. Don't you trust the stream checksum? -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] ZFS send and receive corruption across a WAN link?
no, but I'm slightly paranoid that way. ;) -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] ZFS send and receive corruption across a WAN link?
Can a ZFS send stream become corrupt when piped between two hosts across a WAN link using 'ssh'? For example a host in Australia sends a stream to a host in the UK as follows: # zfs send tank/f...@now | ssh host.uk receive tank/bar -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
