Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Jun 10, 2011, at 17:25, Volodymyr Kostyrko wrote: Am I missing something? How about using fletcher[24] for dedup? Fletcher is fairly weak as things go, and so even though two checksums are the same, there's a decent chance that the data is actually different. At least with recent releases of (Open)Solaris, when you enable do a 'dedup=on' the has used is SHA-256, which has very, very, very, low odds of having the same value occur from two different blocks of data. When ZFS dedupe originally came out you could have one of the following values: . off . on (== sha256) . flecther4 with verify/compare . sha256 (without verify/compare) . sha256 with verify There was a long-ish thread on zfs-discuss fairly recently on whether SHA-256 was good enough where you could trust it, or whether one should do a verify step in addition to SHA-256: http://mail.opensolaris.org/pipermail/zfs-discuss/2011-January/046875.html While some people argued that it was prudent to use verify (especially with your data/job on the line), a good portion of folks though said that it's not worth it (i.e., if you're not worried about being hit by lightning (2^-17 to 2^-18), you shouldn't be worried about a hash collision (2^-128)). ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Jun 10, 2011, at 17:24, Bob Friesenhahn wrote: Dedup can require a huge amount of RAM, or a dedicated L2ARC SSD, depending on the size of your storage. You should not enable it unless you are prepared for the consequences. Under OpenSolaris, each tracking entry for a deduped block (which can be between 512B to 128KB) can be up to 376 bytes (struct ddt_entry): so for one 1 GB (10^9) of deduped data (244140 blocks@4K), you would need ~91MB of overhead to keep track of it; for 1 TB (10^12) of deduped data you would need ~91 GB of space to keep track of all the blocks. And if you can't fit the DDT in RAM, it will have to be saved to disk, which means more I/O to fetch the data. http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/ddt.h?rev=1.2 If your data is in blocks smaller than 4K you'll need more memory for the DDT; if the data is broken up into blocks larger than 4K you'll probably need less. Also remember that even though an L2ARC cache may save you from having to go to spinning rust, you still need to use some RAM (struct arc_buf_hdr; ~178B) to reference the DDT stuff in L2ARC: http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c A few threads on zfs-dicuss on this: http://mail.opensolaris.org/pipermail/zfs-discuss/2011-April/thread.html#48026 http://mail.opensolaris.org/pipermail/zfs-discuss/2011-May/thread.html#48185 Also, the above numbers are for OpenSolaris: someone may want to check the structure sizes for FreeBSD to be sure. They should get you in the right ballpark though. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote: I have merged ZFS version 28 to 8-STABLE (revision 222741) Follow-up, since we're gradually upgrading our ZFS-based RELENG_8 servers to ZFSv28. Committers/those involved should see my very last paragraph. First, server upgrades: We've upgraded 2 of the 4 (including updating zfs and zpools), and so far things are working wonderfully. One of those 2 boxes is our NFS filer (which also does backups via rsync/rsnapshot), so that one's been a big worry-point of mine. The next rsync/rsnapshot runs tonight, so I'll be awake watching intently. All these systems are graphed via bsnmpd (memory, CPU, disk I/O, etc.). Second, performance tweaks: We're testing changing to our tweaks; the following directives have been commented out in our /boot/loader.conf files (e.g. we're now using prefetching): # vfs.zfs.prefetch_disable=1 And the following tunable has been removed completely, because it's now the default in ZFSv28 (see cvsweb, look at line 40 of the relevant commit for src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/txg.c): vfs.zfs.txg.timeout=5 Finally, talking a bit about dedup: In the ZFSv28 commit, did the zfs.8 man page get updated? I find no mention of the dedup property in the zfs(8) man page, and yes I did remove the old /usr/share/man/cat8/zfs.8.gz file. We tried using dedup on one of our systems, but within 10-15 minutes turned it off. I believe the added CPU overhead of dedup was causing the system to act bursty in other non-ZFS-related tasks; e.g. turn on dedup, then in a SSH window hold down the letter q indefinitely, then in another window do some ZFS I/O. The q would stall for 1-2 seconds at times (SSH connectivity was via direct private LAN, so network latency was not what we seen). Without dedup this behaviour wasn't seen at all. I'm happy to try any advice/patches on a locally-accessible box (e.g. private LAN, VGA console is right behind me, etc.). I have not tried tinkering with the following settings to find out what may relieve this issue. I'm referring to the sections titled Trust or verify and Selecting a checksum on Jeff's blog here: http://blogs.oracle.com/bonwick/entry/zfs_dedup All in all, thank you everyone for the work that's gone in to MFC'ing this to RELENG_8. I really do mean that. I'm a harsh bastard on the mailing lists, no question about it. But I always appreciate people doing the grunt work that I myself cannot do (over my head). I state this seriously: if any of you folks who participated in this MFC have donation links (PayPal, etc.), please give them to me. You absolutely will see some worthwhile kick-backs for your efforts, with no strings attached. Just my way of saying thank you. -- | Jeremy Chadwick j...@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB | ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Fri, 10 Jun 2011, Jeremy Chadwick wrote: We tried using dedup on one of our systems, but within 10-15 minutes turned it off. I believe the added CPU overhead of dedup was causing the system to act bursty in other non-ZFS-related tasks; e.g. turn on dedup, then in a SSH window hold down the letter q indefinitely, then in another window do some ZFS I/O. The q would stall for 1-2 seconds at times (SSH connectivity was via direct private LAN, so network latency was not what we seen). Without dedup this behaviour wasn't seen at all. I'm happy to try any advice/patches on a locally-accessible box (e.g. private LAN, VGA console is right behind me, etc.). Dedup can require a huge amount of RAM, or a dedicated L2ARC SSD, depending on the size of your storage. You should not enable it unless you are prepared for the consequences. Solaris 11 Express does not admit to supporting dedup even though it can be enabled in previous OpenSolaris and is supported in Oracle's NAS products (which run a variant of Solaris 11). Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
06.06.2011 13:53, Martin Matuska написав(ла): Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication Am I missing something? How about using fletcher[24] for dedup? -- Sphinx of black quartz judge my vow. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. Beware that the compatibility layer is known broken on big-endian architectures, i.e. powerpc64 and sparc64. If upgrading your boot pool to version 28, please don't forget to read UPDATING and properly update your boot code. Thanks to everyone working on the ZFS port, especially to Pawel Jakub Dawidek (pjd) for doing most of the work! Marius ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Thu, Jun 09, 2011 at 12:46:19AM +0200, C. P. Ghost wrote: On Wed, Jun 8, 2011 at 11:12 PM, Marius Strobl mar...@alchemy.franken.de wrote: On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier ? transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. Beware that the compatibility layer is known broken on big-endian architectures, i.e. powerpc64 and sparc64. Thanks for the heads-up! I was just about to update a couple of sparc64 machines here. Fortunately, the only ZFS file systems there are external file systems (no /, /usr, /var etc...), so it's gonna be painless, I suppose. But what about other layouts? Does it mean that an installkernel, reboot, and installworld won't work? No, using the old zfs binaries with the new kernel module triggers the problem. Using the new zfs binaries with the new kernel module works (at least with CURRENT) so you'd need disregard everything that is documented and do what is not guaranteed to work but usually also does, i.e. do installkernel, installworld and then reboot. If upgrading your boot pool to version 28, please don't forget to read UPDATING and properly update your boot code. Thanks to everyone working on the ZFS port, especially to Pawel Jakub Dawidek (pjd) for doing most of the work! Marius ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Thu, Jun 9, 2011 at 12:55 AM, Marius Strobl mar...@alchemy.franken.de wrote: On Thu, Jun 09, 2011 at 12:46:19AM +0200, C. P. Ghost wrote: On Wed, Jun 8, 2011 at 11:12 PM, Marius Strobl mar...@alchemy.franken.de wrote: On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier ? transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. Beware that the compatibility layer is known broken on big-endian architectures, i.e. powerpc64 and sparc64. Thanks for the heads-up! I was just about to update a couple of sparc64 machines here. Fortunately, the only ZFS file systems there are external file systems (no /, /usr, /var etc...), so it's gonna be painless, I suppose. But what about other layouts? Does it mean that an installkernel, reboot, and installworld won't work? No, using the old zfs binaries with the new kernel module triggers the problem. Using the new zfs binaries with the new kernel module works (at least with CURRENT) so you'd need disregard everything that is documented and do what is not guaranteed to work but usually also does, i.e. do installkernel, installworld and then reboot. Ah, yes, that's what I expected. Thank you! -cpghost. -- Cordula's Web. http://www.cordula.ws/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Wed, Jun 8, 2011 at 11:12 PM, Marius Strobl mar...@alchemy.franken.de wrote: On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. Beware that the compatibility layer is known broken on big-endian architectures, i.e. powerpc64 and sparc64. Thanks for the heads-up! I was just about to update a couple of sparc64 machines here. Fortunately, the only ZFS file systems there are external file systems (no /, /usr, /var etc...), so it's gonna be painless, I suppose. But what about other layouts? Does it mean that an installkernel, reboot, and installworld won't work? If upgrading your boot pool to version 28, please don't forget to read UPDATING and properly update your boot code. Thanks to everyone working on the ZFS port, especially to Pawel Jakub Dawidek (pjd) for doing most of the work! Marius Thanks, -cpghost. -- Cordula's Web. http://www.cordula.ws/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
HEADS UP: ZFS v28 merged to 8-STABLE
Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. If upgrading your boot pool to version 28, please don't forget to read UPDATING and properly update your boot code. Thanks to everyone working on the ZFS port, especially to Pawel Jakub Dawidek (pjd) for doing most of the work! -- Martin Matuska FreeBSD committer http://blog.vx.sk ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On Mon, Jun 06, 2011 at 12:53:11PM +0200, Martin Matuska wrote:
Hi,
I have merged ZFS version 28 to 8-STABLE (revision 222741)
New major features:
- data deduplication
- triple parity RAIDZ (RAIDZ3)
- zfs diff
- zpool split
- snapshot holds
- zpool import -F. Allows to rewind corrupted pool to earlier
transaction group
- possibility to import pool in read-only mode
For updating, there is a compatibility layer so that in the update phase
most functionality of the new zfs binaries can be used with the old
kernel module and old zfs binaries with the new kernel module.
If upgrading your boot pool to version 28, please don't forget to read
UPDATING and properly update your boot code.
Thanks to everyone working on the ZFS port, especially to
Pawel Jakub Dawidek (pjd) for doing most of the work!
Thanks for the work on this, guys!
I've already managed to find something odd. This message only appears
on console, not via pty/tty.
icarus# zpool create backups ada2
Solaris(cont): !created version 28 pool backups using 28
src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/spa_history.c
contains:
490 void
491 spa_history_log_version(spa_t *spa, history_internal_events_t event)
492 {
493 #ifdef _KERNEL
494 uint64_t current_vers = spa_version(spa);
495
496 if (current_vers = SPA_VERSION_ZPOOL_HISTORY) {
497 spa_history_log_internal(event, spa, NULL,
498 pool spa %llu; zfs spa %llu; zpl %d; uts %s %s %s %s,
499 (u_longlong_t)current_vers, SPA_VERSION, ZPL_VERSION,
500 utsname.nodename, utsname.release, utsname.version,
501 utsname.machine);
502 }
503 cmn_err(CE_CONT, !%s version %llu pool %s using %llu,
504 event == LOG_POOL_IMPORT ? imported :
505 event == LOG_POOL_CREATE ? created : accessed,
506 (u_longlong_t)current_vers, spa_name(spa), SPA_VERSION);
507 #endif
508 }
A zpool destroy, etc. does not print any similar message. It only
happens on pool creation. Is this intentional behaviour? What does
Solaris(cont) represent in the context of FreeBSD?
--
| Jeremy Chadwick j...@parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
On 06/06/11 06:53, Martin Matuska wrote: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) New major features: - data deduplication - triple parity RAIDZ (RAIDZ3) - zfs diff - zpool split - snapshot holds - zpool import -F. Allows to rewind corrupted pool to earlier transaction group - possibility to import pool in read-only mode For updating, there is a compatibility layer so that in the update phase most functionality of the new zfs binaries can be used with the old kernel module and old zfs binaries with the new kernel module. If upgrading your boot pool to version 28, please don't forget to read UPDATING and properly update your boot code. Thanks to everyone working on the ZFS port, especially to Pawel Jakub Dawidek (pjd) for doing most of the work! Thanks for everyone's hard work (and the __FreeBSD_version bump!). -Boris ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
Hello, Martin Matuska! On Mon, Jun 06, 2011 at 12:53:11PM +0200 m...@freebsd.org wrote about HEADS UP: ZFS v28 merged to 8-STABLE: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) finally! Thanks! My home server works ok with this update. -- Lystopad Olexandr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
Re: HEADS UP: ZFS v28 merged to 8-STABLE
Thanks. Everything works fine. 2011/6/6 Lystopad Olexandr l...@laa.zp.ua: Hello, Martin Matuska! On Mon, Jun 06, 2011 at 12:53:11PM +0200 m...@freebsd.org wrote about HEADS UP: ZFS v28 merged to 8-STABLE: Hi, I have merged ZFS version 28 to 8-STABLE (revision 222741) finally! Thanks! My home server works ok with this update. -- Lystopad Olexandr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org -- Zhihao Yuan, nickname lichray The best way to predict the future is to invent it. ___ 4BSD -- http://4bsd.biz/ ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to freebsd-stable-unsubscr...@freebsd.org
