Re: [zfs-discuss] Repairing Faulted ZFS pool when zbd doesn't recognize the pool as existing
Hi Chris, Notice your message below, would you mind to share the steps on how the recovery works for you? I have kind of similar issue. Quick update; George has been very helpful, and there is progress with my zpool. I've got partial read ability at this point, and some data is being copied off. It was _way_ beyond my skillset to do anything. Once we have things resolved to a better level, I'll post more details (with a lot of help from George I'd say). - Thanks in advance for your inputs! Regards Kwang Whee Lee EMAIL DISCLAIMER This email message and its attachments are confidential and may also contain copyright or privileged material. If you are not the intended recipient, you may not forward the email or disclose or use the information contained in it. If you have received this email message in error, please advise the sender immediately by replying to this email and delete the message and any associated attachments. Any views, opinions, conclusions, advice or statements expressed in this email message are those of the individual sender and should not be relied upon as the considered view, opinion, conclusions, advice or statement of this company except where the sender expressly, and with authority, states them to be the considered view, opinion, conclusions, advice or statement of this company. Every care is taken but we recommend that you scan any attachments for viruses. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] Benefits of enabling compression in ZFS for the zones
Hi all, By default I'm using ZFS for all the zones: admjoresp@cyd-caszonesrv-15:~$ zfs list NAME USED AVAIL REFER MOUNTPOINT opt 4.77G 45.9G 285M /opt opt/zones 4.49G 45.9G29K /opt/zones opt/zones/glad-gm02-ftcl01 367M 45.9G 367M /opt/zones/glad-gm02-ftcl01 opt/zones/glad-gp02-ftcl01 502M 45.9G 502M /opt/zones/glad-gp02-ftcl01 opt/zones/glad-gp02-ftcl02 1.21G 45.9G 1.21G /opt/zones/glad-gp02-ftcl02 opt/zones/mbd-tcasino-02 257M 45.9G 257M /opt/zones/mbd-tcasino-02 opt/zones/mbd-tcasino-04 281M 45.9G 281M /opt/zones/mbd-tcasino-04 opt/zones/mbfd-gp02-ftcl01 501M 45.9G 501M /opt/zones/mbfd-gp02-ftcl01 opt/zones/mbfd-gp02-ftcl02 475M 45.9G 475M /opt/zones/mbfd-gp02-ftcl02 opt/zones/mbhd-gp02-ftcl01 475M 45.9G 475M /opt/zones/mbhd-gp02-ftcl01 opt/zones/mbhd-gp02-ftcl02 507M 45.9G 507M /opt/zones/mbhd-gp02-ftcl02 However, I have the compression disabled in all of them. According to this Oracle whitepaper http://www.oracle.com/technetwork/server-storage/solaris10/solaris-zfs-in-containers-wp-167903.pdf: The next example demonstrates the compression property. If compression is enabled, Oracle Solaris ZFS will transparently compress all of the data before it is written to disk. The benefits of compression are both saved disk space and possible write speed improvements. What exactly means POSSIBLE write speed improvements? As you can see above I don't use to have any room problems, so if I'm going to enable the compression flag it has to be because of the write speed improvements. BTW, all the zones are containing Glassfish and they're logging a lot. More than that, there's not so much IO activity in the mentioned ZFS fs. -- I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On Tue, Jul 10, 2012 at 4:25 PM, Jordi Espasa Clofent jespa...@minibofh.org wrote: Hi all, By default I'm using ZFS for all the zones: admjoresp@cyd-caszonesrv-15:~$ zfs list NAME USED AVAIL REFER MOUNTPOINT opt 4.77G 45.9G 285M /opt opt/zones 4.49G 45.9G29K /opt/zones opt/zones/glad-gm02-ftcl01 367M 45.9G 367M /opt/zones/glad-gm02-ftcl01 opt/zones/glad-gp02-ftcl01 502M 45.9G 502M /opt/zones/glad-gp02-ftcl01 opt/zones/glad-gp02-ftcl02 1.21G 45.9G 1.21G /opt/zones/glad-gp02-ftcl02 opt/zones/mbd-tcasino-02 257M 45.9G 257M /opt/zones/mbd-tcasino-02 opt/zones/mbd-tcasino-04 281M 45.9G 281M /opt/zones/mbd-tcasino-04 opt/zones/mbfd-gp02-ftcl01 501M 45.9G 501M /opt/zones/mbfd-gp02-ftcl01 opt/zones/mbfd-gp02-ftcl02 475M 45.9G 475M /opt/zones/mbfd-gp02-ftcl02 opt/zones/mbhd-gp02-ftcl01 475M 45.9G 475M /opt/zones/mbhd-gp02-ftcl01 opt/zones/mbhd-gp02-ftcl02 507M 45.9G 507M /opt/zones/mbhd-gp02-ftcl02 However, I have the compression disabled in all of them. According to this Oracle whitepaper http://www.oracle.com/technetwork/server-storage/solaris10/solaris-zfs-in-containers-wp-167903.pdf: The next example demonstrates the compression property. If compression is enabled, Oracle Solaris ZFS will transparently compress all of the data before it is written to disk. The benefits of compression are both saved disk space and possible write speed improvements. What exactly means POSSIBLE write speed improvements? compression = possibly less data to write (depending on the data) = possibly faster writes Some data is not compressible (e.g. mpeg4 movies), so in that case you won't see any improvements. -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On 2012-07-10 11:34, Fajar A. Nugraha wrote: compression = possibly less data to write (depending on the data) = possibly faster writes Some data is not compressible (e.g. mpeg4 movies), so in that case you won't see any improvements. Thanks for your answer Fajar. As I said in my initial mail, those zones are mainly only writing some Glassfish logs. Since they all are text files, I guess I can save a lot of space using compression. Hopefully I can even improve the performance. Can I? However. What's the difference using zfs set compression=on opt/zones/whatever_zone or zfs set compression=gzip-6 opt/zones/whatever_zone or zfs set compression=gzip-9 opt/zones/whatever_zone TIA. -- I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On 07/10/12 09:25 PM, Jordi Espasa Clofent wrote: Hi all, By default I'm using ZFS for all the zones: admjoresp@cyd-caszonesrv-15:~$ zfs list NAME USED AVAIL REFER MOUNTPOINT opt 4.77G 45.9G 285M /opt opt/zones 4.49G 45.9G29K /opt/zones opt/zones/glad-gm02-ftcl01 367M 45.9G 367M /opt/zones/glad-gm02-ftcl01 opt/zones/glad-gp02-ftcl01 502M 45.9G 502M /opt/zones/glad-gp02-ftcl01 opt/zones/glad-gp02-ftcl02 1.21G 45.9G 1.21G /opt/zones/glad-gp02-ftcl02 opt/zones/mbd-tcasino-02 257M 45.9G 257M /opt/zones/mbd-tcasino-02 opt/zones/mbd-tcasino-04 281M 45.9G 281M /opt/zones/mbd-tcasino-04 opt/zones/mbfd-gp02-ftcl01 501M 45.9G 501M /opt/zones/mbfd-gp02-ftcl01 opt/zones/mbfd-gp02-ftcl02 475M 45.9G 475M /opt/zones/mbfd-gp02-ftcl02 opt/zones/mbhd-gp02-ftcl01 475M 45.9G 475M /opt/zones/mbhd-gp02-ftcl01 opt/zones/mbhd-gp02-ftcl02 507M 45.9G 507M /opt/zones/mbhd-gp02-ftcl02 However, I have the compression disabled in all of them. According to this Oracle whitepaper http://www.oracle.com/technetwork/server-storage/solaris10/solaris-zfs-in-containers-wp-167903.pdf: The next example demonstrates the compression property. If compression is enabled, Oracle Solaris ZFS will transparently compress all of the data before it is written to disk. The benefits of compression are both saved disk space and possible write speed improvements. What exactly means POSSIBLE write speed improvements? With compression enabled, less data has to be written to disk, so N bytes writes in N/compress ratio time. On most systems, the performance cost of compressing and uncompressing data is relatively low. As you can see above I don't use to have any room problems, so if I'm going to enable the compression flag it has to be because of the write speed improvements. I always enable compression by default and only turn it off for filesystems I know hold un-compressible data such as media files. -- Ian. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On Tue, Jul 10, 2012 at 4:40 PM, Jordi Espasa Clofent jespa...@minibofh.org wrote: On 2012-07-10 11:34, Fajar A. Nugraha wrote: compression = possibly less data to write (depending on the data) = possibly faster writes Some data is not compressible (e.g. mpeg4 movies), so in that case you won't see any improvements. Thanks for your answer Fajar. As I said in my initial mail, those zones are mainly only writing some Glassfish logs. Since they all are text files, I guess I can save a lot of space using compression. Hopefully I can even improve the performance. Can I? correct. Even normal OS files are usually compressible-enough. For example, this is my root partition (Ubuntu, but uses zfs nontheles) $ sudo zfs get compression,compressratio C/ROOT/precise NAMEPROPERTY VALUE SOURCE C/ROOT/precise compressiongzip inherited from C C/ROOT/precise compressratio 2.48x - so on that dataset I save over 50% I/O read/writes (in bytes) However. What's the difference using zfs set compression=on opt/zones/whatever_zone on = standard LZJB compression (very fast, but doesn't compress much) or zfs set compression=gzip-6 opt/zones/whatever_zone gzip-6 and gzip uses gzip compression. Fast enough, good compression. or zfs set compression=gzip-9 opt/zones/whatever_zone gzip-9 = uses gzip's best, but also slowest, compression -- Fajar ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
Thanks for you explanation Fajar. However, take a look on the next lines: # available ZFS in the system root@sct-caszonesrv-07:~# zfs list NAME USED AVAIL REFER MOUNTPOINT opt 532M 34.7G 290M /opt opt/zones243M 34.7G32K /opt/zones opt/zones/sct-scw02-shared 243M 34.7G 243M /opt/zones/sct-scw02-shared static 104K 58.6G34K /var/www/ # creating a file in /root (UFS) root@sct-caszonesrv-07:~# dd if=/dev/zero of=file.bin count=1024 bs=1024 1024+0 records in 1024+0 records out 1048576 bytes (1.0 MB) copied, 0.0545957 s, 19.2 MB/s root@sct-caszonesrv-07:~# pwd /root # enable compression in some ZFS zone root@sct-caszonesrv-07:~# zfs set compression=on opt/zones/sct-scw02-shared # copying the previos file to this zone root@sct-caszonesrv-07:~# cp /root/file.bin /opt/zones/sct-scw02-shared/root/ # checking the file size in the origin dir (UFS) and the destination one (ZFS with compression enabled) root@sct-caszonesrv-07:~# ls -lh /root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:21 /root/file.bin root@sct-caszonesrv-07:~# ls -lh /opt/zones/sct-scw02-shared/root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:22 /opt/zones/sct-scw02-shared/root/file.bin # the both files has exactly the same cksum! root@sct-caszonesrv-07:~# cksum /root/file.bin 3018728591 1048576 /root/file.bin root@sct-caszonesrv-07:~# cksum /opt/zones/sct-scw02-shared/root/file.bin 3018728591 1048576 /opt/zones/sct-scw02-shared/root/file.bin So... I don't see any size variation with this test. Actually I tried the same with compression=gzip/gzip-6 or even gzip-9 and is always the same. I understand it shouldn't be? What am I missing or doing wrong? More info about the system: root@sct-caszonesrv-07:~# cat /etc/release Oracle Solaris 10 8/11 s10x_u10wos_17b X86 Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved. Assembled 23 August 2011 -- I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
Of course you don't see any difference, this is how it should work. 'ls' will never report the compressed size, because it's not aware of it. Nothing is aware of the compression and decompression that takes place on-the-fly, except of course zfs. That's the reason why you could gain in write and read speed if you use compression, because the actual amount of compressed data that is being written and read from the pool is smaller than the original data. And I think with the checksum test you prooved that zfs checksums the uncompressed data. Regards, Feci On Tue, Jul 10, 2012 at 1:29 PM, Jordi Espasa Clofent jespa...@minibofh.org wrote: Thanks for you explanation Fajar. However, take a look on the next lines: # available ZFS in the system root@sct-caszonesrv-07:~# zfs list NAME USED AVAIL REFER MOUNTPOINT opt 532M 34.7G 290M /opt opt/zones243M 34.7G32K /opt/zones opt/zones/sct-scw02-shared 243M 34.7G 243M /opt/zones/sct-scw02-shared static 104K 58.6G34K /var/www/ # creating a file in /root (UFS) root@sct-caszonesrv-07:~# dd if=/dev/zero of=file.bin count=1024 bs=1024 1024+0 records in 1024+0 records out 1048576 bytes (1.0 MB) copied, 0.0545957 s, 19.2 MB/s root@sct-caszonesrv-07:~# pwd /root # enable compression in some ZFS zone root@sct-caszonesrv-07:~# zfs set compression=on opt/zones/sct-scw02-shared # copying the previos file to this zone root@sct-caszonesrv-07:~# cp /root/file.bin /opt/zones/sct-scw02-shared/** root/ # checking the file size in the origin dir (UFS) and the destination one (ZFS with compression enabled) root@sct-caszonesrv-07:~# ls -lh /root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:21 /root/file.bin root@sct-caszonesrv-07:~# ls -lh /opt/zones/sct-scw02-shared/** root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:22 /opt/zones/sct-scw02-shared/** root/file.bin # the both files has exactly the same cksum! root@sct-caszonesrv-07:~# cksum /root/file.bin 3018728591 1048576 /root/file.bin root@sct-caszonesrv-07:~# cksum /opt/zones/sct-scw02-shared/** root/file.bin 3018728591 1048576 /opt/zones/sct-scw02-shared/**root/file.bin So... I don't see any size variation with this test. Actually I tried the same with compression=gzip/gzip-6 or even gzip-9 and is always the same. I understand it shouldn't be? What am I missing or doing wrong? More info about the system: root@sct-caszonesrv-07:~# cat /etc/release Oracle Solaris 10 8/11 s10x_u10wos_17b X86 Copyright (c) 1983, 2011, Oracle and/or its affiliates. All rights reserved. Assembled 23 August 2011 -- I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. __**_ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/**mailman/listinfo/zfs-discusshttp://mail.opensolaris.org/mailman/listinfo/zfs-discuss ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Jordi Espasa Clofent root@sct-caszonesrv-07:~# zfs set compression=on opt/zones/sct-scw02- shared If you use compression=on, or lzjb, then you're using very fast compression. Should not hurt performance, in fact, may gain performance for highly compressible data. If you use compression=gzip (or any gzip level 1 thru 9) then you're using a fairly expensive compression algorithm. It will almost certainly hurt performance, but you may gain more disk space. (Probably not.) # copying the previos file to this zone ... # checking the file size in the origin dir (UFS) and the destination one (ZFS with compression enabled) ... # the both files has exactly the same cksum! When you enable filesystem compression, it is all done behind the scenes. It is intended to look and behave exactly the same at the file level, so you should not see any difference in your files. Your files should appear to be exactly the same, and have exactly the same size and checksum as you otherwise would have expected. It would be bad if you had to explain to your users, Hey you guys, I've enabled filesystem compression, so you should expect all your files to have different sizes and different checksums. The compression decompression happen at a low level - So all your applications don't know about it. If you want to see the difference in disk usage, I believe df and du will report different things... Or perhaps ls -l versus du ... ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On 07/10/12 12:45, Ferenc-Levente Juhos wrote: Of course you don't see any difference, this is how it should work. 'ls' will never report the compressed size, because it's not aware of it. Nothing is aware of the compression and decompression that takes place on-the-fly, except of course zfs. That's the reason why you could gain in write and read speed if you use compression, because the actual amount of compressed data that is being written and read from the pool is smaller than the original data. And I think with the checksum test you prooved that zfs checksums the uncompressed data. No ZFS checksums are over the data as it is stored on disk so the compressed data. -- Darren J Moffat ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On 2012-07-10 13:45, Ferenc-Levente Juhos wrote: Of course you don't see any difference, this is how it should work. 'ls' will never report the compressed size, because it's not aware of it. Nothing is aware of the compression and decompression that takes place on-the-fly, except of course zfs. Yep, my fault. You are right: root@sct-caszonesrv-07:/opt/test# du -hs u* 1.4Munix1 820Kunix2 Good article in here: https://blogs.oracle.com/observatory/entry/zfs_compression_a_win_win That's the reason why you could gain in write and read speed if you use compression, because the actual amount of compressed data that is being written and read from the pool is smaller than the original data. And I think with the checksum test you prooved that zfs checksums the uncompressed data. Thanks for the explanation. -- I will face my fear. I will permit it to pass over me and through me. And when it has gone past I will turn the inner eye to see its path. Where the fear has gone there will be nothing. Only I will remain. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
On Tue, Jul 10, 2012 at 6:29 AM, Jordi Espasa Clofent jespa...@minibofh.org wrote: Thanks for you explanation Fajar. However, take a look on the next lines: # available ZFS in the system root@sct-caszonesrv-07:~# zfs list NAME USED AVAIL REFER MOUNTPOINT opt 532M 34.7G 290M /opt opt/zones243M 34.7G32K /opt/zones opt/zones/sct-scw02-shared 243M 34.7G 243M /opt/zones/sct-scw02-shared static 104K 58.6G34K /var/www/ # creating a file in /root (UFS) root@sct-caszonesrv-07:~# dd if=/dev/zero of=file.bin count=1024 bs=1024 1024+0 records in 1024+0 records out 1048576 bytes (1.0 MB) copied, 0.0545957 s, 19.2 MB/s root@sct-caszonesrv-07:~# pwd /root # enable compression in some ZFS zone root@sct-caszonesrv-07:~# zfs set compression=on opt/zones/sct-scw02-shared # copying the previos file to this zone root@sct-caszonesrv-07:~# cp /root/file.bin /opt/zones/sct-scw02-shared/root/ # checking the file size in the origin dir (UFS) and the destination one (ZFS with compression enabled) root@sct-caszonesrv-07:~# ls -lh /root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:21 /root/file.bin root@sct-caszonesrv-07:~# ls -lh /opt/zones/sct-scw02-shared/root/file.bin -rw-r--r-- 1 root root 1.0M Jul 10 13:22 /opt/zones/sct-scw02-shared/root/file.bin # the both files has exactly the same cksum! root@sct-caszonesrv-07:~# cksum /root/file.bin 3018728591 1048576 /root/file.bin root@sct-caszonesrv-07:~# cksum /opt/zones/sct-scw02-shared/root/file.bin 3018728591 1048576 /opt/zones/sct-scw02-shared/root/file.bin So... I don't see any size variation with this test. ls(1) tells you how much data is in the file - that is, how many bytes of data that an application will see if it reads the whole file. du(1) tells you how many disk blocks are used. If you look at the stat structure in stat(2), ls reports st_size, du reports st_blocks. Blocks full of zeros are special to zfs compression - it recognizes them and stores no data. Thus, a file that contains only zeros will only require enough space to hold the file metadata. $ zfs list -o compression ./ COMPRESS on $ dd if=/dev/zero of=1gig count=1024 bs=1024k 1024+0 records in 1024+0 records out $ ls -l 1gig -rw-r--r-- 1 mgerdts staff1073741824 Jul 10 07:52 1gig $ du -k 1gig 0 1gig -- Mike Gerdts http://mgerdts.blogspot.com/ ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] S11 zfs snapshot crash
I am toying with Phil Brown's zrep script.
Does anyone have an Oracle BugID for this crashdump?
#!/bin/ksh
srcfs=rpool/testvol
destfs=rpool/destvol
snap=${srcfs}@zrep_00
zfs destroy -r $srcfs
zfs destroy -r $destfs
zfs create -V 100M $srcfs
zfs set foo:bar=foobar $srcfs
zfs create -o readonly=on $destfs
zfs snapshot $snap
zfs send -p $snap| zfs recv -vuF $destfs
# mdb unix.1 vmcore.1
$c
zap_leaf_lookup+0x4d(ff01dd4e1888, ff01e198d500, ff000858e130)
fzap_lookup+0x9a(ff01e198d500, 1, 4b, ff000858e430, 0, 0)
zap_lookup_norm+0x131(ff01dd48c9c0, 1, f7a99e30, 1, 4b,
ff000858e430)
zap_lookup+0x2d(ff01dd48c9c0, 1, f7a99e30, 1, 4b, ff000858e430)
zfs_get_mlslabel+0x56(ff01dd48c9c0, ff000858e430, 4b)
zfs_mount_label_policy+0x62(ff01dd68cb60, ff01ceede200)
zfs_mount+0x499(ff01dd68cb60, ff01e191b500, ff000858ee20,
ff01dbfb23b8)
fsop_mount+0x22(ff01dd68cb60, ff01e191b500, ff000858ee20,
ff01dbfb23b8)
domount+0xd33(0, ff000858ee20, ff01e191b500, ff01dbfb23b8,
ff000858ee18)
mount+0xc0(ff01ce5cf3b8, ff000858ee98)
syscall_ap+0x92()
_sys_sysenter_post_swapgs+0x149()
# pkg info entire| grep Summary
Summary: entire incorporation including Support Repository Update
(Oracle Solaris 11 11/11 SRU 8.5).
John
groenv...@acm.org
___
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
2012-07-10 15:49, Edward Ned Harvey wrote: If you use compression=on, or lzjb, then you're using very fast compression. Should not hurt performance, in fact, may gain performance for highly compressible data. If you use compression=gzip (or any gzip level 1 thru 9) then you're using a fairly expensive compression algorithm. It will almost certainly hurt performance, but you may gain more disk space. (Probably not.) Well, as far as the discussion relates to zones, the WORM (write once - read many) type of data, such as the OS image of the local zone, can suffer gzip-9 compression during installation of the zone and applications. This may make your files consume less disk sectors and further reads can be faster. Then you can enable lzjb on the same dataset, and further writes (of logs) would be compressed faster. In fact, you might want to delegate a dataset to the zone and create several filesystems in it, with different compression options, for your logs, application data and perhaps databases (which may be sensitive to IO block size and dislike external compression in favor of speeds). For these there is a zle compression which only compresses blocks filled with zeroes - that allows to save some space when your DB precreates a huge storage file but only uses a few kilobytes in it. I am not qualified to state whether gzip decompression might be slower during reads than lzjb or not, but remember that all this relies on general assumption that current CPUs are overqualified for their jobs and have lots of spare cycles - so (de)compression has little impact on real work anyway. Also decompression tends to be faster than compression, because there is little to no analysis to do - only matching compressed tags to a dictionary of original data snippets. HTH, //Jim Klimov ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] New fast hash algorithm - is it needed?
Hi guys, I'm contemplating implementing a new fast hash algorithm in Illumos' ZFS implementation to supplant the currently utilized sha256. On modern 64-bit CPUs SHA-256 is actually much slower than SHA-512 and indeed much slower than many of the SHA-3 candidates, so I went out and did some testing (details attached) on a possible new hash algorithm that might improve on this situation. However, before I start out on a pointless endeavor, I wanted to probe the field of ZFS users, especially those using dedup, on whether their workloads would benefit from a faster hash algorithm (and hence, lower CPU utilization). Developments of late have suggested to me three possible candidates: * SHA-512: simplest to implement (since the code is already in the kernel) and provides a modest performance boost of around 60%. * Skein-512: overall fastest of the SHA-3 finalists and much faster than SHA-512 (around 120-150% faster than the current sha256). * Edon-R-512: probably the fastest general purpose hash algorithm I've ever seen (upward of 300% speedup over sha256) , but might have potential security problems (though I don't think this is of any relevance to ZFS, as it doesn't use the hash for any kind of security purposes, but only for data integrity dedup). My testing procedure: nothing sophisticated, I took the implementation of sha256 from the Illumos kernel and simply ran it on a dedicated psrset (where possible with a whole CPU dedicated, even if only to a single thread) - I tested both the generic C implementation and the Intel assembly implementation. The Skein and Edon-R implementations are in C optimized for 64-bit architectures from the respective authors (the most up to date versions I could find). All code has been compiled using GCC 3.4.3 from the repos (the same that can be used for building Illumos). Sadly, I don't have access to Sun Studio. Cheers, -- Saso Hash preformances on 10 GB of data gcc (GCC) 3.4.3 (csl-sol210-3_4-20050802) CFLAGS: -O3 -fomit-frame-pointer -m64 MACHINE #1 CPU: dual AMD Opteron 4234 Options: single thread on no-intr whole-CPU psrset Algorithm Result Improvement sha256 (ASM)21.19 cycles/byte (baseline) sha256 (C) 27.66 cycles/byte -23.34% sha512 (ASM)13.48 cycles/byte 57.20% sha512 (C) 17.35 cycles/byte 22.13% Skein-512 (C) 8.95 cycles/byte136.76% Edon-R-512 (C) 4.94 cycles/byte328.94% MACHINE #2 CPU: single AMD Athlon II Neo N36L Options: single thread on no-intr 1-core psrset Algorithm Result Improvement sha256 (ASM)15.68 cycles/byte (baseline) sha256 (C) 18.81 cycles/byte -16.64% sha512 (ASM)9.95 cycles/byte57.59% sha512 (C) 11.84 cycles/byte 32.43% Skein-512 (C) 6.25 cycles/byte150.88% Edon-R-512 (C) 3.66 cycles/byte328.42% MACHINE #3 CPU: dual Intel Xeon E5645 Options: single thread on no-intr whole-CPU psrset Algorithm Result Improvement sha256 (ASM)15.49 cycles/byte (baseline) sha256 (C) 17.90 cycles/byte -13.46% sha512 (ASM)9.88 cycles/byte56.78% sha512 (C) 11.44 cycles/byte 35.40% Skein-512 (C) 6.88 cycles/byte125.15% Edon-R-512 (C) 3.35 cycles/byte362.39% MACHINE #4 CPU: single Intel Xeon E5405 Options: single thread on no-intr 1-core psrset Algorithm Result Improvement sha256 (ASM)17.45 cycles/byte (baseline) sha256 (C) 18.34 cycles/byte -4.85% sha512 (ASM)10.24 cycles/byte 70.41% sha512 (C) 11.72 cycles/byte 48.90% Skein-512 (C) 7.32 cycles/byte138.39% Edon-R-512 (C) 3.86 cycles/byte352.07% MACHINE #5 CPU: dual Intel Xeon E5450 Options: single thread on no-intr whole-CPU psrset Algorithm Result Improvement sha256 (ASM)16.43 cycles/byte (baseline) sha256 (C) 18.50 cycles/byte -11.19% sha512 (ASM)10.37 cycles/byte 58.44% sha512 (C) 11.85 cycles/byte 38.65% Skein-512 (C) 7.38 cycles/byte122.63% Edon-R-512 (C) 3.88 cycles/byte323.45% ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] Benefits of enabling compression in ZFS for the zones
To amplify what Mike says... On Jul 10, 2012, at 5:54 AM, Mike Gerdts wrote: ls(1) tells you how much data is in the file - that is, how many bytes of data that an application will see if it reads the whole file. du(1) tells you how many disk blocks are used. If you look at the stat structure in stat(2), ls reports st_size, du reports st_blocks. Blocks full of zeros are special to zfs compression - it recognizes them and stores no data. Thus, a file that contains only zeros will only require enough space to hold the file metadata. $ zfs list -o compression ./ COMPRESS on $ dd if=/dev/zero of=1gig count=1024 bs=1024k 1024+0 records in 1024+0 records out $ ls -l 1gig -rw-r--r-- 1 mgerdts staff1073741824 Jul 10 07:52 1gig ls -ls shows the length (as in -l) and size (as in -s, units=blocks) So you can see that it takes only space for metadata. 1 -rw-r--r-- 1 root root 1073741824 Nov 26 06:52 1gig size length -- richard -- ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422 ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] New fast hash algorithm - is it needed?
On 07/10/12 19:56, Sašo Kiselkov wrote: Hi guys, I'm contemplating implementing a new fast hash algorithm in Illumos' ZFS implementation to supplant the currently utilized sha256. On modern 64-bit CPUs SHA-256 is actually much slower than SHA-512 and indeed much slower than many of the SHA-3 candidates, so I went out and did some testing (details attached) on a possible new hash algorithm that might improve on this situation. Is the intent to store the 512 bit hash or truncate to 256 bit? ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] New fast hash algorithm - is it needed?
From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss- boun...@opensolaris.org] On Behalf Of Sašo Kiselkov I'm contemplating implementing a new fast hash algorithm in Illumos' ZFS implementation to supplant the currently utilized sha256. On modern 64-bit CPUs SHA-256 is actually much slower than SHA-512 and indeed much slower than many of the SHA-3 candidates, so I went out and did some testing (details attached) on a possible new hash algorithm that might improve on this situation. As coincidence would have it, I recently benchmarked md5 hashing and AES encryption on systems with and without AES-NI. Theoretically, hashing should be much faster because it's asymmetric, while symmetric encryption has much less speed potential. I found md5 could hash at most several hundred MB/sec, and AES was about half to quarter of that speed ... Which is consistent with the theory. But if I had AES-NI, then AES was about 1.1 GB/sec. Which means we have much *much* more speed potential available untapped in terms of hashing. Now, when you consider that a single disk typically is able to sustain 1.0Gbit (128 MB) per second, it means, very quickly the CPU can become the bottleneck for sustained disk reads in a large raid system. I think a lot of the time, people with a bunch of disks in a raid configuration are able to neglect the CPU load, just because they're using fletcher. Of the SHA3 finalists, I was pleased to see that only one of them was based on AES-NI, and the others are actually faster. So in vague hand-waving terms, yes I believe the stronger faster hash algorithm, in time will be a big win for zfs performance. But only in situations where people have a sufficiently large number of disks and sufficiently high expectation for IO performance. CPU's are not getting much faster. But IO is definitely getting faster. It's best to keep ahead of that curve. ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
