Re: [zfs-discuss] server hang with compression on, ping timeouts from remote machine
Thanks Bill, that looks relevant. Note however this only happens with gzip compression, but it's definiteness something I've experienced. I've decided to wait for the next full release before upgrading. I was just wondering if the problem was resolved. I'll migrate to COMSTAR soon, I hope the kernel mode iscsi will make a difference. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] server hang with compression on, ping timeouts from remote machine
On 01/31/10 07:07, Christo Kutrovsky wrote: I've also experienced similar behavior (short freezes) when running zfs send|zfs receive with compression on LOCALLY on ZVOLs again. Has anyone else experienced this ? Know any of bug? This is on snv117. you might also get better results after the fix to: 6881015 ZFS write activity prevents other threads from running in a timely manner which was fixed in build 129. As a workaround, try a lower gzip compression level -- higher gzip levels usually burn lots more CPU without significantly increasing the compression ratio. - Bill ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] server hang with compression on, ping timeouts from remote machine
Thanks for your replies. I am aware of the 512 bytes concept, thus my selection of 8 KB (matched with 8KB ntfs). Even 20% reduction is still good, that's like having 20% extra ram (for cache). I haven't experimented with the default lzjb compression. If I want to compress something usually I want it compressed well. Originally I had tried 64 Kb, but then I discovered windows does partial reads and writes (not entire clusters), thus I decided to pick 8K something that fits in 9k jumbo frame. Either way, I think it's very bad for an OS compression to cause your server to not respond to pings (other side affects aside). I am running 117, thus the fix should be in place. Nevertheless it does point out that there could be other things wrong with gzip compression and zfs. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] server hang with compression on, ping timeouts from remote machine
On Jan 31, 2010, at 7:21 AM, Henrik Johansson wrote: > Hello Christo, > > On Jan 31, 2010, at 4:07 PM, Christo Kutrovsky wrote: > >> Hello All, >> >> I am running NTFS over iSCSI on a ZFS ZVOL volume with compression=gzip-9 >> and blocksize=8K. The server is 2 core P4 3.0 Ghz with 5 GB of RAM. For NTFS, use recordsize=4 KB, but I wouldn't worry too much about compression at that recordsize. >> Whenever I start copying files from Windows onto the ZFS disk, after about >> 100-200 Mb been copied the server starts to experience freezes. I have >> iostat running, which freezes as well. Even pings on both of the network >> adapters are reporting either 4000 ms or timeouts for when the freeze is >> happening. >> >> I have reproduce the same behavior with a 1 GB test ZVOL. Whenever I do >> sequential writes of 64 Kb with compression=gzip-9 I experience the freezes. >> With compression=off it's all good. gzip-9 is a pig. b115 includes the fix for: CR6586537 async zio taskqs can block out userland commands which greatly reduced this effect. But you might consider the default gzip-6 instead. Back to my note above, compression is done to records, but the size is still 512 byte sectors. In other words, there are 8 sectors in a 4 KB record, so compression is bounded by 12.5% chunks. >> I've also experienced similar behavior (short freezes) when running zfs >> send|zfs receive with compression on LOCALLY on ZVOLs again. > > I think gzip in ZFS have a reputation being somewhat heavy on system > resources, that said it would be nice if it did not have such a large impact > on low level functions. Have a look in the archive, search for example > death-spriral or Death-spriral revisited. Have you tried using the default > compression algorithm also (lzjb, compresison=on)? Good idea, and for small records compression gains less. -- richard ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Re: [zfs-discuss] server hang with compression on, ping timeouts from remote machine
Hello Christo, On Jan 31, 2010, at 4:07 PM, Christo Kutrovsky wrote: > Hello All, > > I am running NTFS over iSCSI on a ZFS ZVOL volume with compression=gzip-9 and > blocksize=8K. The server is 2 core P4 3.0 Ghz with 5 GB of RAM. > > Whenever I start copying files from Windows onto the ZFS disk, after about > 100-200 Mb been copied the server starts to experience freezes. I have iostat > running, which freezes as well. Even pings on both of the network adapters > are reporting either 4000 ms or timeouts for when the freeze is happening. > > I have reproduce the same behavior with a 1 GB test ZVOL. Whenever I do > sequential writes of 64 Kb with compression=gzip-9 I experience the freezes. > With compression=off it's all good. > > I've also experienced similar behavior (short freezes) when running zfs > send|zfs receive with compression on LOCALLY on ZVOLs again. I think gzip in ZFS have a reputation being somewhat heavy on system resources, that said it would be nice if it did not have such a large impact on low level functions. Have a look in the archive, search for example death-spriral or Death-spriral revisited. Have you tried using the default compression algorithm also (lzjb, compresison=on)? Regards Henrik http://sparcv9.blogspot.com ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
[zfs-discuss] server hang with compression on, ping timeouts from remote machine
Hello All, I am running NTFS over iSCSI on a ZFS ZVOL volume with compression=gzip-9 and blocksize=8K. The server is 2 core P4 3.0 Ghz with 5 GB of RAM. Whenever I start copying files from Windows onto the ZFS disk, after about 100-200 Mb been copied the server starts to experience freezes. I have iostat running, which freezes as well. Even pings on both of the network adapters are reporting either 4000 ms or timeouts for when the freeze is happening. I have reproduce the same behavior with a 1 GB test ZVOL. Whenever I do sequential writes of 64 Kb with compression=gzip-9 I experience the freezes. With compression=off it's all good. I've also experienced similar behavior (short freezes) when running zfs send|zfs receive with compression on LOCALLY on ZVOLs again. Has anyone else experienced this ? Know any of bug? This is on snv117. -- This message posted from opensolaris.org ___ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
