Nobody can comment on this?
-Brian
Brian H. Nelson wrote:
> I noticed this issue yesterday when I first started playing around with
> zfs send/recv. This is on Solaris 10U6.
>
> It seems that a zfs send of a zvol issues 'volblocksize' reads to the
> physical devices. This doesn't make any sense to me, as zfs generally
> consolidates read/write requests to improve performance. Even the dd
> case with the same snapshot does not exhibit this behavior. It seems to
> be specific to zfs send.
>
> I checked with 8k, 64k, and 128k volblocksize, and the reads generated
> by zfs send always seem to follow that size, while the reads with dd do not.
>
> The small reads seems to hurt performance of zfs send. I tested with a
> mirror, but on another machine with a 7 disk raidz, the performance is
> MUCH worse because the 8k reads get broken up into even smaller reads
> and spread across the raidz.
>
> Is this a bug, or can someone explain why this is happening?
>
> Thanks
> -Brian
>
> Using 8k volblocksize:
>
> -bash-3.00# zfs send pool1/vo...@now > /dev/null
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 4.01G 274G 1.88K 0 15.0M 0
> mirror 4.01G 274G 1.88K 0 15.0M 0
> c0t9d0 - - 961 0 7.46M 0
> c0t11d0 - - 968 0 7.53M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~8k reads to pool and drives
>
> -bash-3.00# dd if=/dev/zvol/dsk/pool1/vo...@now of=/dev/null bs=8k
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 4.01G 274G 2.25K 0 17.9M 0
> mirror 4.01G 274G 2.25K 0 17.9M 0
> c0t9d0 - - 108 0 9.00M 0
> c0t11d0 - - 109 0 8.92M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~8k reads to pool, ~85k reads to drives
>
>
> Using volblocksize of 64k:
>
> -bash-3.00# zfs send pool1/vol...@now > /dev/null
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 6.01G 272G 378 0 23.5M 0
> mirror 6.01G 272G 378 0 23.5M 0
> c0t9d0 - - 189 0 11.8M 0
> c0t11d0 - - 189 0 11.7M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~64k reads to pool and drives
>
> -bash-3.00# dd if=/dev/zvol/dsk/pool1/vol...@now of=/dev/null bs=64k
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 6.01G 272G 414 0 25.7M 0
> mirror 6.01G 272G 414 0 25.7M 0
> c0t9d0 - - 107 0 12.9M 0
> c0t11d0 - - 106 0 12.8M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~64k reads to pool, ~124k reads to drives
>
>
> Using volblocksize of 128k:
>
> -bash-3.00# zfs send pool1/vol1...@now > /dev/null
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 4.01G 274G 188 0 23.3M 0
> mirror 4.01G 274G 188 0 23.3M 0
> c0t9d0 - - 94 0 11.7M 0
> c0t11d0 - - 93 0 11.7M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~128k reads to pool and drives
>
> -bash-3.00# dd if=/dev/zvol/dsk/pool1/vol1...@now of=/dev/null bs=128k
>
> capacity operations bandwidth
> pool used avail read write read write
> ----------- ----- ----- ----- ----- ----- -----
> pool1 4.01G 274G 247 0 30.8M 0
> mirror 4.01G 274G 247 0 30.8M 0
> c0t9d0 - - 122 0 15.3M 0
> c0t11d0 - - 123 0 15.5M 0
> ----------- ----- ----- ----- ----- ----- -----
> == ~128k reads to pool and drives
>
>
--
---------------------------------------------------
Brian H. Nelson Youngstown State University
System Administrator Media and Academic Computing
bnelson[at]cis.ysu.edu
---------------------------------------------------
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
