On Nov 16, 2011, at 7:35 AM, David Dyer-Bennet wrote:
> On Tue, November 15, 2011 17:05, Anatoly wrote:
>> Good day,
>> The speed of send/recv is around 30-60 MBytes/s for initial send and
>> 17-25 MBytes/s for incremental. I have seen lots of setups with 1 disk
>> to 100+ disks in pool. But the speed doesn't vary in any degree. As I
>> understand 'zfs send' is a limiting factor. I did tests by sending to
>> /dev/null. It worked out too slow and absolutely not scalable.
>> None of cpu/memory/disk activity were in peak load, so there is of room
>> for improvement.
> What you're probably seeing with incremental sends is that the disks being
> read are hitting their IOPS limits. Zfs send does random reads all over
> the place -- every block that's changed since the last incremental send is
> read, in TXG order. So that's essentially random reads all of the disk.
Not necessarily. I've seen sustained zfs sends in the 600+ MB/sec range
for modest servers. It does depend on how the data is used more than the
hardware it is stored upon.
ZFS and performance consulting
LISA '11, Boston, MA, December 4-9
zfs-discuss mailing list