Torrey McMahon wrote:
Matthew Ahrens wrote:
I'm only doing an initial investigation now so I have no test data at
this point. The reason I asked, and I should have tacked this on at the
end of the last email, was a blog entry that stated zfs send was slow
http://www.lethargy.org/~jesus/archives/80-ZFS-sendHHHH-trickle..html
Looking back through the discuss archives I didn't see anything else
mentioned but some others mentioned it to me off line as well. It could
be we all read the same blog entry so I figured I'd ask if anyone had
seen such behavior recently. Hopefully, I can get a test bed setup
fairly quickly and see how it works myself.
I'll address some of the points in the above-mentioned blog:
The author notes that it took 9 days to do a full zfs send. Elsewhere
they note that they have "about 1TB of information on ZFS", so I'm left
to guess that their zfs send went at about 1.3MB/s. Without knowing
their underlying storage hardware, I couldn't say what a reasonable
expectation would be, but even a single modern spindle could do more
sequential reads. Random I/O is another matter, so the layout of the
data on disk would play in. Any other load on the system would also
impact the time that the 'zfs send' would be expected to take.
That said, I'd still guess that they are right -- 'zfs send' could be a
lot faster if it issued more i/o in parallel. Finding the right balance
of 'zfs send' performance vs. other i/o priority will be tricky, but
it's something we're going to work on.
Based on the time it took to do a full zfs send, the author says
"Somehow I think that doing daily incremental backups is out of the
question." However, the data does not support this conclusion. If the
amount of data changed is small (which the author claims: "very very
large files ... that have minimal changes to them"), then the
incremental zfs send will be quite fast.
While I agree that large improvements are possible, the data presented
does not support the conclusion that zfs send is not an acceptable
solution for daily incremental backups for this workload.
--matt
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
