> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
> boun...@opensolaris.org] On Behalf Of Sašo Kiselkov
> I'm contemplating implementing a new fast hash algorithm in Illumos' ZFS
> implementation to supplant the currently utilized sha256. On modern
> 64-bit CPUs SHA-256 is actually much slower than SHA-512 and indeed much
> slower than many of the SHA-3 candidates, so I went out and did some
> testing (details attached) on a possible new hash algorithm that might
> improve on this situation.
As coincidence would have it, I recently benchmarked md5 hashing and AES
encryption on systems with and without AES-NI. Theoretically, hashing should
be much faster because it's asymmetric, while symmetric encryption has much
less speed potential. I found md5 could hash at most several hundred MB/sec,
and AES was about half to quarter of that speed ... Which is consistent with
the theory. But if I had AES-NI, then AES was about 1.1 GB/sec. Which means
we have much *much* more speed potential available untapped in terms of hashing.
Now, when you consider that a single disk typically is able to sustain 1.0Gbit
(128 MB) per second, it means, very quickly the CPU can become the bottleneck
for sustained disk reads in a large raid system. I think a lot of the time,
people with a bunch of disks in a raid configuration are able to neglect the
CPU load, just because they're using fletcher.
Of the SHA3 finalists, I was pleased to see that only one of them was based on
AES-NI, and the others are actually faster. So in vague hand-waving terms, yes
I believe the stronger & faster hash algorithm, in time will be a big win for
zfs performance. But only in situations where people have a sufficiently large
number of disks and sufficiently high expectation for IO performance.
CPU's are not getting much faster. But IO is definitely getting faster. It's
best to keep ahead of that curve.
zfs-discuss mailing list