On 12/11/21, Piper H <pott...@gmail.com> wrote:
> I read this article from Reddit:
> https://www.phoronix.com/scan.php?page=article&item=bsd-linux-eo2021&num=1
>
> I am surprised to see that the BSD cluster today has much worse performance
> than Linux.
> What do you think of this?
>

There is a lot to say here.

One has to own up to Linux likely being a little bit (or even more so)
faster for some of the legitimate tests. One, there are certain
multicore scalability issues compared Linux, which should be pretty
mild given the scale (16 cores/32 threads). A more important problem
is userspace which fails to take advantage of SIMD instructions for
core primitives like memset, memcpy et al. However, if the difference
is more than few %, the result is likely bogus. Key thing to do when
benchmarking is being able to explain the result, most notably if you
run into huge discrepancies.

I had a look at the most egregious result -- zstd and spoiler, it is a
bug in result reporting in zstd.

I got FreeBSD and Linux (Ubuntu Focal) vms running on:
Intel(R) Xeon(R) Platinum 8275CL CPU @ 3.00GHz

Their zstd test ultimately ends up spawning: zstd -T24 -S -i15 -b19
FreeBSD-12.2-RELEASE-amd64-memstick.img (yes, they compress a ~1GB
FreeBSD image).

Side note, it does not matter, but I happen to have CURRENT kernel
running on the FreeBSD 13 vm right now.

[16:37] freebsd13:~ # time zstd -T24 -S -i15 -b19
FreeBSD-12.2-RELEASE-amd64-memstick.img
19#md64-memstick.img :1055957504 -> 692662162 (1.524),  3.97 MB/s ,2156.8 MB/s
zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
274.10s user 12.90s system 763% cpu 37.602 total

In contrast:

[16:37] ubuntu:...tem/compress-zstd-1.5.0 (130) # time zstd -T24 -S
-i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
19#md64-memstick.img :1055957504 -> 692662162 (1.524),  60.1 MB/s ,2030.6 MB/s
zstd -T24 -S -i15 -b19 FreeBSD-12.2-RELEASE-amd64-memstick.img
328.65s user 3.48s system 850% cpu 39.070 total

This is repeatable. If anything, FreeBSD did it *faster*. Yet zstd reports:
FreeBSD: 3.97 MB/s ,2156.8 MB/s [total time real time of  37.602 seconds]
Linux:  60.1 MB/s ,2030.6 MB/s [total time real time of 39.070 seconds]

I don't know what these numbers are supposed to be, but it is pretty
clear Phoronix grabs the first one.

I'll look into sorting this out some time later.

TL;DR don't drink and benchmark
-- 
Mateusz Guzik <mjguzik gmail.com>

Reply via email to