On Tue, Jan 07, 2020 at 09:33:46AM -0600, Edgar Pettijohn wrote:
> In reality, when you dig down, often you find that there's another
> reason for the issue.?? I was recently trying to substitute libressl
> into an openssl environment.?? Performance tanked.?? Some checking
> showed the speed of "speed -evp aes-256-gcm" was way off.?? It looked
> to me like it was an issue with not using AES-NI.?? I'm not going to
> blame libressl for that, I just lacked the time to do a deep dive on
> it to figure out what was (hopefully!) configured wrong.?? Probably
> something with ia32cap or whatever the libressl equivalent is.
>
> ... JG
I believe it has something to do with actually zeroing out memory
before freeing it. Which seems like a good thing to do for crypto
stuff.
My apologies. I posted an insufficient description of the issue as it
was intended as an argument refuting the OP. If we want to discuss my
issue, that's fine and I welcome the input. I normally manage to
resolve these things eventually but this stumped me a bit.
This appears to be an i386-specific issue and it is perhaps a 5:1
performance difference.
Compiled on a FreeBSD 12.1R-amd64 VM, I see exactly what I would hope
to see:
--Begin-FreeBSD-12.1R-amd64--------------------------------------------
# uname -r; uname -m
12.1-RELEASE
amd64
# libressl-3.0.2/apps/openssl/openssl speed -evp aes-256-gcm
WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf
Doing aes-256-gcm for 3s on 16 size blocks: 42776805 aes-256-gcm's in
3.00s
Doing aes-256-gcm for 3s on 64 size blocks: 28274190 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 9382555 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 2636912 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 334132 aes-256-gcm's in
3.01s
LibreSSL 3.0.2
built on: date not available
options:bn(64,64) rc4(16x,int) des(idx,cisc,16,int) aes(partial)
idea(int) blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192
bytes
aes-256-gcm 228204.73k 601456.74k 798353.68k 897432.60k
909765.10k
# openssl speed -evp aes-256-gcm
Doing aes-256-gcm for 3s on 16 size blocks: 40297566 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 64 size blocks: 27287454 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 10106391 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 2858781 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 368695 aes-256-gcm's in
3.00s
Doing aes-256-gcm for 3s on 16384 size blocks: 184909 aes-256-gcm's in
3.01s
OpenSSL 1.1.1d-freebsd 10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) idea(int)
blowfish(ptr)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes
8192 bytes 16384 bytes
aes-256-gcm 214362.12k 580620.32k 860172.00k 973262.71k
1006783.15k 1007226.70k
--End-FreeBSD-12.1R-amd64--------------------------------------------
Okay, so that looks fantastic to me. Now running it on i386 on
a VM "right next door" on the same hypervisor.
--Begin-FreeBSD-12.1R-i386--------------------------------------------
# uname -r; uname -m
12.1-RELEASE
i386
# libressl-3.0.2/apps/openssl/openssl speed -evp aes-256-gcm
WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf
Doing aes-256-gcm for 3s on 16 size blocks: 8904897 aes-256-gcm's in
3.00s
Doing aes-256-gcm for 3s on 64 size blocks: 2387064 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 603284 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 153381 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 19041 aes-256-gcm's in
3.01s
LibreSSL 3.0.2
built on: date not available
options:bn(64,32) rc4(ptr,int) des(idx,cisc,16,long) aes(partial)
idea(int) blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192
bytes
aes-256-gcm 47427.78k 50805.69k 51347.19k 52207.47k
51858.50k
# openssl speed -evp aes-256-gcm
Doing aes-256-gcm for 3s on 16 size blocks: 32056370 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 64 size blocks: 21569563 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 8523369 aes-256-gcm's in
3.00s
Doing aes-256-gcm for 3s on 1024 size blocks: 2528081 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 334502 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 16384 size blocks: 167762 aes-256-gcm's in
3.02s
OpenSSL 1.1.1d-freebsd 10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,32) rc4(8x,mmx) des(long) aes(partial) idea(int)
blowfish(ptr)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes
8192 bytes 16384 bytes
aes-256-gcm 170523.24k 458955.48k 727327.49k 860676.97k
911040.96k 911457.03k
# OPENSSL_ia32cap="+0x200000200000000" openssl speed -evp aes-256-gcm
Doing aes-256-gcm for 3s on 16 size blocks: 8813067 aes-256-gcm's in
2.99s
Doing aes-256-gcm for 3s on 64 size blocks: 2483967 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 630526 aes-256-gcm's in
3.02s
Doing aes-256-gcm for 3s on 1024 size blocks: 160181 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 20112 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 16384 size blocks: 9988 aes-256-gcm's in
3.01s
OpenSSL 1.1.1d-freebsd 10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,32) rc4(4x,int) des(long) aes(partial) idea(int)
blowfish(ptr)
compiler: clang
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes
8192 bytes 16384 bytes
aes-256-gcm 47125.75k 52853.66k 53526.10k 54533.10k
54776.52k 54406.11k
# OPENSSL_ia32cap="~0x200000200000000"
libressl-3.0.2/apps/openssl/openssl spee -evp aes-256-gcm
d
WARNING: can't open config file: /usr/local/etc/ssl/openssl.cnf
Doing aes-256-gcm for 3s on 16 size blocks: 9048488 aes-256-gcm's in
2.99s
Doing aes-256-gcm for 3s on 64 size blocks: 2416620 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 256 size blocks: 583915 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 1024 size blocks: 151550 aes-256-gcm's in
3.01s
Doing aes-256-gcm for 3s on 8192 size blocks: 19287 aes-256-gcm's in
3.00s
LibreSSL 3.0.2
built on: date not available
options:bn(64,32) rc4(ptr,int) des(idx,cisc,16,long) aes(partial)
idea(int) blowfish(idx)
compiler: information not available
The 'numbers' are in 1000s of bytes per second processed.
type 16 bytes 64 bytes 256 bytes 1024 bytes 8192
bytes
aes-256-gcm 48365.46k 51406.29k 49682.09k 51580.05k
52638.66k
--End-FreeBSD-12.1R-i386--------------------------------------------
So the first run, with LibreSSL, the performance has fallen off
catastrophically.
The next run, calling the host system's shipped OpenSSL, the numbers
return to reasonable, although they are somewhat slower than the amd64
runs. That's fine.
Now in the third run, calling the host system's OpenSSL but twiddling
ia32cap, I get numbers that are very similar to the LibreSSL numbers
showing a similar catastrophic performance reduction. My conclusion
is that this is somehow an AES-NI detection issue. For whatever
reason,
FreeBSD's openssl gets it right by default.
And the fourth run was "just to see."
