Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64
On Thu, 26 Mar 2020 at 02:04, Zurcher, Christopher J wrote: > > > -Original Message- > > From: devel@edk2.groups.io On Behalf Of Ard > > Biesheuvel > > Sent: Wednesday, March 25, 2020 11:40 > > To: Zurcher, Christopher J > > Cc: edk2-devel-groups-io ; Wang, Jian J > > ; Lu, XiaoyuX ; Eugene Cohen > > > > Subject: Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native > > instruction support for IA32 and X64 > > > > On Tue, 17 Mar 2020 at 11:26, Christopher J Zurcher > > wrote: > > > > > > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2507 > > > > > > This patch adds support for building the native instruction algorithms for > > > IA32 and X64 versions of OpensslLib. The process_files.pl script was > > modified > > > to parse the .asm file targets from the OpenSSL build config data struct, > > and > > > generate the necessary assembly files for the EDK2 build environment. > > > > > > For the X64 variant, OpenSSL includes calls to a Windows error handling > > API, > > > and that function has been stubbed out in ApiHooks.c. > > > > > > For all variants, a constructor was added to call the required CPUID > > function > > > within OpenSSL to facilitate processor capability checks in the native > > > algorithms. > > > > > > Additional native architecture variants should be simple to add by > > following > > > the changes made for these two architectures. > > > > > > The OpenSSL assembly files are traditionally generated at build time using > > a > > > perl script. To avoid that burden on EDK2 users, these end-result assembly > > > files are generated during the configuration steps performed by the > > > package > > > maintainer (through process_files.pl). The perl generator scripts inside > > > OpenSSL do not parse file comments as they are only meant to create > > > intermediate build files, so process_files.pl contains additional hooks to > > > preserve the copyright headers as well as clean up tabs and line endings > > > to > > > comply with EDK2 coding standards. The resulting file headers align with > > > the generated .h files which are already included in the EDK2 repository. > > > > > > Cc: Jian J Wang > > > Cc: Xiaoyu Lu > > > Cc: Eugene Cohen > > > Cc: Ard Biesheuvel > > > > > > Christopher J Zurcher (1): > > > CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64 > > > > > > > Hello Christopher, > > > > It would be helpful to understand the purpose of all this. Also, I > > think we could be more picky about which algorithms we enable - DES > > and MD5 don't seem highly useful, and even if they were, what do we > > gain by using a faster (or smaller?) implementation? > > > > The selection of algorithms comes from the default OpenSSL assembly targets; > this combination is validated and widely used, and I don't know all the > consequences of picking and choosing which ones to include. If necessary I > could look into reducing the list. > > The primary driver for this change is enabling the Full Flash Update (FFU) OS > provisioning flow for Windows as described here: > https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/wim-vs-ffu-image-file-formats > This item under "Reliability" is what we are speeding up: "Includes a catalog > and hash table to validate a signature upfront before flashing onto a device. > The hash table is generated during capture, and validated when applying the > image." > This provisioning flow can be performed within the UEFI environment, and the > native algorithms allow significant time savings in a factory setting > (minutes per device). > > We also had a BZ which requested these changes and the specific need was not > provided. Maybe David @HP can provide further insight? > https://bugzilla.tianocore.org/show_bug.cgi?id=2507 > > There have been additional platform-specific benefits identified, for example > speeding up HMAC authentication of communication with other HW/FW components. > OK, so in summary, there is one particular hash that you need to speed up, and this is why you enable every single asm implementation in OpenSSL for X86. I'm not sure I am convinced that this is justified. OpenSSL can easily deal with having just a couple of these accelerated implementations enabled. To me, it seems like a more sensible approach to only enable algorithms based on special instructions (such as
Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64
> -Original Message- > From: devel@edk2.groups.io On Behalf Of Ard Biesheuvel > Sent: Wednesday, March 25, 2020 11:40 > To: Zurcher, Christopher J > Cc: edk2-devel-groups-io ; Wang, Jian J > ; Lu, XiaoyuX ; Eugene Cohen > > Subject: Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native > instruction support for IA32 and X64 > > On Tue, 17 Mar 2020 at 11:26, Christopher J Zurcher > wrote: > > > > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2507 > > > > This patch adds support for building the native instruction algorithms for > > IA32 and X64 versions of OpensslLib. The process_files.pl script was > modified > > to parse the .asm file targets from the OpenSSL build config data struct, > and > > generate the necessary assembly files for the EDK2 build environment. > > > > For the X64 variant, OpenSSL includes calls to a Windows error handling > API, > > and that function has been stubbed out in ApiHooks.c. > > > > For all variants, a constructor was added to call the required CPUID > function > > within OpenSSL to facilitate processor capability checks in the native > > algorithms. > > > > Additional native architecture variants should be simple to add by > following > > the changes made for these two architectures. > > > > The OpenSSL assembly files are traditionally generated at build time using > a > > perl script. To avoid that burden on EDK2 users, these end-result assembly > > files are generated during the configuration steps performed by the package > > maintainer (through process_files.pl). The perl generator scripts inside > > OpenSSL do not parse file comments as they are only meant to create > > intermediate build files, so process_files.pl contains additional hooks to > > preserve the copyright headers as well as clean up tabs and line endings to > > comply with EDK2 coding standards. The resulting file headers align with > > the generated .h files which are already included in the EDK2 repository. > > > > Cc: Jian J Wang > > Cc: Xiaoyu Lu > > Cc: Eugene Cohen > > > > Cc: Ard Biesheuvel > > > > Christopher J Zurcher (1): > > CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64 > > > > Hello Christopher, > > It would be helpful to understand the purpose of all this. Also, I > think we could be more picky about which algorithms we enable - DES > and MD5 don't seem highly useful, and even if they were, what do we > gain by using a faster (or smaller?) implementation? > The selection of algorithms comes from the default OpenSSL assembly targets; this combination is validated and widely used, and I don't know all the consequences of picking and choosing which ones to include. If necessary I could look into reducing the list. The primary driver for this change is enabling the Full Flash Update (FFU) OS provisioning flow for Windows as described here: https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/wim-vs-ffu-image-file-formats This item under "Reliability" is what we are speeding up: "Includes a catalog and hash table to validate a signature upfront before flashing onto a device. The hash table is generated during capture, and validated when applying the image." This provisioning flow can be performed within the UEFI environment, and the native algorithms allow significant time savings in a factory setting (minutes per device). We also had a BZ which requested these changes and the specific need was not provided. Maybe David @HP can provide further insight? https://bugzilla.tianocore.org/show_bug.cgi?id=2507 There have been additional platform-specific benefits identified, for example speeding up HMAC authentication of communication with other HW/FW components. Thanks, Christopher Zurcher > > > > > CryptoPkg/Library/OpensslLib/OpensslLib.inf | > 2 +- > > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf| > 2 +- > > CryptoPkg/Library/OpensslLib/OpensslLibIa32.inf | > 680 ++ > > CryptoPkg/Library/OpensslLib/OpensslLibX64.inf | > 691 ++ > > CryptoPkg/Library/Include/openssl/opensslconf.h | > 3 - > > CryptoPkg/Library/OpensslLib/ApiHooks.c | > 18 + > > CryptoPkg/Library/OpensslLib/OpensslLibConstructor.c | > 34 + > > CryptoPkg/Library/OpensslLib/Ia32/crypto/aes/aesni-x86.nasm | > 3209 > > CryptoPkg/Library/Ope
Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64
On Tue, 17 Mar 2020 at 11:26, Christopher J Zurcher wrote: > > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2507 > > This patch adds support for building the native instruction algorithms for > IA32 and X64 versions of OpensslLib. The process_files.pl script was modified > to parse the .asm file targets from the OpenSSL build config data struct, and > generate the necessary assembly files for the EDK2 build environment. > > For the X64 variant, OpenSSL includes calls to a Windows error handling API, > and that function has been stubbed out in ApiHooks.c. > > For all variants, a constructor was added to call the required CPUID function > within OpenSSL to facilitate processor capability checks in the native > algorithms. > > Additional native architecture variants should be simple to add by following > the changes made for these two architectures. > > The OpenSSL assembly files are traditionally generated at build time using a > perl script. To avoid that burden on EDK2 users, these end-result assembly > files are generated during the configuration steps performed by the package > maintainer (through process_files.pl). The perl generator scripts inside > OpenSSL do not parse file comments as they are only meant to create > intermediate build files, so process_files.pl contains additional hooks to > preserve the copyright headers as well as clean up tabs and line endings to > comply with EDK2 coding standards. The resulting file headers align with > the generated .h files which are already included in the EDK2 repository. > > Cc: Jian J Wang > Cc: Xiaoyu Lu > Cc: Eugene Cohen > Cc: Ard Biesheuvel > > Christopher J Zurcher (1): > CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64 > Hello Christopher, It would be helpful to understand the purpose of all this. Also, I think we could be more picky about which algorithms we enable - DES and MD5 don't seem highly useful, and even if they were, what do we gain by using a faster (or smaller?) implementation? > CryptoPkg/Library/OpensslLib/OpensslLib.inf |2 > +- > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf|2 > +- > CryptoPkg/Library/OpensslLib/OpensslLibIa32.inf | 680 > ++ > CryptoPkg/Library/OpensslLib/OpensslLibX64.inf | 691 > ++ > CryptoPkg/Library/Include/openssl/opensslconf.h |3 - > CryptoPkg/Library/OpensslLib/ApiHooks.c | 18 + > CryptoPkg/Library/OpensslLib/OpensslLibConstructor.c | 34 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/aes/aesni-x86.nasm | 3209 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/aes/vpaes-x86.nasm | 648 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/bn-586.nasm | 1522 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/co-586.nasm | 1259 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/x86-gf2m.nasm| 352 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/x86-mont.nasm| 486 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/des/crypt586.nasm | 887 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/des/des-586.nasm| 1835 > + > CryptoPkg/Library/OpensslLib/Ia32/crypto/md5/md5-586.nasm| 690 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/modes/ghash-x86.nasm| 1264 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/rc4/rc4-586.nasm| 381 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha1-586.nasm | 3977 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha256-586.nasm | 6796 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha512-586.nasm | 2842 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/x86cpuid.nasm | 513 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-mb-x86_64.nasm | 1772 > + > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-sha1-x86_64.nasm | 3271 > > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-sha256-x86_64.nasm | 4709 > +++ > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-x86_64.nasm| 5084 > > CryptoPkg/Library/OpensslLib/X64/crypto/aes/vpaes-x86_64.nasm| 1170 > +++ > CryptoPkg/Library/OpensslLib/X64/crypto/bn/rsaz-avx2.nasm| 1989 > + > CryptoPkg/Library/OpensslLib/X64/crypto/bn/rsaz-x86_64.nasm | 2242 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-gf2m.nasm | 432 + > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-mont.nasm | 1479 > > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-mont5.nasm | 4033 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/md5/md5-x86_64.nasm | 794 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/modes/aesni-gcm-x86_64.nasm | 984 > +++ > CryptoPkg/Library/Ope
Re: [edk2-devel] [PATCH 0/1] CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64
On 03/17/20 11:26, Zurcher, Christopher J wrote: > BZ: https://bugzilla.tianocore.org/show_bug.cgi?id=2507 > > This patch adds support for building the native instruction algorithms for > IA32 and X64 versions of OpensslLib. The process_files.pl script was modified > to parse the .asm file targets from the OpenSSL build config data struct, and > generate the necessary assembly files for the EDK2 build environment. > > For the X64 variant, OpenSSL includes calls to a Windows error handling API, > and that function has been stubbed out in ApiHooks.c. > > For all variants, a constructor was added to call the required CPUID function > within OpenSSL to facilitate processor capability checks in the native > algorithms. > > Additional native architecture variants should be simple to add by following > the changes made for these two architectures. > > The OpenSSL assembly files are traditionally generated at build time using a > perl script. To avoid that burden on EDK2 users, these end-result assembly > files are generated during the configuration steps performed by the package > maintainer (through process_files.pl). The perl generator scripts inside > OpenSSL do not parse file comments as they are only meant to create > intermediate build files, so process_files.pl contains additional hooks to > preserve the copyright headers as well as clean up tabs and line endings to > comply with EDK2 coding standards. The resulting file headers align with > the generated .h files which are already included in the EDK2 repository. > > Cc: Jian J Wang > Cc: Xiaoyu Lu > Cc: Eugene Cohen > Cc: Ard Biesheuvel > > Christopher J Zurcher (1): > CryptoPkg/OpensslLib: Add native instruction support for IA32 and X64 > > CryptoPkg/Library/OpensslLib/OpensslLib.inf |2 > +- > CryptoPkg/Library/OpensslLib/OpensslLibCrypto.inf|2 > +- > CryptoPkg/Library/OpensslLib/OpensslLibIa32.inf | 680 > ++ > CryptoPkg/Library/OpensslLib/OpensslLibX64.inf | 691 > ++ > CryptoPkg/Library/Include/openssl/opensslconf.h |3 - > CryptoPkg/Library/OpensslLib/ApiHooks.c | 18 + > CryptoPkg/Library/OpensslLib/OpensslLibConstructor.c | 34 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/aes/aesni-x86.nasm | 3209 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/aes/vpaes-x86.nasm | 648 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/bn-586.nasm | 1522 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/co-586.nasm | 1259 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/x86-gf2m.nasm| 352 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/bn/x86-mont.nasm| 486 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/des/crypt586.nasm | 887 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/des/des-586.nasm| 1835 > + > CryptoPkg/Library/OpensslLib/Ia32/crypto/md5/md5-586.nasm| 690 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/modes/ghash-x86.nasm| 1264 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/rc4/rc4-586.nasm| 381 + > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha1-586.nasm | 3977 > ++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha256-586.nasm | 6796 > > CryptoPkg/Library/OpensslLib/Ia32/crypto/sha/sha512-586.nasm | 2842 > +++ > CryptoPkg/Library/OpensslLib/Ia32/crypto/x86cpuid.nasm | 513 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-mb-x86_64.nasm | 1772 > + > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-sha1-x86_64.nasm | 3271 > > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-sha256-x86_64.nasm | 4709 > +++ > CryptoPkg/Library/OpensslLib/X64/crypto/aes/aesni-x86_64.nasm| 5084 > > CryptoPkg/Library/OpensslLib/X64/crypto/aes/vpaes-x86_64.nasm| 1170 > +++ > CryptoPkg/Library/OpensslLib/X64/crypto/bn/rsaz-avx2.nasm| 1989 > + > CryptoPkg/Library/OpensslLib/X64/crypto/bn/rsaz-x86_64.nasm | 2242 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-gf2m.nasm | 432 + > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-mont.nasm | 1479 > > CryptoPkg/Library/OpensslLib/X64/crypto/bn/x86_64-mont5.nasm | 4033 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/md5/md5-x86_64.nasm | 794 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/modes/aesni-gcm-x86_64.nasm | 984 > +++ > CryptoPkg/Library/OpensslLib/X64/crypto/modes/ghash-x86_64.nasm | 2077 > + > CryptoPkg/Library/OpensslLib/X64/crypto/rc4/rc4-md5-x86_64.nasm | 1395 > > CryptoPkg/Library/OpensslLib/X64/crypto/rc4/rc4-x86_64.nasm | 784 > ++ > CryptoPkg/Library/OpensslLib/X64/crypto/sha/kec
