The branch, master has been updated
via 1e4c32e cli_credentials: Fix a typo
via c754f48 WHATSNEW: Add Using x86_64 Accelerated AES Crypto
Instructions section.
via 5f87a05 lib: crypto: Add the ability to select Intel AESNI
instruction set at configure time.
via 53ac0f7 lib: crypto: Plumb in the Intel AES instructions.
via 3324b55 lib: crypt: Prepare the existing code to switch to Intel
AES hardware instructions.
via 11a5676 third_party: Add build capability to aesni-intel.
via 421fa34 third_party: Add the Intel Add support for AES-NI
acceleration.
from 8c365c5 python: Enable execution of samba.tests.security with
Python 3.
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 1e4c32e3ff7e8dbf6aef12c737c5330273854510
Author: Volker Lendecke <[email protected]>
Date: Wed Sep 6 14:05:09 2017 +0200
cli_credentials: Fix a typo
Signed-off-by: Volker Lendecke <[email protected]>
Reviewed-by: Jeremy Allison <[email protected]>
Autobuild-User(master): Jeremy Allison <[email protected]>
Autobuild-Date(master): Thu Sep 7 05:56:14 CEST 2017 on sn-devel-144
commit c754f48575a5034242594762d62261a48025cebc
Author: Jeremy Allison <[email protected]>
Date: Wed Sep 6 12:13:32 2017 -0700
WHATSNEW: Add Using x86_64 Accelerated AES Crypto Instructions section.
Describes --accel-aes configure time option.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
commit 5f87a05aa314776135eec1057dd644c0b25a9d14
Author: Jeremy Allison <[email protected]>
Date: Wed Sep 6 11:59:44 2017 -0700
lib: crypto: Add the ability to select Intel AESNI instruction set at
configure time.
Add --accel-aes=[none|intelaesni] to select.
Default is none.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
commit 53ac0f7c59a9192e4fa88bf3d257ad80379ded23
Author: Jeremy Allison <[email protected]>
Date: Wed Sep 6 11:40:02 2017 -0700
lib: crypto: Plumb in the Intel AES instructions.
Causes:
AES_set_encrypt_key()
AES_set_decrypt_key()
AES_encrypt()
AES_decrypt()
to probe for the Intel AES instructions at runtime (only once)
and then call the hardware implementations if so, otherwise
fall back to the software implementations.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Based on original work by Justin Maggard <[email protected]>
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
commit 3324b55bde96d4098df2f1138be5fd243206ceba
Author: Jeremy Allison <[email protected]>
Date: Thu Aug 31 11:41:32 2017 -0700
lib: crypt: Prepare the existing code to switch to Intel AES hardware
instructions.
Rename the old struct aes_key as an intermediate struct aes_key_rj
and wrap it in a union so we can chose an alternate aes_key struct
when using Intel AES hardware.
Rename the original software implementations of:
AES_set_encrypt_key()
AES_set_decrypt_key()
AES_encrypt()
AES_decrypt()
by adding an _rj on the end, and call them via a wrapper
function.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Based on original work by Justin Maggard <[email protected]>
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
commit 11a56768950e1caae302507e15e58eb1c316c30a
Author: Jeremy Allison <[email protected]>
Date: Wed Sep 6 09:58:06 2017 -0700
third_party: Add build capability to aesni-intel.
Minor modifications to code to allow building as a Samba
shared library.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Based on original work by Justin Maggard <[email protected]>
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
commit 421fa346de575ebbed3496dc203011b80537bd8a
Author: Jeremy Allison <[email protected]>
Date: Wed Sep 6 09:44:42 2017 -0700
third_party: Add the Intel Add support for AES-NI acceleration.
This commit takes the Linux kernel AES-NI code, and puts it into a
third_party private library. The Linux kernel code is under GPLv2+
so is compatible with Samba.
This can result in massive speed improvements (up to 200% on some
platforms), by using Intel AES-NI instructions.
These are the pristine check-ins of Linux kernel files for Intel AESNI
crypto.
git show
8691ccd764f9ecc69a6812dfe76214c86ac9ba06:arch/x86/crypto/aesni-intel_asm.S
git show
2baad6121e2b2fa3428ee6cb2298107be11ab23a:arch/x86/include/asm/inst.h
Show the exact Linux kernel git refspecs we have imported.
These files are not yet used.
BUG: https://bugzilla.samba.org/show_bug.cgi?id=13008
Based on original work by Justin Maggard <[email protected]>
Signed-off-by: Jeremy Allison <[email protected]>
Reviewed-by: Stefan Metzmacher <[email protected]>
-----------------------------------------------------------------------
Summary of changes:
WHATSNEW.txt | 18 +
auth/credentials/credentials.c | 2 +-
lib/crypto/aes.c | 220 ++-
lib/crypto/aes.h | 11 +-
lib/crypto/aesni.h | 66 +
lib/crypto/wscript | 6 +
lib/crypto/wscript_build | 3 +
lib/crypto/wscript_configure | 13 +
third_party/aesni-intel/aesni-intel_asm.c | 2812 +++++++++++++++++++++++++++++
third_party/aesni-intel/inst-intel.h | 306 ++++
third_party/aesni-intel/wscript | 23 +
third_party/wscript | 2 +
wscript | 1 +
13 files changed, 3471 insertions(+), 12 deletions(-)
create mode 100644 lib/crypto/aesni.h
create mode 100644 lib/crypto/wscript
create mode 100644 third_party/aesni-intel/aesni-intel_asm.c
create mode 100644 third_party/aesni-intel/inst-intel.h
create mode 100644 third_party/aesni-intel/wscript
Changeset truncated at 500 lines:
diff --git a/WHATSNEW.txt b/WHATSNEW.txt
index 52cca7d..428c291 100644
--- a/WHATSNEW.txt
+++ b/WHATSNEW.txt
@@ -16,6 +16,24 @@ UPGRADING
NEW FEATURES/CHANGES
====================
+Using x86_64 Accelerated AES Crypto Instructions
+================================================
+
+Samba on x86_64 can now be configured to use the Intel accelerated AES
+instruction set, which has the potential to make SMB3 signing and
+encryption much faster on client and server. To enable this, configure
+Samba using the new option --accel-aes=intelaesni.
+
+This is a temporary solution that is being included to allow users
+to enjoy the benefits of Intel accelerated AES on the x86_64 platform,
+but the longer-term solution will be to move Samba to a fully supported
+external crypto library.
+
+The third_party/aesni-intel code will be removed from Samba as soon as
+external crypto library performance reaches parity.
+
+The default is to build without setting --accel-aes, which uses the
+existing Samba software AES implementation.
smb.conf changes
================
diff --git a/auth/credentials/credentials.c b/auth/credentials/credentials.c
index 152df96..1a4ec53 100644
--- a/auth/credentials/credentials.c
+++ b/auth/credentials/credentials.c
@@ -684,7 +684,7 @@ _PUBLIC_ const char *cli_credentials_get_realm(struct
cli_credentials *cred)
/**
* Set the realm for this credentials context, and force it to
- * uppercase for the sainity of our local kerberos libraries
+ * uppercase for the sanity of our local kerberos libraries
*/
_PUBLIC_ bool cli_credentials_set_realm(struct cli_credentials *cred,
const char *val,
diff --git a/lib/crypto/aes.c b/lib/crypto/aes.c
index 800a97e..c226ac1 100644
--- a/lib/crypto/aes.c
+++ b/lib/crypto/aes.c
@@ -37,35 +37,235 @@
#ifdef SAMBA_RIJNDAEL
#include "rijndael-alg-fst.h"
-int
-AES_set_encrypt_key(const unsigned char *userkey, const int bits, AES_KEY *key)
+#if defined(HAVE_AESNI_INTEL)
+
+/*
+ * NB. HAVE_AESNI_INTEL is only defined if -lang-asm is
+ * available.
+ */
+
+static inline void __cpuid(unsigned int where[4], unsigned int leaf)
+{
+ asm volatile("cpuid" :
+ "=a" (where[0]),
+ "=b" (where[1]),
+ "=c" (where[2]),
+ "=d" (where[3]): "a" (leaf));
+}
+
+/*
+ * has_intel_aes_instructions()
+ * return true if supports AES-NI and false if doesn't
+ */
+static bool has_intel_aes_instructions(void)
+{
+ static int has_aes_instructions = -1;
+ unsigned int cpuid_results[4];
+
+ if (has_aes_instructions != -1) {
+ return (bool)has_aes_instructions;
+ }
+
+ __cpuid(cpuid_results, 0);
+ /*
+ * MSB LSB
+ * EBX = 'u' 'n' 'e' 'G'
+ * EDX = 'I' 'e' 'n' 'i'
+ * ECX = 'l' 'e' 't' 'n'
+ */
+ if (memcmp((unsigned char *)&cpuid_results[1], "Genu", 4) != 0 ||
+ memcmp((unsigned char *)&cpuid_results[3],
+ "ineI", 4) != 0 ||
+ memcmp((unsigned char *)&cpuid_results[2],
+ "ntel", 4) != 0) {
+ has_aes_instructions = 0;
+ return (bool)has_aes_instructions;
+ }
+
+ __cpuid(cpuid_results, 1);
+ has_aes_instructions = !!(cpuid_results[2] & (1 << 25));
+ return (bool)has_aes_instructions;
+}
+
+/*
+ * Macro to ensure the AES key schedule starts on a 16 byte boundary.
+ */
+
+#define SET_ACC_CTX(k) \
+ do { \
+ (k)->u.aes_ni.acc_ctx = \
+ (struct crypto_aes_ctx *)(((unsigned
long)(k)->u.aes_ni._acc_ctx + 15) & ~0xfUL); \
+ } while (0)
+
+/*
+ * The next 4 functions call the Intel AES hardware implementations
+ * of:
+ *
+ * AES_set_encrypt_key()
+ * AES_set_decrypt_key()
+ * AES_encrypt()
+ * AES_decrypt()
+ */
+
+static int AES_set_encrypt_key_aesni(const unsigned char *userkey,
+ const int bits,
+ AES_KEY *key)
+{
+ SET_ACC_CTX(key);
+ return aesni_set_key(key->u.aes_ni.acc_ctx, userkey, bits/8);
+}
+
+static int AES_set_decrypt_key_aesni(const unsigned char *userkey,
+ const int bits,
+ AES_KEY *key)
+{
+ SET_ACC_CTX(key);
+ return aesni_set_key(key->u.aes_ni.acc_ctx, userkey, bits/8);
+}
+
+static void AES_encrypt_aesni(const unsigned char *in,
+ unsigned char *out,
+ const AES_KEY *key)
+{
+ aesni_enc(key->u.aes_ni.acc_ctx, out, in);
+}
+
+static void AES_decrypt_aesni(const unsigned char *in,
+ unsigned char *out,
+ const AES_KEY *key)
+{
+ aesni_dec(key->u.aes_ni.acc_ctx, out, in);
+}
+#else /* defined(HAVE_AESNI_INTEL) */
+
+/*
+ * Dummy implementations if no Intel AES instructions present.
+ * Only has_intel_aes_instructions() will ever be called.
+*/
+
+static bool has_intel_aes_instructions(void)
+{
+ return false;
+}
+
+static int AES_set_encrypt_key_aesni(const unsigned char *userkey,
+ const int bits,
+ AES_KEY *key)
+{
+ return -1;
+}
+
+static int AES_set_decrypt_key_aesni(const unsigned char *userkey,
+ const int bits,
+ AES_KEY *key)
{
- key->rounds = rijndaelKeySetupEnc(key->key, userkey, bits);
- if (key->rounds == 0)
+ return -1;
+}
+
+static void AES_encrypt_aesni(const unsigned char *in,
+ unsigned char *out,
+ const AES_KEY *key)
+{
+ abort();
+}
+
+static void AES_decrypt_aesni(const unsigned char *in,
+ unsigned char *out,
+ const AES_KEY *key)
+{
+ abort();
+}
+#endif /* defined(HAVE_AENI_INTEL) */
+
+/*
+ * The next 4 functions are the pure software implementations
+ * of:
+ *
+ * AES_set_encrypt_key()
+ * AES_set_decrypt_key()
+ * AES_encrypt()
+ * AES_decrypt()
+ */
+
+static int
+AES_set_encrypt_key_rj(const unsigned char *userkey, const int bits, AES_KEY
*key)
+{
+ key->u.aes_rj.rounds = rijndaelKeySetupEnc(key->u.aes_rj.key, userkey,
bits);
+ if (key->u.aes_rj.rounds == 0)
return -1;
return 0;
}
-int
-AES_set_decrypt_key(const unsigned char *userkey, const int bits, AES_KEY *key)
+static int
+AES_set_decrypt_key_rj(const unsigned char *userkey, const int bits, AES_KEY
*key)
{
- key->rounds = rijndaelKeySetupDec(key->key, userkey, bits);
- if (key->rounds == 0)
+ key->u.aes_rj.rounds = rijndaelKeySetupDec(key->u.aes_rj.key, userkey,
bits);
+ if (key->u.aes_rj.rounds == 0)
return -1;
return 0;
}
+static void
+AES_encrypt_rj(const unsigned char *in, unsigned char *out, const AES_KEY *key)
+{
+ rijndaelEncrypt(key->u.aes_rj.key, key->u.aes_rj.rounds, in, out);
+}
+
+static void
+AES_decrypt_rj(const unsigned char *in, unsigned char *out, const AES_KEY *key)
+{
+ rijndaelDecrypt(key->u.aes_rj.key, key->u.aes_rj.rounds, in, out);
+}
+
+/*
+ * The next 4 functions are the runtime switch for Intel AES hardware
+ * implementations of:
+ *
+ * AES_set_encrypt_key()
+ * AES_set_decrypt_key()
+ * AES_encrypt()
+ * AES_decrypt()
+ *
+ * If the hardware instructions don't exist, fall back to the software
+ * versions.
+ */
+
+int
+AES_set_encrypt_key(const unsigned char *userkey, const int bits, AES_KEY *key)
+{
+ if (has_intel_aes_instructions()) {
+ return AES_set_encrypt_key_aesni(userkey, bits, key);
+ }
+ return AES_set_encrypt_key_rj(userkey, bits, key);
+}
+
+int
+AES_set_decrypt_key(const unsigned char *userkey, const int bits, AES_KEY *key)
+{
+ if (has_intel_aes_instructions()) {
+ return AES_set_decrypt_key_aesni(userkey, bits, key);
+ }
+ return AES_set_decrypt_key_rj(userkey, bits, key);
+}
+
void
AES_encrypt(const unsigned char *in, unsigned char *out, const AES_KEY *key)
{
- rijndaelEncrypt(key->key, key->rounds, in, out);
+ if (has_intel_aes_instructions()) {
+ return AES_encrypt_aesni(in, out, key);
+ }
+ return AES_encrypt_rj(in, out, key);
}
void
AES_decrypt(const unsigned char *in, unsigned char *out, const AES_KEY *key)
{
- rijndaelDecrypt(key->key, key->rounds, in, out);
+ if (has_intel_aes_instructions()) {
+ return AES_decrypt_aesni(in, out, key);
+ }
+ return AES_decrypt_rj(in, out, key);
}
+
#endif /* SAMBA_RIJNDAEL */
#ifdef SAMBA_AES_CBC_ENCRYPT
diff --git a/lib/crypto/aes.h b/lib/crypto/aes.h
index 48ea764..00bfa3e 100644
--- a/lib/crypto/aes.h
+++ b/lib/crypto/aes.h
@@ -36,6 +36,8 @@
#ifndef LIB_CRYPTO_AES_H
#define LIB_CRYPTO_AES_H 1
+#include "aesni.h"
+
#define SAMBA_RIJNDAEL 1
#define SAMBA_AES_CBC_ENCRYPT 1
#define SAMBA_AES_CFB8_ENCRYPT 1
@@ -59,9 +61,16 @@
#define AES_ENCRYPT 1
#define AES_DECRYPT 0
-typedef struct aes_key {
+struct aes_key_rj {
uint32_t key[(AES_MAXNR+1)*4];
int rounds;
+};
+
+typedef struct aes_key {
+ union {
+ struct aes_key_rj aes_rj;
+ struct crypto_aesni_ctx aes_ni;
+ } u;
} AES_KEY;
#ifdef __cplusplus
diff --git a/lib/crypto/aesni.h b/lib/crypto/aesni.h
new file mode 100644
index 0000000..13d09d2
--- /dev/null
+++ b/lib/crypto/aesni.h
@@ -0,0 +1,66 @@
+/*
+ * Copyright (C) 2008, Intel Corp.
+ * Author: Huang Ying <[email protected]>
+ * Vinodh Gopal <[email protected]>
+ * Kahraman Akdemir
+ *
+ * Ported x86_64 version to x86:
+ * Author: Mathias Krause <[email protected]>
+ *
+ * Modified for use in Samba by Justin Maggard <[email protected]>
+ * and Jeremy Allison <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#ifndef LIB_CRYPTO_AESNI_H
+#define LIB_CRYPTO_AESNI_H 1
+
+#if defined(HAVE_AESNI_INTEL)
+
+#define AES_MAX_KEYLENGTH (15 * 16)
+#define AES_MAX_KEYLENGTH_U32 (AES_MAX_KEYLENGTH / sizeof(uint32_t))
+
+/*
+ * Please ensure that the first two fields are 16-byte aligned
+ * relative to the start of the structure, i.e., don't move them!
+ */
+struct crypto_aes_ctx {
+ uint32_t key_enc[AES_MAX_KEYLENGTH_U32];
+ uint32_t key_dec[AES_MAX_KEYLENGTH_U32];
+ uint32_t key_length;
+};
+
+struct crypto_aesni_ctx {
+ uint8_t _acc_ctx[sizeof(struct crypto_aes_ctx) + 16];
+ struct crypto_aes_ctx *acc_ctx;
+};
+
+/*
+ * These next 4 functions are actually implemented
+ * in the assembly language file:
+ * third_party/aesni-intel/aesni-intel_asm.c
+ */
+
+int aesni_set_key(struct crypto_aes_ctx *ctx,
+ const uint8_t *in_key,
+ unsigned int key_len);
+void aesni_enc(struct crypto_aes_ctx *ctx, uint8_t *dst, const uint8_t *src);
+void aesni_dec(struct crypto_aes_ctx *ctx, uint8_t *dst, const uint8_t *src);
+
+#else /* #if defined(HAVE_AESNI_INTEL) */
+
+/*
+ * We need a dummy definition of struct crypto_aesni_ctx to allow compiles.
+ */
+
+struct crypto_aesni_ctx {
+ int dummy;
+};
+
+#endif /* #if defined(HAVE_AESNI_INTEL) */
+
+#endif /* LIB_CRYPTO_AESNI_H */
diff --git a/lib/crypto/wscript b/lib/crypto/wscript
new file mode 100644
index 0000000..c3708c7
--- /dev/null
+++ b/lib/crypto/wscript
@@ -0,0 +1,6 @@
+#!/usr/bin/env python
+
+def set_options(opt):
+ opt.add_option('--accel-aes',
+ help=("Should we use accelerated AES crypto functions. Options are
intelaesni|none."),
+ action="store", dest='accel_aes', default="none")
diff --git a/lib/crypto/wscript_build b/lib/crypto/wscript_build
index d1f152e..f3257d8 100644
--- a/lib/crypto/wscript_build
+++ b/lib/crypto/wscript_build
@@ -11,6 +11,9 @@ elif bld.CONFIG_SET('HAVE_SYS_MD5_H') and
bld.CONFIG_SET('HAVE_LIBMD'):
elif not bld.CONFIG_SET('HAVE_SYS_MD5_H') and not
bld.CONFIG_SET('HAVE_COMMONCRYPTO_COMMONDIGEST_H'):
extra_source += ' md5.c'
+if bld.CONFIG_SET("HAVE_AESNI_INTEL"):
+ extra_deps += ' aesni-intel'
+
bld.SAMBA_SUBSYSTEM('LIBCRYPTO',
source='''crc32.c hmacmd5.c md4.c arcfour.c sha256.c sha512.c
hmacsha256.c
aes.c rijndael-alg-fst.c aes_cmac_128.c aes_ccm_128.c aes_gcm_128.c
diff --git a/lib/crypto/wscript_configure b/lib/crypto/wscript_configure
index 130acec..9fdaf9f 100644
--- a/lib/crypto/wscript_configure
+++ b/lib/crypto/wscript_configure
@@ -1,3 +1,7 @@
+#!/usr/bin/env python
+import Options
+import Utils
+
if not conf.CHECK_FUNCS_IN('MD5Init', 'bsd', headers='bsd/md5.h',
checklibc=True):
conf.CHECK_FUNCS_IN('MD5Init', 'md5', headers='sys/md5.h',
@@ -13,3 +17,12 @@ if conf.CHECK_FUNCS('SHA256_Update'):
conf.DEFINE('SHA256_RENAME_NEEDED', 1)
if conf.CHECK_FUNCS('SHA512_Update'):
conf.DEFINE('SHA512_RENAME_NEEDED', 1)
+
+#
+# --aes-accel=XXX selects accelerated AES crypto library to use, if any.
+# Default is none.
+#
+if Options.options.accel_aes.lower() == "intelaesni":
+ print("Attempting to compile with runtime-switchable x86_64 Intel AES
instructions. WARNING - this is temporary.")
+elif Options.options.accel_aes.lower() != "none":
+ raise Utils.WafError('--aes-accel=%s is not a valid option. Valid
options are [none|intelaesni]' % Options.options.accel_aes)
diff --git a/third_party/aesni-intel/aesni-intel_asm.c
b/third_party/aesni-intel/aesni-intel_asm.c
new file mode 100644
index 0000000..7baf703
--- /dev/null
+++ b/third_party/aesni-intel/aesni-intel_asm.c
@@ -0,0 +1,2812 @@
+/*
+ * Implement AES algorithm in Intel AES-NI instructions.
+ *
+ * The white paper of AES-NI instructions can be downloaded from:
+ *
http://softwarecommunity.intel.com/isn/downloads/intelavx/AES-Instructions-Set_WP.pdf
+ *
+ * Copyright (C) 2008, Intel Corp.
+ * Author: Huang Ying <[email protected]>
+ * Vinodh Gopal <[email protected]>
+ * Kahraman Akdemir
+ *
+ * Added RFC4106 AES-GCM support for 128-bit keys under the AEAD
+ * interface for 64-bit kernels.
+ * Authors: Erdinc Ozturk ([email protected])
+ * Aidan O'Mahony ([email protected])
+ * Adrian Hoban <[email protected]>
+ * James Guilford ([email protected])
+ * Gabriele Paoloni <[email protected]>
+ * Tadeusz Struk ([email protected])
+ * Wajdi Feghali ([email protected])
+ * Copyright (c) 2010, Intel Corporation.
+ *
+ * Ported x86_64 version to x86:
+ * Author: Mathias Krause <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ */
+
+#define ENTRY(name) \
+ .globl name ; \
+ .align 4,0x90 ; \
+ name:
+#define ENDPROC(name) \
+ .type name, @function ; \
+ .size name, .-name
+
+#define FRAME_BEGIN
+#define FRAME_END
+#define FRAME_OFFSET 0
+
+#include "inst-intel.h"
+
+/*
+ * The following macros are used to move an (un)aligned 16 byte value to/from
+ * an XMM register. This can done for either FP or integer values, for FP use
--
Samba Shared Repository
