Public bug reported:
Hi,
I got to this having a failing systemd testcase for my qemu upload but we
tracked that to a regression with the 4.10 upload IMHO.
TL;DR:
- xts-aes and cbc-aes fail to initialize on power (other arch fine) since
kernel 4.10
socket(AF_ALG, SOCK_SEQPACKET, 0) = 5
bind(5, {sa_family=AF_ALG, sa_data="skcipher\0\0\0\0\0\0"}, 88) = -1 ENOENT
(No such file or directory)
Details:
Failing "storage" subtest of systemd
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-zesty/zesty/ppc64el/s/systemd/20170306_005811_4a060@/log.gz
test_luks_by_devname (__main__.CryptsetupTest)
LUKS device by plain device name, empty ... device-mapper: reload ioctl on
temporary-cryptsetup-4601 failed: No such file or directory
Failed to setup dm-crypt key mapping for device /dev/sda.
Check that kernel supports aes-xts-plain64 cipher (check syslog for more info).
Failed to stop [email protected]: Unit
[email protected] not loaded.
FAIL
[...]
Studying the history of that test shows that it failed rather often with the
same error:
http://autopkgtest.ubuntu.com/packages/systemd/zesty/ppc64el
I discussed with pitti as systemd and autopkgtest expert:
[10:14] <cpaelzer> pitti: hi, I see in the autopkgtest history of systemd that
the "storage" test failed rather often on ppc64el
[10:15] <cpaelzer> pitti: I' currently ran into the same as well, since it
failed/worked so often without any visible difference - are these just "retried
until passing"
[10:15] <cpaelzer> pitti: or is there some background to learn?
[10:15] --> joedborg (~joedborg@2a02:c7f:9638:ab00:438:8be:af59:58e9) has
joined this channel.
[10:15] <cpaelzer> FYI - issues like in
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-zesty/zesty/ppc64el/s/systemd/20170306_005811_4a060@/log.gz
[10:16] <pitti> cpaelzer: indeed, "reload ioctl on temporary-cryptsetup-5254
failed" -- that ioctl seems to be unreliable in cryptsetup somehow
[10:16] <pitti> that should be easy enough to reproduce outside the systemd
test suite too, for a kernel bug report
[10:17] <pitti>
https://anonscm.debian.org/cgit/pkg-systemd/systemd.git/tree/debian/tests/storage#n74
is the cryptsetup luksFormat call
[10:17] <cpaelzer> pitti: ok, so in proposed retrying til pass and in a test
env trying to debug and understand
[10:17] <cpaelzer> pitti: thanks
[10:18] <cpaelzer> depending which of both is faster the debugging might
resolve the retrying :-)
[10:18] <pitti> the test uses scsi_debug as a drive, I suggest to try with both
that and a simple loop device to see whether that makes a difference
[10:18] <cpaelzer> thank you, I will do so
[10:19] <pitti> cpaelzer: y and x seem fine, so might be a regression of linux
4.11?
[10:20] <pitti> actually it started failing with trigger linux-meta/4.10.0.8.10
[10:20] <pitti>
https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-zesty/zesty/ppc64el/s/systemd/20170217_024850_73931@/log.gz
was the last successful one, with autopkgtest [01:48:28]: testbed running
kernel: Linux 4.9.0-15-generic #16-Ubuntu SMP Fri Jan 20 15:28:49 UTC 2017
[10:20] <cpaelzer> yeah I see that this was the "date" it started
[10:20] <pitti> i. e. 4.9 → 4.10
[10:20] <cpaelzer> former fails had other errors
So I went down debugging this:
1. Zesty guest on ppc64el (isolation-machine, so KVM)
2. repro as-is in this guest
pull-lp-source systemd
cd debian/tests
sudo ./storage (needs-roos, so sudo)
=> reproducable
3. debug
Dmesg shows related errors
[ 712.187600] Failed to allocate transformation for 'xts(aes)': -2
[ 712.249614] Failed to allocate transformation for 'xts(aes)': -2
[ 712.250818] device-mapper: table: 253:0: crypt: Error allocating crypto tfm
[ 712.251990] device-mapper: ioctl: error adding target to table
4. simplify repro
modprobe scsi_debug
# in my case /dev/sda
echo fookey | sudo cryptsetup --batch-mode luksFormat --key-file - /dev/sda
Here we can also confirm that it is not the devtype - e.g. also loop
devices fail.
I checked the kernel, but the config has set the algorithm:
$ cat /boot/config-4.10.0-9-generic | grep CRYPTO_XTS
CONFIG_CRYPTO_XTS=y
There are similar older bugs - but they are only "similar" both are fixed in
our kernel and cryptsetup versions:
kernel: https://bugzilla.kernel.org/show_bug.cgi?id=112631
cryptsetup: https://gitlab.com/cryptsetup/cryptsetup/issues/284
Fortunately cryptsetup can check all kind of algorithms easily:
$ cryptsetup benchmark
[...]
# Algorithm | Key | Encryption | Decryption
aes-cbc 128b N/A N/A
serpent-cbc 128b 58,3 MiB/s 68,2 MiB/s
twofish-cbc 128b 118,9 MiB/s 134,6 MiB/s
aes-cbc 256b N/A N/A
serpent-cbc 256b 58,8 MiB/s 68,2 MiB/s
twofish-cbc 256b 119,0 MiB/s 134,4 MiB/s
aes-xts 256b N/A N/A
serpent-xts 256b 62,0 MiB/s 68,1 MiB/s
twofish-xts 256b 127,8 MiB/s 132,6 MiB/s
aes-xts 512b N/A N/A
serpent-xts 512b 62,2 MiB/s 68,1 MiB/s
twofish-xts 512b 128,2 MiB/s 132,5 MiB/s
So aes-xts (the default) is not avail?
Really?
BTW - This at least helps to further simplify the test (no need for a device at
all)
$ cryptsetup benchmark --cipher aes-xts
But in /proc/crypto I can see:
name : xts(aes)
driver : p8_aes_xts
module : vmx_crypto
priority : 2000
refcnt : 1
selftest : passed
internal : no
type : blkcipher
blocksize : 16
min keysize : 32
max keysize : 64
ivsize : 16
geniv : <default>
The test is not failing on x86, check there:
=> Working just fine on x86
$ cat /proc/crypto (from x86)
[...]
name : xts(aes)
driver : xts(ecb(aes-asm))
module : kernel
priority : 200
refcnt : 1
selftest : passed
internal : no
type : skcipher
async : no
blocksize : 16
min keysize : 32
max keysize : 64
ivsize : 16
chunksize : 16
On the power system the listed module "vmx_crypto" is loaded.
The related output in /proc/crypto did not change since xenial.
Stracing to confirm it is not the old accept before bind issue:
Call (xenial):
socket(PF_ALG, SOCK_SEQPACKET, 0) = 5
bind(5, {sa_family=AF_ALG, sa_data="skcipher\0\0\0\0\0\0"}, 88) = 0
Bad (Zesty):
socket(AF_ALG, SOCK_SEQPACKET, 0) = 5
bind(5, {sa_family=AF_ALG, sa_data="skcipher\0\0\0\0\0\0"}, 88) = -1 ENOENT (No
such file or directory)
The old kernel bug was about "bind" needed to be (setting key types) before
accept.
The new crypsetup does that, so it is NOT the old issue (as we could assume
since it was fixed in 4.4 and various stable updates).
Yet it is very close - now the bind itself fails setting the xts-aes type (and
cbc-aes fails too btw).
=> kernel update regression from the move to 4.10
ProblemType: Bug
DistroRelease: Ubuntu 17.04
Package: linux-image-4.10.0-9-generic 4.10.0-9.11
ProcVersionSignature: User Name 4.10.0-9.11-generic 4.10.0
Uname: Linux 4.10.0-9-generic ppc64le
.var.log.platform: Error: [Errno 13] Permission denied: '/var/log/platform'
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Mär 6 09:26 seq
crw-rw---- 1 root audio 116, 33 Mär 6 09:26 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.4-0ubuntu2
Architecture: ppc64el
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq',
'/dev/snd/timer'] failed with exit code 1:
Date: Mon Mar 6 10:25:00 2017
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
Bus 001 Device 003: ID 0627:0001 Adomax Technology Co., Ltd
Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
PciMultimedia:
ProcFB: 0 OFfb vga
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinux-4.10.0-9-generic
root=LABEL=cloudimg-rootfs earlyprintk
ProcLoadAvg: 0.00 0.00 0.00 1/129 24971
ProcLocks:
1: POSIX ADVISORY WRITE 1431 00:13:16508 0 EOF
2: FLOCK ADVISORY WRITE 1423 00:13:16392 0 EOF
3: POSIX ADVISORY WRITE 1422 00:13:16194 0 EOF
4: POSIX ADVISORY WRITE 1387 00:13:15603 0 EOF
5: POSIX ADVISORY WRITE 398 00:13:11036 0 EOF
ProcSwaps: Filename Type Size Used
Priority
ProcVersion: Linux version 4.10.0-9-generic (buildd@bos01-ppc64el-025) (gcc
version 6.3.0 20161229 (User Name 6.3.0-2ubuntu1) ) #11-User Name SMP Mon Feb
20 13:45:11 UTC 2017
RelatedPackageVersions:
linux-restricted-modules-4.10.0-9-generic N/A
linux-backports-modules-4.10.0-9-generic N/A
linux-firmware 1.163
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
cpu_cores: Number of cores present = 1
cpu_coreson: Number of cores online = 1
cpu_smt: Error: command ['ppc64_cpu', '--smt'] failed with exit code 255:
Machine is not SMT capable
** Affects: linux (Ubuntu)
Importance: Undecided
Status: New
** Tags: apport-bug ppc64el uec-images zesty
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1670311
Title:
aes-xts and aes-cbc failing to initialize on power since 4.10
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1670311/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
