[Bug 1891473] Re: cryptsetup ftbfs in focal
** Tags added: fr-236 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
This bug was fixed in the package cryptsetup - 2:2.3.3-1ubuntu5 --- cryptsetup (2:2.3.3-1ubuntu5) groovy; urgency=medium * SECURITY UPDATE: Out-of-bounds write - debian/patches/CVE-2020-14382-*.patch: check segment gaps regardless of heap space in lib/luks2/luks2_json_metadata.c. - CVE-2020-14382 * debian/patches/decrease_memlock_ulimit.patch Fixed FTBFS due a restrict environment in the new Bionic Builder (LP: #1891473) tests/luks2-validation.test, tests/compat-test, tests/tcrypt-compat-test. - Thanks Guilherme G. Piccoli. -- leo.barb...@canonical.com (Leonidas S. Barbosa) Wed, 09 Sep 2020 09:29:17 -0300 ** Changed in: cryptsetup (Ubuntu Groovy) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Changed in: cryptsetup (Ubuntu Groovy) Status: In Progress => Fix Committed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
This bug was fixed in the package cryptsetup - 2:2.2.2-3ubuntu2.2 --- cryptsetup (2:2.2.2-3ubuntu2.2) focal-security; urgency=medium * SECURITY UPDATE: Out-of-bounds write - debian/patches/CVE-2020-14382-*.patch: check segment gaps regardless of heap space in lib/luks2/luks2_json_metadata.c. - CVE-2020-14382 * debian/patches/decrease_memlock_ulimit.patch Fixed FTBFS due a restrict environment in the new Bionic Builder (LP: #1891473) tests/luks2-validation.test, tests/compat-test, tests/tcrypt-compat-test. - Thanks Guilherme G. Piccoli. -- leo.barb...@canonical.com (Leonidas S. Barbosa) Thu, 10 Sep 2020 08:47:50 -0300 ** Changed in: cryptsetup (Ubuntu Focal) Status: In Progress => Fix Released ** CVE added: https://cve.mitre.org/cgi-bin/cvename.cgi?name=2020-14382 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
We also face problems in ppc64el, even more tests fail - in general PowerPC is a more memory-consuming arch, especially given the default page size in such arch is 64K. Three tests fail there due to low host mlock limit in Bionic; I've written a patch to fix that in the package, in case somebody (like security) needs to release cryptsetup urgently due to a bug. I'll attach it here. Also, the consensus on the ubuntu-devel thread on how to fix properly the PPA builder issues was to bump systemd memlock limit on Bionic, making it on-par with more recent releases. This is being addressed in LP #1830746 (https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1830746). Cheers, Guilherme -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Attachment added: "Decrease-memlock-ulimit" https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+attachment/5409874/+files/Decrease-memlock-ulimit -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
Email sent to Ubuntu-devel ML seeking opinions on the approach we should take to fix this issue: https://lists.ubuntu.com/archives/ubuntu- devel/2020-September/041159.html . -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
After some investigation, seems we were able to narrow down the issue.
* tl;dr: After the upgrade of the PPA builder to Bionic, the memlock limit
(ulimit -l) was bumped from a ridiculous low value (64) to something bigger
(16M). Happens that cryptsetup then succeeded in its call to mlockall(), so all
allocations got restricted by such limit, which is still a bit low and it ends
up leading to allocation failures.
When the limit is very low (like in Xenial), the lock procedure fails, and
cryptsetup allocations are not subject to this restriction, so everything just
works.
See section "Conclusion" for alternatives on how to fix this
* Details:
I manage to reproduce that by collecting the luks2-validation images in a local
environment, running a Bionic VM + LXD (a Focal container). By collecting the
strace of luksDump in both environments, we got the following:
### LXD - NOT working
...
openat(AT_FDCWD, "./luks2-metadata-size-4m.img", O_RDONLY|O_DIRECT) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=16777216, ...}) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=16777216, ...}) = 0
lseek(5, 0, SEEK_SET) = 0
read(5, "LUKS\272\276\0\2\0\0\0\0\0@\0\0\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0\0"...,
4096) = 4096
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 EAGAIN (Resource temporarily unavailable)
brk(0x55e789c38000) = 0x55e78982d000
mmap(NULL, 4325376, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
-1 EAGAIN (Resource temporarily unavailable)
lseek(5, 16384, SEEK_SET) = 16384
read(5, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
4096) = 4096
lseek(5, 32768, SEEK_SET) = 32768
read(5, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
4096) = 4096
lseek(5, 65536, SEEK_SET) = 65536
read(5, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
4096) = 4096
...
### VM - working
...
openat(AT_FDCWD, "./luks2-metadata-size-4m.img", O_RDONLY|O_DIRECT) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=16777216, ...}) = 0
fstat(6, {st_mode=S_IFREG|0644, st_size=16777216, ...}) = 0
lseek(5, 0, SEEK_SET) = 0
read(5, "LUKS\272\276\0\2\0\0\0\0\0@\0\0\0\0\0\0\0\0\0\n\0\0\0\0\0\0\0\0"...,
4096) = 4096
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f6b06031000
lseek(5, 4096, SEEK_SET)= 4096
mmap(NULL, 4198400, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) =
0x7f6b05c3
So: as mmap fails, lseeks start to be attempted with wrong sizes, 2K^N,
where N=4,5,...
In cryptsetup code: on luks2_disk_metadata.c, function
LUKS2_disk_hdr_read(), we fail and try all known offsets, as per the
below code:
[...]
* No header size, check all known offsets.
*/
for (r = -EINVAL,i = 0; r < 0 && i < ARRAY_SIZE(hdr2_offsets);
i++)
[...]
This explains why we see that many lseeks in the LXD failing case, with
multiple offsets.
But then, why we fail? In the failing case, on funtion
LUKS2_disk_hdr_read(), we fail right in the first header read, as per
code in lib/luks2/luks2_disk_metadata.c:
[...]
* Read primary LUKS2 header (offset 0).
*/
state_hdr1 = HDR_FAIL;
r = hdr_read_disk(cd, device, _disk1, _area1, 0, 0);
[...]
The failure comes in a malloc(), specifically in hdr_read_disk():
[...]
r = hdr_disk_sanity_check_pre(cd, hdr_disk, _json_size, secondary,
offset);
if (r < 0) {
return r;
}
/*
* Allocate and read JSON area. Always the whole area must be read.
*/
*json_area = malloc(hdr_json_size);
if (!*json_area) {
return -ENOMEM;
}
[...]
Without the json_area allocated we end-up looping, in search of the
proper header size, and failing the test. This malloc is the one
generating the following entry on strace:
mmap(NULL, 4194304, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
0) = -1 EAGAIN (Resource temporarily unavailable)
* Conclusion: we have 2 avenues for fixing that, I personally consider (a)
[below] the more correct one.
(a) We could increase the builders memlock limit to 64M - Focal has that
as a default now. This seems to me the proper approach, given that in
real life cryptsetup is performing the memory lock, so we should
exercise it like that during the build tests.
(b) It's possible to fallback to the same scenario of Xenial builder by
_reducing_ the memlock limit and having cryptsetup not setting the
memory lock at all during the build. The bonus of this approach is its
simplicity - we can decrease such limit from the package itself, but at
the same time, we don't exercise the real life usage anymore during the
build tests.
By following the approach (b) above, I've managed to make the build
work: https://launchpad.net/~gpiccoli/+archive/ubuntu/crypt-
groovy/+build/19913720
I'll
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Changed in: cryptsetup (Ubuntu Focal) Status: Confirmed => In Progress ** Changed in: cryptsetup (Ubuntu Groovy) Status: Confirmed => In Progress ** Changed in: cryptsetup (Ubuntu Focal) Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli) ** Changed in: cryptsetup (Ubuntu Groovy) Assignee: (unassigned) => Guilherme G. Piccoli (gpiccoli) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
I've noticed the problem happens in both Focal and Groovy builds. It's related with the restricted environment (read restricted as "containerized" environment), the failure happens in the "cryptsetup luksDump" against the "luks2-metadata-size-4m.img" image during the metadata validation test. I'm testing in a Bionic VM with Focal LXD container; I was able to dump such image in the local build, and when try to run the lunksDump thing it fails inside the container but works in a Focal native system, with the apt-installed cryptsetup package. Investigation continues. Cheers, Guilherme -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Also affects: cryptsetup (Ubuntu Groovy) Importance: High Status: Confirmed ** Changed in: cryptsetup (Ubuntu Focal) Status: New => Confirmed ** Changed in: cryptsetup (Ubuntu Focal) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Tags added: id-5f358712d1ea131fd27e581f -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
This started after PPA builders got upgraded to Bionic, this worked before. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Tags removed: rls-ff-incoming -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Also affects: cryptsetup (Ubuntu Focal) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1891473] Re: cryptsetup ftbfs in focal
** Changed in: cryptsetup (Ubuntu) Status: New => Confirmed ** Changed in: cryptsetup (Ubuntu) Importance: Undecided => High -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1891473 Title: cryptsetup ftbfs in focal To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cryptsetup/+bug/1891473/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
