Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On 2018-12-20 Daniel Kahn Gillmor wrote: [...] > On Wed 2018-12-19 11:59:46 -0500, Antoine Beaupré wrote: >> On 2018-12-18 14:34:06, Emilio Pozuelo Monfort wrote: >>> libgcrypt is a bit more worrying, even after dropping most of the noise: >>> $ diff libgcrypt20-1.*/ | filterdiff -x '*.pc/*' -x '*/debian/*' -x >>> '*/tests/*' >>> | diffstat | tail -1 >>> 263 files changed, 51927 insertions(+), 14888 deletions(-) >> Yeah, that's my concern as well. >> Daniel, what do you think of that diff? Is that something we can >> reasonably review? How much can we expect stability in that upgrade? >> I know you stated before general principles of gpg vs lib / API >> stability, but I'd be curious to hear your thoughts on gcrypt, in this >> specific case. > I agree that an upgrade to gcrypt is the biggest risk here, and i'm not > sure how to evaluate it other than running what meager rdep test suites > we have in jessie. I don't know whether anyone who has been working on > ci.debian.net is following this discussion, but i think it points to > some really salient use cases for test infrastructure. How nice it > would be if a DD could upload a prospective package and say "please run > all test suites for reverse dependencies!" > Andreas Metzler (cc'ed here) has been a stalwart steward of gcrypt in > debian for many years, even after GnuTLS switched to nettle, and > probably has the best sense of what kind of system integration dangers > might lurk in the proposed upgrade for jessie. Perhaps he can comment > on it? [...] Hello, looking at my mail archive gcrypt updates since 1.6 (i.e. since the last soname bump) have been very painless. The only breakage in rdeps I found was #816104, going from 1.6 to 1.7. http://thread.gmane.org/gmane.comp.encryption.gpg.libgcrypt.devel/4487 cu Andreas -- `What a good friend you are to him, Dr. Maturin. His other friends are so grateful to you.' `I sew his ears on from time to time, sure'
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
First off, thanks to Antoine not only for doing all this work for jessie, but for helping out with getting stretch in better shape. If we aim to support our users for an LTS distro, this is exactly the sort of thing we need done. If we're realistically talking about actually dropping support for enigmail on jessie on the grounds that not many people are using it for destkop systems at all, then i think we should instead consider dropping thunderbird itself from jessie. (alternately, of course, we could just drop jessie entirely and encourage an upgrade to debian stable instead, but that doesn't seem to be the consensus on this debian-lts list) On Wed 2018-12-19 11:59:46 -0500, Antoine Beaupré wrote: > On 2018-12-18 14:34:06, Emilio Pozuelo Monfort wrote: >> libgcrypt is a bit more worrying, even after dropping most of the noise: >> >> $ diff libgcrypt20-1.*/ | filterdiff -x '*.pc/*' -x '*/debian/*' -x >> '*/tests/*' >> | diffstat | tail -1 >> 263 files changed, 51927 insertions(+), 14888 deletions(-) > > Yeah, that's my concern as well. > > Daniel, what do you think of that diff? Is that something we can > reasonably review? How much can we expect stability in that upgrade? > > I know you stated before general principles of gpg vs lib / API > stability, but I'd be curious to hear your thoughts on gcrypt, in this > specific case. I agree that an upgrade to gcrypt is the biggest risk here, and i'm not sure how to evaluate it other than running what meager rdep test suites we have in jessie. I don't know whether anyone who has been working on ci.debian.net is following this discussion, but i think it points to some really salient use cases for test infrastructure. How nice it would be if a DD could upload a prospective package and say "please run all test suites for reverse dependencies!" Andreas Metzler (cc'ed here) has been a stalwart steward of gcrypt in debian for many years, even after GnuTLS switched to nettle, and probably has the best sense of what kind of system integration dangers might lurk in the proposed upgrade for jessie. Perhaps he can comment on it? as rdeps go, systemd is the scariest of the lot (breaking systemd with a dep upgrade would be bad bad bad) but frankly, i'm not worried there. systemd does link against gcrypt, it is used primarily for cryptographic digests (hashes) in any significant way across the codebase, which i'm not worried about breaking across an upgrade. It is only used in any complex way in systemd in two places, afaict: * systemd-journald (its forward-secure pseudorandom generator, and its authenticator function), and * systemd-resolved's DNSSEC verification. These are both pretty advanced systemd features (i don't know how well either of them have ever been tested in jessie at all, fwiw) and i have a hard time imagining that anyone stuck on jessie is actually using either of them. Regards, --dkg signature.asc Description: PGP signature
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On 2018-12-18 14:34:06, Emilio Pozuelo Monfort wrote: [...] > Looking at a jessie -> jessie-new diff, I see that several -dbg packages are > gone in your backports. Yes. That's because they were switched to dbgsym in stretch, but that mecanism wasn't supported in jessie. I did a "fast" backport which meant just dropping those, but I could possibly re-create the -dbg packages for jessie-security, especially considering we might trigger bugs and regressions which could really use dbg/gdb support. > There are some mingw builds as well, which in some cases > don't seemto be installed, but e.g. libgpg-error actually adds a mingw > package. > I would remove all that stuff. Hmm... I *thought* I explicitely removed all that stuff, but i'll make sure to remove that one as well, thanks for the catch! > The npth diff is pretty trivial, basically comes down to this: > > src/npth.c | 132 Neat, I'll explicitely review that one then. > libassuan is a bit larger, but not too bad: > > $ diff libassuan-2.*/src/ | diffstat | tail -1 > 26 files changed, 1492 insertions(+), 510 deletions(-) > > (some of that is Makefile.in) Probably worth reviewing as well... > libgpg-error has some autogenerated stuff, ignoring that it's mostly this: > > estream.c| 1456 > +-- ... and same. > libgcrypt is a bit more worrying, even after dropping most of the noise: > > $ diff libgcrypt20-1.*/ | filterdiff -x '*.pc/*' -x '*/debian/*' -x > '*/tests/*' > | diffstat | tail -1 > 263 files changed, 51927 insertions(+), 14888 deletions(-) Yeah, that's my concern as well. Daniel, what do you think of that diff? Is that something we can reasonably review? How much can we expect stability in that upgrade? I know you stated before general principles of gpg vs lib / API stability, but I'd be curious to hear your thoughts on gcrypt, in this specific case. > FWIW I see that Ubuntu added OpenPGP.js back, and is using gnupg 2.0.x in > trusty. We ruled that out because supporting gnupg 2.0.x is unfeasible or > because we are missing some dependencies for OpenGPG.js ? Can't we just use > the > bundled code inside enigmail? Sorry if these questions have already been > answered. I have looked at the various long threads but wasn't sure. Yeah, I went down that rabbit hole... three months ago now! I documented my work in bug #787774. It's a complicated set of nesting dependencies, and many packages would first need to cross NEW in unstable (let alone stretch / jessie) before this lands in Debian. A summary of the situation is here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=787774#49 I made a wiki page, back then, detailing all those dependencies. I am just re-running the script again to get an accurate picture: https://wiki.debian.org/Javascript/Nodejs/Tasks/openpgp That's all stuff in unstable, mind you. All that would need to trickle down in jessie somehow, and that includes npm/nodejs, which I am not sure are in good health in jessie in the first place. A. -- During times of universal deceit, telling the truth becomes a revolutionary act. - Georges Orwell
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
Hi Antoine, dkg, On Sat, Dec 15, 2018 at 01:09:39PM +0100, Moritz Mühlenhoff wrote: > On Fri, Dec 14, 2018 at 09:08:42AM +0100, Emilio Pozuelo Monfort wrote: > > However given the impact of these library updates, I was wondering > > if we have considered to just mark enigmail as EOL in jessie? Obviously if > > we > > can keep supporting stuff we should do that, but as you say these library > > updates affect important unrelated rdeps so we need to weigh that. > +1 I've read this thread multiple times now and came to conclude that Moritz and Emilio are probably right here, also because - afaics - noone besides you two have tested the packages on https://people.debian.org/~anarcat/debian/jessie-lts/ and also mostly concerning whether they fix enigmail, not so much for subtile breakage in other parts. (Or did I miss something?) Then I also looked at the packages LTS customers (=sponsors) are using and noted neither enigmail nor libgcrypt20 are among those, so while I agree breaking/not fixing/declaring EOL of enigmail will be sad for our jessie users, I also think that most web users are using modern desktops now (though of course those still on jessie are bitten) and that keeping jessie stable should have highest priority. Of course, more tests could probably convince me again in the other direction.. ;) -- cheers, Holger --- holger@(debian|reproducible-builds|layer-acht).org PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C signature.asc Description: PGP signature
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On Tue 2018-12-18 14:34:06 +0100, Emilio Pozuelo Monfort wrote: > FWIW I see that Ubuntu added OpenPGP.js back, and is using gnupg 2.0.x > in trusty. sounds fairly dubious to me, see below: > We ruled that out because supporting gnupg 2.0.x is unfeasible or GnuPG 2.0.x is unsupported upstream, has been entirely EOL for a couple weeks short of a year now. Enigmail claims to work with it (package/gpg.jsm claims MINIMUM_GPG_VERSION = 2.0.14), but i don't recommend trying to use something that far outside of GnuPG upstream's attention. > because we are missing some dependencies for OpenGPG.js ? I tried getting OpenPGP.js packaged for debian properly, and failed. Perhaps someone with more node/npm knowledge and/or stomach for the task could succeed: https://bugs.debian.org/787774 I would welcome it if someone could pick up this work -- we really should have more implementations of OpenPGP in debian. But i'm not convinced that it's the answer for jessie, given the ongoing struggles around npm/gitlab/node in stretch-backports itself. > Can't we just use the bundled code inside enigmail? If you want to use the bundled code inside enigmail, you should be aware that enigmail upstream is not even building the bundle -- they're just copying it raw from whatever OpenPGP.js is shipping in their git repository (apparently in npm-land it's common practice to commit your generated output files to revision control). I've asked upstream whether they'd ever built OpenPGP.js from source, and the last answer i got was that they'd tried it, but had failed, and it was more straightforward just to copy in the bundle. This doesn't sound like a DFSG-compliant situation to me, but i'd be open to listening to an argument for it. Regardless of DFSG-compliance, i'm particularly concerned about responsible maintenance a pre-generated blob, particularly one that sits close to protected material like encrypted messages. All the best, --dkg signature.asc Description: PGP signature
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On 14/12/2018 09:08, Emilio Pozuelo Monfort wrote: > On 13/12/2018 21:14, Antoine Beaupré wrote: >> Hi, >> >> This is the latest update in the Thunderbird / Enigmail changes that are >> happening in jessie. I have built a series of test packages, partly from >> stretch (gnupg2, enigmail) and partly from backports (libassuan, >> libgcrypt, libgpg-error, npth) and uploaded them here: >> >> https://people.debian.org/~anarcat/debian/jessie-lts/ >> >> I need people to test those packages, and not just enigmail users. Some >> of those packages have pernicious and deep ramifications. I am >> particularly worried about libgcrypt, which is used for example by >> cryptsetup. > > I see that your tests have gone well so far (except for enigmail itself for > unrelated reasons as you explain). This is great work, and I don't mean to > push > back on it. However given the impact of these library updates, I was wondering > if we have considered to just mark enigmail as EOL in jessie? Obviously if we > can keep supporting stuff we should do that, but as you say these library > updates affect important unrelated rdeps so we need to weigh that. > > BTW I have briefly looked at the versions you have backported, and I wonder > why > npth and libgpg-error have deb8u3 rather than deb8u1? > > I haven't looked at your changes yet, but I will find some time to look at > them > and give these packages a try. Looking at a jessie -> jessie-new diff, I see that several -dbg packages are gone in your backports. There are some mingw builds as well, which in some cases don't seemto be installed, but e.g. libgpg-error actually adds a mingw package. I would remove all that stuff. The npth diff is pretty trivial, basically comes down to this: src/npth.c | 132 libassuan is a bit larger, but not too bad: $ diff libassuan-2.*/src/ | diffstat | tail -1 26 files changed, 1492 insertions(+), 510 deletions(-) (some of that is Makefile.in) libgpg-error has some autogenerated stuff, ignoring that it's mostly this: estream.c| 1456 +-- libgcrypt is a bit more worrying, even after dropping most of the noise: $ diff libgcrypt20-1.*/ | filterdiff -x '*.pc/*' -x '*/debian/*' -x '*/tests/*' | diffstat | tail -1 263 files changed, 51927 insertions(+), 14888 deletions(-) FWIW I see that Ubuntu added OpenPGP.js back, and is using gnupg 2.0.x in trusty. We ruled that out because supporting gnupg 2.0.x is unfeasible or because we are missing some dependencies for OpenGPG.js ? Can't we just use the bundled code inside enigmail? Sorry if these questions have already been answered. I have looked at the various long threads but wasn't sure. Cheers, Emilio
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On Fri, Dec 14, 2018 at 09:08:42AM +0100, Emilio Pozuelo Monfort wrote: > However given the impact of these library updates, I was wondering > if we have considered to just mark enigmail as EOL in jessie? Obviously if we > can keep supporting stuff we should do that, but as you say these library > updates affect important unrelated rdeps so we need to weigh that. +1 Cheers, Moritz
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On Fri 2018-12-14 09:26:50 -0500, Antoine Beaupré wrote: > I have outlined the tradeoffs of this in the past. For me, the biggest > concern is that users will blindly install Enigmail from the app store > and that actually has security vulnerabilities because the jessie gpg > version is too old, as I understand it. Installing enigmail from addons.mozilla.org (what i think anarcat means by "the app store") raises not only concerns about gpg compatibility on jessie -- it also downloads and runs arbitrary binary code from the Internet: https://bugs.debian.org/891882 This is fixed in debian by a change in the defaults, but upstream appears to have no intention to change those defaults in the version in addons.mozilla.org. Leaving jessie users vulnerable to this would make me pretty sad. I appreciate the work that anarcat is doing here! --dkg signature.asc Description: PGP signature
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On 2018-12-14 09:08:42, Emilio Pozuelo Monfort wrote: > On 13/12/2018 21:14, Antoine Beaupré wrote: >> Hi, >> >> This is the latest update in the Thunderbird / Enigmail changes that are >> happening in jessie. I have built a series of test packages, partly from >> stretch (gnupg2, enigmail) and partly from backports (libassuan, >> libgcrypt, libgpg-error, npth) and uploaded them here: >> >> https://people.debian.org/~anarcat/debian/jessie-lts/ >> >> I need people to test those packages, and not just enigmail users. Some >> of those packages have pernicious and deep ramifications. I am >> particularly worried about libgcrypt, which is used for example by >> cryptsetup. > > I see that your tests have gone well so far (except for enigmail itself for > unrelated reasons as you explain). This is great work, and I don't mean to > push > back on it. However given the impact of these library updates, I was wondering > if we have considered to just mark enigmail as EOL in jessie? Obviously if we > can keep supporting stuff we should do that, but as you say these library > updates affect important unrelated rdeps so we need to weigh that. I have repeatedly considered this, and received almost zero feedback on the idea, other than "we should support our users", which I took as a go ahead to actually complete the backport. I have outlined the tradeoffs of this in the past. For me, the biggest concern is that users will blindly install Enigmail from the app store and that actually has security vulnerabilities because the jessie gpg version is too old, as I understand it. > BTW I have briefly looked at the versions you have backported, and I wonder > why > npth and libgpg-error have deb8u3 rather than deb8u1? An oversight. I also need to use dh-autoreconf in enigmail as I have been told it actually exists in jessie - not sure how I couldn't find it. :) > I haven't looked at your changes yet, but I will find some time to look at > them > and give these packages a try. Thanks! The more testing we get, the better off we'll be. :) A. -- No animal has more liberty than the cat; but it buries the mess it makes. The cat is the best anarchist. Until they learn that from the cat I cannot respect them. - For whom the bell tolls, Ernest Hemingway
Re: HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
On 13/12/2018 21:14, Antoine Beaupré wrote: > Hi, > > This is the latest update in the Thunderbird / Enigmail changes that are > happening in jessie. I have built a series of test packages, partly from > stretch (gnupg2, enigmail) and partly from backports (libassuan, > libgcrypt, libgpg-error, npth) and uploaded them here: > > https://people.debian.org/~anarcat/debian/jessie-lts/ > > I need people to test those packages, and not just enigmail users. Some > of those packages have pernicious and deep ramifications. I am > particularly worried about libgcrypt, which is used for example by > cryptsetup. I see that your tests have gone well so far (except for enigmail itself for unrelated reasons as you explain). This is great work, and I don't mean to push back on it. However given the impact of these library updates, I was wondering if we have considered to just mark enigmail as EOL in jessie? Obviously if we can keep supporting stuff we should do that, but as you say these library updates affect important unrelated rdeps so we need to weigh that. BTW I have briefly looked at the versions you have backported, and I wonder why npth and libgpg-error have deb8u3 rather than deb8u1? I haven't looked at your changes yet, but I will find some time to look at them and give these packages a try. Cheers, Emilio
HEADS UP: upcoming change to libgcrypt and other gnupg libraries for Enigmail backport
Hi, This is the latest update in the Thunderbird / Enigmail changes that are happening in jessie. I have built a series of test packages, partly from stretch (gnupg2, enigmail) and partly from backports (libassuan, libgcrypt, libgpg-error, npth) and uploaded them here: https://people.debian.org/~anarcat/debian/jessie-lts/ I need people to test those packages, and not just enigmail users. Some of those packages have pernicious and deep ramifications. I am particularly worried about libgcrypt, which is used for example by cryptsetup. I am also concerned about the interactions between gpg2 and legacy gpg 1.4. I have made sure that gpg binaries are not clobbered by gpg2, which means it *should* be possible to run both side by side. But this does mean having multiple key storage at once when gpg2 is in used, something we have managed to avoid in the 1.4 -> 2.x migration in stretch so far. I am also specifically concerned about statements such as "[even though co-installability was considered while designing 2.1, in practice 1.4 and 2.1+ don't mix well][gnupg]" that were said elsewhere. [gnupg]: https://lists.gnupg.org/pipermail/gnupg-users/2018-February/059988.html Nevertheless, I have gone through the process of testing the packages against their dependencies in a jessie virtual machine, as much as possible. The following tools were tested, based on [advice from dkg][]: * cryptsetup: no build-time test suite, smoke-tested (luksFormat/Open + mkfs + edit file / close loop), main related change is libgpgerror and libgcrypt bumps * gpgme: build-time test suite passes, no further direct test although covered by later mutt tests, i believe * gmime: untested * libotr: depends on libgcrypt11, so presumed not affected. built, but no build-time test suite * mutt: no test suite, segfaults when hitting "enter" when no key match, but bug already present in jessie before proposed changes. other smoke tests seem okay. * claws: untested * mcabber: untested * enigmail: self-test suite passes at build time, had several problems during account setup (revocation cert failed to create during key init; can encrypt to a client, but not sign or decrypt. so something definitely wrong), related to missing pinentry packages. once pinentry is installed, all functionality seems to be working, including receiving and sending encrypted+signed and encrypted emails. autocrypt not tested. Regarding the latter, it seems like autocrypt caused some problems at least with the [Tails team][15923]. It might be advisable to upgrade to Enigmail 2.0.9 in stretch and jessie before completing this work, as it seems to address those issues specifically. [advice from dkg]: https://lists.debian.org/87ftvrnbyb@fifthhorseman.net [15923]: https://redmine.tails.boum.org/code/issues/15923 I would appreciate code reviews, although the changes to perform the backports are generally trivial: downgrade debhelper from 10 to 9, delete the dh-strip --dbgsym-migration overrides, remove the mingw packages, etc. Those who want to review the changes in code might want to use the git repositories on salsa, because all packages are conveniently available there. I created a debian/jessie-security branch on every repository I had write access to, or on a fork in my own namespace otherwise: https://salsa.debian.org/debian/enigmail https://salsa.debian.org/debian/gnupg2 https://salsa.debian.org/debian/libassuan https://salsa.debian.org/anarcat/libgcrypt https://salsa.debian.org/debian/libgpg-error https://salsa.debian.org/anarcat/npth Unless I get significant pushback on this, I plan on uploading those packages next tuesday. Phew! Maybe we'll get through that one at last. :) A. -- Seul a un caractère scientifique ce qui peut être réfuté. Ce qui n'est pas réfutable relève de la magie ou de la mystique. - Karl Popper
