Re: Why Operating Systems don't always upgrade GnuPG

2018-02-21 Thread Peter Lebbing 
On 21/02/18 17:22, Teemu Likonen wrote:
> default-key FINGERPRINT!

That would help for command-line usage for a user with only one private
key. But anything else might not use the default key.

Peter.

-- 
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at 



signature.asc
Description: OpenPGP digital signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-21 Thread Teemu Likonen 
Daniel Kahn Gillmor [2018-02-20 21:35:12-08] wrote:

> Anyway, here's one concrete example (hinted at above) of a
> programmatic gap that is much easier to achieve by mucking around with
> the internal state rather than by the programmatic interface:
>
>  * I want to introduce a new signing-capable subkey, and i want to
>distribute it widely, but i don't want to start signing with it just
>yet.

It seems to me that there is an easy gpg.conf solution:

default-key FINGERPRINT!

See the ! character which forces exactly that (sub)key for signing. Use
that option to select your old signing (sub)key.

-- 
/// Teemu Likonen   - .-..    //
// PGP: 4E10 55DC 84E9 DFF6 13D7 8557 719D 69D3 2453 9450 ///


signature.asc
Description: PGP signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-21 Thread Werner Koch 
On Tue, 20 Feb 2018 20:36, n...@walfield.org said:

> "uncool".  I left because we (Werner and I) could not work well
> together.  This is the same reason that Justus, Kai and Marcus left.

Okay, you raised it and now my Lavamat wants to reply on this: Secret
negotiations with other companies, promising them things without
consulting with me, working on your own schedule, throwing in large
blobs of code after silently working on them for months, not adhering to
milestones, ignoring requests to write documentation, leaving almost all
of the project work to me, spending half a person year on preparing a
campaign, kind of blackmailing me to re-write GnuPG in Rust, refusing to
prepare the agreed-upon training material.

My take away of that story is that not all hackers are able value the
liberties granted to them.  Too bad.


Shalom-Salam,

   Werner

--
#  Please read:  Daniel Ellsberg - The Doomsday Machine  #
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.


pgpnKtX2OUR5G.pgp
Description: PGP signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-20 Thread Daniel Kahn Gillmor 
On Tue 2018-02-20 16:08:35 +0100, Werner Koch wrote:
> On Mon, 19 Feb 2018 19:45, d...@fifthhorseman.net said:
>
>> GnuPG is under active development, and it has never had a fully-featured
>> stable API (Application Programming Interface).  What i mean is, there
>> are some capabilities that are only available from the user interface
>> (UI), and are not in the stable API.  GnuPG's UI has been constantly
>
> You probably expected that I need to dissent: Modulo a few bugs there
> has always been a stable API in GnuPG and we have taken great caution to
> not break it.

We're not in disagreement, actually :) I explicitly said that what was
has been lacking was a "*fully-featured* stable API".  I didn't say that
the API was not stable.  But the UI has features that the API does not,
and this results in people using the UI as though it was an API, or
mucking about with the innards of the local storage.  As one example, I
know this because i've been through that when working on early versions
of monkeysphere; i was using bash around GnuPG and yeah, in a few cases
i ended up stuffing a pipeline of "commands" into a gpg invocation
because it was there and seemed to do roughly what i wanted. 

The GnuPG stable API is much more fully-featured today (thank you for
the --quick-* commands!), but that has not been the case historically.
And we're paying the cost for that with legacy applications that have
ossified around wrong assumptions about how to interact with GnuPG :/

> Granted some parts of the API are not easy to use but
> nevertheless, they are stable.  For example the --edit-key interactor
> requires the use a state machine to stay compatible with future versions
> of GnuPG.  This has been remarked over and over in the last ~20 years.

Sorry, but building a state machine in external code to model the
internal state of a tool is not a functional API that we can
realistically expect developers to use.  If a module has state, it needs
to express that state to the user and make the transitions clear and
functional, with comprehensible and easily-handlable error cases.  To
wit:

> Unfortunately some folks reject to read manuals, examples and
> description on proper use and just go ahead and do what seems to work.

Even the folks like me, who read manuals and examples and descriptions
of proper use will eventually just go ahead and do what seems to work :/
The goal of using someone else's code is to *relieve* yourself of
needing to know exactly how that code works.  I know that there are some
people that end up sending patches upstream for every project they
touch.  But most developers don't have time or energy for that, and they
need simple, clear interfaces.

This is why (for example) i've argued in the past for making the return
code for gpgv simple and closely aligned with the boolean tests what
people are likely to want (https://dev.gnupg.org/T1537#100523).  Not
everyone wants to write a parser for a text stream to find out if a
signature is valid.  Some folks just want to know whether a signature is
valid, and are happy to punt on the details.

And even if everyone *did* want to write a parser, how many different
parser implementations do you think would need to be created?  how many
of them would be bug-free?  checking a return code is much harder to get
wrong.

> Actually this is not just a gpg thing: I have seen too many scripts and
> programs which neglect to use LC_ALL=C and break as soon as they are
> running under a different locale.

very true.  it's quite difficult to write robust programs, even when
interfaces are simple and clear and don't have sneaky side-effects or
behavioral changes due to rarely-changed environment variables.  And
even if you test with LC_ALL=C, did you test with LC_ALL=C.UTF-8 ? :P

> To avoid such breakage but keeping friendly to the command line user,
> gpg provide a machine interface (--with-colons, --batch, --command-fd)
> to make automation easy.

yes, these are great things!  thank you for them!

> I can't count how often I wrote that the only defined way to access the
> content of keyrings are my means of the --export and --import commands.  

I know.  I can't count how many times i've said it either.  It's pretty
frustrating!  I think it's aggravated further by the fact that we don't
have a clear rule like "do not touch anything inside ~/.gnupg" -- those
kinds of rules end up riddled with caveats like "well, except for
sshcontrol and gpg.conf" or "oh, you want to do custom subkey
management? let me tell you how to poke around in private-keys-v1.d/" :/

> Please give examples.

did you look at the bug reports i already cited?  Those are definitely
examples of overly-brittle wrappers around GnuPG.  You might also be
interested in Request-Tracker's brittle bindings
(https://bugs.debian.org/832829), or pretty much anything in debian that
explicitly Depends: gnupg1 today.

I'd be game to cc you every time i hold a project's hand through some
tricky problem if you'd like.

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-20 Thread Daniel Kahn Gillmor 
On Tue 2018-02-20 13:18:40 +0100, Dashamir Hoxha wrote:
> One solution to this situation may be to install the latest GnuPG
> in a Docker container, where it can have all the required libraries
> and dependencies that it needs, without disturbing the host OS.

I think this misses the point that it's not just *what does gnupg depend
on* but it's also *what depends on gnupg*.  The dependencies work in
both directions.

> Another solution may be to use a "snap", which is a kind of new
> software packaging invented by Ubuntu:

The basic idea behind "snap" and "flatpak" and other similar tools is
what many people call "bundling" or "vendoring" -- you ship the program
together with all its dependencies, regardless of what dependencies are
on the host system.  it's not a new idea at all, and is quite common on
many platforms, including in some flavors of cowboy web development.

As with docker containsers, this approach doesn't address the other
direction of the dependency graph.  In addition, all of these approaches
have maintenance costs and open questions about responsibility.  if
every app ships with its own bundled copy of libfoo, and a flaw is found
in libfoo, then it needs to be fixed.  can you be sure you've found and
fixed all copies?  Who is responsible for fixing each specific copy?  Do
those maintainers have enough time/attention/living expenses to make
sure vulerabilities and software flaws get patched in all of their
dependencies?  are they willing to re-ship the entire bundle/snap/docker
image for each dependency that needs an upgrade?

I recently heard bundling/vendoring/snaps/docker containers
characterized in the following way, which resonated with me:

Hm, maintaining a complex operating system is hard.  I know, we can
fix that by trying to maintain 100 complex operating systems
instead!

To be clear, i believe that there are contexts where bundling is
actually the right approach.  But it is not an obvious win to me in most
cases.

Regards,

--dkg


signature.asc
Description: PGP signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-20 Thread Ben McGinnes 
On Mon, Feb 19, 2018 at 10:45:52AM -0800, Daniel Kahn Gillmor wrote:
> 
> How can GnuPG contribute to fixing this problem?  The traditional way
> that many other projects have taken is to define their core programmatic
> functionality into a library with a strict interface guarantees, and
> have explicitly deprecated other use.  The closest that GnuPG comes to
> this technique is GPGME, which is not feature-complete (as compared to
> the gpg executable), and has its own history of both difficult upgrades
> and unclear/surprising semantics.

True, but what it does have available still covers a *lot* of what
people do mainly want to access programmatically.  My trusty key
counter is now done that way (it takes a while to run, but that's due
to the size of the keybox, not due to GPGME).

> It also doesn't have bindings in many popular programming languages.

There is a cunning plan which may provide a bridge to at least in part
address that.

> When programmers in those language want to use GnuPG, their shortest
> path to "something that works" often involves shelling out to gpg,
> rather than binding to GPGME. :/

Yes

> Another thing that would help would be to explicitly and succinctly
> document the preferred ways of interacting with GnuPG in ways that other
> developers find useful.

That would be brilliant, but it requires convincing developers to not
simply document their work, but to actually step back from their
focussed POV and abstract what they're intending to do so we can
determine what will actually be useful to procide all developers.
Otherwise we'll just end up with a bunch of feature requests specific
to the problem each developer happens to be working on at the time
they notify us.

If you can work out a way to get engineers to understand why that's
important ... you'll become very rich shortly thereafter.  ;)

> Perhaps GnuPG could also itself try to detect when it is being used
> programmatically in an unstable way and produce warnings?

Most of the worst of those examples are in bash, how're you going to
tell the difference between that and a user?

> Yet another complementary approach might be to aggressively police
> the ecosystem by finding other software that deends on GnuPG in any
> of the aforementioned brittle ways, and either ask those developers
> to stop using GnuPG entirely, or to provide them with stable,
> well-supported ways to do what they're looking to do.

And people accuse me of being combative!  Wow ... you're brave ...

Moreseriously, though, let's besure we've got an alternative method to
replace whatever their pet project's method is first.

Arguably gpgme-tool might already do that, but we'll see.

> I welcome discussion/suggestions on how we can improve this situation,
> and i *definitely* welcome help in doing the kind of
> ecosystem-perspective work necessary to make it easier to maintain an
> up-to-date branch of GnuPG.

I'm already laying the groundwork on the API side of things, as well
as working on doumentation.

> But shrugging and suggesting it's uncontroversial to upgrade arbitrary
> machines to the latest version of GnuPG doesn't appreciate the scope of
> the problem involved with software maintenance in an active and
> interdependent ecosystem.

Indeed.  I didn't check your buglist ... but I did recall that one of
the dependencies of GMIME is GPGME.  Pretty sure that'll affect a few
packages and libgcrypt & libgpg-error are a dependencies for lots things.

It's exactly this sort of thing which led to the popularity of
installing things the users wanted in /usr/local or /opt or wherever
specifically to prevent the end user use case adversely affecting the
system's use case.


Regards,
Ben


signature.asc
Description: PGP signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-20 Thread Neal H. Walfield 
At Tue, 20 Feb 2018 16:08:35 +0100,
Werner Koch wrote:
> > Yet another complementary approach might be to aggressively police the
> > ecosystem by finding other software that deends on GnuPG in any of the
> > aforementioned brittle ways, and either ask those developers to stop
> 
> That is what our plan for the time after 2.2 was.  Unfortunately, this
> seems to be a boring job for some hackers and thus some of them opted to
> leave gnupg to work on cool new stuff instead.  Nevertheless, this is
> still high on my task list.

I'd rather not air dirty laundry, but I feel it necessary to correct
misinformation.  I did not leave g10code, because working on gpg was
"uncool".  I left because we (Werner and I) could not work well
together.  This is the same reason that Justus, Kai and Marcus left.

___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-20 Thread Werner Koch 
On Mon, 19 Feb 2018 19:45, d...@fifthhorseman.net said:

> GnuPG is under active development, and it has never had a fully-featured
> stable API (Application Programming Interface).  What i mean is, there
> are some capabilities that are only available from the user interface
> (UI), and are not in the stable API.  GnuPG's UI has been constantly

You probably expected that I need to dissent: Modulo a few bugs there
has always been a stable API in GnuPG and we have taken great caution to
not break it.  Granted some parts of the API are not easy to use but
nevertheless, they are stable.  For example the --edit-key interactor
requires the use a state machine to stay compatible with future versions
of GnuPG.  This has been remarked over and over in the last ~20 years.
Unfortunately some folks reject to read manuals, examples and
description on proper use and just go ahead and do what seems to work.
Actually this is not just a gpg thing: I have seen too many scripts and
programs which neglect to use LC_ALL=C and break as soon as they are
running under a different locale.

To avoid such breakage but keeping friendly to the command line user,
gpg provide a machine interface (--with-colons, --batch, --command-fd)
to make automation easy.

> improving; sometimes that means that older (worse) UI gets deprecated,
> removed, or has its semantics change.

Deprecated, okay.  But I am not aware of semantic changes except for
rare corner cases.

> certain behaviors and features of GnuPG's internal storage (e.g. what
> goes on in ~/.gnupg/), which has never been explicitly part of its API
> (confusingly, there are some exceptions where GnuPG upstream *has*
> encouraged other tools to programmatically interact with some elements

I can't count how often I wrote that the only defined way to access the
content of keyrings are my means of the --export and --import commands.  

Please give examples.

> Simply upgrading GnuPG to the latest available version on a server that
> also ships other complex software is likely to lead to breakage
> elsewhere in the OS because of these brittle assumptions and
> dependencies around GnuPG's UI and internal storage.

Right.  However, this is not due to changing APIs of GnuPG but due to
the wrong use of the API.  Much like we have seen that grep and sed
started to fail on localized systems.

And even today, we see hackers who are breaking working integration of
gpg by replacing the (correct) use of the machine interface by the
supposed easier to use human interface of gpg.  And then encountering
problems with newer versions of gpg.  :-(

> are not part of an explicit, documented API, the ecosystem apparently
> *does* try to manipulate them anyway, with all the attendant brittleness
> that you can imagine.

Right.  That is a bad idea and that is why it took many years to get a
modern version in widespread use.

> that many other projects have taken is to define their core programmatic
> functionality into a library with a strict interface guarantees, and

Library interfaces are as much abused as command line interfaces.  This
is not really the point.  A library may have the advantage that it
exposes only a smaller set of interfaces and thus lessens the risk of
wrong usage.

> have explicitly deprecated other use.  The closest that GnuPG comes to
> this technique is GPGME, which is not feature-complete (as compared to

That is on purpose; see above.

> developers find useful.  Perhaps GnuPG could also itself try to detect
> when it is being used programmatically in an unstable way and produce

Those tools won't notice these warnings because they hide away the
actual output of gpg. 

> Yet another complementary approach might be to aggressively police the
> ecosystem by finding other software that deends on GnuPG in any of the
> aforementioned brittle ways, and either ask those developers to stop

That is what our plan for the time after 2.2 was.  Unfortunately, this
seems to be a boring job for some hackers and thus some of them opted to
leave gnupg to work on cool new stuff instead.  Nevertheless, this is
still high on my task list.

> machines to the latest version of GnuPG doesn't appreciate the scope of
> the problem involved with software maintenance in an active and
> interdependent ecosystem.

Well said.


Salam-Shalom,

   Werner

-- 
#  Please read:  Daniel Ellsberg - The Doomsday Machine  #
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.


pgp5vybKgZDEZ.pgp
Description: PGP signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-20 Thread Kristian Fiskerstrand 
On 02/20/2018 01:18 PM, Dashamir Hoxha wrote:
> If anybody is willing to give a try to any of these solutions I would
> like to help.

I would be generally cautious for both approaches without proper support
in the surrounding infrastructure. In particular an upgrade to a
depending library would need to automatically cause a rebuild of the
container in the case of a security upgrade when such embedding happens,
which is generally a bad thing unless you have a large scale deployment
and defined QA processes for terminating and replacing containers with
new deployments regularly.

-- 

Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk

Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3

Manus manum lavat
One hand washes the other



signature.asc
Description: OpenPGP digital signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-20 Thread Dashamir Hoxha 
On Mon, Feb 19, 2018 at 7:45 PM, Daniel Kahn Gillmor 
wrote:

> On Sat 2018-02-17 17:06:54 -0600, helices wrote:
> > I will probably never understand why wanting to run the most current
> > version of gnupg on a plethora of servers is controversial.
>
> Here's one last try to explain the situation.
>
> GnuPG (and the libraries it depends on) are used by (aka "depended on
> by") other libraries and tools, both those integrated into the operating
> system itself, and those that might be externally installed.  Some of
> these dependencies are "brittle".
>

One solution to this situation may be to install the latest GnuPG
in a Docker container, where it can have all the required libraries
and dependencies that it needs, without disturbing the host OS.

But I am aware that this may present some challenges for  normal
usage and may not be suitable except for testing.

Another solution may be to use a "snap", which is a kind of new
software packaging invented by Ubuntu:
 - https://snapcraft.io/
 - https://docs.snapcraft.io/snaps/intro
The idea is that a software is shipped with all the dependencies,
so it does not matter in which OS it is installed, it will always work.

I don't know the details of snaps. Since it is a "containerized software
package" maybe it is not much different from the docker solution
above and maybe has the same challenges/problems.

If anybody is willing to give a try to any of these solutions I would like
to help.

Regards,
Dashamir
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG

2018-02-20 Thread Peter Lebbing 
On 19/02/18 19:45, Daniel Kahn Gillmor wrote:
> But shrugging and suggesting it's uncontroversial to upgrade arbitrary
> machines to the latest version of GnuPG doesn't appreciate the scope of
> the problem involved with software maintenance in an active and
> interdependent ecosystem.

You are right and I feel stupid for suggesting it is uncontroversial.
Hell, you'd think running Debian stretch/stable (with its 2.1.18) on a
plethora of servers would be uncontroversial, but even that isn't
totally free of controversy. There are people having problems with
adjusting their process to use GnuPG 2.1+.

I am very grateful for all the work you put in to not only fix programs
in Debian depending on /usr/bin/gpg2 being 2.0, but also fix programs
depending on /usr/bin/gpg being 1.4. Because even though
co-installability was considered while designing 2.1, in practice 1.4
and 2.1+ don't mix well.

Thank you.

If done with care and attention, there are still situations where
installing GnuPG 2.2 on what is the most recent version of CentOS/RHEL
is a good thing to do. You have to carefully consider which software
will be using GnuPG, though.

Peter.

-- 
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at 



signature.asc
Description: OpenPGP digital signature
___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-19 Thread MFPA 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hi


On Monday 19 February 2018 at 8:51:08 PM, in a message with no id,
ed...@pettijohn-web.com wrote:-


> I think gpgme is the answer here as well. If you mean
> specifically
> a python interface to gpgme then it's probably up to
> a python developer to get that rolling.

Python bindings are among those maintained within the GPGME
repository.

See https://wiki.gnupg.org/APIs



- --
Best regards

MFPA  

After all is said and done, a lot more will be said than done.
-BEGIN PGP SIGNATURE-

iNUEARYKAH0WIQSWDIYo1ZL/jN6LsL/g4t7h1sju+gUCWotpwF8UgAAuAChp
c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0OTYw
Qzg2MjhENTkyRkY4Q0RFOEJCMEJGRTBFMkRFRTFENkM4RUVGQQAKCRDg4t7h1sju
+rFDAP9aIDxhmkvC8xFKh2TI/JWefO9UNIDUh+0iYom167DTKQD+OQ/iUn5MW6fu
PfJfdRiAMR5fMB7NpFvzcrRbkoXzQQCJApMEAQEKAH0WIQRSX6konxd5jbM7JygT
DfUWES/A/wUCWotpwF8UgAAuAChpc3N1ZXItZnByQG5vdGF0aW9ucy5vcGVu
cGdwLmZpZnRoaG9yc2VtYW4ubmV0NTI1RkE5Mjg5RjE3Nzk4REIzM0IyNzI4MTMw
REY1MTYxMTJGQzBGRgAKCRATDfUWES/A/0IVD/0ZWM6fz12is1z2FrZq9S9RX/dE
pz4UedrGjWcufMzv39R2V8TS03u0dx+vAzRo8KT05DGtvIna+3FhosnXSRBu3JSZ
jfT3pfA8hIPY2p2lJK0s2haeL98S2Yu3wJ2Yt+KgS9MmyFOSL/022fVDvYFwi+9Y
/oJlQaE5BXBXvS3n/0WsMrZzDcY3nLf5S0eSK9MSd9rM/M4z09KxS2dngj1Sifgv
GIiBYD7MwnZaaIi8WLGYTj6YeXSVMtlHzXa83fNiZtNOmLRI6p8VkMQUTh5Tjt3F
qlDd+zTSkn801IGiivEYfaOQcRylfTqORhSEFZqIUXI4W/uzJ6NL165SMNgfUa9m
UAinMeKWuC8SdJwKU3BcaqhqhdwoI5HjNq4RQm5K833dkjvn8XoBZ/Tk+Xy918oW
5UCsF3w+AZ9pCG1li0ZnCRkyK7BL0LE8qmotRfknyd9pbGkmljbQYYrxBFKtQdjx
QFvV5OymnxeJcFLqa/cWv7Mz2jPNkA5dSIdPEp0Gsq1Of9hfwS8sddJ6Z/bFONa9
WfSam337dgrZv9aa6I7sO9AP28gkKbB+W9TRVi5WW1m3tHqTm9WOXv30HKLuuyeO
f+K9v0NbKbhoUu1KzYPmXbzBzqeRlYgtGac85I/UvVWV4XzN80G8sxoHsg/fPyd5
sMEr+A2cUaWR7F0DUQ==
=VNMJ
-END PGP SIGNATURE-


___
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users

Re: Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-19 Thread edgar 

On Feb 19, 2018 12:45 PM, Daniel Kahn Gillmor  wrote:
>
> On Sat 2018-02-17 17:06:54 -0600, helices wrote:
> > I will probably never understand why wanting to run the most current
> > version of gnupg on a plethora of servers is controversial.
>
> Here's one last try to explain the situation.
>
> GnuPG (and the libraries it depends on) are used by (aka "depended on
> by") other libraries and tools, both those integrated into the operating
> system itself, and those that might be externally installed.  Some of
> these dependencies are "brittle".
>
> Brittle software dependencies
> -
>
> GnuPG is under active development, and it has never had a fully-featured
> stable API (Application Programming Interface).  What i mean is, there
> are some capabilities that are only available from the user interface
> (UI), and are not in the stable API.  GnuPG's UI has been constantly
> improving; sometimes that means that older (worse) UI gets deprecated,
> removed, or has its semantics change.
>
> For historical reasons, there are a number of tools that were built
> around some element of the GnuPG UI that was current at the time the
> tool was built.  Even worse, there are a number of tools that assume
> certain behaviors and features of GnuPG's internal storage (e.g. what
> goes on in ~/.gnupg/), which has never been explicitly part of its API
> (confusingly, there are some exceptions where GnuPG upstream *has*
> encouraged other tools to programmatically interact with some elements
> within its internal storage).  Newer versions of GnuPG do different
> things with its internal storage (and as users we get benefits from
> those improvements).
>
> Simply upgrading GnuPG to the latest available version on a server that
> also ships other complex software is likely to lead to breakage
> elsewhere in the OS because of these brittle assumptions and
> dependencies around GnuPG's UI and internal storage.
>
> A case study
> 
>
> For example, the current stable version of the Debian operating system
> is Debian 9 ("stretch"), and it ships a version of the "modern" branch
> of GnuPG.
>
> As one of the GnuPG maintainers for Debian, i was hoping at one point to
> backport the "modern" version of GnuPG to the previous version of Debian
> (Debian 8, "jessie"), which some people still run on servers.  I found
> that such an upgrade would break at least a half-dozen other packages in
> Debian jessie *that i knew of* [0] -- and their breakage would in turn
> likely affect some number of other packages.  This was not an exhaustive
> survey of all possible bugs, just the most visible ones. :/
>
> I have personally given up on the project of backporting modern GnuPG to
> "jessie", because i think what time i can devote to GnuPG maintenance is
> better-spent elsewhere.  I don't have the bandwidth to cope with the
> resultant bug reports in other packages that such a backport would
> produce.  Generally, i encourage users of "jessie" to uprade their
> entire OS to the current version of debian stable, and to take advantage
> of the improvements to GnuPG that way.
>
> What can we do?
> ---
>
> The problems described above point to problems in the ecosystem *around*
> GnuPG, but it also points to concerns about GnuPG's presentation of its
> capabilities *to* the rest of the ecosystem.  To the extent that GnuPG
> offers features that other tools might want to use, when those features
> are not part of an explicit, documented API, the ecosystem apparently
> *does* try to manipulate them anyway, with all the attendant brittleness
> that you can imagine.
>
> How can GnuPG contribute to fixing this problem?  The traditional way
> that many other projects have taken is to define their core programmatic
> functionality into a library with a strict interface guarantees, and
> have explicitly deprecated other use.  The closest that GnuPG comes to
> this technique is GPGME, which is not feature-complete (as compared to
> the gpg executable), and has its own history of both difficult upgrades
> and unclear/surprising semantics.  It also doesn't have bindings in many
> popular programming languages.  When programmers in those language want
> to use GnuPG, their shortest path to "something that works" often
> involves shelling out to gpg, rather than binding to GPGME. :/ 

That doesn't sound secure. Probably shouldn't use such programs on a server. 
Gpgme looks to do everything that it needs to do. Can you specify a function it 
is missing? 

>
> Another thing that would help would be to explicitly and succinctly
> document the preferred ways of interacting with GnuPG in ways that other
> developers find useful.  Perhaps GnuPG could also itself try to detect
> when it is being used programmatically in an unstable way and produce
> warnings?
>
> Yet another complementary approach might be to aggressively police the
> ecosystem by finding other software that deends on GnuPG in

Why Operating Systems don't always upgrade GnuPG [was: Re: How can we utilize latest GPG from RPM repository?]

2018-02-19 Thread Daniel Kahn Gillmor 
On Sat 2018-02-17 17:06:54 -0600, helices wrote:
> I will probably never understand why wanting to run the most current
> version of gnupg on a plethora of servers is controversial.

Here's one last try to explain the situation.

GnuPG (and the libraries it depends on) are used by (aka "depended on
by") other libraries and tools, both those integrated into the operating
system itself, and those that might be externally installed.  Some of
these dependencies are "brittle".

Brittle software dependencies
-

GnuPG is under active development, and it has never had a fully-featured
stable API (Application Programming Interface).  What i mean is, there
are some capabilities that are only available from the user interface
(UI), and are not in the stable API.  GnuPG's UI has been constantly
improving; sometimes that means that older (worse) UI gets deprecated,
removed, or has its semantics change.

For historical reasons, there are a number of tools that were built
around some element of the GnuPG UI that was current at the time the
tool was built.  Even worse, there are a number of tools that assume
certain behaviors and features of GnuPG's internal storage (e.g. what
goes on in ~/.gnupg/), which has never been explicitly part of its API
(confusingly, there are some exceptions where GnuPG upstream *has*
encouraged other tools to programmatically interact with some elements
within its internal storage).  Newer versions of GnuPG do different
things with its internal storage (and as users we get benefits from
those improvements).

Simply upgrading GnuPG to the latest available version on a server that
also ships other complex software is likely to lead to breakage
elsewhere in the OS because of these brittle assumptions and
dependencies around GnuPG's UI and internal storage.

A case study


For example, the current stable version of the Debian operating system
is Debian 9 ("stretch"), and it ships a version of the "modern" branch
of GnuPG.

As one of the GnuPG maintainers for Debian, i was hoping at one point to
backport the "modern" version of GnuPG to the previous version of Debian
(Debian 8, "jessie"), which some people still run on servers.  I found
that such an upgrade would break at least a half-dozen other packages in
Debian jessie *that i knew of* [0] -- and their breakage would in turn
likely affect some number of other packages.  This was not an exhaustive
survey of all possible bugs, just the most visible ones. :/

I have personally given up on the project of backporting modern GnuPG to
"jessie", because i think what time i can devote to GnuPG maintenance is
better-spent elsewhere.  I don't have the bandwidth to cope with the
resultant bug reports in other packages that such a backport would
produce.  Generally, i encourage users of "jessie" to uprade their
entire OS to the current version of debian stable, and to take advantage
of the improvements to GnuPG that way.

What can we do?
---

The problems described above point to problems in the ecosystem *around*
GnuPG, but it also points to concerns about GnuPG's presentation of its
capabilities *to* the rest of the ecosystem.  To the extent that GnuPG
offers features that other tools might want to use, when those features
are not part of an explicit, documented API, the ecosystem apparently
*does* try to manipulate them anyway, with all the attendant brittleness
that you can imagine.

How can GnuPG contribute to fixing this problem?  The traditional way
that many other projects have taken is to define their core programmatic
functionality into a library with a strict interface guarantees, and
have explicitly deprecated other use.  The closest that GnuPG comes to
this technique is GPGME, which is not feature-complete (as compared to
the gpg executable), and has its own history of both difficult upgrades
and unclear/surprising semantics.  It also doesn't have bindings in many
popular programming languages.  When programmers in those language want
to use GnuPG, their shortest path to "something that works" often
involves shelling out to gpg, rather than binding to GPGME. :/ 

Another thing that would help would be to explicitly and succinctly
document the preferred ways of interacting with GnuPG in ways that other
developers find useful.  Perhaps GnuPG could also itself try to detect
when it is being used programmatically in an unstable way and produce
warnings?

Yet another complementary approach might be to aggressively police the
ecosystem by finding other software that deends on GnuPG in any of the
aforementioned brittle ways, and either ask those developers to stop
using GnuPG entirely, or to provide them with stable, well-supported
ways to do what they're looking to do.

I welcome discussion/suggestions on how we can improve this situation,
and i *definitely* welcome help in doing the kind of
ecosystem-perspective work necessary to make it easier to maintain an
up-to-date branch of GnuPG.

But