Re: Need better release validation documentation/strategy.

[Bob Friesenhahn]

On Fri, 8 Apr 2022, ckeader wrote:


The key server network as we knew it is dead and buried, and I would not
expect any of them to provide complete or indeed reliable information.
This article explains why:
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f.
There was some discussion at the time over on gnupg-users also.


This was facinating reading, and I was not aware of any of it before. 
Unfortunately, I have not figured out how to follow its advice yet.


Everything related to OpenPGP is extremely obtuse with massive amounts 
of documentation.


OpenSSH 8 and later offer a facility which allows validating a file's 
origin and integrity given a certificate (see 
https://www.agwa.name/blog/post/ssh_signatures). I gave this a try and 
it was remarkably simple.  It is several orders of magnitude less 
complex than OpenPGP and many people use OpenSSH.  Unfortunately, not 
all systems have OpenSSH 8 yet (or will ever have OpenSSH).  Another 
issue is that users could be confused by ".sig" files and won't know 
if they should use OpenSSH or gpg to validate with them without 
looking at the content.



Providing the signer's pub keys on a (secured) web site seems to be the
best option for now.


I have been using several mechanisms, including an insecure URL link 
as is shown in my email signature text.



An important question has not been asked yet, IMHO - why are maintainers
using this relatively obscure method for hashing?


Yes, this is very obscure and it defeats the purpose, which should be 
to encourage verification.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

Re: Need better release validation documentation/strategy.

[ckeader]

> For some reason key servers are not very helpful these days with some 
> of them offering distorted behavior or appearing to be severely 
> overloaded.  It may be that the key server used by default by Ubuntu 
> Linux imposes additional limitations such as regarding exposing email 
> addresses. The user might need to know how to specify a different 
> server and know which ones to use.
 
The key server network as we knew it is dead and buried, and I would not
expect any of them to provide complete or indeed reliable information.
This article explains why:
https://gist.github.com/rjhansen/67ab921ffb4084c865b3618d6955275f.
There was some discussion at the time over on gnupg-users also.

Providing the signer's pub keys on a (secured) web site seems to be the
best option for now.

The key to verifying the base64 encoded checksums is a conversion from
binary to hex and vice versa. od from coreutils and xxd from vim can be
used.

| Here are the SHA1 and SHA256 checksums:
|
| 91fa501ada319c4dc8f796208440d45a3f48ed13  gzip-1.12.tar.gz
| W0+xTTgxTgny/IocUQ581UCj6g4+ubBCAEa4LDv0EIU  gzip-1.12.tar.gz
| 318107297587818c8f1e1fbb55962f4b2897bc0b  gzip-1.12.tar.xz
| zl4D5Rn2N+H4FAEazjXE+HszwLur7sNbr1+9NHnpGVY  gzip-1.12.tar.xz

$ echo "zl4D5Rn2N+H4FAEazjXE+HszwLur7sNbr1+9NHnpGVY" |base64 -d |xxd -p |tr -d 
'\n'
base64: invalid input
ce5e03e519f637e1f814011ace35c4f87b33c0bbabeec35baf5fbd3479e91956$
$ echo "zl4D5Rn2N+H4FAEazjXE+HszwLur7sNbr1+9NHnpGVY" |base64 -d | od -A n -v -t 
x1 |tr -d ' \n'
base64: invalid input
ce5e03e519f637e1f814011ace35c4f87b33c0bbabeec35baf5fbd3479e91956$
$ sha256sum gzip-1.12.tar.xz
ce5e03e519f637e1f814011ace35c4f87b33c0bbabeec35baf5fbd3479e91956  
gzip-1.12.tar.xz
$ 

For the reverse operation, use xxd -r -p, or openssl sha256 -binary. The
"invalid input" message above is due to missing padding.

An important question has not been asked yet, IMHO - why are maintainers
using this relatively obscure method for hashing?

Re: Need better release validation documentation/strategy.

[Bob Friesenhahn]

On Fri, 8 Apr 2022, Jim Meyering wrote:


On Fri, Apr 8, 2022 at 6:30 AM Bob Friesenhahn
 wrote:

Today I saw an announcement for a new version of gzip.  It provided
lots of data for how to verify the downloaded tarballs.  I recently
saw a very similar announcement for a new version of libtool. I am not
sure where the template of this announcement text is coming from, and
if anyone has validated that recipients will be able to make sense of
it.

The problem is that the advice in the announcements regarding use of
'gpg' just doesn't work (commands fail),


That was my mistake, and I fixed it last night. We updated the
generated and recommended gpg-key-fetching command to be a wget
command that fetches from savannah. I presumed that it would just
work, but that was not true. I fixed it by adding my gpg key in the
"public information" section of each project for which I'm already
listed as an authorized uploader.


For some reason key servers are not very helpful these days with some 
of them offering distorted behavior or appearing to be severely 
overloaded.  It may be that the key server used by default by Ubuntu 
Linux imposes additional limitations such as regarding exposing email 
addresses. The user might need to know how to specify a different 
server and know which ones to use.


This one failed:

% gpg --locate-external-key j...@meyering.net
gpg: error retrieving 'j...@meyering.net' via WKD: General error
gpg: error reading key: General error

and this one confusingly did not succeed:

% gpg --recv-keys 7FD9FCCB000B
gpg: key 7FD9FCCB000B: new key but contains no user ID - skipped
gpg: Total number processed: 1
gpg:   w/o user IDs: 1

% gpg --verify gzip-1.12.tar.xz.sig
gpg: assuming signed data in 'gzip-1.12.tar.xz'
gpg: Signature made Thu 07 Apr 2022 11:59:54 AM CDT
gpg:using RSA key 155D3FC500C834486D1EEA677FD9FCCB000B
gpg: Can't check signature: No public key

The next problem is this:

% sha256sum --version
sha256sum (GNU coreutils) 8.30
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
.

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Ulrich Drepper, Scott Miller, and David Madore.

It is possible that newer versions of this utility do support the 
base64 output but this one does not appear to.  This is from Ubuntu 
20.04LTS, which is Ubuntu's current LTS offering.


For the recent libtool announcement, the gpg issues were not identical 
but it was also not possible to retrieve the key using the 
instructions provided.  The libtool maintainer tried and he was not 
able to get the instructions to work either.


It is definitely preferable to verify using gpg so anything which 
makes this easier for users.


I did post my mail to the Automake list since it seems that Automake 
may be able to help make some of these aspects better by providing 
helpful rules and boiler-plate pertaining to signing files and 
verifying that it is possible to validate the signature.


For GraphicsMagick I added rules to Makefile.am so that if I am doing 
a "release" (a distcheck plus a few more steps) the build 
automatically signs the release file and generates sha256 sums.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

Re: Need better release validation documentation/strategy.

[Jim Meyering]

On Fri, Apr 8, 2022 at 6:30 AM Bob Friesenhahn
 wrote:
> Today I saw an announcement for a new version of gzip.  It provided
> lots of data for how to verify the downloaded tarballs.  I recently
> saw a very similar announcement for a new version of libtool. I am not
> sure where the template of this announcement text is coming from, and
> if anyone has validated that recipients will be able to make sense of
> it.
>
> The problem is that the advice in the announcements regarding use of
> 'gpg' just doesn't work (commands fail),

That was my mistake, and I fixed it last night. We updated the
generated and recommended gpg-key-fetching command to be a wget
command that fetches from savannah. I presumed that it would just
work, but that was not true. I fixed it by adding my gpg key in the
"public information" section of each project for which I'm already
listed as an authorized uploader.

Re: Need better release validation documentation/strategy.

[Carlos O'Donell]

On 4/8/22 09:30, Bob Friesenhahn wrote:
> Today I saw an announcement for a new version of gzip.  It provided
> lots of data for how to verify the downloaded tarballs.  I recently
> saw a very similar announcement for a new version of libtool. I am
> not sure where the template of this announcement text is coming from,
> and if anyone has validated that recipients will be able to make
> sense of it.

We make no such statements for glibc, and it's arguably more central to any
whole system validation that you're making.

However, because of the requirements for gpg signatures to upload to the FSF
servers we end up with a signature against the uploaded binary from the GNU 
Project
maintainer.

You can verify that I uploded glibc 2.35 to the FSF servers, and you have to
have a web of trust for me:

gpg --verify glibc-2.35.tar.xz.sig glibc-2.35.tar.xz
gpg: Signature made Thu 03 Feb 2022 01:35:30 AM EST
gpg:using RSA key ...
gpg: Good signature from "Carlos O'Donell "
gpg: aka "Carlos O'Donell (Work) "
gpg: aka "Carlos O'Donell (Work) "

> It seems like Automake and GNU in general should be trying to help
> with producing releases and release announcements which assist users
> with verifying the release tarballs rather than just leaving them
> royally confused.

In general this is documented here for the GNU Project:

Information for maintainers of GNU software
https://www.gnu.org/prep/maintain/

> I am not sure who the target audience is for GNU releases these days,
> but if it is not normal people who are still willing to compile
> software from source code on popular systems such as GNU/Linux, then
> there is a problem.

Can you expand a bit on the problem that you see?

-- 
Cheers,
Carlos.

Need better release validation documentation/strategy.

[Bob Friesenhahn]

Today I saw an announcement for a new version of gzip.  It provided 
lots of data for how to verify the downloaded tarballs.  I recently 
saw a very similar announcement for a new version of libtool. I am not 
sure where the template of this announcement text is coming from, and 
if anyone has validated that recipients will be able to make sense of 
it.


The problem is that the advice in the announcements regarding use of 
'gpg' just doesn't work (commands fail), and even the SHA256 checksum 
is described as "SHA256 checksum is base64 encoded" which I was 
previously only seeing from the BSD-oriented OpenSSH project which 
might be using a BSD tool which produces such checksums.


It seems like Automake and GNU in general should be trying to help 
with producing releases and release announcements which assist users 
with verifying the release tarballs rather than just leaving them 
royally confused.


If ordinary people are not able to use the data provided with the 
release announcement, then they will not be validating the tarballs 
that they run-across.  Download statistics suggest that the vast 
majority of source-code tarball downloads are not being validated at 
all.


If 'gpg' commands are provided, then they should be able to work by 
default on popular OS platforms.  Likewise, if a SHA256 checksum is 
provided and something new like "SHA256 checksum is base64 encoded", 
then instructions should be provided for how to use mature GNU tools 
(and/or popular non-GNU tools) to reproduce such a checksum.


While I was able to figure out how to use a combination of openssl and 
base64 to create matching SHA256 checksums, I doubt that most people 
would be willing to spend a half hour researching and figuring out how 
to do this.  I was not able to figure out how to produce a similar 
SHA256 checksum using the GNU software provided by the OS I am using.


I am not sure who the target audience is for GNU releases these days, 
but if it is not normal people who are still willing to compile 
software from source code on popular systems such as GNU/Linux, then 
there is a problem.


Bob
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt