Re: Problems verifying signed github releases (Re: Q: uscan with GitHub)
Note that kernel.org signs the raw tar file and not the compressed file. This way, they avoid issues like that and also allow conversion into different compression formats while the signature stays valid. Downside is that you have to decompress it first and then hash quite a big file for validation. Regards signature.asc Description: This is a digitally signed message part
Re: Problems verifying signed github releases (Re: Q: uscan with GitHub)
* FC Stegerman , 2023-02-19 21:08: (There was a recent LWN article covering this, see https://lwn.net/Articles/921787/.) That seems to be subscribers-only :( Here you go: https://lwn.net/SubscriberLink/921787/ff1350f40f12fb8e/ -- Jakub Wilk
Re: Problems verifying signed github releases (Re: Q: uscan with GitHub)
On 19.02.23 21:08, FC Stegerman wrote:
* Guillem Jover [2023-02-19 20:50]:
My upstream creates a tarball with git-archive, creates a signature and
uploads it (as described in the wiki[3]). This used to work to verify
the github-created tarball, but fails now - while creating my own
tarball like upstream and verifying it with upstream's signature works.
The uncompressed .tar files are identical (same hashsum), just the tar.gz
differ. Does anyone know why, and how to fix it? I tried non-default
compression levels for gzip with git-archive, but that didn't help to get an
identical tar.gz like the one from github.
I'd like to avoid having my upstream downloading the github-created
tarball, verify it and then upload this signature.
I assume you (or whatever service or tool is failing the verification
while creating a local tarball) might be seeing issues with git having
switched implementation for gzip, and a mismatch with the implementation
being used in either side. Perhaps try to set git's
tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use
the external command instead of the internal implementation? Or perhaps
you are using an old git that defaults to the external gzip but upstream
uses the internal one?
I was going to suggest that might be the issue, but you were faster :)
I do have some relevant links:
https://reproducible-builds.org/reports/2023-01/#news
https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002709.html
https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002710.html
(There was a recent LWN article covering this, see
https://lwn.net/Articles/921787/.)
That seems to be subscribers-only :(
- FC
Thank you both, that solved it!
After further digging into the details I updated the wiki to suggest:
git -c tar.tar.gz.command='gzip -cn' \
archive --format=tar.gz --prefix="${tag}/" \
-o "../${tag}.tar.gz" "${tag}"
Re: Problems verifying signed github releases (Re: Q: uscan with GitHub)
* Guillem Jover [2023-02-19 20:50]: > > My upstream creates a tarball with git-archive, creates a signature and > > uploads it (as described in the wiki[3]). This used to work to verify > > the github-created tarball, but fails now - while creating my own > > tarball like upstream and verifying it with upstream's signature works. > > > > The uncompressed .tar files are identical (same hashsum), just the tar.gz > > differ. Does anyone know why, and how to fix it? I tried non-default > > compression levels for gzip with git-archive, but that didn't help to get an > > identical tar.gz like the one from github. > > > > I'd like to avoid having my upstream downloading the github-created > > tarball, verify it and then upload this signature. > > I assume you (or whatever service or tool is failing the verification > while creating a local tarball) might be seeing issues with git having > switched implementation for gzip, and a mismatch with the implementation > being used in either side. Perhaps try to set git's > tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use > the external command instead of the internal implementation? Or perhaps > you are using an old git that defaults to the external gzip but upstream > uses the internal one? I was going to suggest that might be the issue, but you were faster :) I do have some relevant links: https://reproducible-builds.org/reports/2023-01/#news https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002709.html https://lists.reproducible-builds.org/pipermail/rb-general/2022-October/002710.html > (There was a recent LWN article covering this, see > https://lwn.net/Articles/921787/.) That seems to be subscribers-only :( - FC
Re: Problems verifying signed github releases (Re: Q: uscan with GitHub)
Hi! On Sun, 2023-02-19 at 18:34:56 +0100, Jens Reyer wrote: > [This is a followup to the thread "Q: uscan with GitHub" at > https://lists.debian.org/debian-devel/2022/10/msg00313.html. I manually set > in-reply-to, but am not sure if it'll work.] > My upstream creates a tarball with git-archive, creates a signature and > uploads it (as described in the wiki[3]). This used to work to verify > the github-created tarball, but fails now - while creating my own > tarball like upstream and verifying it with upstream's signature works. > > The uncompressed .tar files are identical (same hashsum), just the tar.gz > differ. Does anyone know why, and how to fix it? I tried non-default > compression levels for gzip with git-archive, but that didn't help to get an > identical tar.gz like the one from github. > > I'd like to avoid having my upstream downloading the github-created > tarball, verify it and then upload this signature. I assume you (or whatever service or tool is failing the verification while creating a local tarball) might be seeing issues with git having switched implementation for gzip, and a mismatch with the implementation being used in either side. Perhaps try to set git's tar.tar.gz.command="gzip -c" (or/and «tgz» instead of «tar.gz») to use the external command instead of the internal implementation? Or perhaps you are using an old git that defaults to the external gzip but upstream uses the internal one? (There was a recent LWN article covering this, see https://lwn.net/Articles/921787/.) Thanks, Guillem
Problems verifying signed github releases (Re: Q: uscan with GitHub)
[This is a followup to the thread "Q: uscan with GitHub" at
https://lists.debian.org/debian-devel/2022/10/msg00313.html. I manually
set in-reply-to, but am not sure if it'll work.]
> On Tue, Oct 11, 2022 at 10:23 AM Stephan Lachnit
wrote:
>> On Sun, Oct 9, 2022 at 7:06 PM Volans wrote:
[...]
For releases I use something like (w/o signatures): ``` version=4
opts="searchmode=plain,\
filenamemangle=s%v?@ANY_VERSION@%@PACKAGE@-$1.tar.gz%" \
https://api.github.com/repos///releases?per_page=100
\ https://api.github.com/repos/[^/]+/[^/]+/tarball/v?@ANY_VERSION@
```
[...]
To add some additional burden I noticed that the archive downloaded
from the API [1] is different from the one downloaded from the
browser [2] because they have a different top-level directory name,
while the API one has: {OWNER}-{REPO}-{REF_SHA1}/ where REF_SHA1 is
the SHA1 of refs/tags/{TAG_NAME}, the browser one has:
{OWNER}-{VERSION}/ where, at least in my case, VERSION is the tag
name without the leading 'v'. For a tag name of 'v0.2.0' I get
'gjson-py-0.2.0.tar.gz' that extracted creates 'gjson-py-0.2.0/'
(note the lack of 'v' in the names).
Because of the two different directory structures, the tar.gz files
are different so any md5 or gpg signature verification made on one
would fail with the other, adding additional confusion or complexity.
For now I'll probably upload to the release page the signatures of
both files, but this is definitely suboptimal.
Riccardo
[1] https://api.github.com/repos/{OWNER}/{REPO}/tarball/{RELEASE}
[2]
https://codeload.github.com/{OWNER}/{REPO}/tar.gz/refs/tags/{RELEASE}
I use the mentioned tarball from the browser release page, but it still
fails to verify with upstream's locally created signature:
My upstream creates a tarball with git-archive, creates a signature and
uploads it (as described in the wiki[3]). This used to work to verify
the github-created tarball, but fails now - while creating my own
tarball like upstream and verifying it with upstream's signature works.
The uncompressed .tar files are identical (same hashsum), just the
tar.gz differ. Does anyone know why, and how to fix it? I tried
non-default compression levels for gzip with git-archive, but that
didn't help to get an identical tar.gz like the one from github.
I'd like to avoid having my upstream downloading the github-created
tarball, verify it and then upload this signature.
Greets
jre
[3]The "alternative local workflow" described at
https://wiki.debian.org/Creating%20signed%20GitHub%20releases
