Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Mon, Mar 09, 2015 at 03:39:29PM -0400, Todd Zullinger wrote: Those on the list can check the PGP signature on the announcement mail and then use the included SHA1 to check the tarball, but doing that as a non-list member isn't as easy due to many list archives stripping or mangling PGP signatures. I tried doing this with the 0.11 announcement from the Mailman and Gmane archives now and wasn't successful. It turns out that GMane mangles the list address in the message, so it is possible to validate it but it's not straightforward: curl http://article.gmane.org/gmane.comp.version-control.cgit/2387/raw | sed -e 's/cgit[^ ]*@public.gmane.org/cgit@lists.zx2c4.com/' | gpg --verify Posting a detached PGP signature for the tarball would improve the ability for users to trust and verify the cgit tarball. It's not a blocker for your patch, but it would make it significantly more useful, so I thought I would broach the subject. ;) It seems that Jason currently relies on CGit to generate the tarballs by pointing to http://git.zx2c4.com/cgit/refs/tags, which means that a signature isn't guaranteed to remain correct (Git has subtly changed the tar encoding in the past and could do so again). There's a recent thread on the Git mailing list about a way to handle this better[0], but there isn't any code yet AFAIK. [0] http://thread.gmane.org/gmane.comp.version-control.git/264533 ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
John Keeping wrote: On Sat, Mar 07, 2015 at 06:35:10PM -0500, Todd Zullinger wrote: But while we're on the subject, are there PGP signatures available for the cgit tarballs themselves? I know the git tags are signed, but I don't think I've seen detached signatures for the tarballs. In this case, how does a user become happy that the CGit distribution they have is trustworthy? The cgit tarball download isn't available via https either, which might be a reasonable answer in the absence of a detached git signature. Without a signature on the tarball or some other method to verify the cgit tarball, the sha256 of the git tarball included in the cgit Makefile is more or less only useful as a basic download integrity check (in which case sha256 is mild overkill). None of this is to say that this patch isn't a step in the right direction. It certainly helps to display a nicer error message if a user receives a corrupted git tarball. It's just important that users don't confuse this with providing any real authentication of the git tarball. I'm not sure this is true. Providing that the CGit tarball is trusted, then I think this does provide sufficient authentication of the Git tarball. If the CGit tarball isn't trusted, then all bets are off anyway. Agreed. The caveat is that I'm not sure there is a convenient method for end-users or packagers to verify the authenticity of a cgit tarball. Those on the list can check the PGP signature on the announcement mail and then use the included SHA1 to check the tarball, but doing that as a non-list member isn't as easy due to many list archives stripping or mangling PGP signatures. I tried doing this with the 0.11 announcement from the Mailman and Gmane archives now and wasn't successful. Posting a detached PGP signature for the tarball would improve the ability for users to trust and verify the cgit tarball. It's not a blocker for your patch, but it would make it significantly more useful, so I thought I would broach the subject. ;) Thank you for all of your work on cgit. It's very nice to see it continue to improve, with even the smallest details getting attention. -- ToddOpenPGP - KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp ~~ Now don't say you can't swear off drinking; it's easy. I've done it a thousand times. -- W.C. Fields ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
Oh, hah, my pipermail does the same. That's annoying. I'll change up the release announcement next time to avoid that. On Mar 9, 2015 11:32 PM, Jason A. Donenfeld ja...@zx2c4.com wrote: On Mar 9, 2015 9:49 PM, John Keeping j...@keeping.me.uk wrote: It turns out that GMane mangles the list address in the message, Better archives: http://lists.zx2c4.com/pipermail/cgit/ ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Mar 8, 2015 12:35 AM, Todd Zullinger t...@pobox.com wrote: But while we're on the subject, are there PGP signatures available for the cgit tarballs themselves? I include a sha256 of the tarball in the announcement emails. Those emails are pgp signed. My pgp key is embedded in the repo, as well, and it's verifiable that all announce emails have been signed with the same key. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
Jason A. Donenfeld wrote: On Mar 8, 2015 12:35 AM, Todd Zullinger t...@pobox.com wrote: But while we're on the subject, are there PGP signatures available for the cgit tarballs themselves? I include a sha256 of the tarball in the announcement emails. Those emails are pgp signed. My pgp key is embedded in the repo, as well, and it's verifiable that all announce emails have been signed with the same key. (It's a SHA1, isn't it? Not that I care terribly about that part, other than a general preference for SHA256. :) More importantly is that verifying the PGP signature from an archive is not always easy. More often than not, list archives introduce subtle whitespace damage or worse. The other point that John made is more interesting. If cgit generates a tarball on demand, aren't there opportunities for the hash in the announcement mail (or a detactch signature) to become invalid? I belive that git archive has made changes in the past to avoid including the timestamp in the gzip archive, which helps. I don't know if there are other ways this could change. In the end, I don't know if it's a problem that can be solved in a way that doesn't cause more work for you as a maintainer or the other fine folks who are contributing. That's certainly not my intention. ;) On Mar 9, 2015 9:49 PM, John Keeping j...@keeping.me.uk wrote: It turns out that GMane mangles the list address in the message, Better archives: http://lists.zx2c4.com/pipermail/cgit/ I tried that earlier, before posting and found that it munges things too. Mailman's munging is often due to whitespace changes and are hard to avoid. Maybe the change to hyperkitty in Mailman 3 will improve this aspect of the archives. ;) -- ToddOpenPGP - KeyID: 0xBEAF0CE3 | URL: www.pobox.com/~tmz/pgp ~~ Damn you and your estrogenical treachery! -- Stewie Griffin ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Sat, Mar 07, 2015 at 06:35:10PM -0500, Todd Zullinger wrote: John Keeping wrote: I still think we can't rely on `gpg --recv-keys` though, we would have to distribute the key with CGit and possible also do something to avoid importing it into the user's keyring by default. If the check was to be run from a cgit clone, the key Junio uses to sign git tarballs could be included as a blob, similarly to how it's done in git.git. My assumption is that if people have cloned CGit then they will probably clone Git as well, at which point they check out an explicit SHA-1. (See the junio-gpg-pub tag in git.git for anyone unfamiliar with this already. The key can be extracted via: git cat-file blob junio-gpg-pub I've always thought that was a neat use of git, but certainly not a common one. I can't manage to make github display this tagged blob, which is also amusing. The cgit-hosted kernel.org repo displays it easily though: http://git.kernel.org/cgit/git/git.git/tag/?id=junio-gpg-pub) This method does nothing for users who have downloaded a cgit tarball, of course, which I expect is more likely to be the use case you're targeting. Precisely. I think a hash is more appropriate for the situation we're in - we are assuming that the user is happy that the CGit distribution they have is trustworthy but we must verify that the Git distribution we download is also correct. I don't think this is unreasonable at all. Trust has to start somewhere. For users that want to go to the source, they can always download git directly (or just the detached PGP signature) and verify the tarball. When I updated cgit packages in Fedora and EPEL, this is what I always did. I don't know if the current maintainers follow that process still, but hopefully they do. ;) But while we're on the subject, are there PGP signatures available for the cgit tarballs themselves? I know the git tags are signed, but I don't think I've seen detached signatures for the tarballs. In this case, how does a user become happy that the CGit distribution they have is trustworthy? The cgit tarball download isn't available via https either, which might be a reasonable answer in the absence of a detached git signature. Without a signature on the tarball or some other method to verify the cgit tarball, the sha256 of the git tarball included in the cgit Makefile is more or less only useful as a basic download integrity check (in which case sha256 is mild overkill). None of this is to say that this patch isn't a step in the right direction. It certainly helps to display a nicer error message if a user receives a corrupted git tarball. It's just important that users don't confuse this with providing any real authentication of the git tarball. I'm not sure this is true. Providing that the CGit tarball is trusted, then I think this does provide sufficient authentication of the Git tarball. If the CGit tarball isn't trusted, then all bets are off anyway. ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Sat, 07 Mar 2015 at 18:02:59, John Keeping wrote:
[...]
I'm not sure what benefit it has if it's optional. Will anyone check?
Maybe we could do something like:
if type sha256sum /dev/null 21
then
sha256sum --check git.sha256sum $(GIT_FILE)
else
echo 2 'WARNING: sha256sum not found so we cannot verify'
echo 2 'WARNING: the integrity of the Git archive!'
fi
That is exactly what I meant by suggesting to make it optional. Sorry
for being vague...
On a related note, can we download a signature and use `gpg --verify`
instead (should probably be optional as well, to avoid a dependency on
GnuPG)?
I thought about that, but we'd have to embed a key with CGit for it to
work reliably and how do we choose what key to use (given that
individual Git archives are not signed - the list of SHA256 checksums
is)?
Huh? This works fine for me:
$ gpg --recv-keys 96AFE6CB
gpg: key 713660A7: public key Junio C Hamano gits...@pobox.com imported
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: Total number processed: 1
gpg: imported: 1
$ curl -OO
https://www.kernel.org/pub/software/scm/git/git-2.3.2.tar.{xz,sign}
% Total% Received % Xferd Average Speed TimeTime Time
Current
Dload Upload Total SpentLeft
Speed
100 3529k 100 3529k0 0 1133k 0 0:00:03 0:00:03 --:--:--
1133k
100 543 100 5430 0 3404 0 --:--:-- --:--:-- --:--:--
3404
$ unxz git-2.3.2.tar.xz
$ gpg --verify git-2.3.2.tar.sign
gpg: assuming signed data in 'git-2.3.2.tar'
gpg: Signature made Sat 07 Mar 2015 12:10:41 AM CET using RSA key ID
96AFE6CB
gpg: Good signature from Junio C Hamano gits...@pobox.com [unknown]
gpg: aka Junio C Hamano j...@google.com [unknown]
gpg: aka Junio C Hamano ju...@pobox.com [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 96E0 7AF2 5771 9559 80DA D100 20D0 4E5A 7136 60A7
Subkey fingerprint: E1F0 36B1 FEE7 221F C778 ECEF B0B5 E886 96AF E6CB
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Sat, Mar 07, 2015 at 06:49:32PM +0100, Lukas Fleischer wrote:
On Sat, 07 Mar 2015 at 18:02:59, John Keeping wrote:
[...]
I'm not sure what benefit it has if it's optional. Will anyone check?
Maybe we could do something like:
if type sha256sum /dev/null 21
then
sha256sum --check git.sha256sum $(GIT_FILE)
else
echo 2 'WARNING: sha256sum not found so we cannot verify'
echo 2 'WARNING: the integrity of the Git archive!'
fi
That is exactly what I meant by suggesting to make it optional. Sorry
for being vague...
On a related note, can we download a signature and use `gpg --verify`
instead (should probably be optional as well, to avoid a dependency on
GnuPG)?
I thought about that, but we'd have to embed a key with CGit for it to
work reliably and how do we choose what key to use (given that
individual Git archives are not signed - the list of SHA256 checksums
is)?
Huh? This works fine for me:
$ gpg --recv-keys 96AFE6CB
gpg: key 713660A7: public key Junio C Hamano gits...@pobox.com
imported
gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model
gpg: depth: 0 valid: 1 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: Total number processed: 1
gpg: imported: 1
$ curl -OO
https://www.kernel.org/pub/software/scm/git/git-2.3.2.tar.{xz,sign}
% Total% Received % Xferd Average Speed TimeTime Time
Current
Dload Upload Total SpentLeft
Speed
100 3529k 100 3529k0 0 1133k 0 0:00:03 0:00:03 --:--:--
1133k
100 543 100 5430 0 3404 0 --:--:-- --:--:-- --:--:--
3404
$ unxz git-2.3.2.tar.xz
Oh, I tried this earlier but completely missed the fact that the
signature was on the bare tar file not one of the compressed versions.
I still think we can't rely on `gpg --recv-keys` though, we would have
to distribute the key with CGit and possible also do something to avoid
importing it into the user's keyring by default.
I think a hash is more appropriate for the situation we're in - we are
assuming that the user is happy that the CGit distribution they have is
trustworthy but we must verify that the Git distribution we download is
also correct.
$ gpg --verify git-2.3.2.tar.sign
gpg: assuming signed data in 'git-2.3.2.tar'
gpg: Signature made Sat 07 Mar 2015 12:10:41 AM CET using RSA key ID
96AFE6CB
gpg: Good signature from Junio C Hamano gits...@pobox.com [unknown]
gpg: aka Junio C Hamano j...@google.com [unknown]
gpg: aka Junio C Hamano ju...@pobox.com [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the
owner.
Primary key fingerprint: 96E0 7AF2 5771 9559 80DA D100 20D0 4E5A 7136
60A7
Subkey fingerprint: E1F0 36B1 FEE7 221F C778 ECEF B0B5 E886 96AF
E6CB
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
Re: [PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
On Sat, 07 Mar 2015 at 15:46:41, John Keeping wrote: This requires that we save the downloaded file explicitly rather than piping it straight to tar, but that is advisable anyway since it allows us to check the exit status of curl and make sure that we have downloaded the file successfully. Also add a test to make sure we don't forget to update the file when updating our Git version in the future. Signed-off-by: John Keeping j...@keeping.me.uk --- Makefile | 8 ++-- git.sha256sum| 1 + tests/t0001-validate-git-versions.sh | 11 +++ 3 files changed, 18 insertions(+), 2 deletions(-) create mode 100644 git.sha256sum [...] I like the idea, however, sha256sum is not available on all platforms. This breaks `make get-git` under OpenBSD, for example (OpenBSD has a utility called sha256 with a different command line interface). Maybe we can make the check optional, though? On a related note, can we download a signature and use `gpg --verify` instead (should probably be optional as well, to avoid a dependency on GnuPG)? ___ CGit mailing list CGit@lists.zx2c4.com http://lists.zx2c4.com/mailman/listinfo/cgit
[PATCH] Check SHA256 sum of git-$VER.tar.gz after downloading
This requires that we save the downloaded file explicitly rather than
piping it straight to tar, but that is advisable anyway since it allows
us to check the exit status of curl and make sure that we have
downloaded the file successfully.
Also add a test to make sure we don't forget to update the file when
updating our Git version in the future.
Signed-off-by: John Keeping j...@keeping.me.uk
---
Makefile | 8 ++--
git.sha256sum| 1 +
tests/t0001-validate-git-versions.sh | 11 +++
3 files changed, 18 insertions(+), 2 deletions(-)
create mode 100644 git.sha256sum
diff --git a/Makefile b/Makefile
index ed329e8..807879f 100644
--- a/Makefile
+++ b/Makefile
@@ -15,7 +15,8 @@ pdfdir = $(docdir)
mandir = $(prefix)/share/man
SHA1_HEADER = openssl/sha.h
GIT_VER = 2.3.2
-GIT_URL = https://www.kernel.org/pub/software/scm/git/git-$(GIT_VER).tar.gz
+GIT_FILE = git-$(GIT_VER).tar.gz
+GIT_URL = https://www.kernel.org/pub/software/scm/git/$(GIT_FILE)
INSTALL = install
COPYTREE = cp -r
MAN5_TXT = $(wildcard *.5.txt)
@@ -146,7 +147,10 @@ clean-doc:
$(RM) cgitrc.5 cgitrc.5.html cgitrc.5.pdf cgitrc.5.xml cgitrc.5.fo
get-git:
- curl -L $(GIT_URL) | tar -xzf - rm -rf git mv git-$(GIT_VER) git
+ curl -L $(GIT_URL) --output $(GIT_FILE) \
+ sha256sum --check git.sha256sum \
+ tar -xzf $(GIT_FILE) \
+ rm -rf git mv git-$(GIT_VER) git
tags:
$(QUIET_TAGS)find . -name '*.[ch]' | xargs ctags
diff --git a/git.sha256sum b/git.sha256sum
new file mode 100644
index 000..1214d3d
--- /dev/null
+++ b/git.sha256sum
@@ -0,0 +1 @@
+a35aea3a0f63f4cc3dd38fa32127e97273f335a14ea2586b649eb759ecf675a3
git-2.3.2.tar.gz
diff --git a/tests/t0001-validate-git-versions.sh
b/tests/t0001-validate-git-versions.sh
index a65b35e..3325c77 100755
--- a/tests/t0001-validate-git-versions.sh
+++ b/tests/t0001-validate-git-versions.sh
@@ -9,6 +9,12 @@ test_expect_success 'extract Git version from Makefile' '
s/^GIT_VER[ ]*=[]*//
p
} ../../Makefile makefile_version
+ GIT_VER=$(cat makefile_version)
+ sed -n -e /^GIT_FILE[ ]*=/ {
+ s/^GIT_FILE[]*=[]*//
+ s/\$(GIT_VER)/$GIT_VER/
+ p
+ } ../../Makefile makefile_file
'
# Note that Git's GIT-VERSION-GEN script applies s/-/./g to the version
@@ -38,4 +44,9 @@ test_expect_success 'test submodule version matches Makefile'
'
fi
'
+test_expect_success 'git.sha256sum version matches Makefile' '
+ sed -e s/[0-9a-z]* *// ../../git.sha256sum sha256sum_file
+ test_cmp sha256sum_file makefile_file
+'
+
test_done
--
2.3.1.308.g754cd77
___
CGit mailing list
CGit@lists.zx2c4.com
http://lists.zx2c4.com/mailman/listinfo/cgit
