Re: Formats of SHA/MD5 checksums
On Fri, 30 Nov 2012, Roman Shaposhnik wrote: Date: Fri, 30 Nov 2012 03:05:15 +0100 From: Roman Shaposhnik r...@apache.org To: general@incubator.apache.org, infrastruct...@apache.org Subject: Re: Formats of SHA/MD5 checksums Sender: shaposh...@gmail.com On Sun, Nov 25, 2012 at 9:29 PM, Roman Shaposhnik r...@apache.org wrote: On Tue, Nov 20, 2012 at 3:50 PM, sebb seb...@gmail.com wrote: Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 I assume 'it' is md5 cheksum files generated with gpg --print-md MD5 [fileName] [fileName].md5 I am +1 on suggesting (on that page) a 'normal' form for the content of a .md5 file. I am definitedly -1 on removing the gpg line above, or suggesting that only one form of .md5 files is allowed. The reason given I ended up writing a Perl script doesn't make sense ; .md5 files come in many forms but the algorithm to verify is the same for all of them (there are no 'variations.') : verify (checksum md5, .md5-file fff) : -- tmp = lowercase cat fff -- md5 = lowercase cat md5 -- squeeze non-hex ([^a-f0-9]) out of tmp (and md5) -- match md5 ~ tmp HPP _ Henk P. Penning, ICT-beta R Uithof WISK-412 _/ \_ Faculty of Science, Utrecht UniversityT +31 30 253 4106 / \_/ \ Budapestlaan 6, 3584CD Utrecht, NLF +31 30 253 4553 \_/ \_/ http://www.staff.science.uu.nl/~penni101/ M penn...@uu.nl \_/ - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Roman Shaposhnik r...@apache.org wrote: +infra Ping! I would really like this annoyance to be resolved one way or the other. Could somebody more experienced with Apache web properties answer the question? Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? No vote is required to change that web page. Those pages are infra owned but any member can change them. Since infra owns the page, it would be polite to suggest the alternative text on infrastructure@. Once that is agreed, any member can apply it. With my infra hat on, I have no objection to standardising on the md5sum format for MD5 checksums. A couple of side issues: Firstly, you removed far to much context when adding infra to the thread. I had to go digging through the archives to figure out what the problem was. Secondly, votes are not the way to drive change. Discussion is. The process is not: - see an issue - pick a solution - start a vote for implementing that solution - drive through the change but: - see an issue - discuss options to fix it - agree the way forward (consensus) - make the change Releases and new committers are pretty much the only time I'd expect to see votes in an Apache community. Mark - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
That page is part of the Apache CMS and ASF members can edit that page by using the following http://www.apache.org/dev/cms.html#usage. Non ASF members can create a ticket within jira under the infra project and attach a patch for the changes they would like to make. The repo and file you are looking for is https://svn.apache.org/repos/asf/infrastructure/site/trunk/content/dev/release-signing.mdtext -Jake Roman Shaposhnik November 29, 2012 9:05 PM +infra Ping! I would really like this annoyance to be resolved one way or the other. Could somebody more experienced with Apache web properties answer the question? Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. On Sun, Nov 25, 2012 at 9:29 PM, Roman Shaposhnik r...@apache.org wrote: On Tue, Nov 20, 2012 at 3:50 PM, sebb seb...@gmail.com wrote: Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman.
Re: Formats of SHA/MD5 checksums
Henk P. Penning wrote on Fri, Nov 30, 2012 at 10:08:33 +0100:
The reason given I ended up writing a Perl script doesn't
make sense ; .md5 files come in many forms but the algorithm
to verify is the same for all of them (there are no 'variations.') :
verify (checksum md5, .md5-file fff) :
-- tmp = lowercase cat fff
-- md5 = lowercase cat md5
-- squeeze non-hex ([^a-f0-9]) out of tmp (and md5)
md5(1) on FreeBSD produces the literal text MD5 as part of its output.
md5sum(1) on Linux prints the filename as part of its output. The
filename usually contains a hex digit (such as the a in tar or d
in hadoop).
Therefore just stripping non-hex-digits won't work with the standard md5
computation tools on those two platforms.
-- match md5 ~ tmp
HPP
_
Henk P. Penning, ICT-beta R Uithof WISK-412 _/ \_
Faculty of Science, Utrecht UniversityT +31 30 253 4106 / \_/ \
Budapestlaan 6, 3584CD Utrecht, NLF +31 30 253 4553 \_/ \_/
http://www.staff.science.uu.nl/~penni101/ M penn...@uu.nl \_/
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Jake Farrell wrote on Thu, Nov 29, 2012 at 22:02:16 -0500: That page is part of the Apache CMS and ASF members can edit that page by using the following http://www.apache.org/dev/cms.html#usage. Non ASF members can create a ticket within jira under the infra project and attach a patch for the changes they would like to make. Actually, if there's consensus, any ASF member can edit the page directly (but not publish the change). The repo and file you are looking for is https://svn.apache.org/repos/asf/infrastructure/site/trunk/content/dev/release-signing.mdtext -Jake Roman Shaposhnik mailto:r...@apache.org November 29, 2012 9:05 PM +infra Ping! I would really like this annoyance to be resolved one way or the other. Could somebody more experienced with Apache web properties answer the question? Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. On Sun, Nov 25, 2012 at 9:29 PM, Roman Shaposhnikr...@apache.org wrote: On Tue, Nov 20, 2012 at 3:50 PM, sebbseb...@gmail.com wrote: Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Daniel Shahaf wrote on Sat, Dec 01, 2012 at 07:25:54 +0200: Jake Farrell wrote on Thu, Nov 29, 2012 at 22:02:16 -0500: That page is part of the Apache CMS and ASF members can edit that page by using the following http://www.apache.org/dev/cms.html#usage. Non ASF members can create a ticket within jira under the infra project and attach a patch for the changes they would like to make. Actually, if there's consensus, any ASF member can edit the page directly (but not publish the change). s/member/committer/ - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Fri, Nov 30, 2012 at 1:08 AM, Henk P. Penning penn...@uu.nl wrote: I am +1 on suggesting (on that page) a 'normal' form for the content of a .md5 file. I'll take a crack at it now that I know where the source is ;-) I am definitedly -1 on removing the gpg line above, or suggesting that only one form of .md5 files is allowed. Why? The output it generates is not really similar to anything else and can be confusing. Is there a platform where GPG can be the only tool capable of generating MD5/SHA? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Thu, Nov 29, 2012 at 11:49 PM, Mark Thomas ma...@apache.org wrote: Releases and new committers are pretty much the only time I'd expect to see votes in an Apache community. Understood. My question was specifically about the process used by ASF to manage its top level web pages. Now I know. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Hi all, Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 I am fine with a change of the format. But at the moment we (Celix) still have a pending release. Seeing that many other project use different formats, I personally don't see this as a show stopper for our current release.. Can we somehow reach a consensus that for a next release the format will be different? (ie the format used by md5sum). -- Met vriendelijke groet, Alexander Broekhuis
Re: Formats of SHA/MD5 checksums
On Thu, Nov 29, 2012 at 12:12 AM, Alexander Broekhuis a.broekh...@gmail.com wrote: I am fine with a change of the format. But at the moment we (Celix) still have a pending release. Seeing that many other project use different formats, I personally don't see this as a show stopper for our current release.. Can we somehow reach a consensus that for a next release the format will be different? (ie the format used by md5sum). I think that would be a fine choice. I'm fine with releasing it as is for now +1 (binding). That said -- I'd like to see the next release take into account the feedback that has been provided to the project so far. Nothing there is blocking, but not taking it into account would, in my opinion, make it more difficult to review and thus diminish the chances of getting enough eyeballs to look at it in time. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
+infra Ping! I would really like this annoyance to be resolved one way or the other. Could somebody more experienced with Apache web properties answer the question? Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. On Sun, Nov 25, 2012 at 9:29 PM, Roman Shaposhnik r...@apache.org wrote: On Tue, Nov 20, 2012 at 3:50 PM, sebb seb...@gmail.com wrote: Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Nov 30, 2012, at 3:03 AM, Roman Shaposhnik r...@apache.org wrote: On Thu, Nov 29, 2012 at 12:12 AM, Alexander Broekhuis a.broekh...@gmail.com wrote: I am fine with a change of the format. But at the moment we (Celix) still have a pending release. Seeing that many other project use different formats, I personally don't see this as a show stopper for our current release.. Can we somehow reach a consensus that for a next release the format will be different? (ie the format used by md5sum). I think that would be a fine choice. I'm fine with releasing it as is for now +1 (binding). Thanks. That said -- I'd like to see the next release take into account the feedback that has been provided to the project so far. Nothing there is blocking, but not taking it into account would, in my opinion, make it more difficult to review and thus diminish the chances of getting enough eyeballs to look at it in time. +1 For a very first release attempt, I think the project did a good job, and it would definitely demonstrate getting the Apache way by taking all the suggestions and incorporating them in the next release. Yes, the incubator rules are not always written down well/correctly so releasing something will always trigger some amount of discussion. In that sense the incubator itself is also constantly improving. Greetings, Marcel - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Hi, I think that would be a fine choice. I'm fine with releasing it as is for now +1 (binding). Thanks! Could you post your vote to the formal release thread as well? That said -- I'd like to see the next release take into account the feedback that has been provided to the project so far. Nothing there is blocking, but not taking it into account would, in my opinion, make it more difficult to review and thus diminish the chances of getting enough eyeballs to look at it in time. Agreed. Most of the issues are already fixed in trunk. -- Met vriendelijke groet, Alexander Broekhuis
Re: Formats of SHA/MD5 checksums
Roman Shaposhnik wrote on Thu, Nov 29, 2012 at 18:05:15 -0800: +infra Ping! I would really like this annoyance to be resolved one way or the other. Could somebody more experienced with Apache web properties answer the question? Question: how do we go about discouraging it then? Do we need a vote Don't know. What's it ? to modify the content of: http://www.apache.org/dev/release-signing#md5 Ask your question in terms of release practices / policies, not in terms of web pages documenting them, please. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Tue, Nov 20, 2012 at 3:50 PM, sebb seb...@gmail.com wrote: Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Question: how do we go about discouraging it then? Do we need a vote to modify the content of: http://www.apache.org/dev/release-signing#md5 Or even more basic question -- where's the source for that webpage? Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
Hi all, while reviewing Celix RC I came across a peculiar format for the SHA/MD5 checksums which I've never seen before: https://dist.apache.org/repos/dist/dev/incubator/celix/celix-0.0.1-incubating/ There are several other projects who do use the same tools. Cloudstack even list on their site how to verify the hashes: http://incubator.apache.org/cloudstack/docs/en-US/Apache_CloudStack/4.0.0-incubating/html/Installation_Guide/sect-source-verify.html So here's the question: are we making it less likely for folks to actually verify MD5/SHA checksums if we don't have a consistent format for them? +1 to this question -- Met vriendelijke groet, Alexander Broekhuis
Re: Formats of SHA/MD5 checksums
On Tue, Nov 20, 2012 at 12:27 PM, Ted Dunning ted.dunn...@gmail.com wrote: What does gpg --verify do? http://www.gnupg.org/download/integrity_check.en.html It verifies the GPG sig (as it should). I wasn't able to find (well, by spending 2 minutes searching through the man page on my Linux) an easy way to ask it to verify its own output in case of MD5/SHA. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Tue, Nov 20, 2012 at 12:22 PM, Roman Shaposhnik r...@apache.org wrote: Hi! while reviewing Celix RC I came across a peculiar format for the SHA/MD5 checksums which I've never seen before: https://dist.apache.org/repos/dist/dev/incubator/celix/celix-0.0.1-incubating/ Turns out, this is the output of gpg and it seems that there's no way to ask gpg to verify it (unlike -c for md5sum lets say). Worse yet, because of the spaces it is not easy to reconcile the output with the more typical md5sum one. And yet we actually explicitly document gpg as one of the tools: http://www.apache.org/dev/release-signing#md5 So here's the question: are we making it less likely for folks to actually verify MD5/SHA checksums if we don't have a consistent format for them? FWIW, any platform with Perl on it can generate md5sum-compatible and shasum-compatible checksums using incantations like these: perl -MDigest -e '$d = Digest-new(MD5); \ open $fh, apache-lucy-0.4.0.tar.gz or die; \ $d-addfile($fh); print $d-hexdigest; print \ apache-lucy-0.4.0.tar.gz\n' apache-lucy-0.4.0.tar.gz.md5 perl -MDigest -e '$d = Digest-new(SHA-512); \ open $fh, apache-lucy-0.4.0.tar.gz or die; \ $d-addfile($fh); print $d-hexdigest; print \ apache-lucy-0.4.0.tar.gz\n' apache-lucy-0.4.0.tar.gz.sha (Perl is more widespread than md5sum.) Marvin Humphrey - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On Tue, Nov 20, 2012 at 1:51 PM, sebb seb...@gmail.com wrote: (Though installing it just for hash creation may be overkill). Maven generates hashes which are easy to check. It's easy to use Ant to generate hashes in a suitable format. And this is the crux of the question -- what IS suitable format? Personally, I find it difficult to verify the GPG generated checksums. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
Re: Formats of SHA/MD5 checksums
On 20 November 2012 23:22, Roman Shaposhnik r...@apache.org wrote: On Tue, Nov 20, 2012 at 1:51 PM, sebb seb...@gmail.com wrote: (Though installing it just for hash creation may be overkill). Maven generates hashes which are easy to check. It's easy to use Ant to generate hashes in a suitable format. And this is the crux of the question -- what IS suitable format? Ideally, whatever is supported by most (many) hash checking utilities. Personally, I find it difficult to verify the GPG generated checksums. Ditto. It's particularly awkward when the hash is wrapped over several lines. I ended up writing a Perl script to handle all the variations. If I'm not alone perhaps we should discourage the use of this format and modify the release FAQ page. +1 Thanks, Roman. - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org - To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org For additional commands, e-mail: general-h...@incubator.apache.org
