Humppa Validation
Humppa Validation While testing out the ami.c patch for dlg@, I once again needed validate files are being written correctly. Keeping with the OpenBSD theme, I used the MP3 files created from my Elakelaiset Werbung Baby CD for testing out the 480GB RAID5 volume. Once you've got your humppa on a BSD box cksum(1) and friends can take care of creating message digests (cksum, sum, sysvsum, md4, md5, rmd160, sha1, sha256, sha384. sha512), storing them into a text file and then validating the original files were written correctly. $ md5 *.mp3 bsdmd5.md5 You end up with a file in the standard format we all know and love: MD5 (01-Hump.mp3) = 1ecc9c0165043bda9ac07ae35a627cc4 MD5 (02-Humppakonehumppa.mp3) = 0c1cb2e6347b541830125dfcc41a6f00 MD5 (03-Humppamedia.mp3) = d0ee034280d835894e859258a5df4d8f MD5 (04-Punakka_Humppa.mp3) = b768d3491261f1b256ba2d0d70167b3d MD5 (05-Peljatty_Humppa.mp3) = a789d1f7ad6b979f9dd6d4d07e5f6849 Of course the above command is the same thing as: $ cksum -a MD5 *.mp3 cksum.md5 The problems start if you're dumb enough to keep the original copy of your humppa on something other than a BSD system... Since you can install OpenSSL on a number of different types of systems, my first thought was to use OpenSSL to generate the checksum files. $ find . -type f -name '*.mp3' -print0 | xargs -0 openssl md5 -out ossl.md5 That get's you fairly close but the resulting format is wrong. MD5(./01-Hump.mp3)= 1ecc9c0165043bda9ac07ae35a627cc4 MD5(./02-Humppakonehumppa.mp3)= 0c1cb2e6347b541830125dfcc41a6f00 MD5(./03-Humppamedia.mp3)= d0ee034280d835894e859258a5df4d8f MD5(./04-Punakka_Humppa.mp3)= b768d3491261f1b256ba2d0d70167b3d MD5(./05-Peljatty_Humppa.mp3)= a789d1f7ad6b979f9dd6d4d07e5f6849 Notice we're missing two of the required spaces as per cksum(1) Sure, you can edit the above rather easily to produce the correct format for BSD md5/cksum but why should we be doing that all of the time. The GNU has their own incomplete and incompatible file format for MD5 checksums which is used with their ``md5sum'' tool. $ md5sum *.mp3 md5sum.md5 Which results in: 1ecc9c0165043bda9ac07ae35a627cc4 *01-Hump.mp3 0c1cb2e6347b541830125dfcc41a6f00 *02-Humppakonehumppa.mp3 d0ee034280d835894e859258a5df4d8f *03-Humppamedia.mp3 b768d3491261f1b256ba2d0d70167b3d *04-Punakka_Humppa.mp3 a789d1f7ad6b979f9dd6d4d07e5f6849 *05-Peljatty_Humppa.mp3 The gnu md5sum tool lacks designation of the message digest used. Would it be worthwhile to add a format switch (maybe -f) to cksum so we can handle different file formats? $ cksum -f openssl -c ossl.md5 $ cksum -f md5sum -c md5sum.md5 Is there some unstated reasoning why we don't support the other formats? Since the format we use in OpenBSD quite nice: Would it be better to fix/augment the output of OpenSSL? Would it be better to fix/augment the output of md5sum? Is there a better way to handle cross-os file validation? And lastly NO, I don't actually believe my liver would survive listening to 480GB of Humppa MP3's but the experiment might be, well, intoxicating. (; Thanks, JCR -- Free, Open Source CAD, CAM and EDA Tools http://www.DesignTools.org
Re: Humppa Validation
On Wed, May 31, 2006 at 10:58:42PM -0700, J.C. Roberts wrote: The problems start if you're dumb enough to keep the original copy of your humppa on something other than a BSD system... Since you can install OpenSSL on a number of different types of systems, my first thought was to use OpenSSL to generate the checksum files. $ find . -type f -name '*.mp3' -print0 | xargs -0 openssl md5 -out ossl.md5 That get's you fairly close but the resulting format is wrong. why not use openssl on both platforms? jmc
Re: Humppa Validation
On Thu, 1 Jun 2006 10:11:07 +0100, Jason McIntyre [EMAIL PROTECTED] wrote: On Wed, May 31, 2006 at 10:58:42PM -0700, J.C. Roberts wrote: The problems start if you're dumb enough to keep the original copy of your humppa on something other than a BSD system... Since you can install OpenSSL on a number of different types of systems, my first thought was to use OpenSSL to generate the checksum files. $ find . -type f -name '*.mp3' -print0 | xargs -0 openssl md5 -out ossl.md5 That get's you fairly close but the resulting format is wrong. why not use openssl on both platforms? jmc I've read through the openssl man page more than a few times looking for the required magic to input such a text file and get openssl to validate the listed files but no such luck. As far as I know, openssl is only able to produce the output file but is unable to accept said file as input. To make sure the files were transferred correctly to the openbsd box, I resorted to editing the openssl output file so it could be used with cksum/md5. jcr -- Free, Open Source CAD, CAM and EDA Tools http://www.DesignTools.org
Re: Humppa Validation
J.C. Roberts 1-Jun-06 06:58 : Sure, you can edit the above rather easily to produce the correct format for BSD md5/cksum but why should we be doing that all of the time. Don't edit it manually, use a one-line sed, awk or perl script. Would it be worthwhile to add a format switch (maybe -f) to cksum so we can handle different file formats? $ cksum -f openssl -c ossl.md5 $ cksum -f md5sum -c md5sum.md5 Is there some unstated reasoning why we don't support the other formats? Yes. Each tool should do one thing, and do it well. cksum does. Having multiple different formats in the program, particularly those that can be generated from each other by simple sed scripts, is insane. Sorry. Tom (If you really want this, you could write a shell script that does exactly that - even taking your suggested -f option.)
Re: Humppa Validation
On Thu, 1 Jun 2006 10:59:56 +0100 (BST), Tom Cosgrove [EMAIL PROTECTED] wrote: Sure, you can edit the above rather easily to produce the correct format for BSD md5/cksum but why should we be doing that all of the time. Don't edit it manually, use a one-line sed, awk or perl script. yep, exactly what I did. Would it be worthwhile to add a format switch (maybe -f) to cksum so we can handle different file formats? $ cksum -f openssl -c ossl.md5 $ cksum -f md5sum -c md5sum.md5 Is there some unstated reasoning why we don't support the other formats? Yes. Each tool should do one thing, and do it well. cksum does. Having multiple different formats in the program, particularly those that can be generated from each other by simple sed scripts, is insane. Sorry. Tom (If you really want this, you could write a shell script that does exactly that - even taking your suggested -f option.) I see you point. My suggestion kind of seems like feature creep. jcr -- Free, Open Source CAD, CAM and EDA Tools http://www.DesignTools.org
Re: Humppa Validation
On Thu, 1 Jun 2006 11:19:43 +0059, Jason McIntyre [EMAIL PROTECTED] wrote: On Thu, Jun 01, 2006 at 10:59:56AM +0100, Tom Cosgrove wrote: Is there some unstated reasoning why we don't support the other formats? Yes. Each tool should do one thing, and do it well. cksum does. Having multiple different formats in the program, particularly those that can be generated from each other by simple sed scripts, is insane. Sorry. still, it's odd that openssl appears to have no way to do something similar...(the checklist bit, i mean). jmc The interesting thought is since openssl is unable to accept as input the checksum/digest output file it created, it almost makes some sense to alter the format of openssl output (-out) file so it can be used with cksum and friends. The unanswerable question is how many things count on the current openssl output format and would the addition of missing two spaces break them? jcr -- Free, Open Source CAD, CAM and EDA Tools http://www.DesignTools.org
Re: Humppa Validation
J.C. Roberts wrote: MD5 (01-Hump.mp3) = 1ecc9c0165043bda9ac07ae35a627cc4 MD5 (02-Humppakonehumppa.mp3) = 0c1cb2e6347b541830125dfcc41a6f00 MD5 (03-Humppamedia.mp3) = d0ee034280d835894e859258a5df4d8f MD5 (04-Punakka_Humppa.mp3) = b768d3491261f1b256ba2d0d70167b3d MD5 (05-Peljatty_Humppa.mp3) = a789d1f7ad6b979f9dd6d4d07e5f6849 [snip] The problems start if you're dumb enough to keep the original copy of your humppa on something other than a BSD system... [snip] Is there a better way to handle cross-os file validation? Check out Perl's Digest::MD5 module, specifically the addfile/hexdigest methods and associated example code. You could use that to generate or validate bsd-style /bin/md5 output on other platforms, including Win32. -- Stephen Takacs [EMAIL PROTECTED] http://perlguru.net/ 4149 FD56 D078 C988 9027 1EB4 04CC F80F 72CB 09DA
