Re: checking integrity of already written CD/DVD
On 31.03.09 08:18, Paul E Condon wrote: A second comment: In my experience, the iso files that I download from Debian always have lengths that are integral multiples of 1024 bytes. I think there is already some padding going on in the creation of these files, so partial sectors in the iso is probably not an explanation for whatever difficulties one may be having in verifying a CD/DVD. (On doing a little quick research, I think the sector size on CD/DVD may be 2048 bytes. I don't make a claim for integral multiple of 2048 because that is not what I actually tested. I don't remember whether the integer was odd or even, just that there was no remainder.) Good, since the OP mentioned debian installation CD, this should be the point. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Nothing is fool-proof to a talented fool. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD more info
On 31.03.09 12:57, Paul E Condon wrote: I did an experiment with the drive that always truncated the dd read of the CD. The iso is lenny business card. This iso is 18133 blocks of 2048 bytes each. The read back using dd on the short read drive is 18104 2kblocks long, which is 29 block short of a full load. [...] I think this is good news for people who are unlucky enough to have only disk drives that give too short a read-back from a CD. shouldn't they replace their CD/DVD drives instead? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Atheism is a non-prophet organization. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD more info
On 2009-04-01_10:34:41, Matus UHLAR - fantomas wrote: On 31.03.09 12:57, Paul E Condon wrote: I did an experiment with the drive that always truncated the dd read of the CD. The iso is lenny business card. This iso is 18133 blocks of 2048 bytes each. The read back using dd on the short read drive is 18104 2kblocks long, which is 29 block short of a full load. [...] I think this is good news for people who are unlucky enough to have only disk drives that give too short a read-back from a CD. shouldn't they replace their CD/DVD drives instead? -- Matus UHLAR Everyone should replace their hardware whenever a new model becomes available, but some of us are slackers at supporting the world economy. ;-) The drives are serviceable for reading and burning CD/DVDs, just not so good at reading to EOF. The iso standard defines a format for data on disk that tells the reading software where exactly every datum is. In normal operation there is never any need to test for EOF. Maybe the reason that there is no confusion in more modern hardware is that there is only one factory left in the world actually making these drives, so there is de-facto consensus on how they should operate. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD
On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Is there a way to check a written DVD against the checksum of the iso image written on it? In 20090329202842.ga3...@think.homelan, Andrei Popescu wrote: $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso On 29.03.09 16:27, Boyd Stephen Smith Jr. wrote: Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. afaik, if the same image is written to multiple CDs/DVDs, they all should have the same md5sum, independently on its size. That is the one md5sum shjould report. The same for sha1sum. -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Save the whales. Collect the whole set. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD
On 29.03.09 16:27, Boyd Stephen Smith Jr. wrote: Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. On 31.03.09 09:53, Matus UHLAR - fantomas wrote: afaik, if the same image is written to multiple CDs/DVDs, they all should have the same md5sum, independently on its size. That is the one md5sum shjould report. The same for sha1sum. ... could this problem come out of fact that there was something different burned on those medias before? -- Matus UHLAR - fantomas, uh...@fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. A day without sunshine is like, night. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD
In 20090331075641.gb18...@fantomas.sk, Matus UHLAR - fantomas wrote: On 29.03.09 16:27, Boyd Stephen Smith Jr. wrote: Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. On 31.03.09 09:53, Matus UHLAR - fantomas wrote: afaik, if the same image is written to multiple CDs/DVDs, they all should have the same md5sum, independently on its size. That is the one md5sum shjould report. The same for sha1sum. While this may be true, the comparison asked about (and that I normally do) is between an ISO file and a physical DVD. ISO files is want to burn generally have not been produced by dd'ing a physical DVD, so they may not take into account the physical limitations on the medium. I would love it if 'md5sum /path/to/iso /dev/dvd' gave two lines with the same sum. It does not usually happen that way, here. I have to trim the /dev/dvd device to have exactly the same byte count as the iso file first. ... could this problem come out of fact that there was something different burned on those medias before? No, my experience is mainly with WORM optical media. -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: checking integrity of already written CD/DVD
On 2009-03-31_09:53:51, Matus UHLAR - fantomas wrote: On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Is there a way to check a written DVD against the checksum of the iso image written on it? In 20090329202842.ga3...@think.homelan, Andrei Popescu wrote: $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso On 29.03.09 16:27, Boyd Stephen Smith Jr. wrote: Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. afaik, if the same image is written to multiple CDs/DVDs, they all should have the same md5sum, independently on its size. That is the one md5sum shjould report. The same for sha1sum. Are you saying that two files that have different lengths (size) should have the same md5 or sha1? If you mean something else by size, ignore this post, but ... The design goal of both md5 and sha1 is to provide, for any file, a message digest that is different from that of any other file that is different from the first file in any way. If the file that is read from CD/DVD device is longer, or shorter, than the iso that was used to burn the CD/DVD then the two files are different in a way that is significant to the message digest idea. For two files to have the same message digest, they must be bit for bit identical. That means, at a very minimum, they must have the same length. The motivation for the invention of sha1 was that there was growing evidence that md5 was failing to meet the design goal of identical message digest only if the files are identical. IOW, a message digest algorithm must NOT ignore trailing zero bytes, or trailing garbage bytes that have no effect on the meaning of the file in its intended use. Trailing zero bytes are easy to truncate. If the truncate file has a matching message digest, one can be reasonably confident that the file with the trailing zeros will function properly. For trailing garbage bytes it is difficult to assert that those lost bytes at the end really are garbage that may safely be ignored. I have two CD/DVD devices one consistently reports longer files than the other on reading the same CD/DVD. Luckily, for me, for the one reporting the longer file, its reading is always longer than the length of the iso that I used to burn. I truncate the longer file to match the iso and a always get a matching message digest. If the message digest of the leading part of the file matches the whole of the iso, and if the iso is a well constructed image of what should be on a CD/DVD, then a CD/DVD reader should never look at those extra bytes that dd reports at the end. A second comment: In my experience, the iso files that I download from Debian always have lengths that are integral multiples of 1024 bytes. I think there is already some padding going on in the creation of these files, so partial sectors in the iso is probably not an explanation for whatever difficulties one may be having in verifying a CD/DVD. (On doing a little quick research, I think the sector size on CD/DVD may be 2048 bytes. I don't make a claim for integral multiple of 2048 because that is not what I actually tested. I don't remember whether the integer was odd or even, just that there was no remainder.) My problems with the beautiful one or two line checking script are indicative of a little extra complexity here. Manufacturers don't, apparently, guarantee reliable, accurate end-of-file checking. If you are unlucky and have hardware with unreliable EOF sensing, you need to take extra measures in verifying the accuracy of a burn. HTH -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD more info
On 2009-03-31_08:18:16, Paul E Condon wrote: On 2009-03-31_09:53:51, Matus UHLAR - fantomas wrote: On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Is there a way to check a written DVD against the checksum of the iso image written on it? In 20090329202842.ga3...@think.homelan, Andrei Popescu wrote: $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso On 29.03.09 16:27, Boyd Stephen Smith Jr. wrote: Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. afaik, if the same image is written to multiple CDs/DVDs, they all should have the same md5sum, independently on its size. That is the one md5sum shjould report. The same for sha1sum. Are you saying that two files that have different lengths (size) should have the same md5 or sha1? If you mean something else by size, ignore this post, but ... The design goal of both md5 and sha1 is to provide, for any file, a message digest that is different from that of any other file that is different from the first file in any way. If the file that is read from CD/DVD device is longer, or shorter, than the iso that was used to burn the CD/DVD then the two files are different in a way that is significant to the message digest idea. For two files to have the same message digest, they must be bit for bit identical. That means, at a very minimum, they must have the same length. The motivation for the invention of sha1 was that there was growing evidence that md5 was failing to meet the design goal of identical message digest only if the files are identical. IOW, a message digest algorithm must NOT ignore trailing zero bytes, or trailing garbage bytes that have no effect on the meaning of the file in its intended use. Trailing zero bytes are easy to truncate. If the truncate file has a matching message digest, one can be reasonably confident that the file with the trailing zeros will function properly. For trailing garbage bytes it is difficult to assert that those lost bytes at the end really are garbage that may safely be ignored. I have two CD/DVD devices one consistently reports longer files than the other on reading the same CD/DVD. Luckily, for me, for the one reporting the longer file, its reading is always longer than the length of the iso that I used to burn. I truncate the longer file to match the iso and a always get a matching message digest. If the message digest of the leading part of the file matches the whole of the iso, and if the iso is a well constructed image of what should be on a CD/DVD, then a CD/DVD reader should never look at those extra bytes that dd reports at the end. A second comment: In my experience, the iso files that I download from Debian always have lengths that are integral multiples of 1024 bytes. I think there is already some padding going on in the creation of these files, so partial sectors in the iso is probably not an explanation for whatever difficulties one may be having in verifying a CD/DVD. (On doing a little quick research, I think the sector size on CD/DVD may be 2048 bytes. I don't make a claim for integral multiple of 2048 because that is not what I actually tested. I don't remember whether the integer was odd or even, just that there was no remainder.) My problems with the beautiful one or two line checking script are indicative of a little extra complexity here. Manufacturers don't, apparently, guarantee reliable, accurate end-of-file checking. If you are unlucky and have hardware with unreliable EOF sensing, you need to take extra measures in verifying the accuracy of a burn. I did an experiment with the drive that always truncated the dd read of the CD. The iso is lenny business card. This iso is 18133 blocks of 2048 bytes each. The read back using dd on the short read drive is 18104 2kblocks long, which is 29 block short of a full load. Using /dev/zero, dd, and cat, I create another copy of lenny business that has 40 blocks of zeros appended at the end. Then I burn a CD from this iso file. Then I read back from this newly burnt CD. The read-back is 18168 blocks, which is quite a bit longer than the read-back of that of the original iso. It is 64 blocks longer which doesn't seem to add up. But notice that it is longer than the original iso, so I truncate this longer file to the same length as the original, and compute the md5 of this augmented-then-truncated file. This file has the same md5 as the original iso from Debian, so I think it is rational to believe that the CD does have all the bits from the original accurately reproduced, and an unknown amount of junk at the end where the isofs software will never look. All this work was done using 2048byte block size in dd. The original iso
checking integrity of already written CD/DVD
Greetings, I'm about to lend my entire DVD set with Debian 5.0 to a friend. I have no doubt in my friend, but this occasion awakened the curious person in me and here is the question: Is there a way to check that the DVD I give to somebody is the same that the DVD I'll get later? More complex: Is there a way to check a written DVD against the checksum of the iso image written on it? (For example to check that Debian 5.0 Lenny Official i386 Binary-1 DVD I got from someone has written on it exactly the .iso file from debian.org) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Re: checking integrity of already written CD/DVD
On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Greetings, I'm about to lend my entire DVD set with Debian 5.0 to a friend. I have no doubt in my friend, but this occasion awakened the curious person in me and here is the question: Is there a way to check that the DVD I give to somebody is the same that the DVD I'll get later? More complex: Is there a way to check a written DVD against the checksum of the iso image written on it? (For example to check that Debian 5.0 Lenny Official i386 Binary-1 DVD I got from someone has written on it exactly the .iso file from debian.org) $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso (which should be the same as the one published by Debian). If they don't match the DVD was not written correctly. Regards, Andrei -- If you can't explain it simply, you don't understand it well enough. (Albert Einstein) signature.asc Description: Digital signature
Re: checking integrity of already written CD/DVD
In 20090329202842.ga3...@think.homelan, Andrei Popescu wrote: On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Is there a way to check a written DVD against the checksum of the iso image written on it? $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. head -c $(wc -c image) /dev/dvd | md5sum should be the same as md5sum image though. -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: checking integrity of already written CD/DVD
On 2009-03-29_16:27:56, Boyd Stephen Smith Jr. wrote: In 20090329202842.ga3...@think.homelan, Andrei Popescu wrote: On Sun,29.Mar.09, 20:28:44, Angelin Lalev wrote: Is there a way to check a written DVD against the checksum of the iso image written on it? $ md5sum /dev/dvd This should result in *exactly* the same checksum as the iso Not in my experience. Both DVDs and CDs have a physical sector size. If the image is not a multiple of that sector size, the md5sum of the block device and the image will differ, because of the extra bits in the last physical sector. head -c $(wc -c image) /dev/dvd | md5sum should be the same as md5sum image though. This doesn't work for me, but something similar does. Don't know why my system behaves as it does. Here is my situation: I have two CD/DVD drives hdc and hdd. hdc is read only, hdd also is a burner. I burn an iso on hdd and read it back using dd. I always get a read back that is 5 to 10 ik blocks shorter than the iso file that I burned from. I put the newly burnt CD into hdc and read with dd, and I alway get a read back the is 5 to 10 1k block longer that the original iso file. As you might expect, the exact command given above never works for me on either drive but ... I can read back on drive hdc using dd into a file on disk, call it cd.iso I then use dd to copy cd.iso to cd2.iso and specify a block count that is equal to that of the original iso from which the CD was burnt. The MD5SUM of cd2.iso invariably agrees with the MD5SUM of the original iso. It seems to me this is something wrong in Linux, the kernel, and ought to be fixable. In the past it did work without all the fussiness, but not since Etch, or maybe Sarge. And it is hardware dependent. It should be possible to automate with a script, but I haven't tried. I only burn CDs when Debian issues a new release, so its not high on my to-do list. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
