Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie
On Wed, 10 Jan 2007, Matt Boge wrote: PS. I think I'd like to consider creating multiple partitions in my new setup (two 400GB drives that I will be setting up with RAID0). Watch out with RAID0, if you do that, if either one of the drives die you lost the data from both. And recovery is much more complicated because half the data is on each drive. I'd install Windows XP on one partition and I think I'd like to install a Linux variant on the other to play around with and get more familiar with this OS (hey, an old dog can ALWAYS learn new tricks, right?). Any suggestions on how I should do this and which variant of Linux I should install? Create the partitions outside linux, first create the Windows partition (type doesn't matter, as you will see). Then create the Linux partition, and then finally delete the windows one. You are doing it in this strange order to make sure the second partition was also the second one created. Don't forget to create a swap partition if you will use one (you can also swap onto a data partition). Don't install linux yet (since potentially windows will erase it, and then you'll have wasted your time) boot windows setup, and ask to partition the drive - windows should complain about some mystery partition on the drive, but ignore that, and let window partition the free space on the drive. Finish windows install. Now install linux, first of all linux will not get confused about the extra partition, second the installer will (should) notice windows and create an entry for it. And finally linux will do the right thing in regard to making sure you can actual boot (window can't handle it). Anyway, as far as what variant (called a distribution) I like Debian, but try these pages: http://www.tuxs.org/chooser/ http://www.zegeniestudios.net/ldc/ -Ariel PS. Another option (if your chosen distib supports it), is to be dumb, and let windows own the whole drive, then have the distrib shrink the partition to make room for linux. If you use RAID on the drives you are complicating things, and I just don't know what linux will do. If it's hardware raid it should work, but if you use windows raid, I'm just not sure. ___ Bug-ddrescue mailing list Bug-ddrescue@gnu.org http://lists.gnu.org/mailman/listinfo/bug-ddrescue
Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie
Thanks, Ariel. I just set off the process again with, but splitting the errors I'm prepared to wait a while. After this, I plan on running SpinRite on the damaged disk to see if I can eek any more data off of it and then run ddrescue again for the final time on that drive. Then I will copy the data to a new, partitioned drive as you suggested and the source drive will be my safe copy and the target my scratch copy. I then will then run scandisk on the scratch copy to attempt to recover any file names and folders lost in the process and hopefully find the lost directories. And you're right... no plans to boot (or even directly access) the drive with Windows. Once completed, I'll attempt to access it through the network (another Linux challenge for me) and copy the files to my new Windows XP machine that way. I'll keep you posted. - Matt - Original Message From: Ariel [EMAIL PROTECTED] To: Matt Boge [EMAIL PROTECTED] Cc: David Burton [EMAIL PROTECTED]; bug-ddrescue@gnu.org Sent: Sunday, February 4, 2007 4:22:06 AM Subject: Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie On Fri, 2 Feb 2007, Matt Boge wrote: (By the way, I know I should have copied it to /dev/hdb1, but should I have created that partition with a tool like cfdisk first or would including that in the ddrescue command have created the partition itself?) You should have created the partition first (with cfdisk) (rebooted, probably), and then copied to that partition. Anyway, to capture the MBR, I then ran (with some helpful guidance from here): ddrescue -B -n -s 63 /dev/hda rescued_mbr.ima rescued_mbr.log The MBR is not really useful to you. So, I've got a copy of my MBR in as an image om my USB drive, a rescued copy of my damaged partition on a new, unpartitioned drive (/dev/hdb). Based on some more help here, I created a new partition on another new drive (/dev/sda1) with the IDENTICAL specs as the failed one (I used cfdisk to do this). I then ran the following two commands: ddrescue -B rescued_mbr.ima /dev/sda1 ddrescue -B /dev/hdb /dev/sda1 Does that look right to you? Is that all I really need to do to /dev/sda1? Is it now what it would have looked like if I had run ddrescue correctly the first time? Is this my best-good-copy now and should I duplicate this back to my /dev/hdb drive, overwriting the unpartitioned data from the first ddrescue run? No no!! Just run ddrescue -B /dev/hdb /dev/sda1 (you don't technically need ddrescue to do that copy, you can use dd, or even cp, but that's not important). Do not copy the MBR! First of all the MBR is only meant for the start of the bare drive, not the partition. Second an MBR from one drive will not (necessarily) work in a different one unless they have identical sizes. The MBR holds a couple of things: the partition table, and the DOS boot stuff. The partition table you have already, and the DOS boot stuff would be better created with windows. (Setup should do it, but I have a feeling you are not going to be booting that drive anyway, so it doesn't actually matter.) Also, when I rerun ddrescue with autosplit turned on can I run it to the new partition (i.e. /dev/sda1) or will the ddrescue log file be off by 63 sectors? Hmm. At first I was going to tell you it will be off, but actually - at first you were copying TO /dev/hda (or whatever it is), now you are copying to /dev/hdb1 - but you moved all the data downward into the partition, it makes no difference to ddrescue where the data is physically, logically in both cases the data starts at the top of the output device! Now if you had done FROM /dev/hda you would be in trouble. Also, if there was a difference it would not be 63 sectors. The exact amount depends on your drive. I'm sorry this is all so confusing, but I'm just not sure how much of a mistake I made originally by not creating that partition on /dev/hdb/ and if the relatively simple process above of merging the MBR and rescued data to a new partition on my /dev/sda drive was all I needed to do to correct that mistake. You're OK. Don't merge the MBR, create it fresh, then copy from the data from /dev/hdb to /dev/sda1. Be aware - the drive sizes are NOT the same, i.e. the partition on /dev/sda1 is smaller then the total size of /dev/hdb, you WILL get an error while doing this, but as long as all the data was copied, it doesn't matter. Just check that the amount of data copied is EXACTLY equal to the size of the partition on the original drive, if it is, you know you got all of it. Really, thanks again for your time and patience dealing with me on this... I'm sure you have much better things to do than to hand-hold a Linux newbie. But you all have been such a lot of help and really appreciate it. You're welcome. Sorry about the delay with responding, be sure to let us know how it turns out
Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie
Yea... I'm aware of the inherent vulnerability of RAID0, but I don't want to lose the HD space by using RAID1. However, this time I'm prepared to implement a robust backup procedure and utilize either a USB or NAS (or most probably both) for the backups and to decentralize my data (i.e. pics and vids on my NAS, MyDocs and Preferences backed up on the USB drive and routine backups ups to DVDs). I am also looking into paying an on-line service for backups (i.e. X-drive or Carbonite) just in case of a house fire! (Can you tell I don't want to go through this again ;)) Thanks for the Linux info... I will look into that. - Matt - Original Message From: Ariel [EMAIL PROTECTED] To: Matt Boge [EMAIL PROTECTED] Cc: bug-ddrescue@gnu.org Sent: Sunday, February 4, 2007 4:35:44 AM Subject: Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie On Wed, 10 Jan 2007, Matt Boge wrote: PS. I think I'd like to consider creating multiple partitions in my new setup (two 400GB drives that I will be setting up with RAID0). Watch out with RAID0, if you do that, if either one of the drives die you lost the data from both. And recovery is much more complicated because half the data is on each drive. I'd install Windows XP on one partition and I think I'd like to install a Linux variant on the other to play around with and get more familiar with this OS (hey, an old dog can ALWAYS learn new tricks, right?). Any suggestions on how I should do this and which variant of Linux I should install? Create the partitions outside linux, first create the Windows partition (type doesn't matter, as you will see). Then create the Linux partition, and then finally delete the windows one. You are doing it in this strange order to make sure the second partition was also the second one created. Don't forget to create a swap partition if you will use one (you can also swap onto a data partition). Don't install linux yet (since potentially windows will erase it, and then you'll have wasted your time) boot windows setup, and ask to partition the drive - windows should complain about some mystery partition on the drive, but ignore that, and let window partition the free space on the drive. Finish windows install. Now install linux, first of all linux will not get confused about the extra partition, second the installer will (should) notice windows and create an entry for it. And finally linux will do the right thing in regard to making sure you can actual boot (window can't handle it). Anyway, as far as what variant (called a distribution) I like Debian, but try these pages: http://www.tuxs.org/chooser/ http://www.zegeniestudios.net/ldc/ -Ariel PS. Another option (if your chosen distib supports it), is to be dumb, and let windows own the whole drive, then have the distrib shrink the partition to make room for linux. If you use RAID on the drives you are complicating things, and I just don't know what linux will do. If it's hardware raid it should work, but if you use windows raid, I'm just not sure.___ Bug-ddrescue mailing list Bug-ddrescue@gnu.org http://lists.gnu.org/mailman/listinfo/bug-ddrescue
Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie
Thanks Ariel, Yea, I've pretty much accepted the idea that this will not be a quick and easy process and I'm OK with that. Fortunately, I've got a secondary computer I can use for this purpose and am willing to be diligent and patient. When I first ran ddrescue, I mistakenly copied the partition to a non-partitioned disk: ddrescue -B -n /dev/hda1 /dev/hdb rescued.log (By the way, I know I should have copied it to /dev/hdb1, but should I have created that partition with a tool like cfdisk first or would including that in the ddrescue command have created the partition itself?) Anyway, to capture the MBR, I then ran (with some helpful guidance from here): ddrescue -B -n -s 63 /dev/hda rescued_mbr.ima rescued_mbr.log So, I've got a copy of my MBR in as an image om my USB drive, a rescued copy of my damaged partition on a new, unpartitioned drive (/dev/hdb). Based on some more help here, I created a new partition on another new drive (/dev/sda1) with the IDENTICAL specs as the failed one (I used cfdisk to do this). I then ran the following two commands: ddrescue -B rescued_mbr.ima /dev/sda1 ddrescue -B /dev/hdb /dev/sda1 Does that look right to you? Is that all I really need to do to /dev/sda1? Is it now what it would have looked like if I had run ddrescue correctly the first time? Is this my best-good-copy now and should I duplicate this back to my /dev/hdb drive, overwriting the unpartitioned data from the first ddrescue run? Also, when I rerun ddrescue with autosplit turned on can I run it to the new partition (i.e. /dev/sda1) or will the ddrescue log file be off by 63 sectors? I'm sorry this is all so confusing, but I'm just not sure how much of a mistake I made originally by not creating that partition on /dev/hdb/ and if the relatively simple process above of merging the MBR and rescued data to a new partition on my /dev/sda drive was all I needed to do to correct that mistake. Really, thanks again for your time and patience dealing with me on this... I'm sure you have much better things to do than to hand-hold a Linux newbie. But you all have been such a lot of help and really appreciate it. - Matt - Original Message From: Ariel [EMAIL PROTECTED] To: Matt Boge [EMAIL PROTECTED] Cc: bug-ddrescue@gnu.org; David Burton [EMAIL PROTECTED] Sent: Thursday, January 25, 2007 5:29:04 PM Subject: Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie On Thu, 25 Jan 2007, Matt Boge wrote: Hey guys, I don't want to be a pest, but if there is any chance someone could take a few minutes and look at my questions below, I would definitely appreciate it. Again, I really appreciate what you have all done already, but I'm kinda dead in the water right now and I'm a little anxious about moving forward on my own. I have been checking out the rescued data using Knoppix, and while a lot of data was saved, I noticed a huge glaring hole: the My Pictures folder was missing. As you can probably imagine, this is one of the few folders I really cared about rescuing and while I was expecting that some of the data might be lost, I certainly wasn't expecting the entire folder to not show up. Seems that you had the disk failure where new writes fail, I guess you were writing a file to that directory? Anyway the directory itself may be gone, but most likely all the files in it are still there. If you are using ntfs (not fat32) there is a change yu can save the file names, but otherwise you will lose the file names, but at least you will have the data. Run checkdisk on it, and MAKE SURE you tell it to save lost clusters. Anyway, I think I'd like to give SpinRite a shot on the damaged drive to see if I can eek out anymore data, but before I do (and possibly damage the drive further), I want to be sure that the stuff I did pull off already with ddrescue is safe. In that vein, I want to have two copies of the rescued data: one scratch drive (to add any potential new data saved by SpinRite and updated by a second running of ddrescue) and one safe copy that I can revert to if my scratch drive fiddlings go bad. There is another reason to have a second disk: when you run checkdisk you may end up with less data then you would like, so if that happens you can revert back to your saved disk. Also, before running spinrite, you may want to consider running ddrescue again, using the same logfile and settings etc (watch out if you moved the data because of your partition issue, I don't remember what you did). It's possible ddrescue will be able to find more data - also this time turn on autosplit mode, I remember you had that off the first time. And finally, after spinrite finishes, run ddrescue on the disk, again with the logfile - if spinrite managed to recover any bad sectors, then ddrescue will copy them, but it won't bother copying data that it already managed to get. BTW: The first time I had a disk error
Re: [Bug-ddrescue] ddrescue 1.3 - questions from a newbie
Hi Matt, Your recovery process is obviously going rather slowly, but, take heart, sometimes the process speeds up after it gets past the worst error area. Unless there are only a few errors, the copy time is almost entirely proportional to the number of errors, because error-free sectors copy so quickly and bad sectors take so long to fail. (Unfortunately, I don't know of any way to stop the drive's very slow internal error recover process, for a quicker first pass.) Your strategy seems pretty good except for a couple of small mistakes. First, your recovery command should have been: ddrescue -B -n /dev/hda /dev/hdb rescued.log instead of ddrescue -B -n /dev/hda1 /dev/hdb rescued.log Unfortunately, the command you used will omit the MBR, so there'll be no partition table on /dev/hdb. Oops! I do NOT recommend that you start over. Once you've scraped some data from a drive, it is never a good idea to throw it away and start over. But it would be a good idea to go ahead and grab the part that you missed: ddrescue -B -n -s 63 /dev/hda rescued_mbr.ima rescued_mbr.log (That's assuming that /dev/hda1 starts at sector 63. You can check that with fdisk -lu /dev/hda.) You're gonna have a little chore, eventually, putting the two pieces back together... you'll have to do something like this (assuming that /dev/hdb is a scratch drive, and /dev/hdc is the eventual new drive): # Copy the MBR / primary partition table: ddrescue -B rescued_mbr.ima /dev/hdc # Copy the first (only?) data partition: ddrescue -B /dev/hdb /dev/hdc1 Secondly, I think you mistake how SpinRite works. Your step #4 needs to run SpinRite on the original (failing) drive, not on the copy. (Note: my ddr2sr.pl ddrescue-to-SpinRite script will help you target just the bad areas of the drive with SpinRite.) When ddrescue finishes, before you run SpinRite, the new copy will have only two kinds of sectors: those which contain perfect data, and those which have nothing at all (probably zeros, if /dev/hdb was initially all zeros). Then, when SpinRite has finished working on the bad sectors, you'll have two kinds of formerly-bad sectors: Those which were fully recovered (probably about 10% of them), and those which were only partially recovered (the other 90%). You can then use ddrescue to add the (fully partially) recovered sectors to /dev/hdb. But be sure to save a copy of your rescued.log file first, because ddrescue can't tell the difference between the fully recovered sectors and the partially recovered sectors -- they will all read successfully when ddrescue sees them. So, after you use ddrescue to add the SpinRite-recovered sectors to /dev/hdb, the rescued.log file will no longer be an accurate record of which parts if the drive are damaged. (Dd-rescue's logfile has no mechanism for representing damaged/partially-recovered sectors.) But the combination of the new and old ddrescue log files, plus the SpinRite logfile, encapsulates that information. My ddr2nfi.pl tool uses all three logfiles to generate NFI commands for the truly damaged sectors, for trying to deduce which files are impacted by the damage. (Note: run the resulting .bat file of NFI commands on a scratch copy of the recovered drive, not the main/best copy, since Windows always messes with a drive when it mounts it.) For hours of entertainment, download my Perl scripts that I use when doing dd-rescue recoveries. One http://www.burtonsys.com/download/ddr2sr.zip It contains: clustersize.pl -- Shows cluster size(s) for FAT and NTFS partitions ddrsummarize.pl -- Summarize a ddrescue log file ddrsplit.pl -- DDRescue logfile SPLITter ddrcombine.pl -- DDRescue logfile COMBINEr ddrlogand.pl-- DDR LOGical .AND. ddrlogor.pl -- DDR LOGical .OR. ddrlognot.pl-- DDR LOGical .NOT. ddr2sr.pl -- ddrescue-to-SpinRite ddr2nfi.pl -- DDRescue to NFI nficruncher.pl -- Process the output of many NFI commands fat32inf.pl -- FAT32 info (this one is still under construction) foreach.pl -- Repeat a command for each filename or argument ddrcopyincrement.pl -- Copy additional recovered sectors ddrwipe.pl -- Obliterate all the GOOD sectors on a failing disk drive samplescripts.zip -- some shell scripts ddrescue_perl_helper_programs.txt -- documentation (see also comments in the scripts) There are examples in the samplescripts.zip of using a raw device for a later pass to read sector-at-a-time, so that the unrecovered parts will be down to a granularity of one sector, instead of a full Linux buffered disk block. Unfortunately, raw devices vary in implementation between Linux versions, so you might have to fiddle with the scripts to make that part work. Unfortunately, some of the other scripts assume that the destination is a copy of the full drive, including MBR, and expect the logfile to describe a whole drive. After you put the two
