RE: vinum striped volume has corrupt plex, help needed (resolved)
Final update on this chapter for all of you breathlessly waiting for word on what happened... :) turned out to be a couple of issues. The scsi cable seemed to be sensitive to it's route through the innards of my case, a little more interference in some spots perhaps. But the bigger issue was that the extra drive I added taxed my power supply just a bit too much. When the 5 baracudda drives were operational under vinum (and striping made them *all* active at once) the resultant strain on the PS lead to signal errors on the scsi bus (fast and wide but not differential). Powering off the new drive allowed me to restore vinum (thanks for the setstate, i needed it several times) and copy off the data to a network share. One think I noticed and I'm wondering if anyone else has seen this. When my scsi drives were having difficulity, they would spin down to a lower speed. Sometimes they would spin down to several lower speeds and even stop on occasion before spinning back up. This was sometimes (but not always) accompanied by a bus free in data-in phase error message on the console. In severe cases there would be an entire scsi dump state on the console. Was the scsi driver telling the drives to spin down or were the drives doing it themselves? I've never seen that behaviour in other OSs. I'm wondering if the drives (i couldn't figure out which one was changing speed) were responding to PS voltage drops or something? Another thing One forgets the march of technology. I remember coding scsi drivers waay back when it was called sasi. Reverting my fast-wide-disconnectenabled-synchronous drives back to narrow-slow-nodisconnect and asynchronous (just like the original spec) made them very slow. took forever to dd them. And of course my scsi drives are slow by todays scsi standards... anyway, thanks all. -lee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Lee Dilkie Sent: Sunday, May 09, 2004 9:50 AM To: 'Greg 'groggy' Lehey' Cc: [EMAIL PROTECTED] Subject: RE: vinum striped volume has corrupt plex, help needed Update on my progress. The setstate up allows me to read my array now. I seem to have a read error on /dev/da1s1e about 262M from the start of the disk. (i'm using dd if=/dev/da*s1e to copy the contents of each drive to separate files, in case i screw something up and I need to restore a drive and re-try the vinum... don't know if that a dumb idea or not but it seemed logical). anyway, i have to figure out a way to get around this read error. or find out what file(s) it affects so i can avoid trying to copy them. Any pointers would be welcome (as I fire up google... where would we be without search engines?) When i tried to vinum start striped.p0.s1, most of the time i would get an error Input/output error (5) but a couple of times the command hung (as it is right now). It would be interesting to see the ps -l output for that process and any other Vinum-related processes. Darn. I wish i had done that for you but I rebooted my server and vinum is running correctly now. When this is all finished, I'll see if I can get vinum stuck again and retrieve anything you wish. thanks for the help folks(greg), the saga continues. -lee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Greg 'groggy' Lehey Sent: Saturday, May 08, 2004 6:40 PM To: Lee Dilkie Cc: [EMAIL PROTECTED] Subject: Re: vinum striped volume has corrupt plex, help needed On Saturday, 8 May 2004 at 13:37:42 -0400, Lee Dilkie wrote: Hi there, I've been running a 5 disk vinum array (sripted, no redundancy) for a few months now. It's composed of 5 scsi drives of 4G each. I bought a new 120G ide drive, with the intention of copying over all the files from the vinum array and retiring the array (the scsi drives are really loud). All was fine until the file copy part. Shortly after starting, i started to get scsi errors and the scsi system reset the drives and re-spun them up in an attempt to provide data (this i could hear). Eventually vinum reported a read error. My machine kinda locked up because there were swap partitions on the scsi drives and things just went south when the OS couldn't swap properly. I rebooted and fsck'd my other partitions just fine but vinum reported that the plex was corrupt and one of the subdisks was stale (see vinum list output below). I also include the output from the command to read and parse the vium table on each drive ( as describe at http://www.vinumvm.org/vinum/how-to-debug.html ). it sure looks to me like all the disks have the same vinum info. When i tried to vinum start striped.p0.s1, most of the time i would get an error Input/output error (5) but a couple of times the command hung (as it is right now). It would be interesting to see the ps -l output for that process and any other Vinum-related processes. Also, I reconfured my scsi (2940uw) to the lowest transfer speed, disabled wide negotation
RE: vinum striped volume has corrupt plex, help needed
Update on my progress. The setstate up allows me to read my array now. I seem to have a read error on /dev/da1s1e about 262M from the start of the disk. (i'm using dd if=/dev/da*s1e to copy the contents of each drive to separate files, in case i screw something up and I need to restore a drive and re-try the vinum... don't know if that a dumb idea or not but it seemed logical). anyway, i have to figure out a way to get around this read error. or find out what file(s) it affects so i can avoid trying to copy them. Any pointers would be welcome (as I fire up google... where would we be without search engines?) When i tried to vinum start striped.p0.s1, most of the time i would get an error Input/output error (5) but a couple of times the command hung (as it is right now). It would be interesting to see the ps -l output for that process and any other Vinum-related processes. Darn. I wish i had done that for you but I rebooted my server and vinum is running correctly now. When this is all finished, I'll see if I can get vinum stuck again and retrieve anything you wish. thanks for the help folks(greg), the saga continues. -lee -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Greg 'groggy' Lehey Sent: Saturday, May 08, 2004 6:40 PM To: Lee Dilkie Cc: [EMAIL PROTECTED] Subject: Re: vinum striped volume has corrupt plex, help needed On Saturday, 8 May 2004 at 13:37:42 -0400, Lee Dilkie wrote: Hi there, I've been running a 5 disk vinum array (sripted, no redundancy) for a few months now. It's composed of 5 scsi drives of 4G each. I bought a new 120G ide drive, with the intention of copying over all the files from the vinum array and retiring the array (the scsi drives are really loud). All was fine until the file copy part. Shortly after starting, i started to get scsi errors and the scsi system reset the drives and re-spun them up in an attempt to provide data (this i could hear). Eventually vinum reported a read error. My machine kinda locked up because there were swap partitions on the scsi drives and things just went south when the OS couldn't swap properly. I rebooted and fsck'd my other partitions just fine but vinum reported that the plex was corrupt and one of the subdisks was stale (see vinum list output below). I also include the output from the command to read and parse the vium table on each drive ( as describe at http://www.vinumvm.org/vinum/how-to-debug.html ). it sure looks to me like all the disks have the same vinum info. When i tried to vinum start striped.p0.s1, most of the time i would get an error Input/output error (5) but a couple of times the command hung (as it is right now). It would be interesting to see the ps -l output for that process and any other Vinum-related processes. Also, I reconfured my scsi (2940uw) to the lowest transfer speed, disabled wide negotation (these are wide fast drives), disabled disconnect and disabled synchronous transfers. Basicly, i slowed them down as slow as they can go. I am able to successfully read each drive (tested the first 1G of each using dd if=/dev/da*s1e of=/dev/null bs=1m count=1000). That might work for a while. There were no write operations to the vinum volume when things crashed. I'm hoping i can get vinum up and running again so i can copy off this data. Vinum protects you by making it difficult to access data of dubious integrity. Question to the group. Would a vinum create using the original configuration (i have the file) recover this situation so i could mount and read the disk? Yes. Is there something else to do that will help? Yes. Do: vinum - setstate up striped.p0.s1 striped.p0 When you're happy with the data, do: vinum - setdaemon 4 vinum - saveconfig Greg -- When replying to this message, please copy the original recipients. If you don't, I may ignore the reply or reply to the original recipients. For more information, see http://www.lemis.com/questions.html Note: I discard all HTML mail unseen. Finger [EMAIL PROTECTED] for PGP public key. See complete headers for address and phone numbers. ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: vinum striped volume has corrupt plex, help needed
On Saturday, 8 May 2004 at 13:37:42 -0400, Lee Dilkie wrote: Hi there, I've been running a 5 disk vinum array (sripted, no redundancy) for a few months now. It's composed of 5 scsi drives of 4G each. I bought a new 120G ide drive, with the intention of copying over all the files from the vinum array and retiring the array (the scsi drives are really loud). All was fine until the file copy part. Shortly after starting, i started to get scsi errors and the scsi system reset the drives and re-spun them up in an attempt to provide data (this i could hear). Eventually vinum reported a read error. My machine kinda locked up because there were swap partitions on the scsi drives and things just went south when the OS couldn't swap properly. I rebooted and fsck'd my other partitions just fine but vinum reported that the plex was corrupt and one of the subdisks was stale (see vinum list output below). I also include the output from the command to read and parse the vium table on each drive ( as describe at http://www.vinumvm.org/vinum/how-to-debug.html ). it sure looks to me like all the disks have the same vinum info. When i tried to vinum start striped.p0.s1, most of the time i would get an error Input/output error (5) but a couple of times the command hung (as it is right now). It would be interesting to see the ps -l output for that process and any other Vinum-related processes. Also, I reconfured my scsi (2940uw) to the lowest transfer speed, disabled wide negotation (these are wide fast drives), disabled disconnect and disabled synchronous transfers. Basicly, i slowed them down as slow as they can go. I am able to successfully read each drive (tested the first 1G of each using dd if=/dev/da*s1e of=/dev/null bs=1m count=1000). That might work for a while. There were no write operations to the vinum volume when things crashed. I'm hoping i can get vinum up and running again so i can copy off this data. Vinum protects you by making it difficult to access data of dubious integrity. Question to the group. Would a vinum create using the original configuration (i have the file) recover this situation so i could mount and read the disk? Yes. Is there something else to do that will help? Yes. Do: vinum - setstate up striped.p0.s1 striped.p0 When you're happy with the data, do: vinum - setdaemon 4 vinum - saveconfig Greg -- When replying to this message, please copy the original recipients. If you don't, I may ignore the reply or reply to the original recipients. For more information, see http://www.lemis.com/questions.html Note: I discard all HTML mail unseen. Finger [EMAIL PROTECTED] for PGP public key. See complete headers for address and phone numbers. pgpI7BzE2YxRl.pgp Description: PGP signature