Re: Re: About USB hard drives and errors
-- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100610013235.ga9...@big.lan.gnu
Re: About USB hard drives and errors
Celejar wrote: On Sat, 10 Apr 2010 22:06:06 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... I got a little less timid and tried running smartctl even though I was quite unsure of what to expect. It ran. Each of the three USB HD gave somewhat different output, but none gave output that claimed there was a working SMART on the drive. These drives are Western Digital (WD). The WD web site mentions SMART and also uses the words Smart Drive to mean something else that is a proprietary marketing thing, AFAICT. I was unable to find a list of part #s for drives that support S.M.A.R.T. I think I should be in the market for a better class of drives, but not this weekend. Thanks for the help. My understanding is that S.M.A.R.T. doesn't generally work over USB. As Wikipedia puts it: For example, few external drives connected via USB and Firewire correctly send S.M.A.R.T. data over those interfaces. http://en.wikipedia.org/wiki/S.M.A.R.T.#Standards_and_implementation I can use smart on my USB drive enclosure with an ATA drive but not with my USB drive enclosure with a SATA drive. Hugo -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/hpvvd4$4p...@dough.gmane.org
Re: About USB hard drives and errors
On 20100412_152156, Hugo Vanwoerkom wrote: Celejar wrote: On Sat, 10 Apr 2010 22:06:06 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... I got a little less timid and tried running smartctl even though I was quite unsure of what to expect. It ran. Each of the three USB HD gave somewhat different output, but none gave output that claimed there was a working SMART on the drive. These drives are Western Digital (WD). The WD web site mentions SMART and also uses the words Smart Drive to mean something else that is a proprietary marketing thing, AFAICT. I was unable to find a list of part #s for drives that support S.M.A.R.T. I think I should be in the market for a better class of drives, but not this weekend. Thanks for the help. My understanding is that S.M.A.R.T. doesn't generally work over USB. As Wikipedia puts it: For example, few external drives connected via USB and Firewire correctly send S.M.A.R.T. data over those interfaces. http://en.wikipedia.org/wiki/S.M.A.R.T.#Standards_and_implementation I can use smart on my USB drive enclosure with an ATA drive but not with my USB drive enclosure with a SATA drive. Hugo Very interesting --- but My quick googling reveals many USB drive enclosures for both ATA and SATA but the marketers don't make mention of S.M.A.R.T . What is the brand of your enclosure. I want to look for it, in particular, and see if it claims the feature that you have discovered. Actually, on looking a little more closely, I'm not seeing any case of a single enclosure that can be used with both ATA and SATA. Do I read your post correctly - your enclosure can contain either a SATA or an ATA, and work over USB? But SMART only works when it is enclosing an ATA? -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100412223607.gl10...@big.lan.gnu
Re: About USB hard drives and errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul E Condon wrote: My understanding is that S.M.A.R.T. doesn't generally work over USB. So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? A quick partial solution could be: If you just want to read the SMART tables once, say to see how many sectors have been remapped, you can simply take the HD out of its enclosure and temporarily connect it directly inside your PC. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvDrRAACgkQ+VSRxYk440/TwACg6US5ORZajsE0+8fIgAdsnD2o T9sAn2+wxSvbTdJ4G1RpGcF29J2coTVh =7LV8 -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc3ad10.6080...@web.de
Re: About USB hard drives and errors
Original Message From: pecon...@mesanetworks.net To: debian-user@lists.debian.org Subject: Re: About USB hard drives and errors Date: Sun, 11 Apr 2010 15:47:40 -0600 On 20100411_115203, Andrei Popescu wrote: On Sat,10.Apr.10, 16:24:45, Paul E Condon wrote: The errors that I am experiencing are all similar. The first indication of a problem is a message from the kernel (I think). An example is: kernel: [78454.939948] journal commit I/O error [...] When this happens, all the USB drives (3 of them) disappear from /dev/disks/by-label (they are all labeled by me). I have not Just a long shot, bun since you are connecting 3 USB drives to the same computer you might experience power issues. If you connect only one drive do you get the same issues? Or you could try a powered USB hub if you have one. This is a good thing to look into. Now I am experiencing a interlude of error free operation, so I can't test it. Thanks -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411214740.gg10...@big.lan.gnu while its working you might try and plug in yet another USB device or two(if you have the ports) and see if it fails. Larry -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/380-22010421312134...@netptc.net
Re: About USB hard drives and errors
On Sat,10.Apr.10, 16:24:45, Paul E Condon wrote: The errors that I am experiencing are all similar. The first indication of a problem is a message from the kernel (I think). An example is: kernel: [78454.939948] journal commit I/O error [...] When this happens, all the USB drives (3 of them) disappear from /dev/disks/by-label (they are all labeled by me). I have not Just a long shot, bun since you are connecting 3 USB drives to the same computer you might experience power issues. If you connect only one drive do you get the same issues? Or you could try a powered USB hub if you have one. Regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: About USB hard drives and errors
On Sat, 10 Apr 2010 23:48:51 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... Your comment was/is helpful to me. Thanks. Please don't drop off this thread because I'm sometimes too terse. Following some links from article No offense taken - I just didn't want you to get your expectations up of the likelihood of much more in the way of any helpful contributions to this thread from me ;) Celejar -- foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411090426.71776c9a.cele...@gmail.com
Re: About USB hard drives and errors
On 2010-04-08 19:44, Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. http://smartmontools.sourceforge.net/faq.html#testinghelp As for USB and FireWire (IEEE 1394) disks and tape drives, the news is not good. They appear to the operating system as SCSI devices but their implementations do not usually support those SCSI commands needed by smartmontools. -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc21942.3010...@cox.net
Re: About USB hard drives and errors
On 20100411_005025, Stan Hoeppner wrote: Paul E Condon put forth on 4/10/2010 11:41 PM: So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? You're got 3 USB hard drives already, and you're throwing them out and looking for another solution, just because they don't do S.M.A.R.T? If neither USB nor firewire do smart, you choices are very limited. Well, the sad story is that I don't really want to do S.M.A.R.T. I was experiencing disk errors on the drives which caused something in the kernel to throw a fit. When this happened, the only recovery I could find was to reboot. This is slow and not really a way to learn how to fix the problem. So I ask for advice on this list. I work down the list of suggestions, not having much success, and arrive at smartctl. But now we know that that doesn't work for reasons that I might have known if I had read the whole of wikipedia and had total recall --- but I hadn't and I don't. Since my last post, I have succeeded in doing a successful, error free, backup to each of the disks. One notable fact about all the errors is that after reboot it would appear that no data was lost and the backup could be resumed from where it advanced to just before the crash. The only thing that I can think that I did differently in these new successful runs is that I did not sit at the computer and watch. Can software detect being watched? I think not, but ... My reading about eSATA gives me suspicion that it has its own poorly documented problems. I'm not convinced that I cannot make USB work without S.M.A.R.T, but I really don't have any good ideas as to how. And now, without errors happening fairly frequently, I doubt that I will be able to test and debug any approach that I dream up. The only other realistic option I know of is eSATA, but a quick scan of Newegg shows only 6 devices total, 2 DVR expander drives and 4 eSATA RAID enclosures. I have no idea if the DVR expander drives will work with a standard PC setup. They should. The two DVR drives are both 1TB and both just under $130 USD, one Iomega and one Western Digital: http://www.newegg.com/Product/Product.aspx?Item=N82E16822186175 http://www.newegg.com/Product/Product.aspx?Item=N82E16822136384 Are you planning on connecting this drive to a laptop or desktop? To multiple computers or just one computer? This is for a LAN of desktop systems, three computers in all. One computer acts as a collector of backups from the itself and the other two, and is responsible for getting the files onto an external drive in a timely way. All of the computers are hand-me-downs. None have eSATA capability. So far I have not convinced myself that spending money would help solve the problem. Perhaps in a few years, computers with eSATA will start showing up in dumpsters. Maybe I should just wait. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411214050.gf10...@big.lan.gnu
Re: About USB hard drives and errors
On 20100411_115203, Andrei Popescu wrote: On Sat,10.Apr.10, 16:24:45, Paul E Condon wrote: The errors that I am experiencing are all similar. The first indication of a problem is a message from the kernel (I think). An example is: kernel: [78454.939948] journal commit I/O error [...] When this happens, all the USB drives (3 of them) disappear from /dev/disks/by-label (they are all labeled by me). I have not Just a long shot, bun since you are connecting 3 USB drives to the same computer you might experience power issues. If you connect only one drive do you get the same issues? Or you could try a powered USB hub if you have one. This is a good thing to look into. Now I am experiencing a interlude of error free operation, so I can't test it. Thanks -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411214740.gg10...@big.lan.gnu
Re: About USB hard drives and errors
Paul E Condon put forth on 4/11/2010 4:40 PM: All of the computers are hand-me-downs. None have eSATA capability. So far I have not convinced myself that spending money would help solve the problem. Perhaps in a few years, computers with eSATA will start showing up in dumpsters. Maybe I should just wait. Hand-me-downs aren't well known for their reliability, especially if they weren't, ahem, treated/handled properly during the decommissioning process. If these external USB hard drives were new-in-box, I'd suspect the USB cables/interface more than I would the new drives. I assume you've swapped the USB cables? If you're powering the external USB drives via the USB cable, DON'T. Plug in the wall wart transformer to power the drives. You could be having issues with dirty/insufficient power on the USB cable to the drive. Have you purchased and installed a ~$10 PCI USB 2.0 card, loaded the drivers, and tested the reliability of the drives with the PCI USB card? If not, I'd recommend that as your next step. -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc28212.90...@hardwarefreak.com
Re: About USB hard drives and errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. You can see the internal count of the remapped sectors with SMART, as others have already pointed out here. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvAJssACgkQ+VSRxYk4408n6gCgrLFCZRpvissG4/Q8WXoAHnHU E1sAnRmcsZbxX1Bei7JYD0ZNhGaXVCrC =GN/u -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc026cc.8000...@web.de
Re: About USB hard drives and errors
On 2010-04-10 02:20, Clive McBarton wrote: Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. You can see the internal count of the remapped sectors with SMART, as others have already pointed out here. Interesting. So what is /badblocks/ for, and should it be removed in order to remove useless complexity? -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc080f6.60...@cox.net
Re: About USB hard drives and errors
Ron Johnson wrote: On 2010-04-10 02:20, Clive McBarton wrote: Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. You can see the internal count of the remapped sectors with SMART, as others have already pointed out here. Interesting. So what is /badblocks/ for, and should it be removed in order to remove useless complexity? And I thought I had something good when all the badblock counts were 0... Hugo -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/hpq16b$79...@dough.gmane.org
Re: About USB hard drives and errors
On Sat, Apr 10, 2010 at 08:45:26 -0500, Ron Johnson wrote: On 2010-04-10 02:20, Clive McBarton wrote: [...] Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. You can see the internal count of the remapped sectors with SMART, as others have already pointed out here. Interesting. So what is /badblocks/ for, I would say it is useful to make the drive access every single block; afterwards you can check in the SMART log if that caused any remappings. and should it be removed in order to remove useless complexity? I would not consider a command-line utility that can simply be ignored to be useless complexity. -- Regards,| Florian | -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100410154659.ga11...@isar.localhost
Re: About USB hard drives and errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Florian Kulzer wrote: Interesting. So what is /badblocks/ for, I would say it is useful to make the drive access every single block; afterwards you can check in the SMART log if that caused any remappings. That's a good idea. Another application is if you suspect that part of the HD surface is bad (old age, you dropped the HD, etc.) but the HD does not know it yet because it has not accessed the damaged sectors yet. Then badblocks (or dd) will force it to do so, probably bricking the HD in the process (sometimes with audible clicking noises or such) which, if you think about it, can be very useful if you intended to store valuable data on it later. should it be removed in order to remove useless complexity? I would not consider a command-line utility that can simply be ignored to be useless complexity. Indeed, and that is true for thousands of CLI commands. ImageMagick alone installs dozens which add complexity to your system if unused. Back to the topic of HDs, I even remember seeing a unix program for low-level formatting them, though formattable HDs probably have ceased to be sold a long time ago. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvAsE0ACgkQ+VSRxYk440/yowCgj8/SSjj3PtgHVdq3BWuUG6MF txMAn3HMUtUM+tTnZ0PyYhzx6qUy0V2p =D76w -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc0b04d.3090...@web.de
Re: About USB hard drives and errors
On 10-04-10 03:20:44, Clive McBarton wrote: Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. Not quite true. If the data in a sector was not readable, the sector will be listed as Pending. Pending sectors are much worse than Reallocated sectors, as Pending sectors mean lost data (if the sector was in actual use, which SMART does not know -- and figuring out which file might have been affected is, umm, tedious). I keep SMART's Offline Surface Scan enabled on my drives, to have the best chance that any failing sectors will be noticed early while they can still be recovered. I don't mind if there are a few Reallocated sectors, as long as there are never any Pending sectors. I'd mind if the number of Reallocated sectors kept increasing. Of course, I also keep backups. In other words, scanning for bad blocks on a HD cannot work. Or at least normally won't, unless Data Has Been Lost. You can see the internal count of the remapped sectors with SMART, as others have already pointed out here. Agree. Use smartctl. -- TonyN.:' mailto:tonynel...@georgeanelson.com ' http://www.georgeanelson.com/ -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1270922015.2998...@localhost.localdomain
Re: About USB hard drives and errors
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Tony Nelson wrote: If the data in a sector was not readable, the sector will be listed as Pending. Pending sectors are much worse than Reallocated sectors, as Pending sectors mean lost data (if the sector was in actual use, which SMART does not know -- and figuring out which file might have been affected is, umm, tedious). OK. Usually (during regular use) the internal errors probably increase more slowly. If a single sector is already really unreadable, then every last one of the internal error correction mechanisms has already tried and failed. Such many errors probably indicate that either the HD is terminally worn out, or that a sector got damaged due to external influences. Either way, I would not use such a HD any more. I keep SMART's Offline Surface Scan enabled on my drives, to have the best chance that any failing sectors will be noticed early while they can still be recovered. I don't mind if there are a few Reallocated sectors, as long as there are never any Pending sectors. I'd mind if the number of Reallocated sectors kept increasing. Of course, I also keep backups. Good practice. But I believe that HDs always try to recover failing sectors whenever possible, with or without offline surface scan. Presumably sectors are remapped when the number of errors is still way below the maximum of correctable errors. The world outside the HD never hears anything about it unless they ask the HD to report the SMART tables. In other words, scanning for bad blocks on a HD cannot work. Or at least normally won't, unless Data Has Been Lost. Yes, that's what I meant. It does work for proving for sure that the HD is broken. And to push a HD over the edge which is about to break (useful particularly for people with backups). I was assuming that the HD does not contain data, since the write test of badblocks deletes everything anyway and does not restore the original. Speaking of, does anybody know why the programmers of badblocks left out the ability to write the original content back after a read-write scan? That makes no sense to me. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkvAybYACgkQ+VSRxYk4408JNgCg59NgXTrJd+LbdzS+x/1QgAJm 6WIAoI66+djV1dAA7aVCe1VbLsdHn8U8 =g2yo -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc0c9b6.8090...@web.de
Re: About USB hard drives and errors
will be listed as Pending. Pending sectors are much worse than Reallocated sectors, as Pending sectors mean lost data (if the sector Indeed. OTOH Pending sectors can be eliminated by turning them into Reallocated sectors (just write to the corresponding sector), whereas Reallocated sectors will stay forever. Stefan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/jwvfx3312sd.fsf-monnier+gmane.linux.debian.u...@gnu.org
Re: About USB hard drives and errors
On 20100410_092044, Clive McBarton wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. Thanks Clive. Your post has been invaluable in fixing some faulty thinking on my part, and in provoking other useful posts. But I want more ... The errors that I am experiencing are all similar. The first indication of a problem is a message from the kernel (I think). An example is: kernel: [78454.939948] journal commit I/O error This appears on all xterm windows on the affected machine. On the xterm that is controlling a process that is using one of the USB drives, there follows a long sequence of error messages about the drive being read-only, which stops after a while. Or sometimes I stop it by typing ^C on that xterm. When this happens, all the USB drives (3 of them) disappear from /dev/disks/by-label (they are all labeled by me). I have not discovered any to make them re-appear, short of rebooting the computer. After reboot, I run e2fsck on all of them, and always get a longish delay on each while e2fsck commits (or whatever) the journal. This can take a few seconds or up to half a minute. Then I manually mount them using pmount, and all data upto the point where the crash happened seems to be present. I have installed smartmontools, but I think there is some incompatibility between the installed version and the installed docs. The README.Debian makes reference to editing some lines in the config file that are not present in the default, package installed, config file. There is (apparently) some incompatibility between using the daemon and using smartctl. The problem host is running Lenny, but the docs seem to be the same as on a different host that is running Squeeze. I would very much appreciate some help in understanding the docs. What is a safe thing for a nubie to type as a first command to smartctl? -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100410222445.gc5...@big.lan.gnu
Re: About USB hard drives and errors
On 20100410_162445, Paul E Condon wrote: On 20100410_092044, Clive McBarton wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Paul E Condon wrote: dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Every HD that is even remotely close to being usable will always have zero bad blocks when seen from outside the HD. All HDs have error recognition and error correction and automatic replacement of faulty sectors with spare ones. A HD will only show bad blocks after all of its remapping area is used, at which point it is far beyond being usable. In other words, scanning for bad blocks on a HD cannot work. Thanks Clive. Your post has been invaluable in fixing some faulty thinking on my part, and in provoking other useful posts. But I want more ... The errors that I am experiencing are all similar. The first indication of a problem is a message from the kernel (I think). An example is: kernel: [78454.939948] journal commit I/O error This appears on all xterm windows on the affected machine. On the xterm that is controlling a process that is using one of the USB drives, there follows a long sequence of error messages about the drive being read-only, which stops after a while. Or sometimes I stop it by typing ^C on that xterm. When this happens, all the USB drives (3 of them) disappear from /dev/disks/by-label (they are all labeled by me). I have not discovered any to make them re-appear, short of rebooting the computer. After reboot, I run e2fsck on all of them, and always get a longish delay on each while e2fsck commits (or whatever) the journal. This can take a few seconds or up to half a minute. Then I manually mount them using pmount, and all data upto the point where the crash happened seems to be present. I have installed smartmontools, but I think there is some incompatibility between the installed version and the installed docs. The README.Debian makes reference to editing some lines in the config file that are not present in the default, package installed, config file. There is (apparently) some incompatibility between using the daemon and using smartctl. The problem host is running Lenny, but the docs seem to be the same as on a different host that is running Squeeze. I would very much appreciate some help in understanding the docs. What is a safe thing for a nubie to type as a first command to smartctl? I'm answering my own post in order to bring some closure on this issue. If anyone has suggestions, please come forward. But here is where things stand with me: I got a little less timid and tried running smartctl even though I was quite unsure of what to expect. It ran. Each of the three USB HD gave somewhat different output, but none gave output that claimed there was a working SMART on the drive. These drives are Western Digital (WD). The WD web site mentions SMART and also uses the words Smart Drive to mean something else that is a proprietary marketing thing, AFAICT. I was unable to find a list of part #s for drives that support S.M.A.R.T. I think I should be in the market for a better class of drives, but not this weekend. Thanks for the help. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411040606.gc10...@big.lan.gnu
Re: About USB hard drives and errors
On Sat, 10 Apr 2010 22:06:06 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... I got a little less timid and tried running smartctl even though I was quite unsure of what to expect. It ran. Each of the three USB HD gave somewhat different output, but none gave output that claimed there was a working SMART on the drive. These drives are Western Digital (WD). The WD web site mentions SMART and also uses the words Smart Drive to mean something else that is a proprietary marketing thing, AFAICT. I was unable to find a list of part #s for drives that support S.M.A.R.T. I think I should be in the market for a better class of drives, but not this weekend. Thanks for the help. My understanding is that S.M.A.R.T. doesn't generally work over USB. As Wikipedia puts it: For example, few external drives connected via USB and Firewire correctly send S.M.A.R.T. data over those interfaces. http://en.wikipedia.org/wiki/S.M.A.R.T.#Standards_and_implementation Celejar -- foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411002510.020a145a.cele...@gmail.com
Re: About USB hard drives and errors
On 20100411_002510, Celejar wrote: On Sat, 10 Apr 2010 22:06:06 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... I got a little less timid and tried running smartctl even though I was quite unsure of what to expect. It ran. Each of the three USB HD gave somewhat different output, but none gave output that claimed there was a working SMART on the drive. These drives are Western Digital (WD). The WD web site mentions SMART and also uses the words Smart Drive to mean something else that is a proprietary marketing thing, AFAICT. I was unable to find a list of part #s for drives that support S.M.A.R.T. I think I should be in the market for a better class of drives, but not this weekend. Thanks for the help. My understanding is that S.M.A.R.T. doesn't generally work over USB. As Wikipedia puts it: For example, few external drives connected via USB and Firewire correctly send S.M.A.R.T. data over those interfaces. http://en.wikipedia.org/wiki/S.M.A.R.T.#Standards_and_implementation Celejar So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411044157.gd10...@big.lan.gnu
Re: About USB hard drives and errors
On Sat, 10 Apr 2010 22:41:57 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? I'm sorry, I don't really know much about this stuff. I'm just repeating what I've heard (and seen, in my very limited experience). Celejar -- foffl.sourceforge.net - Feeds OFFLine, an offline RSS/Atom aggregator mailmin.sourceforge.net - remote access via secure (OpenPGP) email ssuds.sourceforge.net - A Simple Sudoku Solver and Generator -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411005504.af4f8227.cele...@gmail.com
Re: About USB hard drives and errors
On 20100411_005504, Celejar wrote: On Sat, 10 Apr 2010 22:41:57 -0600 Paul E Condon pecon...@mesanetworks.net wrote: ... So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? I'm sorry, I don't really know much about this stuff. I'm just repeating what I've heard (and seen, in my very limited experience). Celejar Your comment was/is helpful to me. Thanks. Please don't drop off this thread because I'm sometimes too terse. Following some links from article you cited lead me to some threads on SATA and eSATA, so I am not all at my wits end now. But it is well past my bed time. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100411054850.ge10...@big.lan.gnu
Re: About USB hard drives and errors
Paul E Condon put forth on 4/10/2010 11:41 PM: So, the fact that my WD drives don't play well with S.M.A.R.T doesn't make them special, and I should not spend much, if any, time looking for a USB solution. What other options are there for external HD? You're got 3 USB hard drives already, and you're throwing them out and looking for another solution, just because they don't do S.M.A.R.T? If neither USB nor firewire do smart, you choices are very limited. The only other realistic option I know of is eSATA, but a quick scan of Newegg shows only 6 devices total, 2 DVR expander drives and 4 eSATA RAID enclosures. I have no idea if the DVR expander drives will work with a standard PC setup. They should. The two DVR drives are both 1TB and both just under $130 USD, one Iomega and one Western Digital: http://www.newegg.com/Product/Product.aspx?Item=N82E16822186175 http://www.newegg.com/Product/Product.aspx?Item=N82E16822136384 Are you planning on connecting this drive to a laptop or desktop? To multiple computers or just one computer? -- Stan -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bc16321.8000...@hardwarefreak.com
Re: About USB hard drives and errors
On Thu, 08 Apr 2010 18:44:33 -0600, Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. I do get errors. So I decided to do scans for bad blocks. The drives I'm using are all Western Digital because they have been the lowest cost at the times I buy at Costco. Also all are 500GB. (...) Try passing the SMART test (by using the manufacturer's test disk or smartctl, but as they are attached to a USB port, smartmontools may have problems to detect and scan the device). It won't tell you a concrete number of bad blocks, but higher values in Reallocated Sectors Count could give you an idea about the remapping operations performed by the disk (the lower the better). Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/pan.2010.04.09.06.38...@gmail.com
Re: About USB hard drives and errors
On Thursday 08 April 2010 04:44:33 pm Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. I do get errors. So I decided to do scans for bad blocks. The drives I'm using are all Western Digital because they have been the lowest cost at the times I buy at Costco. Also all are 500GB. e2fsck -c device is supposed to scan for bad blocks and allocate them to a special inode so that they cannot be used. It runs for 3 to 4 hours and then says its finished with no indication of how many bad blocks it found. dumpe2fs -b device is supposed to print the bad blocks that have been marked on a device. When I run it, it prints nothing. I find it hard to believe that a 500GB HD contains ZERO bad blocks. Especially one on which I have witnessed error messages about I/O errors in writing the journal. I can find no information about what, exactly, dumpe2fs is supposed to print. The wording seems to be that it prints the contents of the bad blocks. But in other places it seems that it prints a list of block numbers, or maybe cylinder/sector/head. Since I see nothing, I don't have a clue as to what I should see. Has anyone ever used these programs? Have you ever seen useful output? What SHOULD they do (with a little more specificity and believability)? -- Paul E Condon pecon...@mesanetworks.net I used 'smartmontools' AFAIKT, lots of drives have errors, errors get re-mapped by the hd firmware, some drives are more accurate than others. Never did get straight, but the drives keep working. smartmontools uses badblocks , if memory serves. -- Greg Madden Precision Air Balance, Inc. Phone: (907)276-0461 -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201004082141.26567.p...@gci.net
Re: About USB hard drives and errors
On 2010-04-08 20:27, Ron Johnson wrote: On 2010-04-08 19:44, Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. I do get errors. So I decided to do scans for bad blocks. The drives I'm using are all Western Digital because they have been the lowest cost at the times I buy at Costco. Also all are 500GB. e2fsck -c device is supposed to scan for bad blocks and allocate them to a special inode so that they cannot be used. It runs for 3 to 4 hours and then says its finished with no indication of how many bad blocks it found. [snip] Has anyone ever used these programs? Have you ever seen useful output? What SHOULD they do (with a little more specificity and believability)? Not a direct answer to your question, but: I never leave home without -vfFC0. (The Unix Way is to say something only upon failure, but I like continuous feedback.) (The dumpe2fs -b command should show you all your bad blocks.) Here's the result of me checking for bad blocks: # e2fsck -c -vfFC0 /dev/mapper/main_huge_vg-main_huge_lv e2fsck 1.41.10 (10-Feb-2009) Checking for bad blocks (read-only test): done BIG_LV: Updating bad block inode. Note!! Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information BIG_LV: * FILE SYSTEM WAS MODIFIED * 301084 inodes used (0.12%) 14172 non-contiguous files (4.7%) 121 non-contiguous directories (0.0%) # of inodes with ind/dind/tind blocks: 0/0/0 Extent depth histogram: 294700/6219 560591687 blocks used (58.03%) 0 bad blocks 128 large files 292926 regular files 7972 directories 0 character device files 0 block device files 0 fifos 0 links 177 symbolic links (155 fast symbolic links) 0 sockets 301075 files -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bbf46ba.1060...@cox.net
Re: About USB hard drives and errors
On 20100409_102442, Ron Johnson wrote: On 2010-04-08 20:27, Ron Johnson wrote: On 2010-04-08 19:44, Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. I do get errors. So I decided to do scans for bad blocks. The drives I'm using are all Western Digital because they have been the lowest cost at the times I buy at Costco. Also all are 500GB. e2fsck -c device is supposed to scan for bad blocks and allocate them to a special inode so that they cannot be used. It runs for 3 to 4 hours and then says its finished with no indication of how many bad blocks it found. [snip] Has anyone ever used these programs? Have you ever seen useful output? What SHOULD they do (with a little more specificity and believability)? Not a direct answer to your question, but: I never leave home without -vfFC0. (The Unix Way is to say something only upon failure, but I like continuous feedback.) (The dumpe2fs -b command should show you all your bad blocks.) Here's the result of me checking for bad blocks: # e2fsck -c -vfFC0 /dev/mapper/main_huge_vg-main_huge_lv e2fsck 1.41.10 (10-Feb-2009) Checking for bad blocks (read-only test): done BIG_LV: Updating bad block inode. Note!! Pass 1: Checking inodes, blocks, and sizes Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information BIG_LV: * FILE SYSTEM WAS MODIFIED * 301084 inodes used (0.12%) 14172 non-contiguous files (4.7%) 121 non-contiguous directories (0.0%) # of inodes with ind/dind/tind blocks: 0/0/0 Extent depth histogram: 294700/6219 560591687 blocks used (58.03%) 0 bad blocks 128 large files 292926 regular files 7972 directories 0 character device files 0 block device files 0 fifos 0 links 177 symbolic links (155 fast symbolic links) 0 sockets 301075 files Thanks Ron. Your option strings will do wonders for future e2fsck runs. But ... Why does the output say that the disk was modified during the run? There were no badblocks found. What needed modification? Do you have similar magic for dumpe2fs? Of course your output presented here indicates that you HD is not one on which you can verify badblock printing, since your disk has no badblocks. -- Paul E Condon pecon...@mesanetworks.net -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100409160423.gb5...@big.lan.gnu
Re: About USB hard drives and errors
On 2010-04-09 11:04, Paul E Condon wrote: [snip] But ... Why does the output say that the disk was modified during the run? There were no badblocks found. What needed modification? Good question. Do you have similar magic for dumpe2fs? Nope. Of course your output presented here indicates that you HD is not one on which you can verify badblock printing, since your disk has no badblocks. I could have done a -cc test (which the man page says is a non-destructive read-write test) but I don't want to take the chance of making a mistake or there being a bug and having it foul the partition. -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bbf6045.6030...@cox.net
Re: About USB hard drives and errors
On 2010-04-08 19:44, Paul E Condon wrote: I want to use the low cost high capacity hard drives that are for sale in places like Best Buy and Costco. I have put ext3 on several of them and started experimenting. The results so far are puzzling. I do get errors. So I decided to do scans for bad blocks. The drives I'm using are all Western Digital because they have been the lowest cost at the times I buy at Costco. Also all are 500GB. e2fsck -c device is supposed to scan for bad blocks and allocate them to a special inode so that they cannot be used. It runs for 3 to 4 hours and then says its finished with no indication of how many bad blocks it found. [snip] Has anyone ever used these programs? Have you ever seen useful output? What SHOULD they do (with a little more specificity and believability)? Not a direct answer to your question, but: I never leave home without -vfFC0. (The Unix Way is to say something only upon failure, but I like continuous feedback.) -- Dissent is patriotic, remember? -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4bbe827d.2000...@cox.net
