Re: [gentoo-user] repair uefi vfat /boot?
On 24/03/20 20:11, Caveman Al Toraboran wrote: > from now on, will start the smart daemon + some raid solution (after replacing > faulty disk). https://raid.wiki.kernel.org/index.php/Linux_Raid Cheers, Wol
Re: [gentoo-user] repair uefi vfat /boot?
‐‐‐ Original Message ‐‐‐ On Monday, March 23, 2020 3:33 PM, Michael wrote: > 'man smartctl' provides some explanation with regards to reading the Attribute > values reported by the firmware of the disk, as does Wikipedia: > > https://en.wikipedia.org/wiki/S.M.A.R.T.#Known_ATA_S.M.A.R.T._attributes > > However, with Seagate drives in particular reported values by the firmware are > counterintuitive and can cause confusion: > > http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html > > Not withstanding the above, if you look under the section "-A --attributes" in > the manual you'll see the following. If an attribute type is of type 'Pre- > fail' and is equal or less than the Threshold value then there is a problem. > If the WHEN_FAILED column shows a dash, this means the drive has not failed > yet with respect to this attribute. > > Looking at your SMART table we can see no attribute has failed completely yet, > but we see some potentially worrying signs too. > > There have been a number of (ID 1) Raw Read Errors and also (ID 195) Hardware > ECC Recovered sectors. However, there are a large number of (ID 187) Reported > Uncorrectable errors - these are sectors the Hardware ECC failed to correct. > > The next value (ID 188) Command Timeout is also of some concern, showing a > count of 30 aborted operations by the HDD. > > There are also some Bad Blocks, with a raw value of 49. If you see this > number increasing over time, it means potentially more and more of your data > can be lost. It would explain for example why some of the files you stored in > the vfat partition are showing a size of zero. The value of (ID 197) Current > Pending Sector of 12 is also worrying - there are 12 sectors waiting to be > remapped to a more healthy part of the disk because of unrecoverable read > errors. The following attribute (ID 198) Offline Uncorrectable Error counts > also shows 12. These are indications your hard disk is failing probably due > to some platter surface damage and you should take all data off it. At some > point it will fail completely and until then loss of data is likely to > increase. amazing help :). thank you very much for walking me throughout this. highly appreciated. from now on, will start the smart daemon + some raid solution (after replacing faulty disk). (side note: and psu's fuse blew up a few days ago. fortunately important data is backed up. but i wonder if this is related? or is it just that i'm unlucky?) rgrds, cm
Re: [gentoo-user] repair uefi vfat /boot?
On Sunday, 22 March 2020 22:51:20 GMT Caveman Al Toraboran wrote: > ‐‐‐ Original Message ‐‐‐ > > On Sunday, March 22, 2020 12:50 PM, Michael wrote: > > What Stefan said - the disk is on its way out and autorecovery of bad > > sectors is failing. You could run: > > > > smartctl -a /dev/sda > > > > to see what errors it reports, but in the first instance if the data on > > this disk is valuable I suggest you get another disk and immediately > > transfer all useful/recoverable files off this drive. If the value of the > > data is not high/irreplaceable, then carry on using it - it may take > > years and years before it fails completely. > > > > To reallocate a bad block on your disk and hope more won't arrive > > overnight, have a read at this page: > > > > https://www.smartmontools.org/wiki/BadBlockHowto > > i get this output: > > https://gist.github.com/Al-Caveman/b3be1a623f20b55de80d0e2eddcda5d4 > > how to read this? seems very cryptic to me. > how is this better than dmest -T? > > thx. 'man smartctl' provides some explanation with regards to reading the Attribute values reported by the firmware of the disk, as does Wikipedia: https://en.wikipedia.org/wiki/S.M.A.R.T.#Known_ATA_S.M.A.R.T._attributes However, with Seagate drives in particular reported values by the firmware are counterintuitive and can cause confusion: http://www.users.on.net/~fzabkar/HDD/Seagate_SER_RRER_HEC.html Not withstanding the above, if you look under the section "-A --attributes" in the manual you'll see the following. If an attribute type is of type 'Pre- fail' and is equal or less than the Threshold value then there is a problem. If the WHEN_FAILED column shows a dash, this means the drive has not failed yet with respect to this attribute. Looking at your SMART table we can see no attribute has failed completely yet, but we see some potentially worrying signs too. There have been a number of (ID 1) Raw Read Errors and also (ID 195) Hardware ECC Recovered sectors. However, there are a large number of (ID 187) Reported Uncorrectable errors - these are sectors the Hardware ECC failed to correct. The next value (ID 188) Command Timeout is also of some concern, showing a count of 30 aborted operations by the HDD. There are also some Bad Blocks, with a raw value of 49. If you see this number increasing over time, it means potentially more and more of your data can be lost. It would explain for example why some of the files you stored in the vfat partition are showing a size of zero. The value of (ID 197) Current Pending Sector of 12 is also worrying - there are 12 sectors waiting to be remapped to a more healthy part of the disk because of unrecoverable read errors. The following attribute (ID 198) Offline Uncorrectable Error counts also shows 12. These are indications your hard disk is failing probably due to some platter surface damage and you should take all data off it. At some point it will fail completely and until then loss of data is likely to increase. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] repair uefi vfat /boot?
"madscientistatlarge" , 23.03.2020, 01:36: > I usually toss them across the room so I know not to trust them. Don't do that! Open the thing, salvage the strong magnets and keep the disk platters. They make for pretty good mirrors in hobby projects and magnets are always useful. I use one of those to keep my office door "almost closed" in the winter. Helps keep the room warm but the dog can still push the door open if he needs me to open the back door for him :-) s.
Re: [gentoo-user] repair uefi vfat /boot?
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Sunday, March 22, 2020 5:54 PM, Wols Lists wrote: > On 22/03/20 22:40, Caveman Al Toraboran wrote: > > > any idea why 1 partition (uefi vfat) is suffering errors, but the other > > ext4 isn't? > > Simple. If the surface is decaying, it will be localised. It's decided > to hit the locality of the uefi partition. > > This is what would, in the old days, lead to a head crash. Once the > magnetic layer started to physically deform (flake), the head would hit > it which would start a cascade of debris scouring the disk surface and > trashing the heads. > > Cheers, > Wol Drives may have failed that way at one time (and still can on some drives if they are thumped hard enough), the main issue, according to drive makers is that the coating starts to crystallize, which changes it's magnetic properties (heads and how they are driven are pretty tightly optimized now). These crystals grow over time, affecting more and more of the platter they are on. This is why altitude and temperature cycles (well within operating specs.) will accelerate this failure, both tend to cause crystal growth. In any case, once the drive is out of spare sectors your' data is going to disappear. On all modern drives, running out of spare sectors means back it up quick, preferably immediately without a shutdown or reboot. I've seen drives dying this way, until they run out of sectors (quietly) you get data changes as it miss copies the sectors involved to backup sectors (they re-read until the CRC is right, in this case it's often until there are multiple errors that make it appear to read correctly). If you can hear your drive, and it's constantly seeking when it should be idle, or if you hear it repeatedly load and unload the head it's time to get a spare quickly, if you care about your' data but don't have it all backed up. High operating temperature also accelerates this. I've found keeping them under 100 Deg F. makes them last a long time, even drives I've pulled out of Tivo units have lasted me years this way, and in cable boxes or Tivo's the drive run criminally hot (at least the older ones, don't know about newer ones). Bus for now, just get a new drive, don't waste effort beating a dying horse. It will only frustrate you. I've even seen a drive that passed the "long" smart test when it was failing, the data would change within a few hours! When a drive fails me, after any possible back up, I usually toss them across the room so I know not to trust them. If drives do get particulate matter in them (sometimes the seals fail, or are just bad, or the air filter get poked etc.) I pulled a drive out of a laptop once, took off the cover (because it was already dead) and found about a quarter teaspoon of black power that was supposed to be on the platters. If this happens the heads get ground and more and more coating gets ground off, it accelerates very quickly. I've taken apart dozens of failed drives, but only found this once on a failed drive. Many drives are rated for 20G of shock while operating (200G when not spinning). I wouldn't take those number too seriously though. Hope you didn't lose valuable data.
Re: [gentoo-user] repair uefi vfat /boot?
On 22/03/20 22:40, Caveman Al Toraboran wrote: > any idea why 1 partition (uefi vfat) is suffering errors, but the other ext4 > isn't? Simple. If the surface is decaying, it will be localised. It's decided to hit the locality of the uefi partition. This is what would, in the old days, lead to a head crash. Once the magnetic layer started to physically deform (flake), the head would hit it which would start a cascade of debris scouring the disk surface and trashing the heads. Cheers, Wol
Re: [gentoo-user] repair uefi vfat /boot?
‐‐‐ Original Message ‐‐‐ On Sunday, March 22, 2020 12:50 PM, Michael wrote: > What Stefan said - the disk is on its way out and autorecovery of bad sectors > is failing. You could run: > > smartctl -a /dev/sda > > to see what errors it reports, but in the first instance if the data on this > disk is valuable I suggest you get another disk and immediately transfer all > useful/recoverable files off this drive. If the value of the data is not > high/irreplaceable, then carry on using it - it may take years and years > before it fails completely. > > To reallocate a bad block on your disk and hope more won't arrive overnight, > have a read at this page: > > https://www.smartmontools.org/wiki/BadBlockHowto i get this output: https://gist.github.com/Al-Caveman/b3be1a623f20b55de80d0e2eddcda5d4 how to read this? seems very cryptic to me. how is this better than dmest -T? thx.
Re: [gentoo-user] repair uefi vfat /boot?
‐‐‐ Original Message ‐‐‐ On Sunday, March 22, 2020 11:53 AM, Stefan Schmiedl wrote: > Messages like > > > > [sda] tag#6 Sense Key : Medium Error [current] > > > > [sda] tag#6 Add. Sense: Unrecovered read error - auto reallocate failed > > usually point towards towards problems with the magnetic layer > on the disk. These do not get better over time, they only get > worse. > > Then we have "auto reallocate failed", which means that the HD > controller tried to reassign the damaged sector to another working > sector, unsuccessfully. > > > what do you think? > > If there is anything of value on the disk, get a new one right now. done (fortunately important data got backed up). any idea why 1 partition (uefi vfat) is suffering errors, but the other ext4 isn't?
Re: [gentoo-user] repair uefi vfat /boot?
Am Sonntag, 22. März 2020, 10:11:38 CET schrieb Dale: > [...] I can't recall how SMART says > this but a drive that failed on me several years ago said the drive was > dying and it was going to die soon, I think 24 hours was mentioned. [...] You're probably thining of what smartctl prints after "SMART overall-health self-assessment test result:" when it thinks the drive is failing (it claims the drive is going to fail within the next 24 hours). Greetings -- Marc Joliet -- "People who think they know everything really annoy those of us who know we don't" - Bjarne Stroustrup signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] repair uefi vfat /boot?
Michael wrote: > On Sunday, 22 March 2020 07:53:18 GMT Stefan Schmiedl wrote: >> "Caveman Al Toraboran" , 22.03.2020, 02:29: >>> On Saturday, March 21, 2020 8:03 PM, Stefan Schmiedl wrote: "Caveman Al Toraboran" toraboracave...@protonmail.com, 21.03.2020, 14:49: > questions: > * what's going on? > * how to find out? "dmesg -T" is your friend. It should show the error messages with their timestamps. > * how to fix? For spinning HDs: If the error messages point towards faulty sectors that can't be written, get a new drive and migrate your data. If the messages don't contain sectors, check and/or replace the cabling. If the problem persists, get a new drive etc... >>> i get this: http://codepad.org/MVeqeBBu >>> it mentions "sector", but not sure if it is what you mean. >> Messages like >> >> >> [sda] tag#6 Sense Key : Medium Error [current] >> >> [sda] tag#6 Add. Sense: Unrecovered read error - auto reallocate failed >> >> usually point towards towards problems with the magnetic layer >> on the disk. These do not get better over time, they only get >> worse. >> >> Then we have "auto reallocate failed", which means that the HD >> controller tried to reassign the damaged sector to another working >> sector, unsuccessfully. >> >>> what do you think? >> If there is anything of value on the disk, get a new one right now. >> >> Good luck, >> s. > What Stefan said - the disk is on its way out and autorecovery of bad sectors > is failing. You could run: > > smartctl -a /dev/sda > > to see what errors it reports, but in the first instance if the data on this > disk is valuable I suggest you get another disk and *immediately* transfer > all > useful/recoverable files off this drive. If the value of the data is not > high/irreplaceable, then carry on using it - it may take years and years > before it fails completely. > > To reallocate a bad block on your disk and hope more won't arrive overnight, > have a read at this page: > > https://www.smartmontools.org/wiki/BadBlockHowto Just to add something to think about. I can't recall how SMART says this but a drive that failed on me several years ago said the drive was dying and it was going to die soon, I think 24 hours was mentioned. I unmounted that drive and removed power until I had a replacement drive. Once the replacement drive came in, I hooked the drive back up and copied the data over to the new drive. I seem to recall I had a couple pics and a couple text files that were corrupted. I might also add, I started running some tests on the drive and it died a few hours later. I could hear it spin but it wouldn't even mount anymore. If you see something similar to that, I suggest you take action since it could be accurate. It was in my case. If the data has no value or you have known good backups, then it may not matter so much. Dale :-) :-)
Re: [gentoo-user] repair uefi vfat /boot?
On Sunday, 22 March 2020 07:53:18 GMT Stefan Schmiedl wrote: > "Caveman Al Toraboran" , 22.03.2020, 02:29: > > On Saturday, March 21, 2020 8:03 PM, Stefan Schmiedl wrote: > >> "Caveman Al Toraboran" toraboracave...@protonmail.com, 21.03.2020, 14:49: > >> > questions: > >> > * what's going on? > >> > * how to find out? > >> > >> "dmesg -T" is your friend. It should show the error messages > >> with their timestamps. > >> > >> > * how to fix? > >> > >> For spinning HDs: > >> > >> If the error messages point towards faulty sectors that can't be > >> written, get a new drive and migrate your data. If the messages > >> don't contain sectors, check and/or replace the cabling. If the > >> problem persists, get a new drive etc... > > > > i get this: http://codepad.org/MVeqeBBu > > it mentions "sector", but not sure if it is what you mean. > > Messages like > > >> [sda] tag#6 Sense Key : Medium Error [current] > >> [sda] tag#6 Add. Sense: Unrecovered read error - auto reallocate failed > > usually point towards towards problems with the magnetic layer > on the disk. These do not get better over time, they only get > worse. > > Then we have "auto reallocate failed", which means that the HD > controller tried to reassign the damaged sector to another working > sector, unsuccessfully. > > > what do you think? > > If there is anything of value on the disk, get a new one right now. > > Good luck, > s. What Stefan said - the disk is on its way out and autorecovery of bad sectors is failing. You could run: smartctl -a /dev/sda to see what errors it reports, but in the first instance if the data on this disk is valuable I suggest you get another disk and *immediately* transfer all useful/recoverable files off this drive. If the value of the data is not high/irreplaceable, then carry on using it - it may take years and years before it fails completely. To reallocate a bad block on your disk and hope more won't arrive overnight, have a read at this page: https://www.smartmontools.org/wiki/BadBlockHowto signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] repair uefi vfat /boot?
"Caveman Al Toraboran" , 22.03.2020, 02:29: > On Saturday, March 21, 2020 8:03 PM, Stefan Schmiedl wrote: >> "Caveman Al Toraboran" toraboracave...@protonmail.com, 21.03.2020, 14:49: >> >> > questions: >> > * what's going on? >> > * how to find out? >> >> "dmesg -T" is your friend. It should show the error messages >> with their timestamps. >> >> > * how to fix? >> >> For spinning HDs: >> >> If the error messages point towards faulty sectors that can't be >> written, get a new drive and migrate your data. If the messages >> don't contain sectors, check and/or replace the cabling. If the >> problem persists, get a new drive etc... > i get this: http://codepad.org/MVeqeBBu > it mentions "sector", but not sure if it is what you mean. Messages like >> [sda] tag#6 Sense Key : Medium Error [current] >> [sda] tag#6 Add. Sense: Unrecovered read error - auto reallocate failed usually point towards towards problems with the magnetic layer on the disk. These do not get better over time, they only get worse. Then we have "auto reallocate failed", which means that the HD controller tried to reassign the damaged sector to another working sector, unsuccessfully. > what do you think? If there is anything of value on the disk, get a new one right now. Good luck, s.
Re: [gentoo-user] repair uefi vfat /boot?
‐‐‐ Original Message ‐‐‐ On Saturday, March 21, 2020 8:03 PM, Stefan Schmiedl wrote: > "Caveman Al Toraboran" toraboracave...@protonmail.com, 21.03.2020, 14:49: > > > questions: > > * what's going on? > > * how to find out? > > "dmesg -T" is your friend. It should show the error messages > with their timestamps. > > > * how to fix? > > For spinning HDs: > > If the error messages point towards faulty sectors that can't be > written, get a new drive and migrate your data. If the messages > don't contain sectors, check and/or replace the cabling. If the > problem persists, get a new drive etc... i get this: http://codepad.org/MVeqeBBu it mentions "sector", but not sure if it is what you mean. what do you think?
Re: [gentoo-user] repair uefi vfat /boot?
"Caveman Al Toraboran" , 21.03.2020, 14:49: > questions: > * what's going on? > * how to find out? "dmesg -T" is your friend. It should show the error messages with their timestamps. > * how to fix? For spinning HDs: If the error messages point towards faulty sectors that can't be written, get a new drive and migrate your data. If the messages don't contain sectors, check and/or replace the cabling. If the problem persists, get a new drive etc... For SSDs: Try to get as much data off the disk as you can without rebooting or power cycling. When these things fail, they tend to fail completely. > symptoms: > * can't write (gives read/write error). > * but files can get created and deleted. > * newly created files, which also have failed writes > have 0 bytes in them. > * mount /dev/sda1 /boot is slow. > * umount /boot is slow. > cave ~ # fsck.vfat -v -a -w /dev/sda1 > fsck.fat 4.1 (2017-01-24) > Checking we can access the last sector of the filesystem > 0x41: Dirty bit is set. Fs was not properly unmounted and some data may be > corrupt. > Automatically removing dirty bit. > Boot sector contents: > System ID "mkfs.fat" > Media byte 0xf8 (hard disk) > 512 bytes per logical sector > 4096 bytes per cluster > 32 reserved sectors > First FAT starts at byte 16384 (sector 32) > 2 FATs, 32 bit entries > 565248 bytes per FAT (= 1104 sectors) > Root directory start at cluster 2 (arbitrary size) > Data area starts at byte 1146880 (sector 2240) > 140520 data clusters (575569920 bytes) > 63 sectors/track, 255 heads > 2048 hidden sectors > 1126400 sectors total > Got 4096 bytes instead of 562088 at 16384 If you're lucky and your hard disks supports it, you could try - migrating data to another drive - write to every sector on the drive, as in dd if=/dev/zero of=/dev/sda or with some other non-zero pattern. - hope that the controller catches the errors and marks the faulty blocks as bad so that they are not accessed in the future. - reformat the drive and trust in your luck s.
Re: [gentoo-user] repair uefi vfat /boot?
Oops! My bad - the title said it: UEFI boot, but somehow I missed it. Same things comments, but since this not a USB drive, check the output of smartclt -a and run a few tests too. On a spinning disk the faults could be related to hard drive failure, or the SATA cable coming loose. Reseat the cable to see if the errors go away. On Saturday, 21 March 2020 15:28:44 GMT Michael wrote: > On Saturday, 21 March 2020 13:49:04 GMT Caveman Al Toraboran wrote: > > questions: > > * what's going on? > > It looks as if your USB stick connector or its microcontroller is faulty. > There is also a smaller probability the USB port on the PC is playing up. > > > * how to find out? > > Look at dmesg -w and syslog for I/O errors. > > > * how to fix? > > If this is a hardware fault, check for dirty/oxidized contacts on the USB > connector and clean these as appropriate. If they look OK, try a different > USB port on the PC. > > > symptoms: > > * can't write (gives read/write error). > > * but files can get created and deleted. > > * newly created files, which also have failed writes > > > >have 0 bytes in them. > > > > * mount /dev/sda1 /boot is slow. > > * umount /boot is slow. > > > > cave ~ # fsck.vfat -v -a -w /dev/sda1 > > fsck.fat 4.1 (2017-01-24) > > Checking we can access the last sector of the filesystem > > 0x41: Dirty bit is set. Fs was not properly unmounted and some data may be > > corrupt. Automatically removing dirty bit. > > Boot sector contents: > > System ID "mkfs.fat" > > Media byte 0xf8 (hard disk) > > > >512 bytes per logical sector > > > > 4096 bytes per cluster > > > > 32 reserved sectors > > > > First FAT starts at byte 16384 (sector 32) > > > > 2 FATs, 32 bit entries > > > > 565248 bytes per FAT (= 1104 sectors) > > > > Root directory start at cluster 2 (arbitrary size) > > Data area starts at byte 1146880 (sector 2240) > > > > 140520 data clusters (575569920 bytes) > > > > 63 sectors/track, 255 heads > > > > 2048 hidden sectors > > > >1126400 sectors total > > > > Got 4096 bytes instead of 562088 at 16384 > > > > > > > > > > thoughts? > > > > rgrds, > > cm. > > > > Sent with ProtonMail Secure Email. > > You could try formatting the USB drive with -v -t and then monitor the logs > to see if the errors persist. If the errors do not come back, then the > problem is unlikely to have been caused by hardware faults. If they do, > its time to destroy the USB drive (unless the data on it does not contain > private information) and throw it away. signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] repair uefi vfat /boot?
On Saturday, 21 March 2020 13:49:04 GMT Caveman Al Toraboran wrote: > questions: > * what's going on? It looks as if your USB stick connector or its microcontroller is faulty. There is also a smaller probability the USB port on the PC is playing up. > * how to find out? Look at dmesg -w and syslog for I/O errors. > * how to fix? If this is a hardware fault, check for dirty/oxidized contacts on the USB connector and clean these as appropriate. If they look OK, try a different USB port on the PC. > symptoms: > * can't write (gives read/write error). > * but files can get created and deleted. > * newly created files, which also have failed writes >have 0 bytes in them. > * mount /dev/sda1 /boot is slow. > * umount /boot is slow. > > > cave ~ # fsck.vfat -v -a -w /dev/sda1 > fsck.fat 4.1 (2017-01-24) > Checking we can access the last sector of the filesystem > 0x41: Dirty bit is set. Fs was not properly unmounted and some data may be > corrupt. Automatically removing dirty bit. > Boot sector contents: > System ID "mkfs.fat" > Media byte 0xf8 (hard disk) >512 bytes per logical sector > 4096 bytes per cluster > 32 reserved sectors > First FAT starts at byte 16384 (sector 32) > 2 FATs, 32 bit entries > 565248 bytes per FAT (= 1104 sectors) > Root directory start at cluster 2 (arbitrary size) > Data area starts at byte 1146880 (sector 2240) > 140520 data clusters (575569920 bytes) > 63 sectors/track, 255 heads > 2048 hidden sectors >1126400 sectors total > Got 4096 bytes instead of 562088 at 16384 > > > > > thoughts? > > rgrds, > cm. > > Sent with ProtonMail Secure Email. You could try formatting the USB drive with -v -t and then monitor the logs to see if the errors persist. If the errors do not come back, then the problem is unlikely to have been caused by hardware faults. If they do, its time to destroy the USB drive (unless the data on it does not contain private information) and throw it away. signature.asc Description: This is a digitally signed message part.
