Re: [bug-ddrescue] ddrescue strange read behaviour
Hi, Adding up some detail related to my "-b" usage flag. When I define value for the "-b" flag I am careful enough to choose multiple of 512 values. So in my experiments and trials I started with 4096 and so on. I have settled with the value 10240 ... Although I do not see any difference by using the "-d" ( direct disk acess ) mode .. Most of the time I get this kind of ata error : ata8.00: cmd c8/00:08:00:4b:f9/00:00:00:00:00/ed tag 0 dma 10240 in res 51/04:08:00:4b:f9/00:00:00:00:00/ed Emask 0x1 (device error) With the value after the "dma" defined in the ddrescue -b flag evocation ... Although I get sometimes different entries mostly with "dma 4096 in" Despite the fact that am I still no recovering any of the marked zones ... So at the moment I still have 2.1 GiB of data to "scrap" and with the default block ( 512) and stopping for each badblock plus powercycling manually the HDD It would take an enormous amount of time ... Any idea how to speed things up are welcome .. Thanks for the fast answers and support Marco Marques Although this weekend I started fiddling a little bit with the "-b " flag, order to mark some bad parts ... As far as I tried it is working as expected as I get more bad blocks marked faster ( bigger volume ) at the expense of not recovering the data. Can I use that way or I am doing it completly wrong ? You are doing it completely wrong. :-) '-b' must be (a multiple of) the real sector size, or all reads will instantly fail in direct disc access mode.
Re: [bug-ddrescue] ddrescue strange read behaviour
Marco Marques wrote: The idea of a counter in the mapfile it might be useful in case of tests and trials ( as I am currently ) where each time I get an error the ddrescue simply exits writing the file . As sometimes try a new zone I copy the full domain mapfile aside and make some tests , returning to it later , still keeping writing to the same fullfile In this case the datestamp is not enough to aid ... If the timestamp is not enough ¿?, then you may devise a naming convention for test mapfiles that allow you to keep track of what you have done. Making a backwards-incompatible change to add a counter seems too much truble. Although this weekend I started fiddling a little bit with the "-b " flag , order to mark some bad parts ... As far as I tried it is working as expected as I get more bad blocks marked faster ( bigger volume ) at the expense of not recovering the data. Can I use that way or I am doing it completly wrong ? You are doing it completely wrong. :-) '-b' must be (a multiple of) the real sector size, or all reads will instantly fail in direct disc access mode. Another comment is the "-i" only after several analysis I have found that ddrescue starts to read in the next domain mapfile section, explaining why if I call "-i 4G" ddrescue started to read at the "9G" section. This should not happen. If there is a pending area at pos X, '-i X' should try to read it. Please verify the exact numbers you pass to -i. Another detail, refering the "line type runtime map" I understand the difiiculty of condensing the million datablocks but the idea would be to have a visual aid even it would be rough with at least the reading position in the full domain mapfile would be nice... OK. I'll see what I can do. Best regards, Antonio.
Re: [bug-ddrescue] ddrescue strange read behaviour
Hi, Sorry again for the delayed answer ... The idea of a counter in the mapfile it might be useful in case of tests and trials ( as I am currently ) where each time I get an error the ddrescue simply exits writing the file . As sometimes try a new zone I copy the full domain mapfile aside and make some tests , returning to it later , still keeping writing to the same fullfile In this case the datestamp is not enough to aid ... Although this weekend I started fiddling a little bit with the "-b " flag , order to mark some bad parts ... As far as I tried it is working as expected as I get more bad blocks marked faster ( bigger volume ) at the expense of not recovering the data . Can I use that way or I am doing it completly wrong ? Another comment is the "-i" only after several analysis I have found that ddrescue starts to read in the next domain mapfile section, explaining why if I call "-i 4G" ddrescue started to read at the "9G" section. Perhaps a small info warning at the header would be useful to avoid such confusion ... And idea would be something like "Unable to read from starting reading at map area in " I know that now It gives a line indicating the start and end of the domain in full bytes but it is not imediatly understable . Another detail, refering the "line type runtime map" I understand the difiiculty of condensing the million datablocks but the idea would be to have a visual aid even it would be rough with at least the reading position in the full domain mapfile would be nice... Nonetheless thank you
Re: [bug-ddrescue] ddrescue strange read behaviour
Hello Marco, Marco Marques wrote: Nonetheless I do have some other details to ask : if there is a way to keep the marked bad sectores when "clearing" the trimming phase "-M" flag . The goal of the '-M, --retrim' option is to mark all failed blocks inside the rescue domain as non-trimmed in order to try them again. If you want to try them again without unmarking them, you may use '--retry-passes=1'. See http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Invoking-ddrescue Another detail is related to the "-K" flag ... Is it applied in the Trimming or Scrapping phases ? Ddrescue only skips during the copying phase. It does not make sense to blindly skip ahead when the bad areas have already been delimited. See http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Algorithm 2) (First phase; Copying) Copying is done in up to five passes. The first pass reads the non-tried parts of the input file, marking the failed blocks as non-trimmed and skipping beyond them. The second pass delimits the blocks skipped by the first pass. The first two passes also skip beyond slow areas. The skipped areas are tried later in one or three additional passes (before trimming). The copying direction is reversed after each pass until all the rescue domain is tried. Although I am tented to edit manually the mapfile I will try to avoid it. You do not need to edit the mapfile manually. Instead use the domain setting options (-i and -s) to restrict the rescue to the area of interest. An idea would be to use something like a "status" bar ( something like a line 80 columns ) with some info to give a rough idea where the current reading place is being done. Something like this : ...-R..*...+... Marking the current reading place is easy. The problem is deciding what to print in the other 79 columns, given that each character must condensate the status of (in your case) 25 million sectors. Another detail that might be usefull in the mapfile would be somekind of a counter, indicating how may times the file was written or similar . What would be the use of this feature? Best regards, Antonio.
Re: [bug-ddrescue] ddrescue strange read behaviour
Are you familiar with ddrescueview? Robert Trevellyan On Wed, Feb 12, 2020 at 11:10 AM Marco Marques wrote: > Changing subject I think that the information os the status could be > improved a litle bit. > > An idea would be to use something like a "status" bar ( something like a > line 80 columns ) > with some info to give a rough idea where the current reading place is > being done. > Something like this : > > > ...-R..*...+... > Updated at the same time as the main status interface . > ( the symbols could be the same as mapfile in order to keep consistency ) > >
Re: [bug-ddrescue] ddrescue strange read behaviour
Hi, sorry for the delayed answer... After some tests I am using only the Jmicron SATA controller as it fully stops when it reads an error . In attach are the ddrescue readlog file and the map file for the full HDD ... Although because the readlog file i s so small here it is : - # Reads Logfile. Created by GNU ddrescue version 1.24 # Command line: ddrescue -r 1 -O -d -c 10M --log-reads=ddrescue.log /dev/sdb /mnt/mk1/marcio1Tb.hdd marcio3.map # Start time: 2020-02-10 20:28:33 # Ipos Size Copied_size Error_size # 0s Scraping failed blocks... (forwards) 0xC9EFBC00 512 0 512 # 9s 0xC9EFBE00 512 0 512 0xC9EFC000 512 0 512 0xC9EFC200 512 0 512 0xC9EFC400 512 0 512 0xC9EFC600 512 0 512 0xC9EFC800 512 0 512 0xC9EFCA00 512 0 512 0xC9EFCC00 512 0 512 0xC9EFCE00 512 0 512 0xC9EFD000 512 0 512 0xC9EFD200 512 0 512 0xC9EFD400 512 0 512 0xC9EFD600 512 0 512 0xC9EFD800 512 0 512 0xC9EFDA00 512 0 512 0xC9EFDC00 512 0 512 0xC9EFDE00 512 0 512 0xC9EFE000 512 0 512 0xC9EFE200 512 0 512 # End time: 2020-02-10 20:28:42 So as far as I can see nothing out of the ordinary Although ddrescue exits with the read error message and the HDD "disappears" from the system ... If I try to run imediatly I get the "no medium available" message as supposed. Relating to the "-i" flag please ignore my previous reference as I noticed that the strange behaviour were related to the -"R" flag ... Nonetheless I do have some other details to ask : if there is a way to keep the marked bad sectores when "clearing" the trimming phase "-M" flag . I am still recovering the same HDD that each time it gets to a incorrect read error it "disappears" from the system and ddrescue exits . In order to power cycle the HDD I build a manual power switch to the HDD to help with the recovery. Nonetheless I do need some help as if I use the "-M" flag I loose all the bad blocks found previously ... :( In this latest experience (a 1Tb HDD), in a initial approach I had 9Gb at the trimming part, afterwards applied the "-A" and "-M" flags it decreased to the 6Gb and now after some more tries I get 2.5Gb . Although the ammount of bad blocks is still in the 100Kb amount ... So my question is : am I doing the right thing by using the "-M" flag ? or is there any other form to keep the badblocks marked ? Another detail is related to the "-K" flag ... Is it applied in the Trimming or Scrapping phases ? Although I am tented to edit manually the mapfile I will try to avoid it. Changing subject I think that the information os the status could be improved a litle bit. An idea would be to use something like a "status" bar ( something like a line 80 columns ) with some info to give a rough idea where the current reading place is being done. Something like this : ...-R..*...+... Updated at the same time as the main status interface . ( the symbols could be the same as mapfile in order to keep consistency ) Another detail that might be usefull in the mapfile would be somekind of a counter, indicating how may times the file was written or similar . THanks in advance Marco Marques ddrescuelogs.7z Description: Binary data
Re: [bug-ddrescue] ddrescue strange read behaviour
Your hard disk gets into an error condition when trying to read some very
bad patch of sectors, and drops offline.
You could reduce the timeout and number of retries, so that ddrescue doesn?t
try too much to read bad sectors at the first pass, or if the bad patch is
contained into a small area, try to skip that area and read the rest of the
disk.
Or, get ready to disconnect/reconnect power to the disk ( a switch in the
middle of the power cord helps ) and re-detect it into the sata controllers.
Just keep in mind that trying too much to read those bad sectors can damage
the heads, then things get wor$e.
[ ]
- Original Message -
From: "Marco Marques"
To: "Antonio Diaz Diaz"
Cc:
Sent: Friday, January 10, 2020 6:09 PM
Subject: Re: [bug-ddrescue] ddrescue strange read behaviour
Hi Antonio,
in order to answer a litle more about the reading issues I am facing,
here are the kernel ATA messages that I got in this particular HDD.
[ 1.078615] ata7: SATA max UDMA/133 abar m512@0xf7a1 port
0xf7a10100 irq 41
[ 1.551176] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.553009] ata7.00: ATA-8: ST31000340AS, SD1A, max UDMA/133
[ 1.553012] ata7.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 32)
[ 1.555255] ata7.00: configured for UDMA/133
[ 106.013191] ata7.00: READ LOG DMA EXT failed, trying PIO
[ 106.032859] ata7: failed to read log page 10h (errno=-5)
[ 106.032907] ata7.00: exception Emask 0x1 SAct 0x8000 SErr 0x0 action
0x0
[ 106.032947] ata7.00: irq_stat 0x4008
[ 106.032974] ata7.00: failed command: READ FPDMA QUEUED
[ 106.033010] ata7.00: cmd 60/00:78:00:54:ab/04:00:2c:00:00/40 tag 15
ncq dma 524288 in
[ 106.033095] ata7.00: status: { DRDY }
[ 106.066187] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 106.066190] ata7.00: revalidation failed (errno=-2)
[ 106.066233] ata7: hard resetting link
[ 106.534514] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 106.557603] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 106.557606] ata7.00: revalidation failed (errno=-2)
[ 111.721004] ata7: hard resetting link
[ 112.190978] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 112.221389] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 112.221392] ata7.00: revalidation failed (errno=-2)
[ 112.221433] ata7.00: disabled
[ 112.221469] ata7: EH complete
This is the result in the JMicron controller...
As soon as I have the time to get the in th Intel I will report them
here ...
Thanks in advance ...
Marco Marques
Citando Antonio Diaz Diaz :
Hi Marco,
Marco Marques wrote: > In the Intel once a bad read sector is found I
get several "Kernel ATA"
errors and ddrescue continues to try to read.
Despite the fact the HDD is now "offline and out of the system",
ddrescue tries to continue with no sucess.
After every read error ddrescue verifies that the input file is still
there. If the name exists and the read() call does not set 'errno' to
EINVAL, ddrescue continues reading. Maybe ddrescue should also stop on
EBADF and maybe on ESPIPE and ENXIO? (I have no idea what value the
device driver will return to mean that the drive is "offline and out of
the system").
In the Jmicron controller I get different situation :
Although still getting the same "Kernel ATA" errors but now ddrescue
aborts with :
"ddrescue: Unaligned read error. Is sector size correct ?"
This is because in this case read() is setting 'errno' to EINVAL, which
in direct mode usually means an incorrect sector size.
So in order to proceed I have to completly shutdown the system in order
to access the HDD and proceed with the data recovery .
( Rebooting does not allow the system to recover access to the HDD
...)
I think drescue can't help here.
With this specific HDD I did found some other strange behaviour where
the -i flag was being ignored ...
In one of the trials I call the "-i 500GiB" but ddrescue simply
ignored
it as it started to read from a lower value
This is a bug in ddrescue. As you can read in the manual:
http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Basic-concepts
"Ddrescue will never try to read any data outside of the rescue domain
except when unaligned direct disc access is requested (see Direct disc
access). If it does, please, report it as a bug."
Please, try to reproduce the problem using the option
'--log-reads=FILE', and then send me the exact command line you used and
the FILE (compressed) so that I can find out what the problem is. Thanks.
Is the "-s" flag mandatory ?
No. If you do not specify a size, ddrescue reads up to the end of the
file.
Before ddrescue aborts I get "read errors: yyy " but in the next
ddrescue call it becomes 0 .
it would be useful to
Re: [bug-ddrescue] ddrescue strange read behaviour
Hi Antonio,
in order to answer a litle more about the reading issues I am facing,
here are the kernel ATA messages that I got in this particular HDD.
[ 1.078615] ata7: SATA max UDMA/133 abar m512@0xf7a1 port
0xf7a10100 irq 41
[ 1.551176] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 1.553009] ata7.00: ATA-8: ST31000340AS, SD1A, max UDMA/133
[ 1.553012] ata7.00: 1953525168 sectors, multi 0: LBA48 NCQ (depth 32)
[ 1.555255] ata7.00: configured for UDMA/133
[ 106.013191] ata7.00: READ LOG DMA EXT failed, trying PIO
[ 106.032859] ata7: failed to read log page 10h (errno=-5)
[ 106.032907] ata7.00: exception Emask 0x1 SAct 0x8000 SErr 0x0 action 0x0
[ 106.032947] ata7.00: irq_stat 0x4008
[ 106.032974] ata7.00: failed command: READ FPDMA QUEUED
[ 106.033010] ata7.00: cmd 60/00:78:00:54:ab/04:00:2c:00:00/40 tag 15
ncq dma 524288 in
[ 106.033095] ata7.00: status: { DRDY }
[ 106.066187] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 106.066190] ata7.00: revalidation failed (errno=-2)
[ 106.066233] ata7: hard resetting link
[ 106.534514] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 106.557603] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 106.557606] ata7.00: revalidation failed (errno=-2)
[ 111.721004] ata7: hard resetting link
[ 112.190978] ata7: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
[ 112.221389] ata7.00: both IDENTIFYs aborted, assuming NODEV
[ 112.221392] ata7.00: revalidation failed (errno=-2)
[ 112.221433] ata7.00: disabled
[ 112.221469] ata7: EH complete
This is the result in the JMicron controller...
As soon as I have the time to get the in th Intel I will report them
here ...
Thanks in advance ...
Marco Marques
Citando Antonio Diaz Diaz :
Hi Marco,
Marco Marques wrote: > In the Intel once a bad read sector is
found I get several "Kernel ATA"
errors and ddrescue continues to try to read.
Despite the fact the HDD is now "offline and out of the system",
ddrescue tries to continue with no sucess.
After every read error ddrescue verifies that the input file is
still there. If the name exists and the read() call does not set
'errno' to EINVAL, ddrescue continues reading. Maybe ddrescue should
also stop on EBADF and maybe on ESPIPE and ENXIO? (I have no idea
what value the device driver will return to mean that the drive is
"offline and out of the system").
In the Jmicron controller I get different situation :
Although still getting the same "Kernel ATA" errors but now ddrescue
aborts with :
"ddrescue: Unaligned read error. Is sector size correct ?"
This is because in this case read() is setting 'errno' to EINVAL,
which in direct mode usually means an incorrect sector size.
So in order to proceed I have to completly shutdown the system in order
to access the HDD and proceed with the data recovery .
( Rebooting does not allow the system to recover access to the HDD ...)
I think drescue can't help here.
With this specific HDD I did found some other strange behaviour where
the -i flag was being ignored ...
In one of the trials I call the "-i 500GiB" but ddrescue simply ignored
it as it started to read from a lower value
This is a bug in ddrescue. As you can read in the manual:
http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Basic-concepts
"Ddrescue will never try to read any data outside of the rescue
domain except when unaligned direct disc access is requested (see
Direct disc access). If it does, please, report it as a bug."
Please, try to reproduce the problem using the option
'--log-reads=FILE', and then send me the exact command line you used
and the FILE (compressed) so that I can find out what the problem
is. Thanks.
Is the "-s" flag mandatory ?
No. If you do not specify a size, ddrescue reads up to the end of the file.
Before ddrescue aborts I get "read errors: yyy " but in the next
ddrescue call it becomes 0 .
it would be useful to have some kind of totaliser (failed , read error )
as also have some kind of human readable info
( number of opening tries to the file ) in the mapfile related to
previous ddrescue calls .
This requires a change of format in the mapfile and may be
confusing or even annoying. For example, if "read errors:" keeps the
previous value, the 'N' in '--max-read-errors=N' needs to be
increased in each call. (Or the syntax of 'N' be also changed). So
it needs some justification and careful thinking.
Best regards,Antonio.
Re: [bug-ddrescue] ddrescue strange read behaviour
Hi Marco, Marco Marques wrote: In the Intel once a bad read sector is found I get several "Kernel ATA" errors and ddrescue continues to try to read. Despite the fact the HDD is now "offline and out of the system", ddrescue tries to continue with no sucess. After every read error ddrescue verifies that the input file is still there. If the name exists and the read() call does not set 'errno' to EINVAL, ddrescue continues reading. Maybe ddrescue should also stop on EBADF and maybe on ESPIPE and ENXIO? (I have no idea what value the device driver will return to mean that the drive is "offline and out of the system"). In the Jmicron controller I get different situation : Although still getting the same "Kernel ATA" errors but now ddrescue aborts with : "ddrescue: Unaligned read error. Is sector size correct ?" This is because in this case read() is setting 'errno' to EINVAL, which in direct mode usually means an incorrect sector size. So in order to proceed I have to completly shutdown the system in order to access the HDD and proceed with the data recovery . ( Rebooting does not allow the system to recover access to the HDD ...) I think drescue can't help here. With this specific HDD I did found some other strange behaviour where the -i flag was being ignored ... In one of the trials I call the "-i 500GiB" but ddrescue simply ignored it as it started to read from a lower value This is a bug in ddrescue. As you can read in the manual: http://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html#Basic-concepts "Ddrescue will never try to read any data outside of the rescue domain except when unaligned direct disc access is requested (see Direct disc access). If it does, please, report it as a bug." Please, try to reproduce the problem using the option '--log-reads=FILE', and then send me the exact command line you used and the FILE (compressed) so that I can find out what the problem is. Thanks. Is the "-s" flag mandatory ? No. If you do not specify a size, ddrescue reads up to the end of the file. Before ddrescue aborts I get "read errors: yyy " but in the next ddrescue call it becomes 0 . it would be useful to have some kind of totaliser (failed , read error ) as also have some kind of human readable info ( number of opening tries to the file ) in the mapfile related to previous ddrescue calls . This requires a change of format in the mapfile and may be confusing or even annoying. For example, if "read errors:" keeps the previous value, the 'N' in '--max-read-errors=N' needs to be increased in each call. (Or the syntax of 'N' be also changed). So it needs some justification and careful thinking. Best regards, Antonio.
