Re: Suggestion: A No-write Option, e.g. --scrub.

2020-03-06 Thread Antonio Diaz Diaz

Ralph Corderoy wrote:

 $ strace -c tee <$f 2>&1 >/dev/null | sed 4q
 % time seconds  usecs/call callserrors syscall
 -- --- --- - - 
  56.730.126982  33  3750   read
  43.230.096779  25  3743   write


25 µs/call? It seems that your hardware is very slow. I have tried the 
command above on my AMD Athlon64 and it is 38000 times faster (0.65 ns/call).


$ strace -c tee < enwik8 2>&1 > /dev/null
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 82.980.39   0 12210   read
 17.020.08   0 12208   write

Even my AMD K6-II 450 MHz from 1999 is almost one hundred times faster (0.28 
µs/call).




The extra writes to /dev/null are adding 0.1 s and that's about 75% of
the time taken by all the read()s.


I guess this is because you are reading a file cached in RAM. Reading from a 
real USB pendrive in the K6 gives:


$ strace -c tee < binutils-2.32.tar.lz 2>&1 > /dev/null
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 98.770.056898  23  2494   read
  1.230.000708   0  2490   write



So it might be a measurable saving to avoid the write()s when scrubbing
1 TB, say.


On my Athlon64, ddrescue copies 1 TB from /dev/zero to /dev/null in 2m40s 
(6.25 GB/s), which makes the savings insignificant. When reading from a real 
device the savings are even more insignificant. Ddrescue spends much more 
time managing the mapfile than writing to /dev/null. (Note that the total 
read/write/seek time is itself an insignificant fraction of total run time):


$ strace -c ddrescue -f -s1TB /dev/zero /dev/null 2>&1 > /dev/null
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 91.860.215880   0  15258794   read
  5.160.012119   0  30517582   lseek
  2.980.007014   0  15259740   write


Best regards,
Antonio.




Re: Suggestion: A No-write Option, e.g. --scrub.

2020-03-06 Thread Ralph Corderoy
Hi Antonio,

> > I should have also asked: is using /dev/null a special value that
> > avoids the write(2) system calls, e.g. as shown by 'strace -c'?
>
> No. Ddrescue does not implement special values. I have not even tested
> if avoiding the write calls would speed up the operation significantly
> because I think this use case has not been suggested here before.

/dev/null obviously implements write(2) cheaply, but there's still the
context switch associated with the system call.  A very rough test here:

$ f=/boot/initramfs-linux-lts-fallback.img
$ stat -c %s $f
30658665
$
$ strace -c tee <$f 2>&1 >/dev/null | sed 4q
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 56.730.126982  33  3750   read
 43.230.096779  25  3743   write
$
$ strace -c tee /dev/null <$f 2>&1 >/dev/null | sed 4q
% time seconds  usecs/call callserrors syscall
-- --- --- - - 
 59.840.194545  25  7486   write
 39.700.129078  34  3750   read
$

The extra writes to /dev/null are adding 0.1 s and that's about 75% of
the time taken by all the read()s.

(0.126982 + 0.129078) / 2 = 0.12803   # Mean read.
0.194545 - 0.096779 = 0.097766# Difference write.
0.097766 / 0.12803 = 0.76361790205420604545   # Difference / mean.

So it might be a measurable saving to avoid the write()s when scrubbing
1 TB, say.

-- 
Cheers, Ralph.



Re: Suggestion: A No-write Option, e.g. --scrub.

2020-03-06 Thread Antonio Diaz Diaz

Ralph Corderoy wrote:

Thanks.  I should have also asked: is using /dev/null a special value
that avoids the write(2) system calls, e.g. as shown by 'strace -c'?


No. Ddrescue does not implement special values. I have not even tested if 
avoiding the write calls would speed up the operation significantly because 
I think this use case has not been suggested here before.




Re: Suggestion: A No-write Option, e.g. --scrub.

2020-03-06 Thread Ralph Corderoy
Hi Antonio,

> > It occurred to me that ddrescue(1) would do a better job, but I
> > wouldn't want a copy created in an output file.  I skimmed
> > https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html
> > but didn't see a way to do this, or a special meaning to an output
> > file of /dev/null.
>
> Thanks. I'll document this. As you suggest, /dev/null (combined with a
> mapfile) can be used to perform this task efficiently with a command
> like this one:
>
>ddrescue -f -r-1 /dev/sdcard /dev/null mapfile

Thanks.  I should have also asked: is using /dev/null a special value
that avoids the write(2) system calls, e.g. as shown by ‘strace -c’?

-- 
Cheers, Ralph.



Re: Suggestion: A No-write Option, e.g. --scrub.

2020-03-06 Thread Antonio Diaz Diaz

Hi Ralph,

Ralph Corderoy wrote:

I've just used dd(1) to repeatedly read an SD card until it got all the
way through without an I/O error.  Quite a bit of the run time was
reading what had already been successfully read because I was lazy, or
had a better way to use my time.

It occurred to me that ddrescue(1) would do a better job, but I wouldn't
want a copy created in an output file.  I skimmed
https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html but
didn't see a way to do this, or a special meaning to an output file of
/dev/null.


Thanks. I'll document this. As you suggest, /dev/null (combined with a 
mapfile) can be used to perform this task efficiently with a command like 
this one:


  ddrescue -f -r-1 /dev/sdcard /dev/null mapfile



A --scrub would be quite handy to read repeatedly until the media's
controller succeeds and re-maps the block internally.


The option '-r-1' (infinite retry passes) in the command above achieves 
exactly that.



Best regards,
Antonio.