[OFFTOPIC] Fun with pv and broken drive (was: `pv` rates)
Stefan Monnier [2021-08-02 15:09:38] wrote: > E.g. the average has moved up to ~80kB/s since my last message). For your entertainment: I managed to bring the average rate up to about 1MB/s by running `lvs` in a loop at the same time (and that let me see occasionally a transfer rate around 10MB/s). I have no idea what it is that makes `lvs`s interference so beneficial. I tried a few other operations (like `du`, and `dd`) but to no avail. Without any interference from `lvs` I get a long term average slightly below 100kB/s. Stefan
Re: `pv` rates
On Mon 02 Aug 2021 at 15:09:38 (-0400), Stefan Monnier wrote: > >> If it weren't for the first sometimes changing to > >> 44.xKiB/s it'd be hard to know which is which (IIUC the average is > >> higher because occasionally the drive gives a more reasonable transfer > >> rate than that measly 45kB/s). > > > > So now we're left wondering how you came by this situation. Perhaps > > you slumped onto the Return key, then woke up after a few minutes, > > having missed the initial burst that gave rise to the average being > > more than 50% faster than the current rate. > > I don't know either. Reading the whole drive would take a few years, so > I'm only fetching the few parts which have changed since the last > backup, and when I look the rate seems to be always ~45kB/s, but > obviously there have to be bursts at higher speeds (presumably because > the drive's defect doesn't affect every cylinder in the same way or > something like that. E.g. the average has moved up to ~80kB/s since my > last message). Yes, perhaps because the drive is broken, there was no initial burst (which usually makes it obvious). 45kB/s is so slow that any normal-speed bursts might be too short to notice unless you're watching like a hawk. Most transfers are more likely to look something like below (narrowed). Nice command though (I too had never noticed it), especially the wall time, though I would say that having both I&e is less sensible than a&r when using the command for real. Perhaps I should have used -i myself … Cheers, David. $ pv -parIe <2021-01-20-inauguration-biden-harris-C-qYgs_yOXA.webm >/tmp/copy [86.8MiB/s] [86.8MiB/s] [> ] 1% ETA 0:01:25 ETA 18:51:46 [93.2MiB/s] [89.9MiB/s] [> ] 2% ETA 0:01:22 ETA 18:51:44 [91.9MiB/s] [90.6MiB/s] [> ] 3% ETA 0:01:20 ETA 18:51:43 [93.1MiB/s] [91.2MiB/s] [=> ] 4% ETA 0:01:18 ETA 18:51:42 [92.1MiB/s] [91.4MiB/s] [==>] 6% ETA 0:01:17 ETA 18:51:42 [85.2MiB/s] [90.4MiB/s] [==>] 7% ETA 0:01:17 ETA 18:51:43 [41.6MiB/s] [82.5MiB/s] [==>] 7% ETA 0:01:22 ETA 18:51:49 [51.6MiB/s] [79.2MiB/s] [===> ] 8% ETA 0:01:27 ETA 18:51:55 [3.90MiB/s] [66.4MiB/s] [===> ] 8% ETA 0:01:37 ETA 18:52:06 [10.7MiB/s] [64.5MiB/s] [===> ] 8% ETA 0:01:47 ETA 18:52:17 [3.83MiB/s] [58.6MiB/s] [===> ] 8% ETA 0:01:57 ETA 18:52:28 [3.56MiB/s] [53.5MiB/s] [===> ] 8% ETA 0:02:06 ETA 18:52:38 [3.80MiB/s] [50.4MiB/s] [===> ] 8% ETA 0:02:16 ETA 18:52:49 [5.78MiB/s] [47.3MiB/s] [===> ] 8% ETA 0:02:26 ETA 18:53:00 [5.86MiB/s] [44.4MiB/s] [===> ] 8% ETA 0:02:34 ETA 18:53:09 [2.54MiB/s] [41.6MiB/s] [===> ] 8% ETA 0:02:44 ETA 18:53:20 [2.79MiB/s] [39.4MiB/s] [===> ] 8% ETA 0:02:53 ETA 18:53:30 [72.2MiB/s] [41.0MiB/s] [===> ] 9% ETA 0:02:46 ETA 18:53:24 [93.1MiB/s] [43.7MiB/s] [> ] 10% ETA 0:02:34 ETA 18:53:13 [92.4MiB/s] [46.2MiB/s] [=> ] 12% ETA 0:02:24 ETA 18:53:04 [67.3MiB/s] [47.2MiB/s] [==>] 13% ETA 0:02:19 ETA 18:53:00 [92.9MiB/s] [49.2MiB/s] [==>] 14% ETA 0:02:11 ETA 18:52:53 [87.6MiB/s] [50.9MiB/s] [===> ] 15% ETA 0:02:05 ETA 18:52:48 [45.0MiB/s] [50.6MiB/s] [===> ] 16% ETA 0:02:04 ETA 18:52:48 [48.6MiB/s] [50.6MiB/s] [===> ] 16% ETA 0:02:04 ETA 18:52:49 [45.1MiB/s] [50.4MiB/s] [> ] 17% ETA 0:02:04 ETA 18:52:50 [20.4MiB/s] [49.2MiB/s] [> ] 17% ETA 0:02:06 ETA 18:52:53 [28.5MiB/s] [48.5MiB/s] [> ] 17% ETA 0:02:08 ETA 18:52:56 [35.2MiB/s] [48.0MiB/s] [> ] 18% ETA 0:02:08 ETA 18:52:57 [4.75MiB/s] [46.5MiB/s] [> ] 18% ETA 0:02:12 ETA 18:53:02 [1.30MiB/s] [45.1MiB/s] [> ] 18% ETA 0:02:16 ETA 18:53:07 [3.25MiB/s] [43.8MiB/s] [> ] 18% ETA 0:02:20 ETA 18:53:12 [2.93MiB/s] [42.5MiB/s] [> ] 18% ETA 0:02:24 ETA 18:53:17 [2.67MiB/s] [41.3MiB/s] [> ] 18% ETA 0:02:28 ETA 18:53:22 [4.17MiB/s] [40.2MiB/s] [> ] 18% ETA 0:02:32 ETA 18:53:27 [2.52MiB/s] [39.3MiB/s] [> ] 18% ETA 0:02:36 ETA 18:53:32 [5.58MiB/s] [38.4MiB/s] [> ] 18% ETA 0:02:40 ETA 18:53:37 [3.75MiB/s] [37.5MiB/s] [> ] 18% ETA 0:02:43 ETA 18:53:41 [3.90MiB/s] [36.6MiB/s] [> ] 18% ETA 0:02:47 ETA 18:53:46 [55.0MiB/s] [37.0MiB/s] [=> ] 19% ETA 0:02:44 ETA 18:53:44 [47.9MiB/s] [37.3MiB/s] [==>] 20% ETA 0:02:41 ETA 18:53:42 [83.6MiB/s] [38.4MiB/s] [==>] 21% ETA 0:02:35 ETA 18:53:37 [72.3MiB/s] [39.2MiB/s] [===> ] 22% ETA 0:02:30 ETA 18:53:33 [ 105MiB/s] [40.7MiB/s] [===> ] 23% ETA 0:02:22 ETA 18:53:26 [93.8MiB/s] [41.9MiB/s] [> ] 24% ETA 0:02:15 ETA 18:53:20 ^C $ Cheers, David.
Re: `pv` rates
>> If it weren't for the first sometimes changing to >> 44.xKiB/s it'd be hard to know which is which (IIUC the average is >> higher because occasionally the drive gives a more reasonable transfer >> rate than that measly 45kB/s). > > So now we're left wondering how you came by this situation. Perhaps > you slumped onto the Return key, then woke up after a few minutes, > having missed the initial burst that gave rise to the average being > more than 50% faster than the current rate. I don't know either. Reading the whole drive would take a few years, so I'm only fetching the few parts which have changed since the last backup, and when I look the rate seems to be always ~45kB/s, but obviously there have to be bursts at higher speeds (presumably because the drive's defect doesn't affect every cylinder in the same way or something like that. E.g. the average has moved up to ~80kB/s since my last message). Stefan
`pv` rates (was: burn iso to usb)
Anssi Saari [2021-08-02 19:04:59] wrote: > David Wright writes: >> On Mon 02 Aug 2021 at 16:14:15 (+0300), Anssi Saari wrote: >>> Stefan Monnier writes: >>> >>> > cp /path/to/file.iso /dev/sdX >>> >>> dd if=whatever.iso of=/dev/sdX >>> >> It's up to taste. >>> > >>> > Not at all. The only right answer is: >>> > >>> > pv -parIe /dev/sdX >>> >>> Actually I'm not sure how good it is to have both -a and -r, pv doesn't >>> really show which rate counter is which... >> >> No need: the rate is far more variable than its average, as time passes. > > I guess it depends. Before that comment I tried it on an old USB > stick. Read speed was pretty much constant if low. So I think it was the > left rate counter that showed current rate but wouldn't bet on it. [ I'm glad my silly intervention brings up a more constructive discussion ;-) ] If they're both pretty much constant, they're presumably both pretty much equal, so it doesn't matter which is which ;-) But indeed, it's not always the case. I'm right now using `pv` to read data off of a broken drive (a 2TB 2½" drive which apparently has problems seeking, resulting in a transfer rate of about 45kB/s), and it's currently showing me 45.0KiB/s and 70.6KiB/s both of which are quite stable. If it weren't for the first sometimes changing to 44.xKiB/s it'd be hard to know which is which (IIUC the average is higher because occasionally the drive gives a more reasonable transfer rate than that measly 45kB/s). Stefan
Re: `pv` rates (was: burn iso to usb)
On Mon 02 Aug 2021 at 13:26:19 (-0400), Stefan Monnier wrote: > Anssi Saari [2021-08-02 19:04:59] wrote: > > David Wright writes: > >> On Mon 02 Aug 2021 at 16:14:15 (+0300), Anssi Saari wrote: > >>> Stefan Monnier writes: > >>> >>> > cp /path/to/file.iso /dev/sdX > >>> >>> dd if=whatever.iso of=/dev/sdX > >>> >> It's up to taste. > >>> > > >>> > Not at all. The only right answer is: > >>> > > >>> > pv -parIe /dev/sdX > >>> > >>> Actually I'm not sure how good it is to have both -a and -r, pv doesn't > >>> really show which rate counter is which... > >> > >> No need: the rate is far more variable than its average, as time passes. > > > > I guess it depends. Before that comment I tried it on an old USB > > stick. Read speed was pretty much constant if low. So I think it was the > > left rate counter that showed current rate but wouldn't bet on it. > > [ I'm glad my silly intervention brings up a more constructive > discussion ;-) ] > > If they're both pretty much constant, they're presumably both pretty > much equal, so it doesn't matter which is which ;-) > > But indeed, it's not always the case. I'm right now using `pv` to read > data off of a broken drive (a 2TB 2½" drive which apparently has > problems seeking, resulting in a transfer rate of about 45kB/s), and > it's currently showing me 45.0KiB/s and 70.6KiB/s both of which are > quite stable. Writing "quite stable" cloaks the information nicely. The average can never move away from the rate: a change in the least significant digit gives the game away. > If it weren't for the first sometimes changing to > 44.xKiB/s it'd be hard to know which is which (IIUC the average is > higher because occasionally the drive gives a more reasonable transfer > rate than that measly 45kB/s). So now we're left wondering how you came by this situation. Perhaps you slumped onto the Return key, then woke up after a few minutes, having missed the initial burst that gave rise to the average being more than 50% faster than the current rate. Cheers, David.
