Re: btrfs send hung in pipe_wait

2018-09-09 Thread Chris Murphy
On Sun, Sep 9, 2018 at 2:16 PM, Stefan Loewen wrote: > I'm not sure about the exact definition of "blocked" here, but I was > also surprised that there were no blocked tasks listed since I'm > definitely unable to kill (SIGKILL) that process. > On the other hand it wakes up hourly to transfer a

Re: btrfs send hung in pipe_wait

2018-09-08 Thread Chris Murphy
I don't see any blocked tasks. I wonder if you were too fast with sysrq w? Maybe it takes a little bit for the block task to actually develop? I suggest also 'btrfs check --mode=lowmem' because that is a separate implementation of btrfs check that tends to catch different things than the

Fwd: btrfs send hung in pipe_wait

2018-09-08 Thread Stefan Loewen
And this one as well. -- Forwarded message - From: Chris Murphy Date: Fr., 7. Sep. 2018 um 23:53 Uhr Subject: Re: btrfs send hung in pipe_wait To: Stefan Loewen Cc: Chris Murphy On Fri, Sep 7, 2018 at 3:19 PM, Stefan Loewen wrote: > Now I also tested with Fedora 28 (li

Fwd: btrfs send hung in pipe_wait

2018-09-08 Thread Stefan Loewen
Oops. Forgot CCing the mailinglist -- Forwarded message - From: Stefan Loewen Date: Fr., 7. Sep. 2018 um 23:19 Uhr Subject: Re: btrfs send hung in pipe_wait To: Chris Murphy No it does not only happen in VirtualBox. I already tested the following: - Manjaro baremetal (btrfs

Re: btrfs send hung in pipe_wait

2018-09-07 Thread Chris Murphy
On Fri, Sep 7, 2018 at 11:07 AM, Stefan Loewen wrote: > List of steps: > - 3.8G iso lays in read-only subvol A > - I create subvol B and reflink-copy the iso into it. > - I create a read-only snapshot C of B > - I "btrfs send --no-data C > /somefile" > So you got that right, yes. OK I can't

Re: btrfs send hung in pipe_wait

2018-09-07 Thread Stefan Loewen
List of steps: - 3.8G iso lays in read-only subvol A - I create subvol B and reflink-copy the iso into it. - I create a read-only snapshot C of B - I "btrfs send --no-data C > /somefile" So you got that right, yes. Unfortunately I don't have any way to connect the drive to a SATA port directly

Re: btrfs send hung in pipe_wait

2018-09-07 Thread Chris Murphy
On Fri, Sep 7, 2018 at 6:47 AM, Stefan Loewen wrote: > Well... It seems it's not the hardware. > I ran a long SMART check which ran through without errors and > reallocation count is still 0. That only checks the drive, it's an internal test. It doesn't check anything else, including

Re: btrfs send hung in pipe_wait

2018-09-07 Thread Stefan Loewen
Well... It seems it's not the hardware. I ran a long SMART check which ran through without errors and reallocation count is still 0. So I used clonezilla (partclone.btrfs) to mirror the drive to another drive (same model). Everything copied over just fine. No I/O error im dmesg. The new disk

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Chris Murphy
On Thu, Sep 6, 2018 at 2:16 PM, Stefan Loewen wrote: > Data,single: Size:695.01GiB, Used:653.69GiB > /dev/sdb1 695.01GiB > Metadata,DUP: Size:4.00GiB, Used:2.25GiB > /dev/sdb1 8.00GiB > System,DUP: Size:40.00MiB, Used:96.00KiB > Does that mean Metadata is duplicated? Yes. Single

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Stefan Loewen
[root@archlinux @data]# btrfs fi us /mnt/intenso_white/ Overall: Device size: 911.51GiB Device allocated:    703.09GiB Device unallocated:  208.43GiB Device missing:  0.00B Used:    658.19GiB Free (estimated):   

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Chris Murphy
On Thu, Sep 6, 2018 at 12:36 PM, Stefan Loewen wrote: > Output of the commands is attached. fdisk Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes smart Sector Sizes: 512 bytes logical, 4096 bytes physical So clearly the case is lying

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Stefan Loewen
Output of the commands is attached. The broken-sector-theory sounds plausible and is compatible with my new findings: I suspected the problem to be in one specific directory, let's call it "broken_dir". I created a new subvolume and copied broken_dir over. - If I copied it with cp --reflink,

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Chris Murphy
On Thu, Sep 6, 2018 at 10:03 AM, Stefan Löwen wrote: > I have one subvolume (rw) and 2 snapshots (ro) of it. > > I just tested 'btrfs send > /dev/null' and that also shows no IO > after a while but also no significant CPU usage. > During this I tried 'ls' on the source subvolume and it hangs as

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Stefan Löwen
I have one subvolume (rw) and 2 snapshots (ro) of it. I just tested 'btrfs send > /dev/null' and that also shows no IO after a while but also no significant CPU usage. During this I tried 'ls' on the source subvolume and it hangs as well. dmesg has some interesting messages I think (see

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Chris Murphy
On Thu, Sep 6, 2018 at 9:04 AM, Stefan Loewen wrote: > Update: > It seems like btrfs-send is not completely hung. It somewhat regularly > wakes up every hour to transfer a few bytes. I noticed this via a > periodic 'ls -l' on the snapshot file. These are the last outputs > (uniq'ed): > >

Re: btrfs send hung in pipe_wait

2018-09-06 Thread Stefan Loewen
Update: It seems like btrfs-send is not completely hung. It somewhat regularly wakes up every hour to transfer a few bytes. I noticed this via a periodic 'ls -l' on the snapshot file. These are the last outputs (uniq'ed): -rw--- 1 root root 1492797759 Sep 6 08:44 intenso_white.snapshot

btrfs send hung in pipe_wait

2018-09-06 Thread Stefan Löwen
Hello linux-btrfs, I'm trying to clone a subvolume with 'btrfs send' but it always hangs for hours. I tested this on multiple systems. All showed the same result: - Manjaro (btrfs-progs v4.17.1; linux v4.18.5-1-MANJARO) - Ubuntu 18.04 in VirtualBox (btrfs-progs v4.15.1; linux