On Mon, Mar 29, 2021 at 5:12 AM Hendrik Friedel <hend...@friedels.name> wrote: > > Hello, > > I have a filesystem which is sometimes very slow, or even currently > hangs deleting a file (plain and simple rm in bash). > > Label: 'DataPool1' uuid: c4a6a2c9-5cf0-49b8-812a-0784953f9ba3 > Total devices 2 FS bytes used 5.65TiB > devid 1 size 7.28TiB used 6.71TiB path /dev/sda1 > devid 2 size 7.28TiB used 6.71TiB path /dev/sdh1 > > I did run a scrub without errors. > > Checking the logs, I find: > dmesg -T |grep -i btrfs > [Mo Mär 29 09:29:16 2021] Btrfs loaded, crc32c=crc32c-intel > [Mo Mär 29 09:29:16 2021] BTRFS: device label DataPool1 devid 2 transid > 649014 /dev/sdh1 scanned by btrfs (213) > [Mo Mär 29 09:29:16 2021] BTRFS: device label DataPool1 devid 1 transid > 649014 /dev/sda1 scanned by btrfs (213) > [Mo Mär 29 09:29:16 2021] BTRFS: device label Daten devid 1 transid > 254377 /dev/sdd2 scanned by btrfs (213) > [Mo Mär 29 09:29:16 2021] BTRFS: device label DockerImages devid 1 > transid 209067 /dev/sdc2 scanned by btrfs (213) > [Mo Mär 29 09:29:21 2021] BTRFS info (device sda1): disk space caching > is enabled > [Mo Mär 29 09:29:21 2021] BTRFS info (device sda1): has skinny extents > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdd2): enabling ssd > optimizations > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdd2): disk space caching > is enabled > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdd2): has skinny extents > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdc2): turning on sync > discard > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdc2): enabling ssd > optimizations > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdc2): disk space caching > is enabled > [Mo Mär 29 09:29:21 2021] BTRFS info (device sdc2): has skinny extents > [Mo Mär 29 09:29:22 2021] BTRFS info (device sda1): bdev /dev/sda1 errs: > wr 133, rd 133, flush 0, corrupt 0, gen 1 > > Maybe, the last line is concerning?
Yes. Do a 'btrfs scrub' and check dmesg for detailed errors. Next 'btrfs check --readonly' (must be done offline ie booted from usb stick). And if it all comes up without errors or problems, you can zero the statistics with 'btrfs dev stats -z'. But otherwise we need to see the errors to know what's going wrong. It's not normal to have either read or write errors. It could be related to the problem, or an additional problem. > > > Syslog tells me: > Mar 28 20:22:19 homeserver kernel: [1297978.357508] task:btrfs-cleaner > state:D stack: 0 pid:20078 ppid: 2 flags:0x00004000 > Mar 28 20:22:19 homeserver kernel: [1297978.357547] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357564] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357577] > btrfs_drop_snapshot+0x90/0x7f0 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357594] ? > btrfs_delete_unused_bgs+0x3e/0x850 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357609] > btrfs_clean_one_deleted_snapshot+0xd7/0x130 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357622] > cleaner_kthread+0xfa/0x120 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.357636] ? > btrfs_alloc_root+0x3d0/0x3d0 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.360473] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.360488] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.360503] > btrfs_create+0x58/0x1f0 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.363057] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.363072] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:22:19 homeserver kernel: [1297978.363086] > btrfs_rmdir+0x5c/0x180 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024321] task:btrfs-cleaner > state:D stack: 0 pid:20078 ppid: 2 flags:0x00004000 > Mar 28 20:26:20 homeserver kernel: [1298220.024382] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024419] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024442] > btrfs_drop_snapshot+0x90/0x7f0 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024476] ? > btrfs_delete_unused_bgs+0x3e/0x850 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024504] > btrfs_clean_one_deleted_snapshot+0xd7/0x130 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024531] > cleaner_kthread+0xfa/0x120 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.024558] ? > btrfs_alloc_root+0x3d0/0x3d0 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.030300] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.030331] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:26:20 homeserver kernel: [1298220.030361] > btrfs_create+0x58/0x1f0 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854109] task:btrfs-cleaner > state:D stack: 0 pid:20078 ppid: 2 flags:0x00004000 > Mar 28 20:28:21 homeserver kernel: [1298340.854151] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854169] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854183] > btrfs_drop_snapshot+0x90/0x7f0 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854202] ? > btrfs_delete_unused_bgs+0x3e/0x850 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854218] > btrfs_clean_one_deleted_snapshot+0xd7/0x130 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854232] > cleaner_kthread+0xfa/0x120 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.854247] ? > btrfs_alloc_root+0x3d0/0x3d0 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.857610] > wait_current_trans+0xc2/0x120 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.857627] > start_transaction+0x46d/0x540 [btrfs] > Mar 28 20:28:21 homeserver kernel: [1298340.857643] > btrfs_create+0x58/0x1f0 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336160] task:btrfs-transacti > state:D stack: 0 pid:20080 ppid: 2 flags:0x00004000 > Mar 28 20:58:34 homeserver kernel: [1300153.336215] > btrfs_commit_transaction+0x92b/0xa50 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336246] > transaction_kthread+0x15d/0x180 [btrfs] > Mar 28 20:58:34 homeserver kernel: [1300153.336273] ? > btrfs_cleanup_transaction+0x590/0x590 [btrfs] > > > What could I do to find the cause? What kernel version? -- Chris Murphy