Re: invalid files names, btrfs check can't repair it

2018-01-16 Thread Sebastian Andrzej Siewior 
On 2018-01-16 19:20:56 [+0800], Qu Wenruo wrote:
> This is a very minor problem.
> 
> btrfs check --repair should handle it without problem.

awesome, thank you! After the --repair completed, the following didn't
report the error anymore. And now I was able to remove the two files.
Thank you.

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-16 Thread Sebastian Andrzej Siewior 
On 2018-01-16 18:22:14 [+0800], Qu Wenruo wrote:
> Btrfs check is always recommended before really mount it.

 # btrfs check -p /dev/sdb4 
 Checking filesystem on /dev/sdb4
 UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
 checking extents [O]
 cache and super generation don't match, space cache will be invalidated
 root 5 inode 57648595 errors 200, dir isize wrong

 ERROR: errors found in fs roots
 found 64009740288 bytes used, error(s) found
 total csum bytes: 61644600
 total tree bytes: 865746944
 total fs tree bytes: 693469184
 total extent tree bytes: 84905984
 btree space waste bytes: 225096302
 file data blocks allocated: 63203807232
  referenced 63142268928

> However as long as there is no other problem, btrfs check should not
> report anything wrong.

Is it important to note that the filesystem is a raid1 one?

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-16 Thread Sebastian Andrzej Siewior 
On 2018-01-16 13:19:50 [+0800], Qu Wenruo wrote:
> Usage:
> ./btrfs-corrupt-block -X 
# ./btrfs-corrupt-block -X /dev/sdb4
Fix is done successfully

may I mount it now or should I run btrfs check or something first?

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-16 Thread Sebastian Andrzej Siewior 
On 16 January 2018 05:51:23 CET, Qu Wenruo  wrote:
>Since there is no other hit, I assume it's root subvolume (5), but I
>still need the extra confirm since the fix will be hard-coded.

Yes, 5 is correct.

>
>Thanks,
>Qu



Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-15 Thread Sebastian Andrzej Siewior 
On 2018-01-15 12:23:05 [+0800], Qu Wenruo wrote:
> Right, I'll fix it soon.
> 
> And BTW what makes the the output different from the original one?
> 
> Sebastaian, did you do extra write or other operation to the fs after
> previous btrfs check?

Well the filesystem is in use but there should be no writtes to it since
initial `check' output. The `check' invalidates the space cache that is
rebuilt, not sure if this has any effect. Those two magic files are in a
subfolder of ccache and ccache shouldn't look into it at all.

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-15 Thread Sebastian Andrzej Siewior 
On 2018-01-15 09:26:27 [+0800], Qu Wenruo wrote:
> Please run the following command too:
> 
> # btrfs inspect dump-tree  | grep -C20 \(57923894

~# btrfs inspect dump-tree /dev/sdb4 | grep -C20 \(57923894
ctime 1515602448.66422211 (2018-01-10 17:40:48)
mtime 1515602448.66422211 (2018-01-10 17:40:48)
otime 1513266995.540343055 (2017-12-14 16:56:35)
item 10 key (57643872 INODE_REF 682) itemoff 3363 itemsize 13
index 58 namelen 3 name: tmp
item 11 key (57648595 INODE_ITEM 0) itemoff 3203 itemsize 160
generation 89045 transid 89423 size 8350 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0xc90(none)
atime 1513267009.164686143 (2017-12-14 16:56:49)
ctime 1513868329.753150507 (2017-12-21 15:58:49)
mtime 1513868329.753150507 (2017-12-21 15:58:49)
otime 1513267009.164686143 (2017-12-14 16:56:49)
item 12 key (57648595 INODE_REF 57643659) itemoff 3192 itemsize 11
index 113 namelen 1 name: d
item 13 key (57648595 DIR_ITEM 3331247447) itemoff 3123 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 8231
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.oq
item 14 key (57648595 DIR_ITEM 3363354030) itemoff 3053 itemsize 70
location key (57923894 INODE_ITEM 0) type DIR_ITEM.33
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 15 key (57648595 DIR_INDEX 435) itemoff 2983 itemsize 70
location key (57923894 INODE_ITEM 0) type FILE
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 16 key (57648595 DIR_INDEX 1137) itemoff 2914 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 39
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.o
item 17 key (57923894 INODE_ITEM 0) itemoff 2754 itemsize 160
generation 89142 transid 89142 size 36092 nbytes 36864
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x91(none)
atime 1513278413.460486168 (2017-12-14 20:06:53)
ctime 1513278413.460486168 (2017-12-14 20:06:53)
mtime 1513278413.460486168 (2017-12-14 20:06:53)
otime 1513278413.460486168 (2017-12-14 20:06:53)
item 18 key (57923894 INODE_REF 57648595) itemoff 2704 itemsize 50
index 435 namelen 40 name: 
2f3f379b2a3d7499471edb74869efe-1948311.d
item 19 key (57923894 EXTENT_DATA 0) itemoff 2651 itemsize 53
generation 89142 type 1 (regular)
extent data disk byte 123290755072 nr 36864
extent data offset 0 nr 36864 ram 36864
extent compression 0 (none)
item 20 key (58191388 INODE_ITEM 0) itemoff 2491 itemsize 160
generation 89259 transid 89259 size 395280 nbytes 397312
block group 0 mode 100644 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0xa4(none)
atime 1513332325.477020047 (2017-12-15 11:05:25)
ctime 1513332325.477020047 (2017-12-15 11:05:25)
mtime 1513332325.477020047 (2017-12-15 11:05:25)
otime 1513332325.477020047 (2017-12-15 11:05:25)
item 21 key (58191388 INODE_REF 40424284) itemoff 2470 itemsize 21
index 1094 namelen 11 name: bzImage.tsc
leaf 146426621952 items 46 free space 11 generation 89680 owner 5
leaf 146426621952 flags 0x1(WRITTEN) backref revision 1
fs uuid b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
chunk uuid 732d73c9-d037-4406-8dcb-dfa101bc5a9b
item 0 key (58191388 EXTENT_DATA 0) itemoff 3942 itemsize 53
generation 87303 type 1 (regular)

> > transid 89142 data_len 0 name_len 40
> > name: 2f3f379b2a3d7499471edb74869efe-1948311.d
> > item 16 key (57648595 DIR_INDEX 1137) itemoff 2914 itemsize 69
> > location key (58472210 INODE_ITEM 0) type FILE
> 
> And this command too:
> 
> # btrfs inspect dump-tree  | grep -C20 \(58472210

~# btrfs inspect dump-tree /dev/sdb4 | grep -C20 \(58472210
generation 89044 transid 89699 size 0 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x603b3(none)
atime 1513266995.540343055 (2017-12-14 16:56:35)
ctime 1515602448.66422211 (2018-01-10 17:40:48)
mtime 1515602448.66422211 (2018-01-10 17:40:48)
otime 1513266995.540343055 (2017-12-14 16:56:35)
item 10 key (57643872

Re: invalid files names, btrfs check can't repair it

2018-01-14 Thread Sebastian Andrzej Siewior 
On 2018-01-14 08:36:41 [+0800], Qu Wenruo wrote:
> Still needs more. (and maybe even more depending on the output)
> 
> The original mode doesn't report error clear enough, so it would help if
> --mode=lowmem can be used.
> 
> # btrfs check --mode=lowmem /dev/sdb4

~# btrfs check --mode=lowmem /dev/sdb4
Checking filesystem on /dev/sdb4
UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
checking extents
checking free space cache
Wanted bytes 4096, found 8192 for off 159178047488
Wanted bytes 1073725440, found 8192 for off 159178047488
cache appears valid but isn't 159178031104
ERROR: errors found in free space cache
found 63691210752 bytes used, error(s) found
total csum bytes: 61339388
total tree bytes: 860540928
total fs tree bytes: 688816128
total extent tree bytes: 84549632
btree space waste bytes: 224428280
file data blocks allocated: 62890483712
 referenced 62828957696

> And the tree dump of the affected dir inode would help too:
> 
> # btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595

~# btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595
generation 88831 type 1 (regular)
extent data disk byte 193705541632 nr 134217728
extent data offset 0 nr 134217728 ram 134217728
extent compression 0 (none)
item 4 key (57022249 EXTENT_DATA 402653184) itemoff 3769 itemsize 53
generation 88831 type 1 (regular)
extent data disk byte 190035439616 nr 60600320
extent data offset 0 nr 60600320 ram 60600320
extent compression 0 (none)
item 5 key (57643659 INODE_ITEM 0) itemoff 3609 itemsize 160
generation 89044 transid 89423 size 2 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x150be(none)
atime 1513266994.264310920 (2017-12-14 16:56:34)
ctime 1513868332.369178014 (2017-12-21 15:58:52)
mtime 1513868332.369178014 (2017-12-21 15:58:52)
otime 1513266994.264310920 (2017-12-14 16:56:34)
item 6 key (57643659 INODE_REF 58477961) itemoff 3598 itemsize 11
index 6 namelen 1 name: 4
item 7 key (57643659 DIR_ITEM 4189400016) itemoff 3567 itemsize 31
location key (57648595 INODE_ITEM 0) type DIR
transid 89045 data_len 0 name_len 1
name: d
item 8 key (57643659 DIR_INDEX 113) itemoff 3536 itemsize 31
location key (57648595 INODE_ITEM 0) type DIR
transid 89045 data_len 0 name_len 1
name: d
item 9 key (57643872 INODE_ITEM 0) itemoff 3376 itemsize 160
generation 89044 transid 89699 size 0 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x603b3(none)
atime 1513266995.540343055 (2017-12-14 16:56:35)
ctime 1515602448.66422211 (2018-01-10 17:40:48)
mtime 1515602448.66422211 (2018-01-10 17:40:48)
otime 1513266995.540343055 (2017-12-14 16:56:35)
item 10 key (57643872 INODE_REF 682) itemoff 3363 itemsize 13
index 58 namelen 3 name: tmp
item 11 key (57648595 INODE_ITEM 0) itemoff 3203 itemsize 160
generation 89045 transid 89423 size 8350 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0xc90(none)
atime 1513267009.164686143 (2017-12-14 16:56:49)
ctime 1513868329.753150507 (2017-12-21 15:58:49)
mtime 1513868329.753150507 (2017-12-21 15:58:49)
otime 1513267009.164686143 (2017-12-14 16:56:49)
item 12 key (57648595 INODE_REF 57643659) itemoff 3192 itemsize 11
index 113 namelen 1 name: d
item 13 key (57648595 DIR_ITEM 3331247447) itemoff 3123 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 8231
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.oq
item 14 key (57648595 DIR_ITEM 3363354030) itemoff 3053 itemsize 70
location key (57923894 INODE_ITEM 0) type DIR_ITEM.33
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 15 key (57648595 DIR_INDEX 435) itemoff 2983 itemsize 70
location key (57923894 INODE_ITEM 0) type FILE
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 16 key (57648595 DIR_INDEX 1137) itemoff 2914 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 39
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.o
item 17 key (57923894 INODE_ITEM 0) itemoff 2754 itemsize 160

Re: invalid files names, btrfs check can't repair it

2018-01-13 Thread Sebastian Andrzej Siewior 
On 2018-01-13 08:56:24 [+0800], Qu Wenruo wrote:
> 'Btrfs check' output please.

~# btrfs check --repair -p /dev/sdb4
enabling repair mode
Checking filesystem on /dev/sdb4
UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
Fixed 0 roots.
checking extents [o]
No device size related problem found
cache and super generation don't match, space cache will be invalidated
checkingunresolved ref dir 57648595 index 435 namelen 40 name 
2f3f379b2a3d7499471edb74869efe-1948311.d filetype 1 errors 80, filetype mismatch
unresolved ref dir 57648595 index 1137 namelen 39 name 
454bf066ddfbf42e0f3b77ea71c82f-878732.o filetype 1 errors 100, name too long

checking csums
checking root refs
found 63691468800 bytes used, no error found
total csum bytes: 61339388
total tree bytes: 860536832
total fs tree bytes: 688816128
total extent tree bytes: 84545536
btree space waste bytes: 224424238
file data blocks allocated: 62890745856
 referenced 62829219840
~# 

> And in fact, btrfs check --repair has the ability to move/re-link such
> file and keeps its content, as long as there is any info found in
> DIR_INDEX/DIR_ITEM/INODE_REF.
> 
> If repair can't handle it, the output could help us to craft such case
> and enhance btrfs-progs to repair it.

Is the output enough or do you want me to do more?

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

invalid files names, btrfs check can't repair it

2018-01-12 Thread Sebastian Andrzej Siewior 
Hi,

so I had bad memory and before I realized it and removed it btrfs took some
damage. Now I have this:

|ls -lh crap/
|ls: cannot access 'crap/2f3f379b2a3d7499471edb74869efe-1948311.d': No such 
file or directory
|ls: cannot access 'crap/454bf066ddfbf42e0f3b77ea71c82f-878732.o': No such file 
or directory
|total 0
|-? ? ? ? ?? 2f3f379b2a3d7499471edb74869efe-1948311.d
|-? ? ? ? ?? 454bf066ddfbf42e0f3b77ea71c82f-878732.o

and in dmesg I see:

| BTRFS critical (device sda4): invalid dir item type: 33
| BTRFS critical (device sda4): invalid dir item name len: 8231

`btrfs check' (from v4.14.1) finds them and prints them but has no idea
what to do with it. Would it be possible to let the check tool rename
the offended filename to something (like its inode number) put it in
lost+found if it has any data attached to it and otherwise simply remove
it? Right now I can't remove that folder.

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [btrfs/rt] lockdep false positive

2017-01-26 Thread Sebastian Andrzej Siewior 
On 2017-01-25 19:29:49 [+0100], Mike Galbraith wrote:
> On Wed, 2017-01-25 at 18:02 +0100, Sebastian Andrzej Siewior wrote:
> 
> > > [  341.960794]CPU0
> > > [  341.960795]
> > > [  341.960795]   lock(btrfs-tree-00);
> > > [  341.960795]   lock(btrfs-tree-00);
> > > [  341.960796] 
> > > [  341.960796]  *** DEADLOCK ***
> > > [  341.960796]
> > > [  341.960796]  May be due to missing lock nesting notation
> > > [  341.960796]
> > > [  341.960796] 6 locks held by kworker/u8:9/2039:
> > > [  341.960797]  #0:  ("%s-%s""btrfs", name){.+.+..}, at: [] 
> > > process_one_work+0x171/0x700
> > > [  341.960812]  #1:  ((>normal_work)){+.+...}, at: [] 
> > > process_one_work+0x171/0x700
> > > [  341.960815]  #2:  (sb_internal){.+.+..}, at: [] 
> > > start_transaction+0x2a7/0x5a0 [btrfs]
> > > [  341.960825]  #3:  (btrfs-tree-02){+.+...}, at: [] 
> > > btrfs_clear_lock_blocking_rw+0x55/0x100 [btrfs]
> > > [  341.960835]  #4:  (btrfs-tree-01){+.+...}, at: [] 
> > > btrfs_clear_lock_blocking_rw+0x55/0x100 [btrfs]
> > > [  341.960854]  #5:  (btrfs-tree-00){+.+...}, at: [] 
> > > btrfs_clear_lock_blocking_rw+0x55/0x100 [btrfs]
> > > 
> > > Attempting to describe RT rwlock semantics to lockdep prevents this.
> > 
> > and this is what I don't get. I stumbled upon this myself [0] but didn't
> > fully understand the problem (assuming this is the same problem colored
> > differently).
> 
> Yeah, [0] looks like it, though I haven't met an 'fs' variant, my
> encounters were always either 'tree' or 'csum' flavors.
> 
> > With your explanation I am not sure if I get what is happening. If btrfs
> > is taking here read-locks on random locks then it may deadlock if
> > another btrfs-thread is doing the same and need each other's locks.
> 
> I don't know if a real RT deadlock is possible.  I haven't met one,
> only variants of this bogus recursion gripe.
>  
> > If btrfs takes locks recursively which it already holds (in the same
> > context / process) then it shouldn't be visible here because lockdep
> > does not account this on -RT.
> 
> If what lockdep gripes about were true, we would never see the splat,
> we'd zip straight through that (illusion) recursive read_lock() with
> lockdep being none the wiser. 
> 
> > If btrfs takes the locks in a special order for instance only ascending
> > according to inode's number then it shouldn't deadlock.
> 
> No idea.  Locking fancy enough to require dynamic key assignment to
> appease lockdep is too fancy for me.

yup, for me, too. As long as nobody from the btrfs camp explains how
that locking workings and if it is safe I am not feeling comfortable to
shut up lockdep here.

>   -Mike

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] btrfs: drop trace_btrfs_all_work_done() from normal_work_helper()

2016-12-21 Thread Sebastian Andrzej Siewior 
On 2016-12-21 16:45:06 [+0800], Qu Wenruo wrote:
> > | DECLARE_EVENT_CLASS(btrfs__work__done,
> > |
> > | TP_PROTO(struct btrfs_work *work),
> > |
> > | TP_ARGS(work),
> > |
> > | TP_STRUCT__entry_btrfs(
> > | __field(void *, work)
> > | ),
> > |
> > | TP_fast_assign_btrfs(btrfs_work_owner(work),
> > | __entry->work   = work;
> > | ),
> > |
> > | TP_printk_btrfs("work->%p", __entry->work)
> > | );
> > 
> > and btrfs_work_owner exapnds to:
> > 
> > | struct btrfs_fs_info *
> > | btrfs_work_owner(struct btrfs_work *work)
> > | {
> > | return work->wq->fs_info;
> > | }
> > 
> > voilà
> 
> Oh I got it, thanks very much.
> 
> The btrfs_work_owner() is newly introduced, no wonder I didn't know that.

It was introduced in cb001095ca70 ("btrfs: plumb fs_info into
btrfs_work") which is v4.8-rc1. The usage in trace points started in
bc074524e123 ("btrfs: prefix fsid to all trace events") which is also
v4.8-rc1. So whatever is done to get it fixed, it should be pushed
stable for v4.8+.

> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/2] btrfs: drop trace_btrfs_all_work_done() from normal_work_helper()

2016-12-21 Thread Sebastian Andrzej Siewior 
On 2016-12-21 08:33:03 [+0800], Qu Wenruo wrote:
> The trace point only uses the pointer, and this helps us to pair with
> btrfs_work_queued/sched.

| /* For situiations that the work is freed */
| DECLARE_EVENT_CLASS(btrfs__work__done,
| 
| TP_PROTO(struct btrfs_work *work),
| 
| TP_ARGS(work),
| 
| TP_STRUCT__entry_btrfs(
| __field(void *, work)
| ),
| 
| TP_fast_assign_btrfs(btrfs_work_owner(work),
| __entry->work   = work;
| ),
| 
| TP_printk_btrfs("work->%p", __entry->work)
| );

and btrfs_work_owner exapnds to:

| struct btrfs_fs_info *
| btrfs_work_owner(struct btrfs_work *work)
| {
| return work->wq->fs_info;
| }
 
voilà

> But I still don't understand why backtrace is triggered.
> Since we're just recording a pointer, not touching it.
> 
> Would you please explain the problem with more details on how it trigger the
> problem?

enabled all events played with the fs which was just an upgrade and git
tree sync + checkout so nothing special.

> > 
> > So I think we should either remove the tracepoint completely or change
> > the arguments to take something else than a potentially freed 'work'.
> 
> I'm mostly OK to remove the tracepoint, but such all_workd_done() trace
> should still help to determine if it's a workqueue stalled.
> 
> Thanks,
> Qu

Sebastian
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC] btrfs: lockdep says "possible recursive locking detected" in btrfs_clear_lock_blocking_rw()

2016-12-14 Thread Sebastian Andrzej Siewior 
With lockdep enabled I managed to trigger the following lockdep splat:
| =
| [ INFO: possible recursive locking detected ]
| 4.9.0-rt0 #804 Tainted: GW  
| -
| kworker/u16:4/154 is trying to acquire lock:
|  (btrfs-fs-00){+.+...}, at: [] 
btrfs_clear_lock_blocking_rw+0x71/0x120
| 
| but task is already holding lock:
|  (btrfs-fs-00){+.+...}, at: [] 
btrfs_clear_lock_blocking_rw+0x71/0x120
| 
| other info that might help us debug this:
|  Possible unsafe locking scenario:
|
|CPU0
|
|   lock(btrfs-fs-00);
|   lock(btrfs-fs-00);
| 
|  *** DEADLOCK ***
|
|  May be due to missing lock nesting notation
|
| 6 locks held by kworker/u16:4/154:
|  #0:  ("%s-%s""btrfs", name){.+.+.+}, at: [] 
process_one_work+0x1f3/0x7b0
|  #1:  ((>normal_work)){+.+.+.}, at: [] 
process_one_work+0x1f3/0x7b0
|  #2:  (sb_internal){.+.+..}, at: [] 
start_transaction+0x2f1/0x590
|  #3:  (btrfs-fs-02){+.+...}, at: [] 
btrfs_clear_lock_blocking_rw+0x71/0x120
|  #4:  (btrfs-fs-01){+.+...}, at: [] 
btrfs_clear_lock_blocking_rw+0x71/0x120
|  #5:  (btrfs-fs-00){+.+...}, at: [] 
btrfs_clear_lock_blocking_rw+0x71/0x120
| 
| stack backtrace:
| CPU: 1 PID: 154 Comm: kworker/u16:4 Tainted: GW   4.9.0-rt1+ #804
| Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
1.9.3-20161025_171302-gandalf 04/01/2014
| Workqueue: btrfs-delalloc btrfs_delalloc_helper
|  c9000123b7d0 8141a2a5 829d6db0 829d6db0
|  c9000123b890 810c19dd 02fe 0006
|  3c272f80 82308200 ce68145f590e60eb 880039c108c0
| Call Trace:
|  [] dump_stack+0x86/0xc1
|  [] __lock_acquire+0x6dd/0x11d0
|  [] lock_acquire+0x116/0x240
|  [] rt_read_lock+0x45/0x60
|  [] btrfs_clear_lock_blocking_rw+0x71/0x120
|  [] btrfs_clear_path_blocking+0x94/0xb0
|  [] btrfs_next_old_leaf+0x3df/0x420
|  [] btrfs_next_leaf+0xb/0x10
|  [] __btrfs_drop_extents+0x1cb/0xd50
|  [] cow_file_range_inline+0x191/0x6c0
|  [] compress_file_range.constprop.68+0x314/0x710
|  [] async_cow_start+0x30/0x50
|  [] btrfs_scrubparity_helper+0xfd/0x620
|  [] btrfs_delalloc_helper+0x9/0x10
|  [] process_one_work+0x26e/0x7b0
|  [] worker_thread+0x46/0x560
|  [] kthread+0xee/0x110
|  [] ret_from_fork+0x2a/0x40

I can trigger it on -RT but it won't show up on a vanilla kernel. I
don't see obvious difference here (between RT and !RT). We do have more
preemption points and a spin_lock() does not disable preemption (so any
assumption on spin_lock() disabling preemption will fail).
With all btrfs events enabled, this did not trigger. With the following
patch

--- a/fs/btrfs/locking.c
+++ b/fs/btrfs/locking.c
@@ -41,6 +41,7 @@ void btrfs_set_lock_blocking_rw(struct extent_buffer *eb, int 
rw)
 */
if (eb->lock_nested && current->pid == eb->lock_owner)
return;
+   trace_printk("eb %p rw %d\n", eb, rw);
if (rw == BTRFS_WRITE_LOCK) {
if (atomic_read(>blocking_writers) == 0) {
WARN_ON(atomic_read(>spinning_writers) != 1);
@@ -73,6 +74,7 @@ void btrfs_clear_lock_blocking_rw(struct extent_buffer *eb, 
int rw)
if (eb->lock_nested && current->pid == eb->lock_owner)
return;
 
+   trace_printk("eb %p rw %d\n", eb, rw);
if (rw == BTRFS_WRITE_LOCK_BLOCKING) {
BUG_ON(atomic_read(>blocking_writers) != 1);
write_lock(>lock);

I manage to collect this (the last few lines from the kworker):

#  _-=> irqs-off
# / _=> need-resched
#|/  _-=> need-resched_lazy
#|| / _---=> hardirq/softirq
#||| / _--=> preempt-depth
# / _-=> preempt-lazy-depth
#| / _-=> migrate-disable   
#|| /delay
#   TASK-PID   CPU#  |||   TIMESTAMP  FUNCTION
#  | |   |   |||  | |
   kworker/u16:4-154   [001] .1160.632361: btrfs_set_lock_blocking_rw: 
eb 880039ebac00 rw 1
   kworker/u16:4-154   [001] ...60.632362: 
btrfs_clear_lock_blocking_rw: eb 880039ebac00 rw 3
   kworker/u16:4-154   [001] .1160.632366: btrfs_set_lock_blocking_rw: 
eb 880039ebac00 rw 1
   kworker/u16:4-154   [001] ...60.632367: 
btrfs_clear_lock_blocking_rw: eb 880039ebac00 rw 3
   kworker/u16:4-154   [001] .1160.632367: btrfs_set_lock_blocking_rw: 
eb 880039ebac00 rw 1
   kworker/u16:4-154   [001] ...60.632368: btrfs_set_lock_blocking_rw: 
eb 880039ebac00 rw 3
   kworker/u16:4-154   [001] ...60.632369: 
btrfs_clear_lock_blocking_rw: eb 880039ebac00 rw 3
   kworker/u16:4-154   [001] .1260.632371: btrfs_set_lock_blocking_rw: 
eb 880039ebb000 rw 1
   kworker/u16:4-154   [001]

[PATCH 2/2] btrfs: swap free() and trace point in run_ordered_work()

2016-12-14 Thread Sebastian Andrzej Siewior 
The previous patch removed a trace point due to a use after free problem
with tracing enabled. While looking at the backtrace it took me a while
to find the right spot. While doing so I noticed that this trace point
could be used after one of two clean-up functions were invoked:
- run_one_async_free()
- async_cow_free()

Both of them free the `work' item so a later use in the tracepoint is
not possible.
This patch swaps the order so we first have the trace point and then
free the struct.

Signed-off-by: Sebastian Andrzej Siewior <bige...@linutronix.de>
---
 fs/btrfs/async-thread.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index d0dfc3d2e199..6f4631bf74f8 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -288,8 +288,8 @@ static void run_ordered_work(struct __btrfs_workqueue *wq)
 * we don't want to call the ordered free functions
 * with the lock held though
 */
-   work->ordered_free(work);
trace_btrfs_all_work_done(work);
+   work->ordered_free(work);
}
spin_unlock_irqrestore(lock, flags);
 }
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 1/2] btrfs: drop trace_btrfs_all_work_done() from normal_work_helper()

2016-12-14 Thread Sebastian Andrzej Siewior 
For btrfs_scrubparity_helper() the ->func() is set to
scrub_parity_bio_endio_worker(). This functions invokes
scrub_free_parity() which kfrees() the `work' object. All is good as
long as trace events are not enabled because we boom with a backtrace
like this:
| Workqueue: btrfs-endio btrfs_endio_helper
| RIP: 0010:[]  [] 
trace_event_raw_event_btrfs__work__done+0x4e/0xa0
| Call Trace:
|  [] btrfs_scrubparity_helper+0x59d/0x780
|  [] btrfs_endio_helper+0x9/0x10
|  [] process_one_work+0x26e/0x7b0
|  [] worker_thread+0x46/0x560
|  [] kthread+0xee/0x110
|  [] ret_from_fork+0x2a/0x40

So in order to avoid this, I remove the trace point.

Signed-off-by: Sebastian Andrzej Siewior <bige...@linutronix.de>
---
 fs/btrfs/async-thread.c | 2 --
 1 file changed, 2 deletions(-)

diff --git a/fs/btrfs/async-thread.c b/fs/btrfs/async-thread.c
index e0f071f6b5a7..d0dfc3d2e199 100644
--- a/fs/btrfs/async-thread.c
+++ b/fs/btrfs/async-thread.c
@@ -318,8 +318,6 @@ static void normal_work_helper(struct btrfs_work *work)
set_bit(WORK_DONE_BIT, >flags);
run_ordered_work(wq);
}
-   if (!need_order)
-   trace_btrfs_all_work_done(work);
 }
 
 void btrfs_init_work(struct btrfs_work *work, btrfs_work_func_t uniq_func,
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

oops in wait_block_group_cache_progress() while accessing removed media

2014-09-10 Thread Sebastian Andrzej Siewior 
I manage to crash now and then by removing the mounted mmc card. This
happens randomly. This log is from -rc3 but it also triggered on -rc4:

| BTRFS: bdev /dev/root errs: wr 0, rd 2, flush 0, corrupt 0, gen 0
| BTRFS error (device mmcblk0p2): error reading free space cache
| BTRFS warning (device mmcblk0p2): failed to load free space cache for block 
group 29360128, rebuild it now
| Alignment trap: not handling instruction e1932f9f at [c015bd64]
| Unhandled fault: alignment exception (0x001) at 0x0059
| Internal error: : 1 [#1] ARM
| Modules linked in:
| CPU: 0 PID: 676 Comm: btrfs-transacti Not tainted 
3.17.0-rc3-00023-g1b1867b-dirty #268
| task: de472000 ti: de4c task.ti: de4c
| PC is at wait_block_group_cache_progress+0x48/0xe4
| LR is at find_free_extent+0x514/0xe18
| pc : [c015bd68]lr : [c0168298]psr: 2013
| sp : de4c1ac8  ip : de4313b8  fp : 
| r10: 0001  r9 : 0001  r8 : de474700
| r7 :   r6 : 00204000  r5 :   r4 : 0020
| r3 : 0059  r2 : 00204000  r1 : de4a1d80  r0 : de474700
| Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment kernel
| Control: 10c5387d  Table: 9e4e0019  DAC: 0015
| Process btrfs-transacti (pid: 676, stack limit = 0xde4c0240)
| Stack: (0xde4c1ac8 to 0xde4c2000)
| [c015bd68] (wait_block_group_cache_progress) from [c0168298] 
(find_free_extent+0x514/0xe18)
| [c0168298] (find_free_extent) from [c0168cbc] 
(btrfs_reserve_extent+0x120/0x1ac)
| [c0168cbc] (btrfs_reserve_extent) from [c016a7a0] 
(btrfs_alloc_free_block+0x148/0x684)
| [c016a7a0] (btrfs_alloc_free_block) from [c01509b4] 
(__btrfs_cow_block+0x130/0x778)
| [c01509b4] (__btrfs_cow_block) from [c01511a4] 
(btrfs_cow_block+0x140/0x2c4)
| [c01511a4] (btrfs_cow_block) from [c0155ac4] 
(btrfs_search_slot+0x208/0xa14)
| [c0155ac4] (btrfs_search_slot) from [c0175330] 
(btrfs_lookup_inode+0x44/0x1a8)
| [c0175330] (btrfs_lookup_inode) from [c01df4e8] 
(__btrfs_update_delayed_inode+0x80/0x320)
| [c01df4e8] (__btrfs_update_delayed_inode) from [c01e083c] 
(__btrfs_run_delayed_items+0x15c/0x1e0)
| [c01e083c] (__btrfs_run_delayed_items) from [c017f890] 
(btrfs_commit_transaction+0x1ac/0xbc0)
| [c017f890] (btrfs_commit_transaction) from [c017d9e0] 
(transaction_kthread+0x140/0x1a8)
| [c017d9e0] (transaction_kthread) from [c005352c] (kthread+0xc8/0xe4)
| [c005352c] (kthread) from [c000e898] (ret_from_fork+0x14/0x20)

Don't get confused by the Alignment trap. This is generated before
NULL pointer excpetion (it tried to access 0x59).

addr2line for c015bd68 gives:
  arch/arm/include/asm/atomic.h:46
  fs/btrfs/extent-tree.c:330
  fs/btrfs/extent-tree.c:6294

Sebastian
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html