Re: Creation of pseudo items leads to (seemingly) duplicate inodes (BUG inside)
On 10/19/2011 09:01 AM, Liu Bo wrote: > On 10/19/2011 12:02 AM, Jan Schmidt wrote: >> Hi there, >> >> while playing with snapshots for btrfs send, I also encountered >> seemingly duplicate inodes, which are multiple >> BTRFS_EMPTY_SUBVOL_DIR_OBJECTID objects within the same directory. I can >> imagine software that feels uncomfortable, here. >> >> Such objects are created by btrfs_lookup_dentry() in fs/btrfs/inode.c >> when ENOENT is encountered. >> > > Hi, > > A similar bug has been fixed by one of my patches(still queued): > sorry, I missed something, this one has been merged in Chris's github tree(for rc6). > http://marc.info/?l=linux-btrfs&m=131409014207238&w=2 > > thanks, > liubo > >> The main purpose for this email is to ask: Is this going to stay like that? >> >> David Sterba sent a related issue some days ago with effects seen on >> dcache level (subject: "WARNING: at fs/dcache.c:1256 >> d_set_d_op+0xaa/0xc0() , nested snapshots"). >> >> I hit a bug when trying to rename(2) one of those objects, and I'm still >> hitting it reproducably on 3.1-rc5. However, I cannot reproduce it with >> the current for-linus branch (3.1-rc6). I don't see a commit I'd expect >> to fix this, but it may be fixed: >> >> -- >> Oct 17 13:28:26 oglaroon kernel: [266945.208573] btrfs failed to delete >> reference to snap2, inode 257 parent 256 >> Oct 17 13:28:26 oglaroon kernel: [266945.208586] [ cut here >> ] >> Oct 17 13:28:26 oglaroon kernel: [266945.264688] kernel BUG at >> fs/btrfs/inode.c:6907! >> Oct 17 13:28:26 oglaroon kernel: [266945.320807] invalid opcode: >> [#1] SMP >> Oct 17 13:28:26 oglaroon kernel: [266945.370907] CPU 2 >> Oct 17 13:28:26 oglaroon kernel: [266945.393831] Modules linked in: >> btrfs mpt2sas scsi_transport_sas raid_class [last unloaded: btrfs] >> Oct 17 13:28:26 oglaroon kernel: [266945.503578] >> Oct 17 13:28:26 oglaroon kernel: [266945.522353] Pid: 29567, comm: mv >> Not tainted 3.1.0-rc4+ #5 Supermicro X8SIL/X8SIL >> Oct 17 13:28:26 oglaroon kernel: [266945.613011] RIP: >> 0010:[] [] btrfs_rename+0x42a/0x770 >> [btrfs] >> Oct 17 13:28:26 oglaroon kernel: [266945.720059] RSP: >> 0018:88022f933c78 EFLAGS: 00010282 >> Oct 17 13:28:26 oglaroon kernel: [266945.784476] RAX: fffe >> RBX: 3b2456a4 RCX: 0054 >> Oct 17 13:28:26 oglaroon kernel: [266945.870675] RDX: 0053 >> RSI: 60fdc00042c0 RDI: 88023433 >> Oct 17 13:28:26 oglaroon kernel: [266945.956874] RBP: 88022f933d48 >> R08: R09: 0001 >> Oct 17 13:28:26 oglaroon kernel: [266946.043072] R10: 0006 >> R11: R12: 4e9c1157 >> Oct 17 13:28:26 oglaroon kernel: [266946.129270] R13: 880234468368 >> R14: R15: 880234446d58 >> Oct 17 13:28:26 oglaroon kernel: [266946.215471] FS: >> 7f3bf0b96700() GS:88023fc8() knlGS: >> Oct 17 13:28:26 oglaroon kernel: [266946.313080] CS: 0010 DS: ES: >> CR0: 80050033 >> Oct 17 13:28:26 oglaroon kernel: [266946.382681] CR2: 7f3bf0072110 >> CR3: 00015c05f000 CR4: 06e0 >> Oct 17 13:28:26 oglaroon kernel: [266946.468880] DR0: >> DR1: DR2: >> Oct 17 13:28:26 oglaroon kernel: [266946.555079] DR3: >> DR6: 0ff0 DR7: 0400 >> Oct 17 13:28:26 oglaroon kernel: [266946.641278] Process mv (pid: 29567, >> threadinfo 88022f932000, task 88023433) >> Oct 17 13:28:26 oglaroon kernel: [266946.737849] Stack: >> Oct 17 13:28:26 oglaroon kernel: [266946.762849] 000a >> 0006 00d0 >> Oct 17 13:28:26 oglaroon kernel: [266946.852574] 88022f933c01 >> 0101 00ff8802346b4158 8802349b81b8 >> Oct 17 13:28:26 oglaroon kernel: [266946.942299] >> 88022f9e7000 88022f9e7000 88019d4b6aa8 >> Oct 17 13:28:26 oglaroon kernel: [266947.032025] Call Trace: >> Oct 17 13:28:26 oglaroon kernel: [266947.062215] [] >> vfs_rename+0x162/0x3e0 >> Oct 17 13:28:26 oglaroon kernel: [266947.126629] [] >> sys_renameat+0x239/0x270 >> Oct 17 13:28:26 oglaroon kernel: [266947.193120] [] ? >> do_page_fault+0x20c/0x450 >> Oct 17 13:28:26 oglaroon kernel: [266947.262722] [] ? >> retint_swapgs+0xe/0x13 >> Oct 17 13:28:26 oglaroon kernel: [266947.329214] [] ? >> trace_hardirqs_on_caller+0xfd/0x190 >> Oct 17 13:28:26 oglaroon kernel: [266947.409185] [] >> sys_rename+0x16/0x20 >> Oct 17 13:28:26 oglaroon kernel: [266947.471527] [] >> system_call_fastpath+0x16/0x1b >> Oct 17 13:28:26 oglaroon kernel: [266947.544240] Code: 68 ff ff ff 48 8b >> 4a 30 44 8b 4a 24 4c 8b 42 28 48 8b 55 90 4c 89 95 58 ff ff ff 44 88 9d >> 50 ff ff ff e8 ba c0 ff ff 85 c0 74 4a <0f> 0b eb fe 48 8b 45 80 48 8b >> b8 20 01 00 00 88 95 48 ff ff ff >> Oct 17 13:28:26 oglaroon kernel: [266947.777216] RIP >> [] btrfs_rename+0x42a/0x770 [btrfs]
Re: Creation of pseudo items leads to (seemingly) duplicate inodes (BUG inside)
On 10/19/2011 12:02 AM, Jan Schmidt wrote: > Hi there, > > while playing with snapshots for btrfs send, I also encountered > seemingly duplicate inodes, which are multiple > BTRFS_EMPTY_SUBVOL_DIR_OBJECTID objects within the same directory. I can > imagine software that feels uncomfortable, here. > > Such objects are created by btrfs_lookup_dentry() in fs/btrfs/inode.c > when ENOENT is encountered. > Hi, A similar bug has been fixed by one of my patches(still queued): http://marc.info/?l=linux-btrfs&m=131409014207238&w=2 thanks, liubo > The main purpose for this email is to ask: Is this going to stay like that? > > David Sterba sent a related issue some days ago with effects seen on > dcache level (subject: "WARNING: at fs/dcache.c:1256 > d_set_d_op+0xaa/0xc0() , nested snapshots"). > > I hit a bug when trying to rename(2) one of those objects, and I'm still > hitting it reproducably on 3.1-rc5. However, I cannot reproduce it with > the current for-linus branch (3.1-rc6). I don't see a commit I'd expect > to fix this, but it may be fixed: > > -- > Oct 17 13:28:26 oglaroon kernel: [266945.208573] btrfs failed to delete > reference to snap2, inode 257 parent 256 > Oct 17 13:28:26 oglaroon kernel: [266945.208586] [ cut here > ] > Oct 17 13:28:26 oglaroon kernel: [266945.264688] kernel BUG at > fs/btrfs/inode.c:6907! > Oct 17 13:28:26 oglaroon kernel: [266945.320807] invalid opcode: > [#1] SMP > Oct 17 13:28:26 oglaroon kernel: [266945.370907] CPU 2 > Oct 17 13:28:26 oglaroon kernel: [266945.393831] Modules linked in: > btrfs mpt2sas scsi_transport_sas raid_class [last unloaded: btrfs] > Oct 17 13:28:26 oglaroon kernel: [266945.503578] > Oct 17 13:28:26 oglaroon kernel: [266945.522353] Pid: 29567, comm: mv > Not tainted 3.1.0-rc4+ #5 Supermicro X8SIL/X8SIL > Oct 17 13:28:26 oglaroon kernel: [266945.613011] RIP: > 0010:[] [] btrfs_rename+0x42a/0x770 > [btrfs] > Oct 17 13:28:26 oglaroon kernel: [266945.720059] RSP: > 0018:88022f933c78 EFLAGS: 00010282 > Oct 17 13:28:26 oglaroon kernel: [266945.784476] RAX: fffe > RBX: 3b2456a4 RCX: 0054 > Oct 17 13:28:26 oglaroon kernel: [266945.870675] RDX: 0053 > RSI: 60fdc00042c0 RDI: 88023433 > Oct 17 13:28:26 oglaroon kernel: [266945.956874] RBP: 88022f933d48 > R08: R09: 0001 > Oct 17 13:28:26 oglaroon kernel: [266946.043072] R10: 0006 > R11: R12: 4e9c1157 > Oct 17 13:28:26 oglaroon kernel: [266946.129270] R13: 880234468368 > R14: R15: 880234446d58 > Oct 17 13:28:26 oglaroon kernel: [266946.215471] FS: > 7f3bf0b96700() GS:88023fc8() knlGS: > Oct 17 13:28:26 oglaroon kernel: [266946.313080] CS: 0010 DS: ES: > CR0: 80050033 > Oct 17 13:28:26 oglaroon kernel: [266946.382681] CR2: 7f3bf0072110 > CR3: 00015c05f000 CR4: 06e0 > Oct 17 13:28:26 oglaroon kernel: [266946.468880] DR0: > DR1: DR2: > Oct 17 13:28:26 oglaroon kernel: [266946.555079] DR3: > DR6: 0ff0 DR7: 0400 > Oct 17 13:28:26 oglaroon kernel: [266946.641278] Process mv (pid: 29567, > threadinfo 88022f932000, task 88023433) > Oct 17 13:28:26 oglaroon kernel: [266946.737849] Stack: > Oct 17 13:28:26 oglaroon kernel: [266946.762849] 000a > 0006 00d0 > Oct 17 13:28:26 oglaroon kernel: [266946.852574] 88022f933c01 > 0101 00ff8802346b4158 8802349b81b8 > Oct 17 13:28:26 oglaroon kernel: [266946.942299] > 88022f9e7000 88022f9e7000 88019d4b6aa8 > Oct 17 13:28:26 oglaroon kernel: [266947.032025] Call Trace: > Oct 17 13:28:26 oglaroon kernel: [266947.062215] [] > vfs_rename+0x162/0x3e0 > Oct 17 13:28:26 oglaroon kernel: [266947.126629] [] > sys_renameat+0x239/0x270 > Oct 17 13:28:26 oglaroon kernel: [266947.193120] [] ? > do_page_fault+0x20c/0x450 > Oct 17 13:28:26 oglaroon kernel: [266947.262722] [] ? > retint_swapgs+0xe/0x13 > Oct 17 13:28:26 oglaroon kernel: [266947.329214] [] ? > trace_hardirqs_on_caller+0xfd/0x190 > Oct 17 13:28:26 oglaroon kernel: [266947.409185] [] > sys_rename+0x16/0x20 > Oct 17 13:28:26 oglaroon kernel: [266947.471527] [] > system_call_fastpath+0x16/0x1b > Oct 17 13:28:26 oglaroon kernel: [266947.544240] Code: 68 ff ff ff 48 8b > 4a 30 44 8b 4a 24 4c 8b 42 28 48 8b 55 90 4c 89 95 58 ff ff ff 44 88 9d > 50 ff ff ff e8 ba c0 ff ff 85 c0 74 4a <0f> 0b eb fe 48 8b 45 80 48 8b > b8 20 01 00 00 88 95 48 ff ff ff > Oct 17 13:28:26 oglaroon kernel: [266947.777216] RIP > [] btrfs_rename+0x42a/0x770 [btrfs] > Oct 17 13:28:26 oglaroon kernel: [266947.856259] RSP > Oct 17 13:28:26 oglaroon kernel: [266947.899234] ---[ end trace > fd19520e3af48c17 ]--- > -- > > Reproducer for the curious: > > # mkfs.btrfs /dev/sdv2 > # mount /dev/sdv2
kernel BUG at fs/btrfs/inode.c:1163
Hi today I hit this Bug, kernel is v3.1-rc10 + josef from today, workload is a ceph osd. Best Regards, Martin [28997.273289] [ cut here ] [28997.282916] kernel BUG at fs/btrfs/inode.c:1163! [28997.290863] invalid opcode: [#1] SMP [28997.290863] CPU 0 [28997.290863] Modules linked in: radeon ttm drm_kms_helper drm psmouse sp5100_tco i2c_piix4 i2c_algo_bit serio_raw edac_core k8temp edac_mce_amd shpchp lp parport pata_atiixp btrfs zlib_deflate e1000e libcrc32c ahci libahci [28997.290863] [28997.290863] Pid: 1220, comm: ceph-osd Tainted: GW 3.1.0-rc10+ #2 MICRO-STAR INTERNATIONAL CO., LTD MS-96B3/MS-96B3 [28997.290863] RIP: 0010:[] [] run_delalloc_nocow+0x7a7/0x7c0 [btrfs] [28997.290863] RSP: 0018:880117357a78 EFLAGS: 00010206 [28997.290863] RAX: 002f RBX: 880116b12a20 RCX: 880117357a38 [28997.290863] RDX: 8800 RSI: 0496 RDI: 8801003851e0 [28997.290863] RBP: 880117357b78 R08: 0497 R09: 880117357a28 [28997.290863] R10: 0030 R11: R12: 00011d3b [28997.290863] R13: 00011d3b R14: 8801003851e0 R15: 0030 [28997.290863] FS: 7ff45ae7b700() GS:88011fc0() knlGS: [28997.507960] CS: 0010 DS: ES: CR0: 8005003b [28997.507960] CR2: 7ff450a2 CR3: 000114b75000 CR4: 06f0 [28997.507960] DR0: DR1: DR2: [28997.507960] DR3: DR6: 0ff0 DR7: 0400 [28997.507960] Process ceph-osd (pid: 1220, threadinfo 880117356000, task 88011526) [28997.507960] Stack: [28997.507960] 880117357aa8 81156e90 880104413af0 880104413af0 [28997.507960] 880117550030 880117357bf0 880117550028 880117550020 [28997.507960] 0040 00010040 880117357d14 00ffa00a973e [28997.507960] Call Trace: [28997.507960] [] ? kmem_cache_free+0x20/0x100 [28997.507960] [] run_delalloc_range+0x334/0x380 [btrfs] [28997.507960] [] __extent_writepage+0x5b5/0x6f0 [btrfs] [28997.507960] [] ? radix_tree_gang_lookup_tag_slot+0x8d/0xd0 [28997.507960] [] extent_write_cache_pages.clone.19.clone.26+0x22a/0x3a0 [btrfs] [28997.507960] [] extent_writepages+0x45/0x60 [btrfs] [28997.507960] [] ? acls_after_inode_item+0xc0/0xc0 [btrfs] [28997.507960] [] ? vfsmount_lock_local_unlock+0x1e/0x30 [28997.507960] [] btrfs_writepages+0x27/0x30 [btrfs] [28997.507960] [] do_writepages+0x21/0x40 [28997.507960] [] __filemap_fdatawrite_range+0x5b/0x60 [28997.507960] [] filemap_fdatawrite_range+0x13/0x20 [28997.507960] [] sys_sync_file_range+0x149/0x180 [28997.835220] [] system_call_fastpath+0x16/0x1b [28997.835220] Code: 8b 7d 80 e8 dc 9e 00 00 41 b9 04 00 00 00 e9 3d fe ff ff 4d 89 ef 41 bc 01 00 00 00 48 c7 45 a8 ff ff ff ff e9 5c fb ff ff 0f 0b <0f> 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 66 0f 1f 84 00 [28997.835220] RIP [] run_delalloc_nocow+0x7a7/0x7c0 [btrfs] [28997.835220] RSP [28997.927402] ---[ end trace a0a1c4a13d975229 ]--- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Creation of pseudo items leads to (seemingly) duplicate inodes (BUG inside)
On Tue, Oct 18, 2011 at 06:02:09PM +0200, Jan Schmidt wrote: > Hi there, > > while playing with snapshots for btrfs send, I also encountered > seemingly duplicate inodes, which are multiple > BTRFS_EMPTY_SUBVOL_DIR_OBJECTID objects within the same directory. I can > imagine software that feels uncomfortable, here. > > Such objects are created by btrfs_lookup_dentry() in fs/btrfs/inode.c > when ENOENT is encountered. Such inodes are created by btrfs_lookup_dentry() when a directry item which is not a first reference to a subvolume is encountered. Subvolumes (and snapshots) can be created anywhere, but only one access point (the first one) is allowed to prevent the same problems that would arise if directory hardlinks were allowed. > The main purpose for this email is to ask: Is this going to stay like that? > > David Sterba sent a related issue some days ago with effects seen on > dcache level (subject: "WARNING: at fs/dcache.c:1256 > d_set_d_op+0xaa/0xc0() , nested snapshots"). I've been meaning to take a look at since then.. > I hit a bug when trying to rename(2) one of those objects, and I'm still > hitting it reproducably on 3.1-rc5. However, I cannot reproduce it with > the current for-linus branch (3.1-rc6). I don't see a commit I'd expect > to fix this, but it may be fixed: > > -- > Oct 17 13:28:26 oglaroon kernel: [266945.208573] btrfs failed to delete > reference to snap2, inode 257 parent 256 > Oct 17 13:28:26 oglaroon kernel: [266945.208586] [ cut here > ] > Oct 17 13:28:26 oglaroon kernel: [266945.264688] kernel BUG at > fs/btrfs/inode.c:6907! > Oct 17 13:28:26 oglaroon kernel: [266945.320807] invalid opcode: > [#1] SMP > Oct 17 13:28:26 oglaroon kernel: [266945.370907] CPU 2 > Oct 17 13:28:26 oglaroon kernel: [266945.393831] Modules linked in: > btrfs mpt2sas scsi_transport_sas raid_class [last unloaded: btrfs] > Oct 17 13:28:26 oglaroon kernel: [266945.503578] > Oct 17 13:28:26 oglaroon kernel: [266945.522353] Pid: 29567, comm: mv > Not tainted 3.1.0-rc4+ #5 Supermicro X8SIL/X8SIL > Oct 17 13:28:26 oglaroon kernel: [266945.613011] RIP: > 0010:[] [] btrfs_rename+0x42a/0x770 > [btrfs] > Oct 17 13:28:26 oglaroon kernel: [266945.720059] RSP: > 0018:88022f933c78 EFLAGS: 00010282 > Oct 17 13:28:26 oglaroon kernel: [266945.784476] RAX: fffe > RBX: 3b2456a4 RCX: 0054 > Oct 17 13:28:26 oglaroon kernel: [266945.870675] RDX: 0053 > RSI: 60fdc00042c0 RDI: 88023433 > Oct 17 13:28:26 oglaroon kernel: [266945.956874] RBP: 88022f933d48 > R08: R09: 0001 > Oct 17 13:28:26 oglaroon kernel: [266946.043072] R10: 0006 > R11: R12: 4e9c1157 > Oct 17 13:28:26 oglaroon kernel: [266946.129270] R13: 880234468368 > R14: R15: 880234446d58 > Oct 17 13:28:26 oglaroon kernel: [266946.215471] FS: > 7f3bf0b96700() GS:88023fc8() knlGS: > Oct 17 13:28:26 oglaroon kernel: [266946.313080] CS: 0010 DS: ES: > CR0: 80050033 > Oct 17 13:28:26 oglaroon kernel: [266946.382681] CR2: 7f3bf0072110 > CR3: 00015c05f000 CR4: 06e0 > Oct 17 13:28:26 oglaroon kernel: [266946.468880] DR0: > DR1: DR2: > Oct 17 13:28:26 oglaroon kernel: [266946.555079] DR3: > DR6: 0ff0 DR7: 0400 > Oct 17 13:28:26 oglaroon kernel: [266946.641278] Process mv (pid: 29567, > threadinfo 88022f932000, task 88023433) > Oct 17 13:28:26 oglaroon kernel: [266946.737849] Stack: > Oct 17 13:28:26 oglaroon kernel: [266946.762849] 000a > 0006 00d0 > Oct 17 13:28:26 oglaroon kernel: [266946.852574] 88022f933c01 > 0101 00ff8802346b4158 8802349b81b8 > Oct 17 13:28:26 oglaroon kernel: [266946.942299] > 88022f9e7000 88022f9e7000 88019d4b6aa8 > Oct 17 13:28:26 oglaroon kernel: [266947.032025] Call Trace: > Oct 17 13:28:26 oglaroon kernel: [266947.062215] [] > vfs_rename+0x162/0x3e0 > Oct 17 13:28:26 oglaroon kernel: [266947.126629] [] > sys_renameat+0x239/0x270 > Oct 17 13:28:26 oglaroon kernel: [266947.193120] [] ? > do_page_fault+0x20c/0x450 > Oct 17 13:28:26 oglaroon kernel: [266947.262722] [] ? > retint_swapgs+0xe/0x13 > Oct 17 13:28:26 oglaroon kernel: [266947.329214] [] ? > trace_hardirqs_on_caller+0xfd/0x190 > Oct 17 13:28:26 oglaroon kernel: [266947.409185] [] > sys_rename+0x16/0x20 > Oct 17 13:28:26 oglaroon kernel: [266947.471527] [] > system_call_fastpath+0x16/0x1b > Oct 17 13:28:26 oglaroon kernel: [266947.544240] Code: 68 ff ff ff 48 8b > 4a 30 44 8b 4a 24 4c 8b 42 28 48 8b 55 90 4c 89 95 58 ff ff ff 44 88 9d > 50 ff ff ff e8 ba c0 ff ff 85 c0 74 4a <0f> 0b eb fe 48 8b 45 80 48 8b > b8 20 01 00 00 88 95 48 ff ff ff > Oct 17 13:28:26 oglaroon kernel: [266947.777216] RIP > [] btrfs_rename+0x42a/0x77
Re: Broken DIR_ITEMs on snapshot
On Tue, Oct 18, 2011 at 06:01:11PM +0200, Jan Schmidt wrote: > Hi, > > while still busy with btrfs send, I came across some strange DIR_ITEMs. > I looked into that briefly, but I'd rather return to implementing btrfs > send, hoping someone is willing to make up his mind on this one :-) > > To reproduce, do the following: > > # mkfs.btrfs /dev/sdv2 > # mount /dev/sdv2 /mnt > # btrfs subvol snap /mnt /mnt/snap1 > > You've a freshly created snapshot. However, file tree 256 (the > snap1-tree) will contain two strange items: > > item 2 key (256 DIR_ITEM 3645318598) itemoff 3788 itemsize 35 > location key (256 ROOT_ITEM 18446744073709551615) type 2 > namelen 5 datalen 0 name: snap1 > item 3 key (256 DIR_INDEX 2) itemoff 3753 itemsize 35 > location key (256 ROOT_ITEM 18446744073709551615) type 2 > namelen 5 datalen 0 name: snap1 > > These items are needed in tree 5 (fs tree) to reference snap1. However, > within snap1, I'd not expect the entries. A brief look into > create_pending_snapshot reveals > >... >btrfs_insert_dir_item() >... >/* some delayed stuff with scary comments */ >... >btrfs_cow_block() >... > > I'm not sure whether cowing earlier would help, I'm particularly > uncertain because of the run_delayed_* code in between. So I haven't > tried to fix this, I'm convinced it should be fixed, though. I don't think it's a bug. Directory item snap1 (the access point) is inserted into /mnt (defaut subvolume) and THEN a snapshot is taken. So snap1 fs-tree contains that directory item, which in Btrfs terms means DIR_ITEM and DIR_INDEX items in the fs-tree. > These items lead to some strange effects: > > # cd /mnt/snap1 > # ls -l > dr-xr-xr-x 1 root root 10 Jan 1 1970 . > dr-xr-xr-x 1 root root 16 Oct 18 15:56 .. > # mkdir snap1 > mkdir: cannot create directory `snap1': File exists > # stat snap1 > File: `snap1' > Size: 0 Blocks: 0 IO Block: 4096 directory > Device: 11h/17d Inode: 2 Links: 1 > # rmdir snap1 > # stat snap1 > stat: cannot stat `snap1': No such file or directory > > Inode number 2 seems to be BTRFS_EMPTY_SUBVOL_DIR_OBJECTID, the pseudo > object is created by btrfs_lookup_dentry() in inode.c when ENOENT is > encountered. Because from the user's point of view that snap1 item shouldn't be there, readdir() skips it and consequently ls doesn't show it. However when you stat() it you force a lookup on it and end up with a special inode which is there to ensure there is only one valid access point to a particular subvolume. > As a side note: the timestamp of the snap-dir item could be prettier. > > No such pseudo items are created when the snapshot is placed outside of > the subvolume to be snapshotted, obviously. In the above example, do ... > > # btrfs subvol snap /mnt/snap1 /mnt/snap2 > > ... and no such items will be created, which makes me quite certain the > existence of above mentioned DIR_ITEMs is a bug, isn't it? And here you are taking a snapshot of snap1 while placing the access point into /mnt. Since in this case the access point is not placed into the subvolume you are taking a snapshot of, snap2 fs-tree doesn't have those items. Thanks, Ilya > -Jan > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: if we have a lot of pinned space, commit the transaction
Mitch kept hitting a panic because he was getting ENOSPC. One of my previous
patches makes it so we are much better at not allocating new metadata chunks.
Unfortunately coupled with the overcommit patch this works us into a bit of a
problem if we are removing a bunch of space and end up chewing up all of our
space with pinned extents. We can allocate chunks fine and overflow is ok, but
the only way to reclaim this space is to commit the transaction. So if we go to
overcommit, first check and see how much pinned space we have. If we have more
than 80% of the free space chewed up with pinned extents, just commit the
transaction, this will free up enough space for our reservation and we won't
have this problem anymore. With this patch Mitch's test doesn't blow up
anymore. Thanks,
Reported-and-tested-by: Mitch Harder
Signed-off-by: Josef Bacik
---
fs/btrfs/extent-tree.c | 15 +++
1 files changed, 15 insertions(+), 0 deletions(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index a5f1421..4eb7d2b 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3510,6 +3510,20 @@ again:
u64 profile = btrfs_get_alloc_profile(root, 0);
u64 avail;
+ /*
+* If we have a lot of space that's pinned, don't bother doing
+* the overcommit dance yet and just commit the transaction.
+*/
+ avail = (space_info->total_bytes - space_info->bytes_used) * 8;
+ do_div(avail, 10);
+ if (space_info->bytes_pinned >= avail && flush && !trans &&
+ !committed) {
+ space_info->flush = 1;
+ flushing = true;
+ spin_unlock(&space_info->lock);
+ goto commit;
+ }
+
spin_lock(&root->fs_info->free_chunk_lock);
avail = root->fs_info->free_chunk_space;
@@ -3581,6 +3595,7 @@ again:
if (trans)
goto out;
+commit:
ret = -ENOSPC;
if (committed)
goto out;
--
1.7.5.2
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: seperate out btrfs_block_rsv_check out into 2 different functions
Currently btrfs_block_rsv_check does 2 things, it will either refill a block
reserve like in the truncate or refill case, or it will check to see if there is
enough space in the global reserve and possibly refill it. However because of
overcommit we could be well overcommitting ourselves just to try and refill the
global reserve, when really we should just be committing the transaction. So
breack this out into btrfs_block_rsv_refill and btrfs_block_rsv_check. Refill
will try to reserve more metadata if it can and btrfs_block_rsv_check will not,
it will only tell you if the factor of the total space is still reserved.
Thanks,
Signed-off-by: Josef Bacik
---
fs/btrfs/ctree.h|4 +++-
fs/btrfs/extent-tree.c | 41 +++--
fs/btrfs/free-space-cache.c |2 +-
fs/btrfs/inode.c|4 ++--
fs/btrfs/relocation.c |6 ++
fs/btrfs/transaction.c |4 ++--
6 files changed, 37 insertions(+), 24 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index ea60897..2276209 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2252,8 +2252,10 @@ int btrfs_block_rsv_add(struct btrfs_root *root,
struct btrfs_block_rsv *block_rsv,
u64 num_bytes);
int btrfs_block_rsv_check(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv, int min_factor);
+int btrfs_block_rsv_refill(struct btrfs_root *root,
struct btrfs_block_rsv *block_rsv,
- u64 min_reserved, int min_factor, int flush);
+ u64 min_reserved);
int btrfs_block_rsv_migrate(struct btrfs_block_rsv *src_rsv,
struct btrfs_block_rsv *dst_rsv,
u64 num_bytes);
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index eb4fe56..a5f1421 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3422,7 +3422,6 @@ static int shrink_delalloc(struct btrfs_trans_handle
*trans,
* @block_rsv - the block_rsv we're allocating for
* @orig_bytes - the number of bytes we want
* @flush - wether or not we can flush to make our reservation
- * @check - wether this is just to check if we have enough space or not
*
* This will reserve orgi_bytes number of bytes from the space info associated
* with the block_rsv. If there is not enough space it will make an attempt to
@@ -3433,7 +3432,7 @@ static int shrink_delalloc(struct btrfs_trans_handle
*trans,
*/
static int reserve_metadata_bytes(struct btrfs_root *root,
struct btrfs_block_rsv *block_rsv,
- u64 orig_bytes, int flush, int check)
+ u64 orig_bytes, int flush)
{
struct btrfs_space_info *space_info = block_rsv->space_info;
struct btrfs_trans_handle *trans;
@@ -3507,7 +3506,7 @@ again:
(orig_bytes * (retries + 1));
}
- if (ret && !check) {
+ if (ret) {
u64 profile = btrfs_get_alloc_profile(root, 0);
u64 avail;
@@ -3742,7 +3741,7 @@ int btrfs_block_rsv_add(struct btrfs_root *root,
if (num_bytes == 0)
return 0;
- ret = reserve_metadata_bytes(root, block_rsv, num_bytes, 1, 0);
+ ret = reserve_metadata_bytes(root, block_rsv, num_bytes, 1);
if (!ret) {
block_rsv_add_bytes(block_rsv, num_bytes, 1);
return 0;
@@ -3752,8 +3751,7 @@ int btrfs_block_rsv_add(struct btrfs_root *root,
}
int btrfs_block_rsv_check(struct btrfs_root *root,
- struct btrfs_block_rsv *block_rsv,
- u64 min_reserved, int min_factor, int flush)
+ struct btrfs_block_rsv *block_rsv, int min_factor)
{
u64 num_bytes = 0;
int ret = -ENOSPC;
@@ -3762,11 +3760,26 @@ int btrfs_block_rsv_check(struct btrfs_root *root,
return 0;
spin_lock(&block_rsv->lock);
- if (min_factor > 0)
- num_bytes = div_factor(block_rsv->size, min_factor);
- if (min_reserved > num_bytes)
- num_bytes = min_reserved;
+ num_bytes = div_factor(block_rsv->size, min_factor);
+ if (block_rsv->reserved >= num_bytes)
+ ret = 0;
+ spin_unlock(&block_rsv->lock);
+ return ret;
+}
+
+int btrfs_block_rsv_refill(struct btrfs_root *root,
+ struct btrfs_block_rsv *block_rsv,
+ u64 min_reserved)
+{
+ u64 num_bytes = 0;
+ int ret = -ENOSPC;
+
+ if (!block_rsv)
+ return 0;
+
+ spin_lock(&block_rsv->lock);
+ num_bytes = min_reserved;
if (block_rsv->reserved >= num_bytes)
ret = 0;
else
@@ -3776,7 +3789,7 @@ int btrfs_block_rsv_check(struct btrfs_root *root,
if (!ret)
return 0;
-
Re: [PATCH 1/2][BTRFS-PROGS] add the "--force" switch to the mkfs.btrfs command
I forgot to mention that you can pull these changes from the following repository: http://cassiopea.homelinux.net/git/btrfs-progs-unstable.git branch force-mkfs Of course these patches are re-based on the latest Hugo's integration (integration-20111012) BR On Tuesday, 18 October, 2011 18:45:13 Goffredo Baroncelli wrote: > As requested by Mitch, I resend the patch related to the "--force" option > for the mkfs.btrfs command. > > This patch allow to bypass the check which ensure that you are formatting a > not mounted partition/disk. > Sometime (eg in a chrooted environment where the /proc filesystem is not > mounted ) you don't have all the information, and the check fails even if > the device is not mounted. > > Passing the --force|-f flag to the mkfs.btrfs command you bypass this check. > > This patch update the mkfs.c program. > > On Monday, 17 October, 2011 15:11:15 you wrote: > > Goffredo: > > > > I was wanting to see if your mkfs.btrfs --force patches could get > > picked up in the Btrfs-progs testing tree that Hugo Mills is trying to > > maintain. > > > > He was OK with the patches, but was hoping for a 'signed-off-by' and > > more description in the commit comment. > > > > Could you resend your patches to the list with a 'signed-off-by' and > > some additional description? > > > > Some suggested text for the comment: > > > > Certain operating environments (such as a chroot environment) can lead > > to mkfs.btrfs failure while checking devices (particularly when all > > devices are not fully described in /dev). > > > > The --force option skips device checking, and forces formating on the > > specified target. > > > > Thanks. -- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/2][BTRFS-PROGS] add the "--force" switch to the mkfs.btrfs command
This patch update the man page -- gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo) Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512>From 82c11672909e5cc784d08d9361c69f0efb122ef1 Mon Sep 17 00:00:00 2001 From: Goffredo Baroncelli Date: Mon, 3 Jan 2011 19:53:05 +0100 Subject: [PATCH 2/2] mkfs.btrfs man page: document the --force option. Add the --force option to not check if a device is already mounted. Signed-off-by: Goffredo Baroncelli --- man/mkfs.btrfs.8.in |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/man/mkfs.btrfs.8.in b/man/mkfs.btrfs.8.in index 432db1b..2610e9d 100644 --- a/man/mkfs.btrfs.8.in +++ b/man/mkfs.btrfs.8.in @@ -6,6 +6,7 @@ mkfs.btrfs \- create an btrfs filesystem [ \fB\-A\fP\fI alloc-start\fP ] [ \fB\-b\fP\fI byte-count\fP ] [ \fB \-d\fP\fI data-profile\fP ] +[ \fB \-f\fP ] [ \fB \-l\fP\fI leafsize\fP ] [ \fB \-L\fP\fI label\fP ] [ \fB \-m\fP\fI metadata profile\fP ] @@ -35,6 +36,9 @@ mkfs.btrfs uses all the available storage for the filesystem. Specify how the data must be spanned across the devices specified. Valid values are raid0, raid1, raid10 or single. .TP +\fB\-f\fR, \fB\-\-force \fR +Don't check if the device is already mounted. +.TP \fB\-l\fR, \fB\-\-leafsize \fIsize\fR Specify the leaf size, the least data item in which btrfs stores data. The default value is the page size. -- 1.7.7 signature.asc Description: This is a digitally signed message part.
[PATCH 1/2][BTRFS-PROGS] add the "--force" switch to the mkfs.btrfs command
As requested by Mitch, I resend the patch related to the "--force" option for
the mkfs.btrfs command.
This patch allow to bypass the check which ensure that you are formatting a
not mounted partition/disk.
Sometime (eg in a chrooted environment where the /proc filesystem is not
mounted ) you don't have all the information, and the check fails even if the
device is not mounted.
Passing the --force|-f flag to the mkfs.btrfs command you bypass this check.
This patch update the mkfs.c program.
On Monday, 17 October, 2011 15:11:15 you wrote:
> Goffredo:
>
> I was wanting to see if your mkfs.btrfs --force patches could get
> picked up in the Btrfs-progs testing tree that Hugo Mills is trying to
> maintain.
>
> He was OK with the patches, but was hoping for a 'signed-off-by' and
> more description in the commit comment.
>
> Could you resend your patches to the list with a 'signed-off-by' and
> some additional description?
>
> Some suggested text for the comment:
>
> Certain operating environments (such as a chroot environment) can lead
> to mkfs.btrfs failure while checking devices (particularly when all
> devices are not fully described in /dev).
>
> The --force option skips device checking, and forces formating on the
> specified target.
>
> Thanks.
--
gpg key@ keyserver.linux.it: Goffredo Baroncelli (ghigo)
Key fingerprint = 4769 7E51 5293 D36C 814E C054 BF04 F161 3DC5 0512>From 037b3d6f01da086dc7d308001289cb79bc08e0bd Mon Sep 17 00:00:00 2001
From: Goffredo Baroncelli
Date: Mon, 3 Jan 2011 19:51:46 +0100
Subject: [PATCH 1/2] Add the --force option.
Add the --force option to not check if a device is already mounted.
Signed-off-by: Goffredo Baroncelli
---
mkfs.c | 46 --
1 files changed, 28 insertions(+), 18 deletions(-)
diff --git a/mkfs.c b/mkfs.c
index e3ced19..be236d0 100644
--- a/mkfs.c
+++ b/mkfs.c
@@ -303,6 +303,7 @@ static void print_usage(void)
fprintf(stderr, "\t -A --alloc-start the offset to start the FS\n");
fprintf(stderr, "\t -b --byte-count total number of bytes in the FS\n");
fprintf(stderr, "\t -d --data data profile, raid0, raid1, raid10 or single\n");
+ fprintf(stderr, "\t -f --force don't check if a device is already mounted\n");
fprintf(stderr, "\t -l --leafsize size of btree leaves\n");
fprintf(stderr, "\t -L --label set a label\n");
fprintf(stderr, "\t -m --metadata metadata profile, values like data profile\n");
@@ -368,6 +369,7 @@ static struct option long_options[] = {
{ "data", 1, NULL, 'd' },
{ "version", 0, NULL, 'V' },
{ "rootdir", 1, NULL, 'r' },
+ { "force", 0, NULL, 'f' },
{ 0, 0, 0, 0}
};
@@ -1191,10 +1193,11 @@ int main(int ac, char **av)
u64 size_of_data = 0;
u64 source_dir_size = 0;
char *pretty_buf;
+ int force=0;
while(1) {
int c;
- c = getopt_long(ac, av, "A:b:l:n:s:m:d:L:r:VM", long_options,
+ c = getopt_long(ac, av, "A:b:l:n:s:m:d:L:r:VMf", long_options,
&option_index);
if (c < 0)
break;
@@ -1240,6 +1243,8 @@ int main(int ac, char **av)
case 'r':
source_dir = optarg;
source_dir_set = 1;
+ case 'f':
+force=1;
break;
default:
print_usage();
@@ -1263,14 +1268,17 @@ int main(int ac, char **av)
if (source_dir == 0) {
file = av[optind++];
- ret = check_mounted(file);
- if (ret < 0) {
- fprintf(stderr, "error checking %s mount status\n", file);
- exit(1);
- }
- if (ret == 1) {
- fprintf(stderr, "%s is mounted\n", file);
- exit(1);
+ if(!force){
+ ret = check_mounted(file);
+ if (ret < 0) {
+fprintf(stderr,
+"error checking %s mount status\n", file);
+exit(1);
+ }
+ if (ret == 1) {
+fprintf(stderr, "%s is mounted\n", file);
+exit(1);
+ }
}
ac--;
fd = open(file, O_RDWR);
@@ -1353,15 +1361,17 @@ int main(int ac, char **av)
int old_mixed = mixed;
file = av[optind++];
- ret = check_mounted(file);
- if (ret < 0) {
- fprintf(stderr, "error checking %s mount status\n",
-file);
- exit(1);
- }
- if (ret == 1) {
- fprintf(stderr, "%s is mounted\n", file);
- exit(1);
+ if(!force){
+ ret = check_mounted(file);
+ if (ret < 0) {
+fprintf(stderr, "error checking %s"
+ " mount status\n",file);
+exit(1);
+ }
+ if (ret == 1) {
+fprintf(stderr, "%s is mounted\n", file);
+exit(1);
+ }
}
fd = open(file, O_RDWR);
if (fd < 0) {
--
1.7.7
signature.asc
Description: This is a digitally signed message part.
Creation of pseudo items leads to (seemingly) duplicate inodes (BUG inside)
Hi there, while playing with snapshots for btrfs send, I also encountered seemingly duplicate inodes, which are multiple BTRFS_EMPTY_SUBVOL_DIR_OBJECTID objects within the same directory. I can imagine software that feels uncomfortable, here. Such objects are created by btrfs_lookup_dentry() in fs/btrfs/inode.c when ENOENT is encountered. The main purpose for this email is to ask: Is this going to stay like that? David Sterba sent a related issue some days ago with effects seen on dcache level (subject: "WARNING: at fs/dcache.c:1256 d_set_d_op+0xaa/0xc0() , nested snapshots"). I hit a bug when trying to rename(2) one of those objects, and I'm still hitting it reproducably on 3.1-rc5. However, I cannot reproduce it with the current for-linus branch (3.1-rc6). I don't see a commit I'd expect to fix this, but it may be fixed: -- Oct 17 13:28:26 oglaroon kernel: [266945.208573] btrfs failed to delete reference to snap2, inode 257 parent 256 Oct 17 13:28:26 oglaroon kernel: [266945.208586] [ cut here ] Oct 17 13:28:26 oglaroon kernel: [266945.264688] kernel BUG at fs/btrfs/inode.c:6907! Oct 17 13:28:26 oglaroon kernel: [266945.320807] invalid opcode: [#1] SMP Oct 17 13:28:26 oglaroon kernel: [266945.370907] CPU 2 Oct 17 13:28:26 oglaroon kernel: [266945.393831] Modules linked in: btrfs mpt2sas scsi_transport_sas raid_class [last unloaded: btrfs] Oct 17 13:28:26 oglaroon kernel: [266945.503578] Oct 17 13:28:26 oglaroon kernel: [266945.522353] Pid: 29567, comm: mv Not tainted 3.1.0-rc4+ #5 Supermicro X8SIL/X8SIL Oct 17 13:28:26 oglaroon kernel: [266945.613011] RIP: 0010:[] [] btrfs_rename+0x42a/0x770 [btrfs] Oct 17 13:28:26 oglaroon kernel: [266945.720059] RSP: 0018:88022f933c78 EFLAGS: 00010282 Oct 17 13:28:26 oglaroon kernel: [266945.784476] RAX: fffe RBX: 3b2456a4 RCX: 0054 Oct 17 13:28:26 oglaroon kernel: [266945.870675] RDX: 0053 RSI: 60fdc00042c0 RDI: 88023433 Oct 17 13:28:26 oglaroon kernel: [266945.956874] RBP: 88022f933d48 R08: R09: 0001 Oct 17 13:28:26 oglaroon kernel: [266946.043072] R10: 0006 R11: R12: 4e9c1157 Oct 17 13:28:26 oglaroon kernel: [266946.129270] R13: 880234468368 R14: R15: 880234446d58 Oct 17 13:28:26 oglaroon kernel: [266946.215471] FS: 7f3bf0b96700() GS:88023fc8() knlGS: Oct 17 13:28:26 oglaroon kernel: [266946.313080] CS: 0010 DS: ES: CR0: 80050033 Oct 17 13:28:26 oglaroon kernel: [266946.382681] CR2: 7f3bf0072110 CR3: 00015c05f000 CR4: 06e0 Oct 17 13:28:26 oglaroon kernel: [266946.468880] DR0: DR1: DR2: Oct 17 13:28:26 oglaroon kernel: [266946.555079] DR3: DR6: 0ff0 DR7: 0400 Oct 17 13:28:26 oglaroon kernel: [266946.641278] Process mv (pid: 29567, threadinfo 88022f932000, task 88023433) Oct 17 13:28:26 oglaroon kernel: [266946.737849] Stack: Oct 17 13:28:26 oglaroon kernel: [266946.762849] 000a 0006 00d0 Oct 17 13:28:26 oglaroon kernel: [266946.852574] 88022f933c01 0101 00ff8802346b4158 8802349b81b8 Oct 17 13:28:26 oglaroon kernel: [266946.942299] 88022f9e7000 88022f9e7000 88019d4b6aa8 Oct 17 13:28:26 oglaroon kernel: [266947.032025] Call Trace: Oct 17 13:28:26 oglaroon kernel: [266947.062215] [] vfs_rename+0x162/0x3e0 Oct 17 13:28:26 oglaroon kernel: [266947.126629] [] sys_renameat+0x239/0x270 Oct 17 13:28:26 oglaroon kernel: [266947.193120] [] ? do_page_fault+0x20c/0x450 Oct 17 13:28:26 oglaroon kernel: [266947.262722] [] ? retint_swapgs+0xe/0x13 Oct 17 13:28:26 oglaroon kernel: [266947.329214] [] ? trace_hardirqs_on_caller+0xfd/0x190 Oct 17 13:28:26 oglaroon kernel: [266947.409185] [] sys_rename+0x16/0x20 Oct 17 13:28:26 oglaroon kernel: [266947.471527] [] system_call_fastpath+0x16/0x1b Oct 17 13:28:26 oglaroon kernel: [266947.544240] Code: 68 ff ff ff 48 8b 4a 30 44 8b 4a 24 4c 8b 42 28 48 8b 55 90 4c 89 95 58 ff ff ff 44 88 9d 50 ff ff ff e8 ba c0 ff ff 85 c0 74 4a <0f> 0b eb fe 48 8b 45 80 48 8b b8 20 01 00 00 88 95 48 ff ff ff Oct 17 13:28:26 oglaroon kernel: [266947.777216] RIP [] btrfs_rename+0x42a/0x770 [btrfs] Oct 17 13:28:26 oglaroon kernel: [266947.856259] RSP Oct 17 13:28:26 oglaroon kernel: [266947.899234] ---[ end trace fd19520e3af48c17 ]--- -- Reproducer for the curious: # mkfs.btrfs /dev/sdv2 # mount /dev/sdv2 /mnt # btrfs subvol snap /mnt /mnt/snap1 # btrfs subvol snap /mnt /mnt/snap2 # btrfs subvol snap /mnt /mnt/snap3 When snap2 was created, there was a dir item for snap1, so this is no surprise: # ls -lai /mnt/snap2 total 8 256 dr-xr-xr-x 1 root root 20 Jan 1 1970 . 256 dr-xr-xr-x 1 root root 30 Jan 1 1970 .. 2 drwxr-xr-x 1 root root 0 Oct 18 16:25 snap1 Inode 2 seems a bit
Broken DIR_ITEMs on snapshot
Hi, while still busy with btrfs send, I came across some strange DIR_ITEMs. I looked into that briefly, but I'd rather return to implementing btrfs send, hoping someone is willing to make up his mind on this one :-) To reproduce, do the following: # mkfs.btrfs /dev/sdv2 # mount /dev/sdv2 /mnt # btrfs subvol snap /mnt /mnt/snap1 You've a freshly created snapshot. However, file tree 256 (the snap1-tree) will contain two strange items: item 2 key (256 DIR_ITEM 3645318598) itemoff 3788 itemsize 35 location key (256 ROOT_ITEM 18446744073709551615) type 2 namelen 5 datalen 0 name: snap1 item 3 key (256 DIR_INDEX 2) itemoff 3753 itemsize 35 location key (256 ROOT_ITEM 18446744073709551615) type 2 namelen 5 datalen 0 name: snap1 These items are needed in tree 5 (fs tree) to reference snap1. However, within snap1, I'd not expect the entries. A brief look into create_pending_snapshot reveals ... btrfs_insert_dir_item() ... /* some delayed stuff with scary comments */ ... btrfs_cow_block() ... I'm not sure whether cowing earlier would help, I'm particularly uncertain because of the run_delayed_* code in between. So I haven't tried to fix this, I'm convinced it should be fixed, though. These items lead to some strange effects: # cd /mnt/snap1 # ls -l dr-xr-xr-x 1 root root 10 Jan 1 1970 . dr-xr-xr-x 1 root root 16 Oct 18 15:56 .. # mkdir snap1 mkdir: cannot create directory `snap1': File exists # stat snap1 File: `snap1' Size: 0 Blocks: 0 IO Block: 4096 directory Device: 11h/17d Inode: 2 Links: 1 # rmdir snap1 # stat snap1 stat: cannot stat `snap1': No such file or directory Inode number 2 seems to be BTRFS_EMPTY_SUBVOL_DIR_OBJECTID, the pseudo object is created by btrfs_lookup_dentry() in inode.c when ENOENT is encountered. As a side note: the timestamp of the snap-dir item could be prettier. No such pseudo items are created when the snapshot is placed outside of the subvolume to be snapshotted, obviously. In the above example, do ... # btrfs subvol snap /mnt/snap1 /mnt/snap2 ... and no such items will be created, which makes me quite certain the existence of above mentioned DIR_ITEMs is a bug, isn't it? -Jan -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: read error: how to fix?
Hallo, Calvin, Du meintest am 16.10.11: [...] > If you have pending sectors, causing the drive to reallocate them is > very simple. Write data (any data) over the sector in question - the > drive will then remap it onto the spare area to do the write. (The > easiest way is to do something like dd if=/dev/zero of=/dev/sdX; but > if you know the exact sector number, "hdparm --write-sector" can > remap it quickly.) I have to try in the next days ... > Keep in mind, though - if you have a single reallocated sector on a > drive, it means that the drive medium is deteriorating. It's very > likely that you will have additional failures in the future, > resulting in more IO errors and lost data. For your sanity, I > recommend replacing a drive as soon as you see any one error on it. Actually "dd if=/dev/sdg of=/dev/zero " tells (in "/var/log/warn") strange things like Oct 18 14:42:48 Arktur kernel: Buffer I/O error on device sdg, logical block 29792786 Oct 18 14:42:48 Arktur kernel: Buffer I/O error on device sdg, logical block 29792787 Oct 18 14:43:04 Arktur kernel: end_request: I/O error, dev sdg, sector 238342224 Oct 18 14:43:04 Arktur kernel: Buffer I/O error on device sdg, logical block 29792778 Oct 18 14:43:20 Arktur kernel: end_request: I/O error, dev sdg, sector 238342224 Oct 18 14:43:20 Arktur kernel: Buffer I/O error on device sdg, logical block 29792778 - >From yesterday to this morning the number of offline uncorrectable has grown from 25 to 26 - no good omen. Maybe there are some files damaged - the disk is filled with about 1.4 TByte, it's part of a btrfs cluster with more than 4 TByte data. What about "btrfsck" - can it help? Or may it lead to one more crash? When I try to copy the whole cluster to another place (I had this problem some days ago) then the system crashes when it tries to access that special file that uses such a defect sector. When I can first detect the name of this file and then exclude it from copying then "cp" works. Nice problems ... Viele Gruesse! Helmut -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Thank you all for your prompt responses. I forced a reboot, remounted the fs and now cannot recreate the behavior. Ls appears to be working as expected. If it hangs later today I will get all the info I can and repost under this subject. Thank you again for your assistance. Jim On 10/18/2011 10:09 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 10:08:30AM -0400, Jim wrote: Yes, I had thought of that as well but still no output. Ls has been hung for about 30 min now and shows as D+ in ps. Do you think it would get us any information if I forced a reboot and redid the entire sequence running sysrq-trigger right after starting ls? Maybe sysrq+t will get it? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Thank you both (Sander and Josef) I can forget anything so all reminders are appreciated. I will try again and also use the t argument. Jim On 10/18/2011 10:09 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 10:08:30AM -0400, Jim wrote: Yes, I had thought of that as well but still no output. Ls has been hung for about 30 min now and shows as D+ in ps. Do you think it would get us any information if I forced a reboot and redid the entire sequence running sysrq-trigger right after starting ls? Maybe sysrq+t will get it? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
On Tue, Oct 18, 2011 at 10:08:30AM -0400, Jim wrote: > Yes, I had thought of that as well but still no output. Ls has been > hung for about 30 min now and shows as D+ in ps. Do you think it > would get us any information if I forced a reboot and redid the > entire sequence running sysrq-trigger right after starting ls? Maybe sysrq+t will get it? Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Josef Bacik wrote (ao): > On Tue, Oct 18, 2011 at 10:03:40AM -0400, Jim wrote: > > I am getting no output from echo 'w' > /proc/sysrq-trigger. Ls is > > hung again so I can try any thing else you suggest to get data. > > Do > > echo 1> /proc/sys/kernel/sysrq > > and then try doing the echo w thing. Thanks, Jim, sorry if I'm stating the obvious here, but the output will be in dmesg (and syslog if possible). Sander -- Humilis IT Services and Solutions http://www.humilis.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Yes, I had thought of that as well but still no output. Ls has been hung for about 30 min now and shows as D+ in ps. Do you think it would get us any information if I forced a reboot and redid the entire sequence running sysrq-trigger right after starting ls? Jim On 10/18/2011 10:04 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 10:03:40AM -0400, Jim wrote: I am getting no output from echo 'w'> /proc/sysrq-trigger. Ls is hung again so I can try any thing else you suggest to get data. Do echo 1> /proc/sys/kernel/sysrq and then try doing the echo w thing. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
On Tue, Oct 18, 2011 at 10:03:40AM -0400, Jim wrote: > I am getting no output from echo 'w' > /proc/sysrq-trigger. Ls is > hung again so I can try any thing else you suggest to get data. Do echo 1> /proc/sys/kernel/sysrq and then try doing the echo w thing. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
I am getting no output from echo 'w' > /proc/sysrq-trigger. Ls is hung again so I can try any thing else you suggest to get data. Jim On 10/18/2011 09:49 AM, Sander wrote: Jim wrote (ao): I am on kernel 3.1.0-rc4. I am ssh'd into a remote machine so have no access to keyboard, any other way to get the info you need? echo 'w'> /proc/sysrq-trigger On 10/18/2011 09:31 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 09:25:46AM -0400, Jim wrote: Good morning btrfs list, I have been rsyncing files from an nfs mount to a btrfs filesystem. After an rsync run I ls random subvols or directorys to check the copy. About 60% to 70% of the time ls completely hangs. Ps aux shows it as running but even when I let it go for up to an hour it never finishes. Kill -9 will not stop the process. Dmesg shows nothing beyond a successful mount at boot. I can't umount the system because "filesystem is busy". I find that a forced reboot is the only way to recapture the system. /var/log/messages has the only indication (that I can find) that anything abnormal is happening. A tail of the file is below. Thank you for any help and advice. Hit sysrq+w when this happens and give us all the tracebacks. Also which kernel you are on would be helpful. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Jim wrote (ao): > I am on kernel 3.1.0-rc4. I am ssh'd into a remote machine so have > no access to keyboard, any other way to get the info you need? echo 'w' > /proc/sysrq-trigger > On 10/18/2011 09:31 AM, Josef Bacik wrote: > >On Tue, Oct 18, 2011 at 09:25:46AM -0400, Jim wrote: > >>Good morning btrfs list, > >>I have been rsyncing files from an nfs mount to a btrfs filesystem. > >>After an rsync run I ls random subvols or directorys to check the > >>copy. About 60% to 70% of the time ls completely hangs. Ps aux > >>shows it as running but even when I let it go for up to an hour it > >>never finishes. Kill -9 will not stop the process. Dmesg shows > >>nothing beyond a successful mount at boot. I can't umount the > >>system because "filesystem is busy". I find that a forced reboot is > >>the only way to recapture the system. /var/log/messages has the > >>only indication (that I can find) that anything abnormal is > >>happening. A tail of the file is below. Thank you for any help and > >>advice. > >Hit sysrq+w when this happens and give us all the tracebacks. Also which > >kernel > >you are on would be helpful. Thanks, > > > >Josef > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Humilis IT Services and Solutions http://www.humilis.net -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
Thank you, I should have thought of that. I will get back as soon as I have info. Jim On 10/18/2011 09:49 AM, Sander wrote: Jim wrote (ao): I am on kernel 3.1.0-rc4. I am ssh'd into a remote machine so have no access to keyboard, any other way to get the info you need? echo 'w'> /proc/sysrq-trigger On 10/18/2011 09:31 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 09:25:46AM -0400, Jim wrote: Good morning btrfs list, I have been rsyncing files from an nfs mount to a btrfs filesystem. After an rsync run I ls random subvols or directorys to check the copy. About 60% to 70% of the time ls completely hangs. Ps aux shows it as running but even when I let it go for up to an hour it never finishes. Kill -9 will not stop the process. Dmesg shows nothing beyond a successful mount at boot. I can't umount the system because "filesystem is busy". I find that a forced reboot is the only way to recapture the system. /var/log/messages has the only indication (that I can find) that anything abnormal is happening. A tail of the file is below. Thank you for any help and advice. Hit sysrq+w when this happens and give us all the tracebacks. Also which kernel you are on would be helpful. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
I am on kernel 3.1.0-rc4. I am ssh'd into a remote machine so have no access to keyboard, any other way to get the info you need? Thanks again. Jim On 10/18/2011 09:31 AM, Josef Bacik wrote: On Tue, Oct 18, 2011 at 09:25:46AM -0400, Jim wrote: Good morning btrfs list, I have been rsyncing files from an nfs mount to a btrfs filesystem. After an rsync run I ls random subvols or directorys to check the copy. About 60% to 70% of the time ls completely hangs. Ps aux shows it as running but even when I let it go for up to an hour it never finishes. Kill -9 will not stop the process. Dmesg shows nothing beyond a successful mount at boot. I can't umount the system because "filesystem is busy". I find that a forced reboot is the only way to recapture the system. /var/log/messages has the only indication (that I can find) that anything abnormal is happening. A tail of the file is below. Thank you for any help and advice. Hit sysrq+w when this happens and give us all the tracebacks. Also which kernel you are on would be helpful. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ls hangs filesystem
On Tue, Oct 18, 2011 at 09:25:46AM -0400, Jim wrote: > Good morning btrfs list, > I have been rsyncing files from an nfs mount to a btrfs filesystem. > After an rsync run I ls random subvols or directorys to check the > copy. About 60% to 70% of the time ls completely hangs. Ps aux > shows it as running but even when I let it go for up to an hour it > never finishes. Kill -9 will not stop the process. Dmesg shows > nothing beyond a successful mount at boot. I can't umount the > system because "filesystem is busy". I find that a forced reboot is > the only way to recapture the system. /var/log/messages has the > only indication (that I can find) that anything abnormal is > happening. A tail of the file is below. Thank you for any help and > advice. Hit sysrq+w when this happens and give us all the tracebacks. Also which kernel you are on would be helpful. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
ls hangs filesystem
Good morning btrfs list, I have been rsyncing files from an nfs mount to a btrfs filesystem. After an rsync run I ls random subvols or directorys to check the copy. About 60% to 70% of the time ls completely hangs. Ps aux shows it as running but even when I let it go for up to an hour it never finishes. Kill -9 will not stop the process. Dmesg shows nothing beyond a successful mount at boot. I can't umount the system because "filesystem is busy". I find that a forced reboot is the only way to recapture the system. /var/log/messages has the only indication (that I can find) that anything abnormal is happening. A tail of the file is below. Thank you for any help and advice. Jim [root@btrfs ~]# tail -20 /var/log/messages Oct 18 05:06:36 btrfs kernel: [] sys_getdents+0x89/0xf0 Oct 18 05:06:36 btrfs kernel: [] system_call_fastpath+0x16/0x1b Oct 18 05:08:36 btrfs kernel: INFO: task ls:7100 blocked for more than 120 seconds. Oct 18 05:08:36 btrfs kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Oct 18 05:08:36 btrfs kernel: ls D 88080bc843d8 0 7100 7047 0x0084 Oct 18 05:08:36 btrfs kernel: 8804f526dd68 0086 8804f526dd18 Oct 18 05:08:36 btrfs kernel: 88080bc84000 00011900 8804f526dfd8 8804f526c010 Oct 18 05:08:36 btrfs kernel: 8804f526dfd8 00011900 88080f64d7f0 88080bc84000 Oct 18 05:08:36 btrfs kernel: Call Trace: Oct 18 05:08:36 btrfs kernel: [] wait_current_trans+0xa5/0x110 [btrfs] Oct 18 05:08:36 btrfs kernel: [] ? wake_up_bit+0x40/0x40 Oct 18 05:08:36 btrfs kernel: [] start_transaction+0x1e0/0x2b0 [btrfs] Oct 18 05:08:36 btrfs kernel: [] btrfs_join_transaction+0x15/0x20 [btrfs] Oct 18 05:08:36 btrfs kernel: [] btrfs_dirty_inode+0x50/0x160 [btrfs] Oct 18 05:08:36 btrfs kernel: [] __mark_inode_dirty+0x3f/0x200 Oct 18 05:08:36 btrfs kernel: [] touch_atime+0x115/0x150 Oct 18 05:08:36 btrfs kernel: [] ? sys_ioctl+0xb0/0xb0 Oct 18 05:08:36 btrfs kernel: [] vfs_readdir+0xce/0xe0 Oct 18 05:08:36 btrfs kernel: [] sys_getdents+0x89/0xf0 Oct 18 05:08:36 btrfs kernel: [] system_call_fastpath+0x16/0x1b -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: btrfs defragment: ioctl failed, ret -1 errno 28
On Tue, Oct 18, 2011 at 5:50 AM, Tomasz Chmielewski wrote: > I'm trying to defragment a 280 GB file when running 3.1.0-rc9 kernel: > > # filefrag /mnt/btrfs/images/srv1-backup.qcow2 > /mnt/btrfs/images/srv1-backup.qcow2: 1139699 extents found > > > # btrfs filesystem defragment /mnt/btrfs/images/srv1-backup.qcow2 > ioctl failed on /mnt/btrfs/images/srv1-backup.qcow2 ret -1 errno 28 > total 1 failures > > > Some files fail to defragment like this; some are defragmented correctly. > > Why does it fail? > Error 28 is ENOSPC (error, no space). Since btrfs will be trying to COW the file, it is looking for at least 280 GB of free space. Also, I've run into situations where btrfs seemed to need way more free space to complete an operation than appeared necessary on the surface. The btrfs defragment command has some capabilities to defragment just segments of a file at a time. But you may want to test that capability on some less important files first. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs defragment: ioctl failed, ret -1 errno 28
I'm trying to defragment a 280 GB file when running 3.1.0-rc9 kernel: # filefrag /mnt/btrfs/images/srv1-backup.qcow2 /mnt/btrfs/images/srv1-backup.qcow2: 1139699 extents found # btrfs filesystem defragment /mnt/btrfs/images/srv1-backup.qcow2 ioctl failed on /mnt/btrfs/images/srv1-backup.qcow2 ret -1 errno 28 total 1 failures Some files fail to defragment like this; some are defragmented correctly. Why does it fail? -- Tomasz Chmielewski http://wpkg.org -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] Btrfs: fix defragmentation regression
16:48, Dan Merillat wrote: > On Fri, Sep 2, 2011 at 4:42 AM, Christoph Hellwig wrote: >> On Fri, Sep 02, 2011 at 03:56:25PM +0800, Li Zefan wrote: >>> There's an off-by-one bug: >>> >>> # create a file with lots of 4K file extents >>> # btrfs fi defrag /mnt/file >>> # sync >>> # filefrag -v /mnt/file >>> Filesystem type is: 9123683e >>> File size of /mnt/file is 1228800 (300 blocks, blocksize 4096) >>>ext logical physical expected length flags >>> 0 0 3372 64 >>> 1 64 3136 3435 1 >>> 2 65 3436 3136 64 >>> 3 129 3201 3499 1 >>> 4 130 3500 3201 64 >>> 5 194 3266 3563 1 >>> 6 195 3564 3266 64 >>> 7 259 3331 3627 1 >>> 8 260 3628 3331 40 eof >>> >>> After this patch: >> >> Can you please create an xfstests testcase for this? > > Did this fix get lost? I don't see it in git, and defragmenting a > file still results in 10x as many fragments as it started with. > (3.1-rc9) > No, it's queued for 3.2, but I think it's a good candidate for 3.1.x. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/4] Btrfs: fix defragmentation regression
On Fri, Sep 2, 2011 at 4:42 AM, Christoph Hellwig wrote: > On Fri, Sep 02, 2011 at 03:56:25PM +0800, Li Zefan wrote: >> There's an off-by-one bug: >> >> # create a file with lots of 4K file extents >> # btrfs fi defrag /mnt/file >> # sync >> # filefrag -v /mnt/file >> Filesystem type is: 9123683e >> File size of /mnt/file is 1228800 (300 blocks, blocksize 4096) >> ext logical physical expected length flags >> 0 0 3372 64 >> 1 64 3136 3435 1 >> 2 65 3436 3136 64 >> 3 129 3201 3499 1 >> 4 130 3500 3201 64 >> 5 194 3266 3563 1 >> 6 195 3564 3266 64 >> 7 259 3331 3627 1 >> 8 260 3628 3331 40 eof >> >> After this patch: > > Can you please create an xfstests testcase for this? Did this fix get lost? I don't see it in git, and defragmenting a file still results in 10x as many fragments as it started with. (3.1-rc9) -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
