Kernel BUG: btrfs send - Incremental backup
Hi there, I tried to perform an incremental backup as described in https://btrfs.wiki.kernel.org/index.php/Incremental_Backup between 2 external USB drives, The 1st "btrfs send foo/snap1 | btrfs receive bar" went well, although it took 5-6 times the time the same workload takes in ZFS. Then "btrfs send -v -p foo/snap1 foo/snap2 | btrfs receive bar" crashed on me. After a reboot and retry, I get a KERNEL BUG again : # uname -a Linux tethys 3.13.5-1-ARCH #1 SMP PREEMPT Sun Feb 23 00:25:24 CET 2014 x86_64 GNU/Linux btrfs send -v -p foo/snap1 foo/snap2 | btrfs receive bar mars 08 08:22:01 tethys kernel: [ cut here ] mars 08 08:22:01 tethys kernel: kernel BUG at fs/btrfs/send.c:4398! mars 08 08:22:02 tethys kernel: invalid opcode: [#1] PREEMPT SMP mars 08 08:22:02 tethys kernel: Modules linked in: usb_storage md5 fuse ecb ecryptfs encrypted_keys hmac trusted tpm ctr ccm intel_rapl x86_pkg_temp_thermal rts5139(C) mars 08 08:22:02 tethys kernel: usbhid hid sr_mod cdrom sd_mod atkbd libps2 crct10dif_pclmul crct10dif_common crc32_pclmul ahci crc32c_intel ghash_clmulni_intel libahc mars 08 08:22:02 tethys kernel: CPU: 0 PID: 2432 Comm: btrfs Tainted: P C O 3.13.5-1-ARCH #1 mars 08 08:22:02 tethys kernel: Hardware name: TOSHIBA SATELLITE L735/Base Board Product Name, BIOS 2.50 06/26/2012 mars 08 08:22:02 tethys kernel: task: 880041724800 ti: 88003b84a000 task.ti: 88003b84a000 mars 08 08:22:02 tethys kernel: RIP: 0010:[] [] changed_cb+0x9e5/0x9f0 [btrfs] mars 08 08:22:02 tethys kernel: RSP: 0018:88003b84bb48 EFLAGS: 00010283 mars 08 08:22:02 tethys kernel: RAX: 000579d5 RBX: 88013279ea00 RCX: 0060 mars 08 08:22:02 tethys kernel: RDX: 88003b84bc65 RSI: 000579cd RDI: 88013279ea00 mars 08 08:22:02 tethys kernel: RBP: 88003b84bbe0 R08: 88003b84bc65 R09: 0002 mars 08 08:22:02 tethys kernel: R10: 88005981a900 R11: R12: 0002 mars 08 08:22:02 tethys kernel: R13: 88003b84bc65 R14: 88007c628800 R15: 2ad1 mars 08 08:22:02 tethys kernel: FS: 7fceade62880() GS:88014f40() knlGS: mars 08 08:22:02 tethys kernel: CS: 0010 DS: ES: CR0: 80050033 mars 08 08:22:02 tethys kernel: CR2: 7fb4cc8c8000 CR3: 41a99000 CR4: 000407f0 mars 08 08:22:02 tethys kernel: Stack: mars 08 08:22:02 tethys kernel: 88003b84bb88 a03206ec 08ca 88005981a900 mars 08 08:22:02 tethys kernel: 88005981a900 88005981a870 mars 08 08:22:02 tethys kernel: 88003b84bbe0 a03161e9 88003b84bc76 88003b84bbe0 mars 08 08:22:02 tethys kernel: Call Trace: mars 08 08:22:02 tethys kernel: [] ? read_extent_buffer+0xbc/0x110 [btrfs] mars 08 08:22:02 tethys kernel: [] ? btrfs_get_token_32+0x59/0xe0 [btrfs] mars 08 08:22:02 tethys kernel: [] ? memcmp_extent_buffer+0xc1/0x120 [btrfs] mars 08 08:22:02 tethys kernel: [] btrfs_compare_trees+0x790/0x970 [btrfs] mars 08 08:22:02 tethys kernel: [] ? finish_inode_if_needed+0x4e0/0x4e0 [btrfs] mars 08 08:22:02 tethys kernel: [] btrfs_ioctl_send+0x8a8/0xc80 [btrfs] mars 08 08:22:02 tethys kernel: [] btrfs_ioctl+0xfee/0x27a0 [btrfs] mars 08 08:22:02 tethys kernel: [] ? __enqueue_entity+0x78/0x80 mars 08 08:22:02 tethys kernel: [] ? enqueue_entity+0x29b/0xa10 mars 08 08:22:02 tethys kernel: [] ? enqueue_task_fair+0x107/0x540 mars 08 08:22:02 tethys kernel: [] ? enqueue_task+0x3a/0x60 mars 08 08:22:02 tethys kernel: [] ? check_preempt_curr+0x75/0xa0 mars 08 08:22:02 tethys kernel: [] do_vfs_ioctl+0x2e0/0x4c0 mars 08 08:22:02 tethys kernel: [] ? do_fork+0x12f/0x320 mars 08 08:22:02 tethys kernel: [] SyS_ioctl+0x81/0xa0 mars 08 08:22:02 tethys kernel: [] system_call_fastpath+0x1a/0x1f mars 08 08:22:02 tethys kernel: Code: 48 8d 75 b6 45 31 c0 b9 01 00 00 00 4c 89 e2 4c 89 f7 e8 2f eb f7 ff 85 c0 0f 89 77 ff ff ff e9 a0 fd ff ff 31 c0 e9 99 fd ff ff < mars 08 08:22:02 tethys kernel: RIP [] changed_cb+0x9e5/0x9f0 [btrfs] mars 08 08:22:02 tethys kernel: RSP mars 08 08:22:02 tethys kernel: ---[ end trace 718b3fc11b4d9241 ]--- HTH, TIA. -- Swâmi Petaramesh http://petaramesh.org PGP 9076E32E -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: ENOSPC errors during raid1 rebalance
Alright! After doing: cd /mymedia; find . -type f | while read file; do mv -v "$file" /dev/shm; f2=`basename "$file"`; mv -v "/dev/shm/$f2" "$file"; done I finally moved whatever files out of the "single" allocation and back onto the new RAID1 profile: oot@ossy:~# /usr/src/btrfs-progs/btrfs fi df /mymedia Data, RAID1: total=1.23TiB, used=1000.43GiB Data, single: total=10.00GiB, used=0.00 System, RAID1: total=32.00MiB, used=184.00KiB Metadata, RAID1: total=3.00GiB, used=1.34GiB And then the rebalance finally was able to move those two block groups and I got no errors: root@ossy:~# btrfs balance start -dconvert=raid1,soft /mymedia Done, had to relocate 2 out of 1264 chunks And now I'm totally RAID1: root@ossy:~# /usr/src/btrfs-progs/btrfs fi df /mymedia Data, RAID1: total=1.23TiB, used=1000.43GiB System, RAID1: total=32.00MiB, used=184.00KiB Metadata, RAID1: total=3.00GiB, used=1.34GiB Yay!! If I want to get back the space I can do another defragment but I don't really care right now. Thanks everyone for all your help on this! Hopefully this thread will assist anyone else in the future if this occurs again. Sincerely, Michael Russo, Systems Engineer PaperSolve, Inc. 268 Watchogue Road Staten Island, NY 10314 Randomly generated quote of the last 5 minutes: A good plan today is better than a perfect plan tomorrow. -- Patton -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 4/9] Btrfs: use bitfield instead of integer data type for the some variants in btrfs_root
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2014 05:08 AM, Miao Xie wrote: > Signed-off-by: Miao Xie --- fs/btrfs/ctree.c > | 25 ++--- fs/btrfs/ctree.h | 39 > +-- fs/btrfs/disk-io.c | 33 > +++-- fs/btrfs/extent-tree.c | 6 > +++--- fs/btrfs/file.c| 4 +++- fs/btrfs/inode.c | 29 > ++--- fs/btrfs/ioctl.c | 4 ++-- > fs/btrfs/relocation.c | 17 + fs/btrfs/root-tree.c > | 2 +- fs/btrfs/transaction.c | 33 > + fs/btrfs/tree-defrag.c | 2 +- > fs/btrfs/tree-log.c| 9 + 12 files changed, 109 > insertions(+), 94 deletions(-) > > diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c index > cbd3a7d..9067d79 100644 --- a/fs/btrfs/ctree.c +++ > b/fs/btrfs/ctree.c @@ -224,7 +224,8 @@ static struct extent_buffer > *btrfs_read_lock_root_node(struct btrfs_root *root) static void > add_root_to_dirty_list(struct btrfs_root *root) { > spin_lock(&root->fs_info->trans_lock); - if (root->track_dirty && > list_empty(&root->dirty_list)) { +if > (test_bit(BTRFS_ROOT_TRACK_DIRTY, &root->state) && + > list_empty(&root->dirty_list)) { list_add(&root->dirty_list, > &root->fs_info->dirty_cowonly_roots); } @@ -246,9 +247,10 @@ int > btrfs_copy_root(struct btrfs_trans_handle *trans, int level; struct > btrfs_disk_key disk_key; > > - WARN_ON(root->ref_cows && trans->transid != - > root->fs_info->running_transaction->transid); - > WARN_ON(root->ref_cows && trans->transid != root->last_trans); + > WARN_ON(test_bit(BTRFS_ROOT_REF_COWS, &root->state) && + > trans->transid != root->fs_info->running_transaction->transid); + > WARN_ON(test_bit(BTRFS_ROOT_REF_COWS, &root->state) && + > trans->transid != root->last_trans); > > level = btrfs_header_level(buf); if (level == 0) @@ -997,14 +999,14 > @@ int btrfs_block_can_be_shared(struct btrfs_root *root, * > snapshot and the block was not allocated by tree relocation, * we > know the block is not shared. */ -if (root->ref_cows && + if > (test_bit(BTRFS_ROOT_REF_COWS, &root->state) && buf != root->node > && buf != root->commit_root && (btrfs_header_generation(buf) <= > btrfs_root_last_snapshot(&root->root_item) || > btrfs_header_flag(buf, BTRFS_HEADER_FLAG_RELOC))) return 1; #ifdef > BTRFS_COMPAT_EXTENT_TREE_V0 - if (root->ref_cows && + if > (test_bit(BTRFS_ROOT_REF_COWS, &root->state) && > btrfs_header_backref_rev(buf) < BTRFS_MIXED_BACKREF_REV) return 1; > #endif @@ -1146,9 +1148,10 @@ static noinline int > __btrfs_cow_block(struct btrfs_trans_handle *trans, > > btrfs_assert_tree_locked(buf); > > - WARN_ON(root->ref_cows && trans->transid != - > root->fs_info->running_transaction->transid); - > WARN_ON(root->ref_cows && trans->transid != root->last_trans); + > WARN_ON(test_bit(BTRFS_ROOT_REF_COWS, &root->state) && + > trans->transid != root->fs_info->running_transaction->transid); + > WARN_ON(test_bit(BTRFS_ROOT_REF_COWS, &root->state) && + > trans->transid != root->last_trans); > > level = btrfs_header_level(buf); > > @@ -1193,7 +1196,7 @@ static noinline int __btrfs_cow_block(struct > btrfs_trans_handle *trans, return ret; } > > - if (root->ref_cows) { + if (test_bit(BTRFS_ROOT_REF_COWS, > &root->state)) { ret = btrfs_reloc_cow_block(trans, root, buf, > cow); if (ret) return ret; @@ -1556,7 +1559,7 @@ static inline int > should_cow_block(struct btrfs_trans_handle *trans, > !btrfs_header_flag(buf, BTRFS_HEADER_FLAG_WRITTEN) && > !(root->root_key.objectid != BTRFS_TREE_RELOC_OBJECTID && > btrfs_header_flag(buf, BTRFS_HEADER_FLAG_RELOC)) && - > !root->force_cow) + !test_bit(BTRFS_ROOT_REF_COWS, > &root->state)) This may or may not be the problem. Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGl2IAAoJEANb+wAKly3Br0EQAKWfEahMQ1ZV9jkgSYF/9c6r ytoH2/Y3u7HYeeqf+Z/j2lkMyvAaWQ9EQCkWSUyCqoU7RPXHEROSCQ11UCk8qYXh fhV2r3plOWIZ/KDHFvqPTN5FTg97OLvDalyvOR6UP/Ws6Z4Ycm0ePm7kb+25iK9u N+PkiqHoQqUVw8Z2EEJ2SN/82SyNnuPDG2RkDD9xW8el5fBplAPgUox8W8Z+ubIo FPiSNv4euly2Zco+Vs3NDFb2tqQBPyAVzE2IC4Nyq1Hci/vyC9k8YfIcCJnsP0Dk 10mVDhSSStWLuqt2L7fUV8nOTyjKT1gBNgoz/eMBeOzWLDbRKD2hpS6wpkw/6/iQ /ff9Spikw7a87epYo4dxft32aQsDIu6JfgFPggL+VkXyn114MK4U5z7KNOPQUSBq neFOVELgN1L75TI9v9/p1qKGeZ47vV1lvd6GP717SDF0yv9wgvHR0Ma47KSaLq79 WlwzmqXDYJYdOedKGQky6GZ7EFji5DDlazx7h1pQTN5rEdiQJTEFxrxtMOKzAttF 3EL9wxVAi2ggD2EYWWsk+SJNJxgU59bxTR9ZiOH+tj29+gFcGUpRsXMsj4KampK1 j0a0BSLj+1yUlLcT8qem05aJk720zxy4UBiZGcRP7qySlrOTwU5Xtk0BXXyfUXLg guOgjR0pj494irbM53RT =H5nB -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 4/9] Btrfs: use bitfield instead of integer data type for the some variants in btrfs_root
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/20/2014 05:08 AM, Miao Xie wrote: > Signed-off-by: Miao Xie --- fs/btrfs/ctree.c > | 25 ++--- fs/btrfs/ctree.h | 39 > +-- fs/btrfs/disk-io.c | 33 > +++-- fs/btrfs/extent-tree.c | 6 > +++--- fs/btrfs/file.c| 4 +++- fs/btrfs/inode.c | 29 > ++--- fs/btrfs/ioctl.c | 4 ++-- > fs/btrfs/relocation.c | 17 + fs/btrfs/root-tree.c > | 2 +- fs/btrfs/transaction.c | 33 > + fs/btrfs/tree-defrag.c | 2 +- > fs/btrfs/tree-log.c| 9 + 12 files changed, 109 > insertions(+), 94 deletions(-) > This patch causes xfstests to lock up the box. I could reproduce by running ./check generic/070 generic/074 generic/075 in a loop. It blows up at 075 and usually happens right away, but sometimes doesn't blow up until the second go around. I'm dropping it for now. Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGlwuAAoJEANb+wAKly3BrE4P+wY0BhYvT1PjMBu6nFSZFxSd 5an4GGW5aoRrQ3G0wZkEpWPJK34exLvuzzaNxHH9FSesz27l1bnibNTUy7BzI7Mv tj92Dxmckpcvl+z3QI+CcPkI6NpVzBNETRxBpRVtkqxWKioEhk4oOwvctBqbqaI1 lEOxylu6/09k2BjjeehJFLjzhVzq9taTUt7ETAaBaJLsPFdD9/E8S1EO/uJyabSo yGSWSHEnXUMjRCaXZ1bZJKDowTZ10wviU4rDPyFi99h4h1m6FZe5jNStHojNcNkc Qt1xv3s2bkuz7yAC9/yJuz4HFiWtiyRHD609vBiIBO93hLCFi5Xc4jXtSRVVC+Xx 891gfD0c8/SVJpviTOFnxewozmIgudcC4y3i2AglRI+IXEkNRr66G6v/H0uZEN+n lhiNTLQ+/Kd+mxUA7Y3To86Hnb55TqX7G8AL2zdFKBb1LJLBCazcEQZkDDraG2MW x77H38VvKyQ+pBgv/lZFnKHadPkveFrGJUYCx49SOHAvWDoDVcIx2liayR8WYBMA reRT6Vd+sH4xkgVXXNhrFcDFsnbJp1R9LkHuk4HW56QASaMcLqwZ/RmwL/3ldX0c 17IyUZkXZx79rbzBwL8EmU9Gb+oDkNeu8nXIzVnBERDBQbDQqSThaLeiPfuJNPjU mGmvLKc/fXNN3RofWrSx =N9yr -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: return EPERM when deleting a default subvolume
The error message is confusing: # btrfs sub delete /mnt/mysub/ Delete subvolume '/mnt/mysub' ERROR: cannot delete '/mnt/mysub' - Directory not empty The error message does not make sense to me: It's not about deleting a directory but it's a subvolume, and it doesn't matter if the subvolume is empty or not. Maybe EPERM or is more appropriate in this case, combined with an explanatory kernel log message. (e.g. "subvolume with ID 123 cannot be deleted because it is configured as default subvolume.") Reported-by: Koen De Wit Signed-off-by: Guangyu Sun --- fs/btrfs/ioctl.c |5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index a6d8efa..c380854 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1797,7 +1797,10 @@ static noinline int may_destroy_subvol(struct btrfs_root *root) if (di && !IS_ERR(di)) { btrfs_dir_item_key_to_cpu(path->nodes[0], di, &key); if (key.objectid == root->root_key.objectid) { - ret = -ENOTEMPTY; + ret = -EPERM; + printk(KERN_ERR "subvolume %llu cannot be deleted " + "because it is configured as default " + "subvolume.\n", key.objectid); goto out; } btrfs_release_path(path); -- 1.7.9.5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] xfstests: add test for btrfs send issuing premature rmdir operations
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/24/2014 06:54 AM, Filipe David Borba Manana wrote: > Regression test for btrfs incremental send issue where a rmdir > instruction is sent against an orphan directory inode which is not > empty yet, causing btrfs receive to fail when it attempts to remove > the directory. > > This issue is fixed by the following linux kernel btrfs patch: > > Btrfs: fix send attempting to rmdir non-empty directories > > Signed-off-by: Filipe David Borba Manana --- Looks good to me, Reviewed-by: Josef Bacik Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGg5bAAoJEANb+wAKly3BFtMQAMDITxFeFKRw5o5g88NUJT+n /AQp584QUb+sBhJnyYVET98S47R4C9d7jAu8fUxUJd2FaZ0UPR+rl/LnpdYXy8oJ 29A2tvbSiOZGQp+kWlGpLqKmmEj/3R/0QZwTrB1jsxKwVsH+OGK1fazOwhkahloL 8R6xBtcvMzkBFEQ08T0dmodDaWKPp8dgztcScrVt+RGkePPVoKE9X4hwTwAl8muk nPv9TvRULOs2emvxuTnEVqZfou0HcOTuKWXAOXN2peXBe324SjfQ0JmAMjtG1Ukw nAs5wu8eII1k9/rjGLarNhpuSuMaM5tth0tIb2ebYwxwpYpY6o4eNN2RR6vWlCEz vULou68VKdOG6/c6uO6cXuI8zOwScWuYyYFXFLTFpAhjdKwY1HaCwSanvnamdYy8 kNITDGkYSqs0Abj9wzQDcJcTrjpHDDe/3olb5LMJjrdTneMyDVXc+Ym+FbBhmQUX d7vC+a7sRVWwN4xGoqxDgZZQLpGBEO60FhA5kIO9lgis2LdG8SXE0ch4sQuWZ8iC S9Aw46dssmiY+d3BkugCOPW/7IlID/obLGUF+CWcWMkxSMbwaPn6RfNJmW+9k+rv fkUVj+U75pBtnuwW3LxV7gUq8HwTz8G1fh+96o41yKXpayBQUoIA47AHCUZtP9Fi RemqwFmZxhi7ztZYfni2 =8Kl8 -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Allow forced conversion of metadata to dup profile on, multiple devices
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/26/2014 09:23 AM, Austin S Hemmelgarn wrote: > Currently, btrfs balance start fails when trying to convert > metadata or system chunks to dup profile on filesystems with > multiple devices. This requires that a conversion from a > multi-device filesystem to a single device filesystem use the > following methodology: 1. btrfs balance start -dconvert=single > -mconvert=single \ -sconvert=single -f / 2. btrfs device delete / > /dev/sdx 3. btrfs balance start -mconvert=dup -sconvert=dup / This > results in a period of time (possibly very long if the devices are > big) where you don't have the protection guarantees of multiple > copies of metadata chunks. > > After applying this patch, one can instead use the following > methodology for conversion from a multi-device filesystem to a > single device filesystem: 1. btrfs balance start -dconvert=single > -mconvert=dup \ -sconvert=dup -f / 2. btrfs device delete / > /dev/sdx This greatly reduces the chances of the operation causing > data loss due to a read error during the device delete. > > Signed-off-by: Austin S. Hemmelgarn --- Fails to build, wait for me to push an updated btrfs-next and rebase and resubmit fs/btrfs/volumes.c: In function ‘btrfs_balance’: fs/btrfs/volumes.c:3223:55: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses] if (((bctl->sys.flags & BTRFS_BALANCE_ARGS_CONVERT) && ^ fs/btrfs/volumes.c:3227:8: error: ‘num_devs’ undeclared (first use in this function) (num_devs > 1)) { ^ fs/btrfs/volumes.c:3227:8: note: each undeclared identifier is reported only once for each function it appears in make[1]: *** [fs/btrfs/volumes.o] Error 1 make: *** [_module_fs/btrfs] Error 2 Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGgp0AAoJEANb+wAKly3BrkkQANOaAAk6LJ4VO86f9/N7Hkom Hf4FK5lQiMkzlLc8pz+RxHeIHSXScXdrVKPv8bIPoGwd171OIrMA1HZa/9iuez8z OInjO6Zwj6l/+N3/QckAJKkNmbInNH2wgdhUrFyUw+gDFvVnl1YGUcTwx6/udqTB kfsls2ivD7S9kfSXyaM4oxVMN+tZWpZtOs1TMpf7BDMIz92gr87VADKIocrlu5qh EZ2YyWGyIP87jz+7zzNPUQ00/BAgC1lPnbZf+ei0L1KQbDtjII3Rl1/rlg78Zdtb VQfYMBz0hpaOE+UNfVsClgDgMjAkphob7BXTrkzJaChOagJ5tIlSGZRUaHPt3qaz so1R/3BaPzlTdv7gPCpMg+nSYdNl0x3w5CauCB4EX3L8PtbxK9tQdFtKmpl5GNrZ gBADD+AAbyPoz4GO2lfQyIN6Hd5FA+drK6cErF/rY+dPlieyihdYfBWRU+nao9RY ju07FN2UHJaAlSh/K8ZLv4lZUULDObr/tOdg5M1h5piSL9H9jOHUdaLmRzSx+7hZ j96qRY9/W4L6avEQQLphGbgn3v+aOHcgxLS0MyO/fhXANah2z69PjpS5J7GOXGZ5 UDGTneQIeRqueYms1yqpmZA0Ctwavq0TGoDvYsOlf/x7vucJE6a02qMRitUJbuvV 6m+8a6PFRudGng7zQYef =KCPn -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2 v2] Btrfs: rename waiting_dir_move
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/27/2014 02:47 AM, Liu Bo wrote: > This is a preparation work, rename waiting_dir_move to > send_dir_node. We'd like to share waiting_dir_move structure in new > did_create_dir() code. > > Signed-off-by: Liu Bo --- v2: fix wrong > patch name. > This isn't applying cleanly, rebase once I push out btrfs-next today and resend. Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGglEAAoJEANb+wAKly3Bm9YP+wSnaq7jN0pQT8UVORFE2BVI rTKGWfSnJQtEAUx3sF0xCp6dCQlkG+BBPHmK3W1yGovtHQrkR0rkGufx0aLzqD2m xpV7qQKa9rAQjK5Xsx0K0vDETqYFzEIGv+bv1Q6jFVKoEBVSSdWDPZoSLkVZ9T0Q aoYUvC/xd4fkp6ikb11vL+3c7WysWaDpSTflZdEFgrd5PgE+r1CoxmTs5NMCxhIK V6GEu/l3yKK2/95CNAfhkkeRXyV2NVxf/AzM0Jay7+zyCUBM4u2B7BAuC7ZlCxG2 kUhqTw4TkFUIW6mW/sszKWJM9bjDLyq7bYzmJ08yFeiiS0Pvr9NaYxkNc55HUve0 ZLnrICzAJbjQV0f01+Tn5G2/jKX6PEeGRcyWJ1RgEFBRDPh5AMsbHeme3JKWkikZ xxtT3gvZMlASm1CFtepoFQiBwtd8KR7VJIS9VAhO7VGyvWu3+DSvGe8di+OKvjIh kBnx7mgfmQChIe4Ccem3iJsNPkR/e9vpoZalBSGTnBGNiztDc64Ht3Xh+YN034n1 4zw6XjYoKo+NJ/8pz2auKjWtZdC3AUSEkaU3koB4U1+uyQJAqDfki8j0Trye5ehn TemIj6kSfOvuuiCaGCqYwJJYG5r0UXALRwpGtiMk+SGzUklbjmg8q1LKKJiagV8K yVTNyMKP8Qldlsw1bAPm =Nkos -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Problems on previously perfect setup
Recently (a week or so back) I started having those problems on my BTRFS system - I. My setup: - SSD /boot ext2defaults0 2 / btrfs rw,noatime,compress=lzo,ssd,space_cache,autodefrag 0 0 (there is a /home subvolume mounted automatically) HDD /mnt/Disk_D btrfs rw,noatime,compress=zlib,autodefrag,nofail 0 0 (I disconnected it, but it changed nothing) -- II. My problems: -- - 1 - . During random boots (seems to be a 50/50 chance) the system won't boot, because it get's stuck on [ OK ] Mounted Configuration File system [ *** ] (1 of 13) A start job is running for dev-disk-by\. It will slowly cycle from 1 to 13, then come back to 1 to cycle again, and again, and after three times, I get this: Timed out waiting for device dev-disk-by\... Dependency failed for /boot Dependency failed for Local File System The computer freezes and I have to shut it down by holding the power button for several seconds. - 2 - . My successful boot times extended considerably. I am using an automatic login and X start without any DM. Before, I could hardly see the login command line because it was so quickly covered by started OpenBox session. Now it lingers several seconds while the disk LED is working hard. - 3 - . For the first 10-15 seeocnds after getting into OpenBox the heaviest processes displayed by Conky are several instances of kworker. Also, it might be an impression, but I feel like the starting almost every program became sluggish. -- III. Recent changes to the system: -- Perhaps they could indicate potential suspects: - upgraded kernel (now 3.13.5-2) - upgraded btrfs tools (always using latest David Sterba's unstable 'integration' branch) - did "btrfs balance /" because it had write erros due to no space even though there was like 30GB free. After doing it, no space problem went away Does anyone have any idea what could be happening? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/3] Btrfs: don't skip the page flush since the enospc is not brought by it
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/27/2014 12:58 AM, Miao Xie wrote: > As we know, btrfs flushes the continuous pages as many as > possible, but if all the free spaces are small, we will allocate > the spaces by several times, and if there is something wrong with > the space reservation, it is very likely that some allocations > succeed and the others fail. But the current code doesn't take this > case into account, and set the error flag for the pages though > their spaces are allocated successfully. It introduces a problem > that we can not umount the fs after the above problem happens > because we need wait for the flush of those pages whose spaces are > allocated. This patch fixes the above problem, and makes the btrfs > developers happy when they investigate the problem of the space > reservation. > > Signed-off-by: Miao Xie --- > fs/btrfs/extent_io.c | 15 +-- 1 file changed, 13 > insertions(+), 2 deletions(-) > > diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index > fbe501d..97595ea 100644 --- a/fs/btrfs/extent_io.c +++ > b/fs/btrfs/extent_io.c @@ -3198,8 +3198,19 @@ static int > __extent_writepage(struct page *page, struct writeback_control > *wbc, &nr_written); /* File system has been set read-only */ if > (ret) { - SetPageError(page); - > goto done; +/* + >* > Private2 means we allocate the space + * for > this page > successfully, and enospc + * error doesn't set > the fs to be > R/O, so + * we can write out the page and > needn't + * set > error flag for this page. If so, we + * can prevent > the umount > task from being + * blocked. + > */ + if (!(ret == -ENOSPC > && PagePrivate2(page))) { + > SetPageError(page); + goto > done; + } So now we just ignore ret == -ENOSPC, we still need to return this back to the caller. Thanks, Josef -BEGIN PGP SIGNATURE- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJTGez/AAoJEANb+wAKly3B9z4QAJ7VWrt7yfW+IpjzVGViPrmb S/Q+nuUEZjFk9huKePz33MaT76gWZlrFM6ZgfSLIEP3gPDaQIz+mrCmDBDSnD0uT ZJypte70l74Rfmp0gBHs+kv9VQPQkeDE17PMGCJBHnqvm/P45PgYU81nWAMhh3uQ Khsu5TDW7Tddi8FHyuD9a8TBdMlZfRLDOvWCTY/9OUPpRhaOhL8RBffw6iKfnq/t M32nGEbZGH1y/kWWc39JNKlUwzW0m0oclznPKp0wXo+lMFmHwl76TGgBl0tvLyAs VGhv101dH0b7vuQOerXSajVjBGFUqovIW3lDPwf4aU4XWSSebbXxeaYiEs5YMFU4 ivzXEMpYBHEteSW9TzJ0Th+D60LhG2vApsG10Ewj7xEblCjcaO8mej6L0j7citw0 WtNiSACyvIlw4zCWT3699UJlmIJ2fsrxEb5SKfE5QFpOXshhSciA8YZJyuX2EBVE je/rUMhTLCqkuY4zKZLzJiqRhNGQQbWZn4+SmRxpg5RsuxzIZzZcWiEEfucdhB6w 7fKhtMBcY4rUzszG2otEbQaPLNvUa/u04448/gsYhvwy9II0WSkLbe3LF0IS1Z6o DM0vUE3FAI80EL/fwyg2VvyxOOSYSE9feoX5ClJOZ5qJamdSqLI9PND04lMz6Dp0 +UUKtPx2wn6CnqO3vBeH =94MT -END PGP SIGNATURE- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 1/3 v2 RFC] btrfs: total_devices vs num_devices
The intended usage of total_devices and num_devices should be recorded in the comments so that these two counters can be used correctly as originally intended. As of now there appears to be slight deviations/bugs from the original intention, the bugs are apparent that num_devices does not count seed devices where as total_devices does, but total_devices does not count the replacing devices where as num_devices does. So in this situation the ioctl(BTRFS_IOC_DEVICES_READY) will fails when there is seed FS. Next, applications using ioctl(BTRFS_IOC_FS_INFO) and ioctl(BTRFS_IOC_DEV_INFO) in conjunction to allocate the slots will also fail as former depend on the num_devices where as latter would match total_devices (except when replace is running). This patch will help have the clarity on usage of these two counter so that bugs related to this can be fixed. Signed-off-by: Anand Jain --- v2: edit commit fs/btrfs/volumes.h | 6 ++ 1 file changed, 6 insertions(+) diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h index 91b7596..99c71aa 100644 --- a/fs/btrfs/volumes.h +++ b/fs/btrfs/volumes.h @@ -121,12 +121,18 @@ struct btrfs_fs_devices { /* the device with this id has the most recent copy of the super */ u64 latest_devid; u64 latest_trans; + /* num_devices contains run time count of devices which are part +* of this FSID. (the FS devices + replacing devices + ? +*/ u64 num_devices; u64 open_devices; u64 rw_devices; u64 missing_devices; u64 total_rw_bytes; u64 num_can_discard; + /* total_devices contains static recorded count of device which +* are part of this FSID. (the FS devices + seed devices + ? +*/ u64 total_devices; struct block_device *latest_bdev; -- 1.8.5.3 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 2/3 RFC] btrfs: total_devices should count replacing devices
ioctl(BTRFS_IOC_FS_INFO) returns num_devices which does not count seed device. num_devices is used to calculate the number of slots during the ioctl(BTRFS_IOC_DEV_INFO) but ioctl(BTRFS_IOC_DEV_INFO) would count seed devices as well. Due to this miss match btrfs_progs get_fs_info() hits the bug.. get_fs_info() :: BUG_ON(ndevs >= fi_args->num_devices); what I notice is total_devices should be returned by the ioctl(BTRFS_IOC_FS_INFO)), however total_devices does not count (transient) replacing device but num_devices does. first, num_devices which appears to be a subset of total_devices helps to count the number of devices under a FSID which excludes the seed devices but includes the transient replacing device. total_devices which includes seed devices is as of now does not count the tansient replacing device, which IMO is a bug or the implementation details are not clear enough to state the role of num_devices and total_devices. The user land on the otherhand would want to know all the devices that would come out of probe against a FSID As of now in this patch I am making the ioctl.c local changes (instead of asking replace thread to update total_devices) to include replacing device. Signed-off-by: Anand Jain --- fs/btrfs/ioctl.c | 13 - 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 5036f9d..ec7d5c6 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -2510,6 +2510,7 @@ static long btrfs_ioctl_fs_info(struct btrfs_root *root, void __user *arg) struct btrfs_device *next; struct btrfs_fs_devices *fs_devices = root->fs_info->fs_devices; int ret = 0; + int dev_replace_running = 0; if (!capable(CAP_SYS_ADMIN)) return -EPERM; @@ -2518,8 +2519,18 @@ static long btrfs_ioctl_fs_info(struct btrfs_root *root, void __user *arg) if (!fi_args) return -ENOMEM; + btrfs_dev_replace_lock(&root->fs_info->dev_replace); + if (btrfs_dev_replace_is_ongoing(&root->fs_info->dev_replace)) + dev_replace_running = 1; + + btrfs_dev_replace_unlock(&root->fs_info->dev_replace); + mutex_lock(&fs_devices->device_list_mutex); - fi_args->num_devices = fs_devices->num_devices; + if (dev_replace_running) + fi_args->num_devices = fs_devices->total_devices + 1; + else + fi_args->num_devices = fs_devices->total_devices; + memcpy(&fi_args->fsid, root->fs_info->fsid, sizeof(fi_args->fsid)); list_for_each_entry_safe(device, next, &fs_devices->devices, dev_list) { -- 1.8.5.3 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3/3 RFC] btrfs: show_devname should not consider seed disk
most of the user level scripts uses /proc/self/mounts for the disk-path to mount-point to fsid mapping. But when seed disk is present which generally has lowest devid, the /proc/self/mounts would show the seed disk, but seed disk has different fsid from the actual fsid that's mounted. Due to this miss match these scripts fails to work. One such example is btrfs-porgs check_mounted_where(). The solution here is not to loop into the seed disks, but still return the lowest devid under the given mount point. Signed-off-by: Anand Jain --- fs/btrfs/super.c | 15 ++- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c index f3c0247..c023ffa 100644 --- a/fs/btrfs/super.c +++ b/fs/btrfs/super.c @@ -1892,15 +1892,12 @@ static int btrfs_show_devname(struct seq_file *m, struct dentry *root) mutex_lock(&fs_info->fs_devices->device_list_mutex); cur_devices = fs_info->fs_devices; - while (cur_devices) { - head = &cur_devices->devices; - list_for_each_entry(dev, head, dev_list) { - if (dev->missing) - continue; - if (!first_dev || dev->devid < first_dev->devid) - first_dev = dev; - } - cur_devices = cur_devices->seed; + head = &cur_devices->devices; + list_for_each_entry(dev, head, dev_list) { + if (dev->missing) + continue; + if (!first_dev || dev->devid < first_dev->devid) + first_dev = dev; } if (first_dev) { -- 1.8.5.3 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Eric Mesa wrote (ao): > Duncan - thanks for this comprehensive explanation. For a huge portion of > your reply...I was all wondering why you and others were saying snapshots > aren't backups. They certainly SEEMED like backups. But now I see that the > problem is one of precise terminology vs colloquialisms. In other words, > snapsshots are not backups in and of themselves. They are like Mac's Time > Machine. BUT if you take these snapshots and then put them on another media > - whether that's local or not - THEN you have backups. Am I right, or am I > still missing something subtle? Snapshots are backups, but only protect you against a limited amount of disasters. Snapshots are very convenient to quickly go back in time for some or all files and directories. But if the filesystem or underlaying disk goes up in flames, the snapshots are toast as well. So you need additional backups, preferably not on the same hardware, for real protection against data loss. The convenience of snapshots is that you can (almost) make them as often as you want, fully automated, with (almost) no impact on performance, without the need for extra hardware, and a restore is no more than a simple copy. Sander -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: ENOSPC errors during raid1 rebalance
Thanks Hugo, that makes sense, and maybe leads to a possible way to fix the issue in future versions of btrfs-convert or a way to handle it in the balance code. What I did to find files with extents: cd /mymedia find . -type f -print0 | xargs -0 filefrag | grep -v 1\ extent | grep -v 0\ extent | awk -F: '{print $1}' > /tmp/extent cat /tmp/extent | while read file; do mv -v "$file" /dev/shm; f2=`basename "$file"`; mv -v "/dev/shm/$f2" "$file"; done I no longer care that even after doing this I still have a file with multiple extents, I just don't want them inside those block groups with >1GB extents. (BTW my terminology may be all off here, I just know that in my syslog btrfs tries to relocate two particular "block groups" and gets 2 enospc errors for them.) But since I'm still getting the errors the files I care about probably aren't in this list so I'll do it for all the files on the system (since I can't find out what files are in those block groups). -Original Message- From: Hugo Mills [mailto:h...@carfax.org.uk] Sent: Friday, March 07, 2014 3:02 AM To: Mike Russo Cc: linux-btrfs@vger.kernel.org Subject: Re: ENOSPC errors during raid1 rebalance The defrag operation, by its nature, _doesn't_ preserve extents, and thus can act to break up the large extents, making it possible to balance the chunks that the offending extents live on. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- If the first-ever performance is the première, is the --- last-ever performance the derrière? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Duncan <1i5t5.duncan cox.net> writes: > *But*, btrfs snapshots by themselves remain on the existing btrfs > filesystem, and thus are subject to many of the same risks as the > filesystem itself. As you mentioned raid is redundancy not backup, > snapshots aren't backup either; snapshots are multiple logical copies > thus protecting you from accidental deletion or bad editing, but pointed > at the same data blocks without redundancy, and if those data blocks or > the entire physical media go bad... > > Which is where real backups, separate copies on separate physical media, > come in, which is where btrfs send/receive, as the ars-technica article > was describing, comes in. > > The idea is to make a read-only snapshot on the local filesystem, read- > only so it can't change while it's being sent, and then use btrfs send to > send that snapshot to be stored on some other media, which can optionally > be over the network to a machine and media at a different site, altho it > can be to a different device on the same machine, as well. > > The first time you do this, there's no existing copy at the other end, so > btrfs send sends a full copy and btrfs receive writes it out. After > that, the receive side has a snapshot identical to the one created on the > send side and further btrfs send/receives to the same set simply > duplicate the differences between the reference and the new snapshot from > the send end to the receive end. As with local snapshots, old ones can > be deleted on both the send and receive ends, as long as at least one > common reference snapshot is maintained on both ends, so diffs taken > against the send side reference can be applied to an appropriately > identical receive side reference, thereby updating the receive side to > match the new read-only snapshot on the send side. > > Hopefully that's clearer now. =:^) > Duncan - thanks for this comprehensive explanation. For a huge portion of your reply...I was all wondering why you and others were saying snapshots aren't backups. They certainly SEEMED like backups. But now I see that the problem is one of precise terminology vs colloquialisms. In other words, snapsshots are not backups in and of themselves. They are like Mac's Time Machine. BUT if you take these snapshots and then put them on another media - whether that's local or not - THEN you have backups. Am I right, or am I still missing something subtle? I think the most important thing you said was at the end and I'd like a little clarification on that if it's OK with you. "As with local snapshots, old ones can > be deleted on both the send and receive ends, as long as at least one > common reference snapshot is maintained on both ends, so diffs taken > against the send side reference can be applied to an appropriately > identical receive side reference, thereby updating the receive side to > match the new read-only snapshot on the send side." So, let's say I have everything set up. This means I created the read-only shot on my home btrfs volume and sent it to the backup drive. I'm making hourly snapshots and after each snapshot is made, it's sent to the backup drive. So, obviously the backup drive needs to be at least as big as the home drive so it can store what's on home plus the snapshot-diffs. Now let's be extreme and say that in the course of a year I touch and somehow change every single file on the home drive. That means if I only had one snapshot I'd need home drive x 2 space. (for used space, not unused space, naturally) So I might want my backups to have last's year's data, but wouldn't want to need to upgrade the size of my actual home drive. So I would want to maintain less snapshots on my home drive than my backup drive. (It's possible I'm missing something here...something subtle that makes this not necessary) So do I only need to make sure I have the latest snapshot or maybe latest plus n-1 on the home drive while the backup drive can have all snapshots since the beginning? I THINK that can be the case based on reading your sentence, but I just want to make sure. In case you were wondering, this is based on what's happened to me with Back in Time. I had to reduce the number of backups I was keeping because my home drive wasn't at 100%, but the backupdrive was at 100% because I'd added and deleted some VMs and other large files (video files I think). And Back in Time intelligently does not remove the oldest backup off the top until it knows it has made a new backup - which it couldn't do because it was at 100%. So I had to delete the top 1 or 2 backups and then tell it to keep less backups. Your description of snapshots makes it seems much less likely that this would be an issue. Although Back in Time is an incremental backup, its takes up more space. If I may venture to see if I've learned something from your response, is it because when I change a file Back in Time stores the entire changed file while btrfs only stores the bi
Re: 3.13.5 btrfs read() oops
On 03/07/2014 05:55 AM, Daniel J Blueman wrote: > With kernel 3.13.5 (Ubuntu mainline), when plugging in a (evidently > twitchy) USB3 stick with a BTRFS filesystem, I hit an oops in read() > [1]. > > Full dmesg output is at: > https://urldefense.proofpoint.com/v1/url?u=http://quora.org/2014/btrfs-oops.txt&k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0A&r=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0A&m=l%2B%2Fvx4VlYB25NI5yUJbBz2UM5ao0KdYuSUnSSAQ4pmY%3D%0A&s=0eee1e5d23ceb6100beb27d5a4f2e050d241e6b53e0db678e81ba9c8b88ad351 I've just uploaded a for-stable-3.13 branch that should fix this: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git for-stable-3.13 -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
3.13.5 btrfs read() oops
With kernel 3.13.5 (Ubuntu mainline), when plugging in a (evidently twitchy) USB3 stick with a BTRFS filesystem, I hit an oops in read() [1]. Full dmesg output is at: http://quora.org/2014/btrfs-oops.txt Thanks, Daniel -- [1] IP: 0010:[] [] memcpy+0x6/0x110 RSP: 0018:88025fa1b910 EFLAGS: 00010207 RAX: 88005c3d906e RBX: 027e RCX: 027e RDX: 027e RSI: 00050800 RDI: 88005c3d906e RBP: 88025fa1b948 R08: 1000 R09: 88025fa1b918 R10: R11: R12: 8800560e6350 R13: 1600 R14: 88005c3d92ec R15: 027e FS: 7f9272f79700() GS:88026f3c() knlGS: CS: 0010 DS: ES: CR0: 80050033 CR2: 7f9264010018 CR3: 00025f79a000 CR4: 001407e0 Stack: a036401c 1000 8800837f3800 8801e041a000 8800763df218 880064c8c4c0 88025fa1ba08 a0348f9c 0f18 1000 Call Trace: [] ? read_extent_buffer+0xbc/0x110 [btrfs] [] btrfs_get_extent+0x91c/0x970 [btrfs] [] __do_readpage+0x357/0x730 [btrfs] [] ? btrfs_real_readdir+0x5b0/0x5b0 [btrfs] [] __extent_readpages.constprop.41+0x2a2/0x2c0 [btrfs] [] ? btrfs_real_readdir+0x5b0/0x5b0 [btrfs] [] extent_readpages+0x1b6/0x1c0 [btrfs] [] ? btrfs_real_readdir+0x5b0/0x5b0 [btrfs] [] ? alloc_pages_current+0xa3/0x160 [] btrfs_readpages+0x1f/0x30 [btrfs] [] __do_page_cache_readahead+0x1b9/0x270 [] ondemand_readahead+0x152/0x2a0 [] page_cache_sync_readahead+0x31/0x50 [] generic_file_aio_read+0x4c5/0x700 [] do_sync_read+0x5a/0x90 [] vfs_read+0x95/0x160 [] SyS_read+0x49/0xa0 [] tracesys+0xe1/0xe6 -- Daniel J Blueman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ENOSPC errors during raid1 rebalance
Hugo Mills posted on Fri, 07 Mar 2014 08:02:13 + as excerpted: > On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: >> Duncan <1i5t5.duncan cox.net> writes: >> >> > But if you're not using compression, /that/ can't explain it... >> > >> > >> Ha! Well while that was an interesting discussion of fragmentation, I >> am only using the default mount options here and so no compression. > >I _think_ the problem here is that there may have been some extents > created during the conversion which were over 1 GiB in size (or at least > which run across two or more chunks). This causes problems, because > there's nowhere that they can be written to by the balance -- > which preserves extents -- because none of the allocation units (chunks) > are big enough. > >The defrag operation, by its nature, _doesn't_ preserve extents, > and thus can act to break up the large extents, making it possible to > balance the chunks that the offending extents live on. Now /that/ would explain the issue, or at least all I've read of it from here. Nice job! =:^) Obviously >1 GB files break/fragment on 1-gig data-chunk lines when (re)written on btrfs, since that's the biggest allocation unit btrfs has, and at least once the filesystem has been (near) full once, btrfs is extremely unlikely to be able to place those gig-chunks contiguously, thus triggering the issue. Meanwhile, the move-to-tmpfs-and-back should indeed break up those >1-gig blocks too, since the that would force a rewrite and thus a reallocation, which would then follow btrfs 1-gig-chunk rules. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Understanding btrfs and backups
Duncan, thank you for this comprehensive post. Really helpful as always! [...] > As for restoring, since a snapshot is a copy of the filesystem as it > existed at that point, and the method btrfs exposes for accessing them is > to mount that specific snapshot, to restore an individual file from a > snapshot, you simply mount the snapshot you want somewhere and copy the > file as it existed in that snapshot over top of your current version > (which will have presumably already been mounted elsewhere, before you > mounted the snapshot to retrieve the file from), then unmount the > snapshot and go about your day. =:^) Please, how do I list mounted snapshots only? [...] > > Since a snapshot is an image of the filesystem as it was at that > particular point in time, and btrfs by nature copies blocks elsewhere > when they are modified, all (well, not "all" as there's metadata like > file owner, permissions and group, too, but that's handled the same way) > the snapshot does is map what blocks composed each file at the time the > snapshot was taken. Is it correct, that e.g. ownership is recorded separately from the data itself, so if I would change the owner of all my files, the respective snapshot would only store the old owner information? [...] > > The first time you do this, there's no existing copy at the other end, so > btrfs send sends a full copy and btrfs receive writes it out. After > that, the receive side has a snapshot identical to the one created on the > send side and further btrfs send/receives to the same set simply > duplicate the differences between the reference and the new snapshot from > the send end to the receive end. As with local snapshots, old ones can > be deleted on both the send and receive ends, as long as at least one > common reference snapshot is maintained on both ends, so diffs taken > against the send side reference can be applied to an appropriately > identical receive side reference, thereby updating the receive side to > match the new read-only snapshot on the send side. Is the receiving side a complete file system in its own right? If so, I only need to maintain one common reference in order to apply the received snapshot, right. If I would in any way get the send and receive side out of sync, such that they do not share a common reference any more, only the send/receive would fail, but I still would have the complete filesystem on the receiving side, and could copy it all over (cp, rscync) to the send side in case of a disaster on the send side. Is this correct? Thank you! Best, Wolfgang -- Wolfgang Mader wolfgang.ma...@fdm.uni-freiburg.de Telefon: +49 (761) 203-7710 Institute of Physics Hermann-Herder Str. 3, 79104 Freiburg, Germany Office: 207 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: ENOSPC errors during raid1 rebalance
On Fri, Mar 07, 2014 at 01:13:53AM +, Michael Russo wrote: > Duncan <1i5t5.duncan cox.net> writes: > > > But if you're not using compression, /that/ can't explain it... > > > > Ha! Well while that was an interesting discussion of fragmentation, > I am only using the default mount options here and so no compression. > The only reason I'm really even looking at the fragmentation > issue is because running the defragment (with varying sizes of > "-t" which I'm not sure why that's even necessary) seemed > to force btrfs to move segments around and let me rebalance > more block groups. I don't even care if the files are defragged, > I just want them all in the RAID1 profile. Hopefully if I move > each file out to some other FS like /dev/shm and then back > it will work, I just gotta hack a script together to do so. I _think_ the problem here is that there may have been some extents created during the conversion which were over 1 GiB in size (or at least which run across two or more chunks). This causes problems, because there's nowhere that they can be written to by the balance -- which preserves extents -- because none of the allocation units (chunks) are big enough. The defrag operation, by its nature, _doesn't_ preserve extents, and thus can act to break up the large extents, making it possible to balance the chunks that the offending extents live on. Hugo. -- === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk === PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk --- If the first-ever performance is the première, is the --- last-ever performance the derrière? signature.asc Description: Digital signature