btrfs wastes disk space after snapshot deletetion.
Hello, If I write large sequential file on snapshot, then create another snapshot, overwrite file with small amount of data and delete first snapshot, second snapshot has very large data extent and only small part of it is used. For example if I use following sequence: mkfs.btrfs /dev/sdn mount -o noatime,nodatacow,nospace_cache /dev/sdn /mnt/b btrfs sub snap /mnt/b /mnt/b/snap1 dd if=/dev/zero of=/mnt/b/snap1/t count=15000 bs=65535 sync btrfs sub snap /mnt/b/snap1 /mnt/b/snap2 dd if=/dev/zero of=/mnt/b/snap2/t seek=3 count=1 bs=2048 sync btrfs sub delete /mnt/b/snap1 btrfs-debug-tree /dev/sdn I see following data extents item 6 key (257 EXTENT_DATA 0) itemoff 3537 itemsize 53 extent data disk byte 1103101952 nr 194641920 extent data offset 0 nr 4096 ram 194641920 extent compression 0 item 7 key (257 EXTENT_DATA 4096) itemoff 3484 itemsize 53 extent data disk byte 2086129664 nr 4096 extent data offset 0 nr 4096 ram 4096 extent compression 0 In item 6: only 4096 from 194641920 are in use. Rest of space is wasted. If I defragment like: btrfs filesystem defragment /mnt/b/snap2/t it release wasted space. But I can't use defragment because if I have few snapshots I need to run defragment on each snapshot and it disconnect relation between snapshot and create multiple copies of same data. In our test that create and delete snapshots while writing data, we end up with few GBs of disk space wasted. Is it possible to limit size of allocated data extents? Is it possible to defragment subvolume without breaking snapshots relations? Any other idea how to recover wasted space? Thanks, Moshe Melnikov -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Does btrfs adapt to size changes of underlying block device(s)?
Hello, I've got a quick question: Does btrfs adapt to size changes of the underlying block device(s)? My specific situation is as follows: I've got a luks-volume on which I want to put btrfs. If this luks-volume grows in the future (i.e. by 'cryptsetup resize'), will btrfs automatically (and reliably) see this and be able to use the additional space? Of course, one can always put an lvm-layer in between, but I would really like to avoid this. And as we're at it: What about shrinking of the underlying block-device? Thanks for your help, Simon -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Does btrfs adapt to size changes of underlying block device(s)?
On Mon, Feb 04, 2013 at 01:37:21PM +0100, Simon Campese wrote: Hello, I've got a quick question: Does btrfs adapt to size changes of the underlying block device(s)? My specific situation is as follows: I've got a luks-volume on which I want to put btrfs. If this luks-volume grows in the future (i.e. by 'cryptsetup resize'), will btrfs automatically (and reliably) see this and be able to use the additional space? Not automatically, you need to invoke btrfs command for that. But shrinking and growing works online and is reliable. btrfs filesystem resize [devid:][+/-]newsize[gkm]|[devid:]max path -- Tomasz Torcz Morality must always be based on practicality. xmpp: zdzich...@chrome.pl-- Baron Vladimir Harkonnen -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs: extend the checksum item as much as possible
For write, we also reserve some space for COW blocks during updating
the checksum tree, and we calculate the number of blocks by checking
if the number of bytes outstanding that are going to need csums needs
one more block for csum.
When we add these checksum into the checksum tree, we use ordered sums
list.
Every ordered sum contains csums for each sector, and we'll first try
to look up an existing csum item,
a) if we don't yet have a proper csum item, then we need to insert one,
b) or if we find one but the csum item is not big enough, then we need
to extend it.
The point is we'll unlock the whole path and then insert or extend.
So others can hack in and update the tree.
Each insert or extend needs update the tree with COW on, and we may need
to insert/extend for many times.
That means what we've reserved for updating checksum tree is NOT enough
indeed.
The case is even more serious with having several write threads at the
same time, it can end up eating our reserved space quickly and starting
eating globle reserve pool instead.
I don't yet come up with a way to calculate the worse case for updating
csum, but extending the checksum item as much as possible can be helpful
in my test.
The idea behind is that it can reduce the times we insert/extend so that
it saves us precious reserved space.
Signed-off-by: Liu Bo bo.li@oracle.com
---
fs/btrfs/file-item.c | 67 ++---
1 files changed, 46 insertions(+), 21 deletions(-)
diff --git a/fs/btrfs/file-item.c b/fs/btrfs/file-item.c
index 94aa53b..ec16020 100644
--- a/fs/btrfs/file-item.c
+++ b/fs/btrfs/file-item.c
@@ -684,6 +684,24 @@ out:
return ret;
}
+static u64 btrfs_sector_sum_left(struct btrfs_ordered_sum *sums,
+struct btrfs_sector_sum *sector_sum,
+u64 total_bytes, u64 sectorsize)
+{
+ u64 tmp = sectorsize;
+ u64 next_sector = sector_sum-bytenr;
+ struct btrfs_sector_sum *next = sector_sum + 1;
+
+ while ((tmp + total_bytes) sums-len) {
+ if (next_sector + sectorsize != next-bytenr)
+ break;
+ tmp += sectorsize;
+ next_sector = next-bytenr;
+ next++;
+ }
+ return tmp;
+}
+
int btrfs_csum_file_blocks(struct btrfs_trans_handle *trans,
struct btrfs_root *root,
struct btrfs_ordered_sum *sums)
@@ -789,20 +807,32 @@ again:
goto insert;
}
- if (csum_offset = btrfs_item_size_nr(leaf, path-slots[0]) /
+ if (csum_offset == btrfs_item_size_nr(leaf, path-slots[0]) /
csum_size) {
- u32 diff = (csum_offset + 1) * csum_size;
+ int extend_nr;
+ u64 tmp;
+ u32 diff;
+ u32 free_space;
- /*
-* is the item big enough already? we dropped our lock
-* before and need to recheck
-*/
- if (diff btrfs_item_size_nr(leaf, path-slots[0]))
- goto csum;
+ if (btrfs_leaf_free_space(root, leaf)
+sizeof(struct btrfs_item) + csum_size * 2)
+ goto insert;
+
+ free_space = btrfs_leaf_free_space(root, leaf) -
+sizeof(struct btrfs_item) - csum_size;
+ tmp = btrfs_sector_sum_left(sums, sector_sum, total_bytes,
+ root-sectorsize);
+ tmp = root-fs_info-sb-s_blocksize_bits;
+ WARN_ON(tmp 1);
+
+ extend_nr = max_t(int, 1, (int)tmp);
+ diff = (csum_offset + extend_nr) * csum_size;
+ diff = min(diff, MAX_CSUM_ITEMS(root, csum_size) * csum_size);
diff = diff - btrfs_item_size_nr(leaf, path-slots[0]);
- if (diff != csum_size)
- goto insert;
+ diff = min(free_space, diff);
+ diff /= csum_size;
+ diff *= csum_size;
btrfs_extend_item(trans, root, path, diff);
goto csum;
@@ -812,19 +842,14 @@ insert:
btrfs_release_path(path);
csum_offset = 0;
if (found_next) {
- u64 tmp = total_bytes + root-sectorsize;
- u64 next_sector = sector_sum-bytenr;
- struct btrfs_sector_sum *next = sector_sum + 1;
+ u64 tmp;
- while (tmp sums-len) {
- if (next_sector + root-sectorsize != next-bytenr)
- break;
- tmp += root-sectorsize;
- next_sector = next-bytenr;
- next++;
- }
- tmp = min(tmp, next_offset - file_key.offset);
+ tmp = btrfs_sector_sum_left(sums, sector_sum, total_bytes,
+
Re: [RESEND][PATCH] btrfs: add no file data flag to btrfs send ioctl
On Wed, Jan 30, 2013 at 02:43:41PM -0800, Mark Fasheh wrote:
@@ -4570,6 +4605,11 @@ long btrfs_ioctl_send(struct file *mnt_file, void
__user *arg_)
INIT_RADIX_TREE(sctx-name_cache, GFP_NOFS);
INIT_LIST_HEAD(sctx-name_cache_list);
+ if (arg-flags ~BTRFS_SEND_FLAG_NO_FILE_DATA)
+ return -EINVAL;
Doing just return skips the cleanup block (goto out;), also the test can
be placed just after the access_ok() call and skip allocation and init
of sctx.
+
+ sctx-flags = arg-flags;
+
sctx-send_filp = fget(arg-send_fd);
if (IS_ERR(sctx-send_filp)) {
ret = PTR_ERR(sctx-send_filp);
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs-progs print more informative error when we fail to open a device
I believe that this patch may have been lost in the noise and I felt it made a very small but meaningful change so I shuffled some test around and made sure it applied to integration-20120201 This patch simply prints more informative message when we fail to open a device. Eric Sandeen (1): Btrfs-progs print more informative error when we fail to open a device utils.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 1.8.1 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] Btrfs-progs print more informative error when we fail to open a device
From: Eric Sandeen sand...@redhat.com
print more informative error when we fail to open a device
If open() fails, we should let the user know why it failed.
Signed-off-by: Eric Sandeen sand...@redhat.com
Signed-off-by: Gene Czarcinski g...@czarc.net
---
utils.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/utils.c b/utils.c
index d92f317..b560a49 100644
--- a/utils.c
+++ b/utils.c
@@ -1212,7 +1212,8 @@ scan_again:
fd = open(fullpath, O_RDONLY);
if (fd 0) {
- fprintf(stderr, failed to read %s\n, fullpath);
+ fprintf(stderr, failed to open %s: %s\n,
+ fullpath, strerror(errno));
continue;
}
ret = btrfs_scan_one_device(fd, fullpath, tmp_devices,
--
1.8.1
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:6185!
On Sun, Feb 03, 2013 at 05:19:29AM -0700, Radek Machulka wrote: Hi guys, my computer suddenly failed to boot. It seams that it is unable to mount /home which is btrfs. Here are the messages I get during the boot (the relevant part; hopefully): Feb 02 13:59:58 Edge kernel: kernel BUG at fs/btrfs/extent-tree.c:6185! Feb 02 13:59:58 Edge kernel: invalid opcode: [#1] PREEMPT SMP Feb 02 13:59:58 Edge kernel: Modules linked in: joydev coretemp kvm_intel kvm arc4 ghash_clmulni_intel aesni_intel aes_x86_64 ablk_helper snd_hda_cod Feb 02 13:59:58 Edge kernel: CPU 1 Feb 02 13:59:58 Edge systemd[1]: home.mount mount process exited, code=killed status=11 Feb 02 13:59:58 Edge kernel: Pid: 314, comm: mount Tainted: G C 3.7.5-1-ARCH #1 LENOVO 129886G/129886G Feb 02 13:59:58 Edge kernel: RIP: 0010:[a024e0a7] [a024e0a7] btrfs_alloc_logged_file_extent+0x1c7/0x1e0 [btrfs] Feb 02 13:59:58 Edge kernel: RSP: 0018:880138b53758 EFLAGS: 00010286 Feb 02 13:59:58 Edge kernel: RAX: fff5 RBX: 880135bc4200 RCX: 0001a7c0 Feb 02 13:59:58 Edge kernel: RDX: ea0004dea100 RSI: 0002 RDI: 880138bdd840 Feb 02 13:59:58 Edge kernel: RBP: 880138b537d8 R08: 880138b53fd8 R09: 880134ace400 Feb 02 13:59:58 Edge kernel: R10: a029aa6c R11: 880138b53fd8 R12: 880138b538a3 Feb 02 13:59:58 Edge kernel: R13: R14: 00017f8f1000 R15: 8801388b6000 Feb 02 13:59:58 Edge kernel: FS: 7f9c55f20780() GS:88013fa4() knlGS: Feb 02 13:59:58 Edge kernel: CS: 0010 DS: ES: CR0: 80050033 Feb 02 13:59:58 Edge kernel: CR2: 0161ceb0 CR3: 000138b5c000 CR4: 000407e0 Feb 02 13:59:58 Edge kernel: DR0: DR1: DR2: Feb 02 13:59:58 Edge systemd[1]: Failed to mount /home. Feb 02 13:59:58 Edge systemd-journal[345]: Allowing runtime journal files to grow to 193.5M. Feb 02 13:59:58 Edge kernel: Process mount (pid: 314, threadinfo 880138b52000, task 880134bcc920) Feb 02 13:59:58 Edge kernel: Stack: Feb 02 13:59:58 Edge kernel: 0001 8000 880138b53788 a023a1ea Feb 02 13:59:58 Edge kernel: 00614000 0430 0005 880137a81000 Feb 02 13:59:58 Edge kernel: 8000 00ff88013788a560 a800017f8f10 8801388b6000 Feb 02 13:59:58 Edge kernel: Call Trace: Feb 02 13:59:58 Edge kernel: [a023a1ea] ? btrfs_free_path+0x2a/0x40 [btrfs] Feb 02 13:59:58 Edge kernel: [a0293990] replay_one_extent+0x620/0x690 [btrfs] Feb 02 13:59:58 Edge kernel: [a0292fc8] ? add_inode_ref+0x638/0x9e0 [btrfs] Feb 02 13:59:58 Edge kernel: [a026e940] ? btrfs_destroy_inode+0x1c0/0x2e0 [btrfs] Feb 02 13:59:58 Edge kernel: [a0294a1b] replay_one_buffer+0x2db/0x3a0 [btrfs] Feb 02 13:59:58 Edge kernel: [a027c9bd] ? alloc_extent_buffer+0x9d/0x490 [btrfs] Feb 02 13:59:58 Edge kernel: [a02913e2] walk_down_log_tree+0x212/0x400 [btrfs] Feb 02 13:59:58 Edge kernel: [a029166d] walk_log_tree+0x9d/0x1f0 [btrfs] Feb 02 13:59:58 Edge kernel: [a02979eb] btrfs_recover_log_trees+0x21b/0x3a0 [btrfs] Feb 02 13:59:58 Edge kernel: [a0294740] ? replay_one_dir_item+0xf0/0xf0 [btrfs] Feb 02 13:59:58 Edge kernel: [a025d9d7] open_ctree+0x1587/0x1ba0 [btrfs] Feb 02 13:59:58 Edge kernel: [81255091] ? disk_name+0x61/0xc0 Feb 02 13:59:58 Edge kernel: [a0236ae3] btrfs_mount+0x633/0x770 [btrfs] Feb 02 13:59:58 Edge kernel: [81165f60] ? alloc_pages_current+0xb0/0x120 Feb 02 13:59:58 Edge kernel: [81188163] mount_fs+0x43/0x1b0 Feb 02 13:59:58 Edge kernel: [811a2974] vfs_kern_mount+0x74/0x110 Feb 02 13:59:58 Edge kernel: [811a2ed4] do_kern_mount+0x54/0x110 Feb 02 13:59:58 Edge kernel: [811a4b55] do_mount+0x315/0x8e0 Feb 02 13:59:58 Edge kernel: [811a46aa] ? copy_mount_options+0x3a/0x180 Feb 02 13:59:58 Edge kernel: [811a51ae] sys_mount+0x8e/0xe0 Feb 02 13:59:58 Edge kernel: [814c111d] system_call_fastpath+0x1a/0x1f Feb 02 13:59:58 Edge kernel: Code: 0b 0f 1f 00 48 8d bb 80 00 00 00 e8 c4 68 ff ff 85 c0 74 1d 48 8b 55 c0 4c 89 f6 48 89 df e8 01 c8 04 00 85 c0 0f 84 29 ff ff ff 0f 0b 0f 0b 0f 0b 0f 0b 0f 0b 66 66 66 66 66 66 2e 0f 1f 84 00 Feb 02 13:59:58 Edge kernel: RIP [a024e0a7] btrfs_alloc_logged_file_extent+0x1c7/0x1e0 [btrfs] Feb 02 13:59:58 Edge kernel: RSP 880138b53758 Feb 02 13:59:58 Edge kernel: ---[ end trace 03c331f97cd8632d ]--- I was however able to save my data with btrfs-restore command (it boots into some systemd emergency shell or I can use recovery cd). I tried to mount it manually and after no success (the mount command does not end) mount it with -o recovery option, but it does not do anything nether (well
Re: btrfs wastes disk space after snapshot deletetion.
On Mon, Feb 04, 2013 at 02:08:01AM -0700, Moshe wrote: Hello, If I write large sequential file on snapshot, then create another snapshot, overwrite file with small amount of data and delete first snapshot, second snapshot has very large data extent and only small part of it is used. For example if I use following sequence: mkfs.btrfs /dev/sdn mount -o noatime,nodatacow,nospace_cache /dev/sdn /mnt/b btrfs sub snap /mnt/b /mnt/b/snap1 dd if=/dev/zero of=/mnt/b/snap1/t count=15000 bs=65535 sync btrfs sub snap /mnt/b/snap1 /mnt/b/snap2 dd if=/dev/zero of=/mnt/b/snap2/t seek=3 count=1 bs=2048 sync btrfs sub delete /mnt/b/snap1 btrfs-debug-tree /dev/sdn I see following data extents item 6 key (257 EXTENT_DATA 0) itemoff 3537 itemsize 53 extent data disk byte 1103101952 nr 194641920 extent data offset 0 nr 4096 ram 194641920 extent compression 0 item 7 key (257 EXTENT_DATA 4096) itemoff 3484 itemsize 53 extent data disk byte 2086129664 nr 4096 extent data offset 0 nr 4096 ram 4096 extent compression 0 In item 6: only 4096 from 194641920 are in use. Rest of space is wasted. If I defragment like: btrfs filesystem defragment /mnt/b/snap2/t it release wasted space. But I can't use defragment because if I have few snapshots I need to run defragment on each snapshot and it disconnect relation between snapshot and create multiple copies of same data. In our test that create and delete snapshots while writing data, we end up with few GBs of disk space wasted. Is it possible to limit size of allocated data extents? Is it possible to defragment subvolume without breaking snapshots relations? Any other idea how to recover wasted space? This is all by design to try and limit the size of the extent tree. Instead of splitting references in the extent tree to account for the split extent we do it in the file tree. In your case it results in a lot of wasted space. This is on the list of things to fix, we will just split the references in the extent tree and deal with the larger extent tree, but it's on the back burner while we get things a bit more stable. Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [BUG] kernel BUG at fs/btrfs/async-thread.c:605!
On 2/3/13 8:39 PM, Miao Xie wrote: Hi, Eric I want to send out my fix patch, but Could I add your Signed-off-by? because you found the key to solving the problem. I don't know if a signed-off-by chain is the right approach, but don't worry about it. You can mention my first patch in the changelog if you like. Thanks, -Eric Thanks Miao On Fri, 01 Feb 2013 14:53:09 +0900, Tsutomu Itoh wrote: Can you please explain similar problems, Miao? Before missing device check, there are several places where we read the metadata, such as reading chunk tree root, btrfs_read_chunk_tree, those functions may fail after submit a bio. If we don't wait until the bio end, and just stop the workers, the same problem will happen. (invalidate_inode_pages2() will wait until the bio end, because it need lock the pages which are going to be invalidated, and the page is locked if it is under disk read IO) I understood. My reproducer is not reproduce this problem yet. But the following messages were displayed when 'rmmod btrfs' command was executed. [76378.723481] = [76378.723901] BUG btrfs_extent_buffer (Tainted: G B ): Objects remaining in btrfs_extent_buffer on kmem_cache_close() [76378.724333] - [76378.724333] [76378.724959] INFO: Slab 0xea00065c3280 objects=23 used=2 fp=0x8801970caac0 flags=0x80004080 [76378.725391] Pid: 9156, comm: rmmod Tainted: G B3.8.0-rc5 #1 [76378.725397] Call Trace: [76378.725403] [8111bc23] slab_err+0xb0/0xd2 I think that this message means there is a possibility that I/O did not end normally. and, after Miao's patch applied, this message is not displayed when rmmod was executed. So, Miao's patch seems to fix the problem for me. [SNIP] diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 0c31d07..d8fd711 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2728,13 +2728,13 @@ fail_cleaner: * kthreads */ filemap_write_and_wait(fs_info-btree_inode-i_mapping); - invalidate_inode_pages2(fs_info-btree_inode-i_mapping); fail_block_groups: btrfs_free_block_groups(fs_info); fail_tree_roots: free_root_pointers(fs_info, 1); + invalidate_inode_pages2(fs_info-btree_inode-i_mapping); fail_sb_buffer: btrfs_stop_workers(fs_info-generic_worker); @@ -2755,7 +2755,6 @@ fail_alloc: fail_iput: btrfs_mapping_tree_free(fs_info-mapping_tree); - invalidate_inode_pages2(fs_info-btree_inode-i_mapping); iput(fs_info-btree_inode); fail_bdi: bdi_destroy(fs_info-bdi); -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: kernel BUG at fs/btrfs/extent-tree.c:1772
On Sun, Feb 03, 2013 at 04:37:58PM -0700, Piotr Pawłow wrote: Hello, 1 week ago I bought a new 2TB hard drive, created 1 partition on the whole disk, and created root and swap LVM volumes on it. I formatted root to btrfs with default options, except meta-data profile which I set to single. I copied all my data to it and unplugged my old hard drives. It worked for a week. Today I installed another 2TB hard drive, created LVM volumes as before, and added to my root filesystem. I started balance with data and metadata conversion to raid1. I left it doing its job, and returned few hours later. When I returned hard disks were idle, screen would not unlock, and I could not log in. I had to reset it. After reset it won't mount. I booted the computer from a live-cd. It has an ancient kernel, so I installed qemu and captured the oops on the current version compiled from git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git: Ok good you have a git tree. Can you run these commands git remote-add josef \ git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git git fetch josef git co josef/piotr and then build that and run that. It will spit out some debug stuff when it mounts before it panics, can you capture all the output and reply to this email with it? Thanks, Josef -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH] btrfs-progs: remove unused bit-radix.[ch] files
fd53de4d Drop bit-radix.[ch] files
removed the files from the Makefile, but not the files themselves.
Signed-off-by: Eric Sandeen sand...@redhat.com
---
bit-radix.c | 211
bit-radix.h | 33 -
2 files changed, 244 deletions(-)
diff --git a/bit-radix.c b/bit-radix.c
deleted file mode 100644
index 57f6f3c..000
--- a/bit-radix.c
+++ /dev/null
@@ -1,211 +0,0 @@
-/*
- * Copyright (C) 2007 Oracle. All rights reserved.
- *
- * This program is free software; you can redistribute it and/or
- * modify it under the terms of the GNU General Public
- * License v2 as published by the Free Software Foundation.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public
- * License along with this program; if not, write to the
- * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
- * Boston, MA 021110-1307, USA.
- */
-
-#include kerncompat.h
-#include radix-tree.h
-
-#define BIT_ARRAY_BYTES 256
-#define BIT_RADIX_BITS_PER_ARRAY ((BIT_ARRAY_BYTES - sizeof(unsigned long)) *
8)
-
-int set_radix_bit(struct radix_tree_root *radix, unsigned long bit)
-{
- unsigned long *bits;
- unsigned long slot;
- int bit_slot;
- int ret;
-
- slot = bit / BIT_RADIX_BITS_PER_ARRAY;
- bit_slot = bit % BIT_RADIX_BITS_PER_ARRAY;
-
- bits = radix_tree_lookup(radix, slot);
- if (!bits) {
- bits = malloc(BIT_ARRAY_BYTES);
- if (!bits)
- return -ENOMEM;
- memset(bits + 1, 0, BIT_ARRAY_BYTES - sizeof(unsigned long));
- bits[0] = slot;
- radix_tree_preload(GFP_NOFS);
- ret = radix_tree_insert(radix, slot, bits);
- radix_tree_preload_end();
- if (ret)
- return ret;
- }
- __set_bit(bit_slot, bits + 1);
- return 0;
-}
-
-int test_radix_bit(struct radix_tree_root *radix, unsigned long bit)
-{
- unsigned long *bits;
- unsigned long slot;
- int bit_slot;
-
- slot = bit / BIT_RADIX_BITS_PER_ARRAY;
- bit_slot = bit % BIT_RADIX_BITS_PER_ARRAY;
-
- bits = radix_tree_lookup(radix, slot);
- if (!bits)
- return 0;
- return test_bit(bit_slot, bits + 1);
-}
-
-int clear_radix_bit(struct radix_tree_root *radix, unsigned long bit)
-{
- unsigned long *bits;
- unsigned long slot;
- int bit_slot;
- int i;
- int empty = 1;
- slot = bit / BIT_RADIX_BITS_PER_ARRAY;
- bit_slot = bit % BIT_RADIX_BITS_PER_ARRAY;
-
- bits = radix_tree_lookup(radix, slot);
- if (!bits)
- return 0;
- __clear_bit(bit_slot, bits + 1);
- for (i = 1; i BIT_ARRAY_BYTES / sizeof(unsigned long); i++) {
- if (bits[i]) {
- empty = 0;
- break;
- }
- }
- if (empty) {
- bits = radix_tree_delete(radix, slot);
- BUG_ON(!bits);
- free(bits);
- }
- return 0;
-}
-
-#define BITOP_WORD(nr) ((nr) / BITS_PER_LONG)
-
-/**
- * __ffs - find first bit in word.
- * @word: The word to search
- *
- * Undefined if no bit exists, so code should check against 0 first.
- */
-static unsigned long __ffs(unsigned long word)
-{
- int num = 0;
-
- if (sizeof(long) == 8 (word 0x) == 0) {
- num += 32;
- word = sizeof(long) * 4;
- }
- if ((word 0x) == 0) {
- num += 16;
- word = 16;
- }
- if ((word 0xff) == 0) {
- num += 8;
- word = 8;
- }
- if ((word 0xf) == 0) {
- num += 4;
- word = 4;
- }
- if ((word 0x3) == 0) {
- num += 2;
- word = 2;
- }
- if ((word 0x1) == 0)
- num += 1;
- return num;
-}
-
-/**
- * find_next_bit - find the next set bit in a memory region
- * @addr: The address to base the search on
- * @offset: The bitnumber to start searching at
- * @size: The maximum size to search
- */
-unsigned long find_next_bit(const unsigned long *addr, unsigned long size,
- unsigned long offset)
-{
- const unsigned long *p = addr + BITOP_WORD(offset);
- unsigned long result = offset ~(BITS_PER_LONG-1);
- unsigned long tmp;
-
- if (offset = size)
- return size;
- size -= result;
- offset %= BITS_PER_LONG;
- if (offset) {
- tmp = *(p++);
- tmp = (~0UL offset);
- if (size BITS_PER_LONG)
- goto found_first;
-
Re: [RESEND][PATCH] btrfs: add no file data flag to btrfs send ioctl
On Mon, Feb 04, 2013 at 04:48:56PM +0100, David Sterba wrote:
On Wed, Jan 30, 2013 at 02:43:41PM -0800, Mark Fasheh wrote:
@@ -4570,6 +4605,11 @@ long btrfs_ioctl_send(struct file *mnt_file, void
__user *arg_)
INIT_RADIX_TREE(sctx-name_cache, GFP_NOFS);
INIT_LIST_HEAD(sctx-name_cache_list);
+ if (arg-flags ~BTRFS_SEND_FLAG_NO_FILE_DATA)
+ return -EINVAL;
Doing just return skips the cleanup block (goto out;), also the test can
be placed just after the access_ok() call and skip allocation and init
of sctx.
Ahh, good catch. Sad I missed that one :( A corrected patch is below.
--Mark
--
Mark Fasheh
From: Mark Fasheh mfas...@suse.de
[PATCH] btrfs: add no file data flag to btrfs send ioctl
This patch adds the flag, BTRFS_SEND_FLAG_NO_FILE_DATA to the btrfs send
ioctl code. When this flag is set, the btrfs send code will never write file
data into the stream (thus also avoiding expensive reads of that data in the
first place). BTRFS_SEND_C_UPDATE_EXTENT commands will be sent (instead of
BTRFS_SEND_C_WRITE) with an offset, length pair indicating the extent in
question.
This patch does not affect the operation of BTRFS_SEND_C_CLONE commands -
they will continue to be sent when a search finds an appropriate extent to
clone from.
Signed-off-by: Mark Fasheh mfas...@suse.de
---
fs/btrfs/ioctl.h |7 +++
fs/btrfs/send.c | 50 ++
fs/btrfs/send.h |1 +
3 files changed, 54 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/ioctl.h b/fs/btrfs/ioctl.h
index 731e287..1f6cfdd 100644
--- a/fs/btrfs/ioctl.h
+++ b/fs/btrfs/ioctl.h
@@ -363,6 +363,13 @@ struct btrfs_ioctl_received_subvol_args {
__u64 reserved[16]; /* in */
};
+/*
+ * Caller doesn't want file data in the send stream, even if the
+ * search of clone sources doesn't find an extent. UPDATE_EXTENT
+ * commands will be sent instead of WRITE commands.
+ */
+#define BTRFS_SEND_FLAG_NO_FILE_DATA 0x1
+
struct btrfs_ioctl_send_args {
__s64 send_fd; /* in */
__u64 clone_sources_count; /* in */
diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index e78b297..058273c 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -85,6 +85,7 @@ struct send_ctx {
u32 send_max_size;
u64 total_send_size;
u64 cmd_send_size[BTRFS_SEND_C_MAX + 1];
+ u64 flags; /* 'flags' member of btrfs_ioctl_send_args is u64 */
struct vfsmount *mnt;
@@ -3707,6 +3708,39 @@ out:
return ret;
}
+/*
+ * Send an update extent command to user space.
+ */
+static int send_update_extent(struct send_ctx *sctx,
+ u64 offset, u32 len)
+{
+ int ret = 0;
+ struct fs_path *p;
+
+ p = fs_path_alloc(sctx);
+ if (!p)
+ return -ENOMEM;
+
+ ret = begin_cmd(sctx, BTRFS_SEND_C_UPDATE_EXTENT);
+ if (ret 0)
+ goto out;
+
+ ret = get_cur_path(sctx, sctx-cur_ino, sctx-cur_inode_gen, p);
+ if (ret 0)
+ goto out;
+
+ TLV_PUT_PATH(sctx, BTRFS_SEND_A_PATH, p);
+ TLV_PUT_U64(sctx, BTRFS_SEND_A_FILE_OFFSET, offset);
+ TLV_PUT_U64(sctx, BTRFS_SEND_A_SIZE, len);
+
+ ret = send_cmd(sctx);
+
+tlv_put_failure:
+out:
+ fs_path_free(sctx, p);
+ return ret;
+}
+
static int send_write_or_clone(struct send_ctx *sctx,
struct btrfs_path *path,
struct btrfs_key *key,
@@ -3742,7 +3776,11 @@ static int send_write_or_clone(struct send_ctx *sctx,
goto out;
}
- if (!clone_root) {
+ if (clone_root) {
+ ret = send_clone(sctx, offset, len, clone_root);
+ } else if (sctx-flags BTRFS_SEND_FLAG_NO_FILE_DATA) {
+ ret = send_update_extent(sctx, offset, len);
+ } else {
while (pos len) {
l = len - pos;
if (l BTRFS_SEND_READ_SIZE)
@@ -3755,10 +3793,7 @@ static int send_write_or_clone(struct send_ctx *sctx,
pos += ret;
}
ret = 0;
- } else {
- ret = send_clone(sctx, offset, len, clone_root);
}
-
out:
return ret;
}
@@ -4559,6 +4594,11 @@ long btrfs_ioctl_send(struct file *mnt_file, void __user
*arg_)
goto out;
}
+ if (arg-flags ~BTRFS_SEND_FLAG_NO_FILE_DATA) {
+ ret = -EINVAL;
+ goto out;
+ }
+
sctx = kzalloc(sizeof(struct send_ctx), GFP_NOFS);
if (!sctx) {
ret = -ENOMEM;
@@ -4570,6 +4610,8 @@ long btrfs_ioctl_send(struct file *mnt_file, void __user
*arg_)
INIT_RADIX_TREE(sctx-name_cache, GFP_NOFS);
INIT_LIST_HEAD(sctx-name_cache_list);
+ sctx-flags = arg-flags;
+
sctx-send_filp = fget(arg-send_fd);
if (IS_ERR(sctx-send_filp)) {
Re: experimental raid5/6 code in git
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+ if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+ printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
goto out;
}
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+ root-fs_info-fs_devices-rw_devices = 2) {
+ printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ ret = -EINVAL;
+ goto out;
+ }
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+ root-fs_info-fs_devices-rw_devices = 3) {
+ printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
-hpa
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Oops when mounting btrfs partition
On Saturday 02 February 2013, Chris Mason wrote: Feb 1 22:57:37 localhost kernel: [ 8561.599482] Kernel BUG at a01fdcf7 [verbose debug info unavailable] Jan 14 19:18:42 localhost kernel: [1060055.746373] btrfs csum failed ino 15619835 off 454656 csum 2755731641 private 864823192 Jan 14 19:18:42 localhost kernel: [1060055.746381] btrfs: bdev /dev/sdb1 errs: wr 0, rd 0, flush 0, corrupt 17, gen 0 ... Jan 21 16:35:40 localhost kernel: [1655047.701147] parent transid verify failed on 17006399488 wanted 54700 found 54764 These aren't good. With a few exceptions for really tight races in fsx use cases, csum errors are bad data from the disk. The transid verify failed shows we wanted to find a metadata block from generation 54700 but found 54764 instead: I've done a full backup of all data now, without any further Ooops messages, but I did get these: [66155.429029] btrfs no csum found for inode 1212139 start 23707648 [66155.429035] btrfs no csum found for inode 1212139 start 23711744 [66155.429039] btrfs no csum found for inode 1212139 start 23715840 [66155.429042] btrfs no csum found for inode 1212139 start 23719936 [66155.452298] btrfs csum failed ino 1212139 off 23707648 csum 4112094897 private 0 [66155.452310] btrfs csum failed ino 1212139 off 23711744 csum 3308812742 private 0 [66155.452316] btrfs csum failed ino 1212139 off 23715840 csum 2566472073 private 0 [66155.452322] btrfs csum failed ino 1212139 off 23719936 csum 2290008602 private 0 [66159.876785] btrfs no csum found for inode 1212139 start 69992448 [66159.876792] btrfs no csum found for inode 1212139 start 69996544 [66159.876797] btrfs no csum found for inode 1212139 start 7640 [66159.876801] btrfs no csum found for inode 1212139 start 70004736 [66159.921506] btrfs csum failed ino 1212139 off 69992448 csum 2290360822 private 0 [66159.921517] btrfs csum failed ino 1212139 off 69996544 csum 954182507 private 0 [66159.921524] btrfs csum failed ino 1212139 off 7640 csum 2594579850 private 0 [66159.921532] btrfs csum failed ino 1212139 off 70004736 csum 25334750 private 0 [66932.289905] btrfs csum failed ino 2461761 off 94208 csum 3824674580 private 1950015541 [92042.101540] btrfs csum failed ino 687755 off 7048040448 csum 2502110259 private 2186199747 [110952.542245] btrfs csum failed ino 5423479 off 475136 csum 490948044 private 3797189576 [122692.216371] btrfs csum failed ino 7959218 off 2818048 csum 1904746846 private 2392844122 [138205.726897] btrfs: sdb1 checksum verify failed on 20495056896 wanted 8C9759CB found 9BFAB73B level 0 Inode 1212139 is the akonadi database that was used by kmail, so it constantly got written to during the crashes. The file was completely corrupt. The other inodes are mostly files that were backed up from the other machine and have been on the drive I started using it, without ever being accessed. I've probably had a few bit flips the entire time I was using the machine, but never noticed before I started using a checksumming file system. Arnd -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: corrupted file size on inline extent conversion?
On Wed, 30 Jan 2013, Josef Bacik wrote: On Wed, Jan 30, 2013 at 11:30:49AM -0700, Mike Lowe wrote: I've been running rsync against a rbd device backed by btrfs filesystems that are about 11% full for about 45 minutes before I checked and noticed the printk message. That was the first go with the patch. Seems like I was able to get by without any problems until the btrfs filesystems got some use and filled up a little bit. Ok since you are seeing the message I'll go ahead and post the patch and get it moving along, let me know if you still see the problem. Thanks, Awesome. Mike still hasn't seen a reocurrence, so it's looking like the patch is good. Thanks so much! sage -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+ if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+ printk(KERN_ERR btrfs: unable to go below three devices
+on raid5 or raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root, char
*device_path)
goto out;
}
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+ root-fs_info-fs_devices-rw_devices = 2) {
+ printk(KERN_ERR btrfs: unable to go below two
+devices on raid5\n);
+ ret = -EINVAL;
+ goto out;
+ }
+ if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+ root-fs_info-fs_devices-rw_devices = 3) {
+ printk(KERN_ERR btrfs: unable to go below three
+devices on raid6\n);
+ ret = -EINVAL;
+ goto out;
+ }
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: experimental raid5/6 code in git
Thank you, that makes a lot of sense :)
It's been a good day, I've learnt something :)
On Tue, Feb 5, 2013 at 4:29 PM, Chester somethingsome2...@gmail.com wrote:
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long :)
But apparently I've made some incredibly newb error.
I used the following two lines to check out the code:
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git
raid56-experimental
git clone git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
raid56-experimental-progs
Then I did not very much to compile both of them (installed lots and
lots of packages that various places told me would be needed so they'd
both compile) finishing up with a sudo make install for both the
kernel and the tools.
Rebooting miracuously it came up with the new kernel and uname -a
assures me that I have a new kernel running:
btrfs@ubuntu:/kernel/raid56-experimental$ uname -a
Linux ubuntu 3.6.0+ #1 SMP Tue Feb 5 12:26:03 EST 2013 x86_64 x86_64
x86_64 GNU/Linux
but 3.6.0 sounds rather low, but it is newer than Ubuntu 12.10's 3.5
so I believe I am running the kernel I just compiled
Where things fail is that I can figure out how to make a raid5 btrfs,
I'm certain I'm using the mkfs.btrfs that I just compiled (by
explicitly calling it in the make folder) but it wont recognise what I
assume the parameter to be:
btrfs@ubuntu:/kernel/raid56-experimental-progs$ ./mkfs.btrfs -m raid5
-d raid5 /dev/sd[bcdef]
Unknown profile raid5
Which flavour of newb am I today?
PS: I use newb in a very friendly way, I feel no shame over that term :)
On Tue, Feb 5, 2013 at 1:26 PM, H. Peter Anvin h...@zytor.com wrote:
Also, a 2-member raid5 or 3-member raid6 are a raid1 and can be treated as
such.
Chris Mason chris.ma...@fusionio.com wrote:
On Mon, Feb 04, 2013 at 02:42:24PM -0700, H. Peter Anvin wrote:
@@ -1389,6 +1392,14 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
}
btrfs_dev_replace_unlock(root-fs_info-dev_replace);
+if ((all_avail (BTRFS_BLOCK_GROUP_RAID5 |
+ BTRFS_BLOCK_GROUP_RAID6) num_devices = 3)) {
+printk(KERN_ERR btrfs: unable to go below three devices
+ on raid5 or raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if ((all_avail BTRFS_BLOCK_GROUP_RAID10) num_devices = 4) {
printk(KERN_ERR btrfs: unable to go below four devices
on raid10\n);
@@ -1403,6 +1414,21 @@ int btrfs_rm_device(struct btrfs_root *root,
char
*device_path)
goto out;
}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID5)
+root-fs_info-fs_devices-rw_devices = 2) {
+printk(KERN_ERR btrfs: unable to go below two
+ devices on raid5\n);
+ret = -EINVAL;
+goto out;
+}
+if ((all_avail BTRFS_BLOCK_GROUP_RAID6)
+root-fs_info-fs_devices-rw_devices = 3) {
+printk(KERN_ERR btrfs: unable to go below three
+ devices on raid6\n);
+ret = -EINVAL;
+goto out;
+}
+
if (strcmp(device_path, missing) == 0) {
struct list_head *devices;
struct btrfs_device *tmp;
This seems inconsistent?
Whoops, missed that one. Thanks!
-chris
--
Sent from my mobile phone. Please excuse brevity and lack of formatting.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Gareth Pye
Level 2 Judge, Melbourne, Australia
Australian MTG Forum: mtgau.com
gar...@cerberos.id.au - www.rockpaperdynamite.wordpress.com
Dear God, I would like to file a bug report
On Tue, Feb 5, 2013 at 4:29 PM, Chester somethingsome2...@gmail.com wrote:
The last argument should be the directory you want to clone into. Use
'-b branch' to specify the branch you want to clone. I'm pretty sure
you've compiled just the master branch of both linux-btrfs and
btrfs-progs.
On Mon, Feb 4, 2013 at 8:59 PM, Gareth Pye gar...@cerberos.id.au wrote:
I felt like having a small play with this stuff, as I've been wanting
it for so long
Re: [PATCH 2/5] btrfs-progs: libify some parts of btrfs-progs
Hi Mark, A week ago I posted a patch to move fs/btrfs/ioctl.h to include/uapi/linux/btrfs.h in the kernel tree: http://thread.gmane.org/gmane.comp.file-systems.btrfs/22763/focus=22764 My main motivation was to export the kernel headers with the ioctl constants/structs so that other programs may use the ioctls, in my case I was interested in implementing strace support for parsing btrfs ioctls (I'm still working on that project.) I see you took a different approach, using the header files present in btrfs-progs and installing those in /usr/include/btrfs (not a linux/ subdirectory.) As your main objective is to export part of btrfs-progs functionality as a library, that makes sense too... I'm not saying that either way is better, just that it probably needs some coordination. I think in the end we'll need a little bit of both approaches, the kernel exporting one of the header files (for ioctls) and the userland exporting some others (for a libbtrfs external interface.) Here are some relevant project ideas mentioned in the Wiki: 1) Provide a library covering 'btrfs' functionality (https://btrfs.wiki.kernel.org/index.php/Project_ideas#Provide_a_library_covering_.27btrfs.27_functionality) -- your patchset implements part of this. 2) Remove ioctl.h in btrfs and btrfs-progs world, create include/linux/btrfs.h with those definitions (https://btrfs.wiki.kernel.org/index.php/Cleanup_ideas#Remove_ioctl.h_in_btrfs_and_btrfs-progs_world.2C_create_include.2Flinux.2Fbtrfs.h_with_those_definitions) -- my patchset implements part of this, the part of the kernel but not the part of btrfs-progs 3) Use the kernel code in user mode (https://btrfs.wiki.kernel.org/index.php/Cleanup_ideas#Use_the_kernel_code_in_user_mode) -- none of them implements this, I've been thinking about this one and I don't have a good solution yet... I'll send a separate e-mail about this one to the list later. Cheers, Filipe On Wed, Jan 30, 2013 at 2:50 PM, Mark Fasheh mfas...@suse.de wrote: External software wanting to use the functionality provided by the btrfs send ioctl has a hard time doing so without replicating tons of work. Of particular interest are functions like btrfs_read_and_process_send_stream() and subvol_uuid_search(). As that functionality requires a bit more than just send-stream.c and send-utils.c we have to pull in some other parts of the progs package. This patch adds code to the Makefile and headers to create a library, libbtrfs which the btrfs command now links to. Signed-off-by: Mark Fasheh mfas...@suse.de --- Makefile | 83 btrfs-list.h |4 +++ crc32c.h |4 +++ ctree.h|9 ++ extent-cache.h |6 extent_io.h|7 + radix-tree.h |4 +++ rbtree.h |4 +++ send-utils.h |5 9 files changed, 96 insertions(+), 30 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
