[PATCH] btrfs-progs: Cleanup unneeded extra variant in btrfs_read_sys_array

2013-06-26 Thread Qu Wenruo
The btrfs_read_sys_array function uses 3 variants to read data from super block. But the three variants are related to each other, so the patch removes unneeded extra variants and make code a little simpler. Signed-off-by: Qu Wenruo Reviewed-by: Miao Xie --- volumes.c | 17 + 1

[bug] mount is successful when check --repair is running

2013-06-26 Thread Anand Jain
Per code it appears that btrfs check doesn't use the EXCL open which means mount can be successful when the check is running. Thanks, Anand -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH] xfstests: fix btrfs/264 to not use the scratch dev pool

2013-06-26 Thread Anand Jain
Reviewed-by: Anand Jain On 06/26/2013 11:52 PM, Josef Bacik wrote: This test doesn't need the scratch dev pool and it also doesn't call _require_scratch_dev_pool, so just kick out the scratch dev pool part of the test. Thanks, Signed-off-by: Josef Bacik --- tests/btrfs/264 |2 +-

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 07:23:41PM -0700, Zach Brown wrote: > > > But in the krealloc() case the rb_erase() will be trying to reference > > > freed memmory because krealloc() frees the old pointer on success. > > > > Yeah, I realize that you're absolutely right, but my box > > didn't complain abou

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
> > But in the krealloc() case the rb_erase() will be trying to reference > > freed memmory because krealloc() frees the old pointer on success. > > Yeah, I realize that you're absolutely right, but my box > didn't complain about the abused old pointers when we're not in int_nodes > case, which is

[PATCH] xfstests: fix SCRATCH_DEV_POOL mount/umount issues

2013-06-26 Thread Josef Bacik
If you use the SCRATCH_DEV_POOL for btrfs you will end up with the command line like this mkfs.btrfs $SCRATCH_DEV_POOL $SCRATCH_DEV and btrfs does this thing where it makes the lowest valued device id show up in /proc/mounts no matter which device you specify at the mount command. So in this cas

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 01:18:29PM -0700, Zach Brown wrote: > On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: > > Several users reported this crash of NULL pointer or general protection, > > the story is that we add a rbtree for speedup ulist iteration, and we > > use krealloc() to address

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Josef Bacik
On Wed, Jun 26, 2013 at 11:47:29PM +0200, Stefan Behrens wrote: > On 06/26/2013 21:55, Zach Brown wrote: > >>+ if (!uuid_root) { > >>+ WARN_ON_ONCE(1); > >>+ ret = -ENOENT; > >>+ goto out; > >>+ } > > > >WARN_ON_ONCE specifically returns the condition so that you c

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 01:18:29PM -0700, Zach Brown wrote: > On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: > > Several users reported this crash of NULL pointer or general protection, > > the story is that we add a rbtree for speedup ulist iteration, and we > > use krealloc() to address

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Liu Bo
On Wed, Jun 26, 2013 at 08:38:21AM -0400, Josef Bacik wrote: > On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: > > Several users reported this crash of NULL pointer or general protection, > > the story is that we add a rbtree for speedup ulist iteration, and we > > use krealloc() to address

[XFSTESTS PATCH] btrfs: Test deduplication

2013-06-26 Thread Gabriel de Perthuis
--- The matching kernel patch is here: https://github.com/g2p/linux/tree/v3.10%2Bextent-same (rebased on 3.10, fixing a small conflict) Requires the btrfs-extent-same command: - http://permalink.gmane.org/gmane.comp.file-systems.btrfs/26579 - https://github.com/markfasheh/duperemove tests/btrf

[PROGS PATCH] Import btrfs-extent-same

2013-06-26 Thread Gabriel de Perthuis
Originally from https://github.com/markfasheh/duperemove/blob/master/btrfs-extent-same.c Signed-off-by: Gabriel de Perthuis --- .gitignore | 1 + Makefile| 2 +- btrfs-extent-same.c | 145 3 files changed, 147 insertio

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Zach Brown
> Please tell me when you are done with the full review. And please > also stop the bikeshedding. I won't commit to a full review, and I won't try and guess which comments you would choose to dismiss as bikeshedding. I'm free to share what occurs to me and you're free to tell me to go jump in a l

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Eric Sandeen
On 6/26/13 5:47 PM, Stefan Behrens wrote: > On 06/26/2013 21:55, Zach Brown wrote: >>> +if (!uuid_root) { >>> +WARN_ON_ONCE(1); >>> +ret = -ENOENT; >>> +goto out; >>> +} >> >> WARN_ON_ONCE specifically returns the condition so that you can write: >> >> if (WARN_O

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Stefan Behrens
On 06/26/2013 21:55, Zach Brown wrote: + if (!uuid_root) { + WARN_ON_ONCE(1); + ret = -ENOENT; + goto out; + } WARN_ON_ONCE specifically returns the condition so that you can write: if (WARN_ON_ONCE(!uuid_root)) { re

some feedbacks seen on btrfs

2013-06-26 Thread Roger Pack
First off, thanks for an awesome file system, it is working well for my purposes of compressing a filesystem on a small VPS. Woot! I thought I'd call out a few things (in the hopes of spurring improvements) I'd seen about btrfs (in case they weren't common knowledge...): http://linux.slashdot.o

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
> ptrdiff_t diff = new_nodes - old; > ulist->root.rb_node += diff; > for (i = 0; i < ulist->nnodes; i++) { > ulist->nodes[i].rb_node.rb_left += diff; > ulist->nodes[i].rb_node.rb_left += diff; > } (

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Zach Brown
On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: > Several users reported this crash of NULL pointer or general protection, > the story is that we add a rbtree for speedup ulist iteration, and we > use krealloc() to address ulist growth, and krealloc() use memcpy to copy > old data to new me

Re: [PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Zach Brown
> + if (!uuid_root) { > + WARN_ON_ONCE(1); > + ret = -ENOENT; > + goto out; > + } WARN_ON_ONCE specifically returns the condition so that you can write: if (WARN_ON_ONCE(!uuid_root)) { ret = -ENOENT; goto out;

Re: [PATCH] xfstests: check if the scratch dev pool is mounted in _require_scratch

2013-06-26 Thread Josef Bacik
On Wed, Jun 26, 2013 at 12:18:04PM -0700, Zach Brown wrote: > > $SCRATCH_DEV, so anybody who wants to just use the scratch mnt with the > > scratch > > dev will fail to work because we never unmount the scratch mount. > > Yeah, this is annoying. > > > Fix this by > > checking to see if the scrat

Re: [PATCH] xfstests: check if the scratch dev pool is mounted in _require_scratch

2013-06-26 Thread Zach Brown
> $SCRATCH_DEV, so anybody who wants to just use the scratch mnt with the > scratch > dev will fail to work because we never unmount the scratch mount. Yeah, this is annoying. > Fix this by > checking to see if the scratch dev pool is mounted at scratch mnt and unmount > it > so we can run our

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-26 Thread Alex Lyakas
Hi Miao, On Mon, Jun 17, 2013 at 4:51 AM, Miao Xie wrote: > On sun, 16 Jun 2013 13:38:42 +0300, Alex Lyakas wrote: >> Hi Miao, >> >> On Thu, Jun 13, 2013 at 6:08 AM, Miao Xie wrote: >>> On wed, 12 Jun 2013 23:11:02 +0300, Alex Lyakas wrote: I reviewed the code starting from: 69aef

[PATCH v2 5/5] Btrfs-progs: Validate super block checksum

2013-06-26 Thread Filipe David Borba Manana
After finding a super block in a device also validate its checksum. This validation is done in the kernel but it was missing in btrfs-progs. The function btrfs_check_super_csum() is imported from the file fs/btrfs/disk-io.c in the kernel source tree. v2: When finding the super block for a device

[PATCH v2 1/5] Btrfs-progs: fix closing of devices

2013-06-26 Thread Filipe David Borba Manana
If a device could not be opened in volumes.c:read_one_dev(), a btrfs_device instance was allocated and added to the list of devices of the fs - however this device instance had its fd, name and label fields not initialized. This is problematic in disk-io.c:close_all_devices() as it tried to sync, f

lz4 status?

2013-06-26 Thread Roger Pack
Any update on the "unmerged" lz4 patches? Have they been merged? Just wondering (and +1'ing my support, obviously). Thank you. -roger- -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.

[PATCH] xfstests: fix btrfs/264 to not use the scratch dev pool

2013-06-26 Thread Josef Bacik
This test doesn't need the scratch dev pool and it also doesn't call _require_scratch_dev_pool, so just kick out the scratch dev pool part of the test. Thanks, Signed-off-by: Josef Bacik --- tests/btrfs/264 |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/tests/btrfs/26

Re: [BULK] Re: [PATCH] xfstests: unmount scratch mnt in test 307

2013-06-26 Thread Josef Bacik
On Fri, May 03, 2013 at 07:27:21PM -0400, Dave Chinner wrote: > On Fri, May 03, 2013 at 03:15:01PM -0500, Eric Sandeen wrote: > > On 5/3/13 3:11 PM, Josef Bacik wrote: > > > So if you have a mount command that doesn't use /etc/mtab then it will > > > spit out > > > a different device for the mount

Re: Filesystem

2013-06-26 Thread Rodrigo Dias Cruz
Alexander Skwar gmail.com> writes: > > Hi > > On Mon, Jun 17, 2013 at 11:43 PM, Alexander Skwar > gmail.com> wrote: > > Hello Josef > > > > On Mon, Jun 17, 2013 at 11:21 PM, Josef Bacik fusionio.com> wrote: > > > >> Pull down my tree > >> > >> git://github.com/josefbacik/btrfs-progs.git > >>

[PATCH] xfstests: check if the scratch dev pool is mounted in _require_scratch

2013-06-26 Thread Josef Bacik
If you use the SCRATCH_DEV_POOL for btrfs you will end up with the command line like this mkfs.btrfs $SCRATCH_DEV_POOL $SCRATCH_DEV and btrfs does this thing where it makes the lowest valued device id show up in /proc/mounts no matter which device you specify at the mount command. So in this cas

[PATCH v4 2/5] Btrfs-progs: add UUID tree lookup methods

2013-06-26 Thread Stefan Behrens
This commit adds UUID tree lookup methods that make use of the search ioctl. The code is based on the kernel code. Signed-off-by: Stefan Behrens --- Makefile| 5 +-- ctree.h | 5 +++ uuid-tree.c | 104 3 files changed, 112

[PATCH v4 4/5] Btrfs-progs: add uuid_tree_gen field to btrfs-show-super

2013-06-26 Thread Stefan Behrens
Signed-off-by: Stefan Behrens --- btrfs-show-super.c | 2 ++ ctree.h| 5 - 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/btrfs-show-super.c b/btrfs-show-super.c index f587f10..c815469 100644 --- a/btrfs-show-super.c +++ b/btrfs-show-super.c @@ -247,6 +247,8 @@ stat

[PATCH v4 3/5] Btrfs-progs: use UUID tree for send/receive

2013-06-26 Thread Stefan Behrens
This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start fast and are independent on the number of subvolules/snapshot that exist. Before this commit, mapping UUIDs to subvolum

[PATCH v4 0/5] Btrfs-progs: speedup btrfs send/receive

2013-06-26 Thread Stefan Behrens
The addressed issue is that Btrfs send / receive does not work as it is today when a high number of subvolumes exist. This commit changes the btrfs send/receive commands to use the UUID tree to map UUIDs to subvolumes, and to use the root tree to map subvolume IDs to paths. Now these tools start f

[PATCH v4 5/5] Btrfs-progs: add option to btrfs-debug-tree to print uuid tree only

2013-06-26 Thread Stefan Behrens
Signed-off-by: Stefan Behrens --- btrfs-debug-tree.c | 31 ++- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/btrfs-debug-tree.c b/btrfs-debug-tree.c index bae7f94..c0eae9b 100644 --- a/btrfs-debug-tree.c +++ b/btrfs-debug-tree.c @@ -30,13 +30,14 @@

[PATCH v4 1/5] Btrfs-progs: Support UUID tree and UUID items in btrfs-debug-tree

2013-06-26 Thread Stefan Behrens
Support printing these things. Signed-off-by: Stefan Behrens --- ctree.h | 15 +++ print-tree.c | 46 +++--- 2 files changed, 58 insertions(+), 3 deletions(-) diff --git a/ctree.h b/ctree.h index 3fe14b0..f959573 100644 --- a/ctree.h +++

[PATCH v6 0/8] Btrfs: introduce a tree for UUID to subvol ID mapping

2013-06-26 Thread Stefan Behrens
Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single subvolume if 10,000 subvolumes exist. But even linear effort would be too

[PATCH v6 8/8] Btrfs: add mount option to force UUID tree checking

2013-06-26 Thread Stefan Behrens
This should never be needed, but since all functions are there to check and rebuild the UUID tree, a mount option is added that allows to force this check and rebuild procedure. Signed-off-by: Stefan Behrens --- fs/btrfs/ctree.h | 1 + fs/btrfs/disk-io.c | 3 ++- fs/btrfs/super.c | 8 +++

[PATCH v6 1/8] Btrfs: introduce a tree for items that map UUIDs to something

2013-06-26 Thread Stefan Behrens
Mapping UUIDs to subvolume IDs is an operation with a high effort today. Today, the algorithm even has quadratic effort (based on the number of existing subvolumes), which means, that it takes minutes to send/receive a single subvolume if 10,000 subvolumes exist. But even linear effort would be too

[PATCH v6 7/8] Btrfs: check UUID tree during mount if required

2013-06-26 Thread Stefan Behrens
If the filesystem was mounted with an old kernel that was not aware of the UUID tree, this is detected by looking at the uuid_tree_generation field of the superblock (similar to how the free space cache is doing it). If a mismatch is detected at mount time, a thread is started that does two things:

[PATCH v6 5/8] Btrfs: fill UUID tree initially

2013-06-26 Thread Stefan Behrens
When the UUID tree is initially created, a task is spawned that walks through the root tree. For each found subvolume root_item, the uuid and received_uuid entries in the UUID tree are added. This is such a quick operation so that in case somebody wants to unmount the filesystem while the task is s

[PATCH v6 3/8] Btrfs: create UUID tree if required

2013-06-26 Thread Stefan Behrens
This tree is not created by mkfs.btrfs. Therefore when a filesystem is mounted writable and the UUID tree does not exist, this tree is created if required. The tree is also added to the fs_info structure and initialized, but this commit does not yet read or write UUID tree elements. Signed-off-by:

[PATCH v6 6/8] Btrfs: introduce uuid-tree-gen field

2013-06-26 Thread Stefan Behrens
In order to be able to detect the case that a filesystem is mounted with an old kernel, add a uuid-tree-gen field like the free space cache is doing it. It is part of the super block and written with each commit. Old kernels do not know this field and don't update it. Signed-off-by: Stefan Behrens

[PATCH v6 4/8] Btrfs: maintain subvolume items in the UUID tree

2013-06-26 Thread Stefan Behrens
When a new subvolume or snapshot is created, a new UUID item is added to the UUID tree. Such items are removed when the subvolume is deleted. The ioctl to set the received subvolume UUID is also touched and will now also add this received UUID into the UUID tree together with the ID of the subvolum

[PATCH v6 2/8] Btrfs: support printing UUID tree elements

2013-06-26 Thread Stefan Behrens
This commit adds support to print UUID tree elements to print-tree.c. Signed-off-by: Stefan Behrens --- fs/btrfs/print-tree.c | 25 + 1 file changed, 25 insertions(+) diff --git a/fs/btrfs/print-tree.c b/fs/btrfs/print-tree.c index dc0024f..a8a1e4d 100644 --- a/fs/btrfs/

Btrfs segfault while trying to reproduce defrag bug

2013-06-26 Thread Ondřej Kunc
Hi Josef, I was trying to reproduce defrag bug to create testcase, which can check if patch https://patchwork.kernel.org/patch/2782621/ helps to mitigiate this bug https://bugzilla.kernel.org/show_bug.cgi?id=59451. But I think, that I discovered another bug, or this is the cause of it, because th

Re: question about transaction-abort-related commits

2013-06-26 Thread Alex Lyakas
Hi Josef, Can you please help me with another question. I am looking at your patch: Btrfs: fix chunk allocation error handling https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=0448748849ef7c593be40e2c1404f7974bd3aac6 Here you changed the order of btrfs_make_block_group()

Re: [PATCH] Btrfs: fix crash regarding to ulist_add_merge

2013-06-26 Thread Josef Bacik
On Wed, Jun 26, 2013 at 12:02:51PM +0800, Liu Bo wrote: > Several users reported this crash of NULL pointer or general protection, > the story is that we add a rbtree for speedup ulist iteration, and we > use krealloc() to address ulist growth, and krealloc() use memcpy to copy > old data to new me

Re: Btrfs lockup during defrag and removing device

2013-06-26 Thread Ondřej Kunc
Hi Josef, I can confirm, that I'm not able to crash it when I didn't run defrag during other operations. So as you wish, I will place "me too" in bugzilla, as I see, there is a patch available, so I will test it asap (just need to patch&compile kernel, because I was testing debian package before).

Re: btrfs send/receive issues

2013-06-26 Thread Miguel Negrão
Hi Josef, Mon, 2013-06-24 às 12:21 -0400, Josef Bacik escreveu: > I'm not sure, I just did it and it worked fine for me. Could you file a > bugzilla at bugzilla.kernel.org and make sure the component is set to btrfs? > Also could you do the send to a file, and then receive from the file so we can