Re: [PATCH v2 1/4] btrfs: Remove userspace transaction ioctls

2018-02-05 Thread Sage Weil
On Mon, 5 Feb 2018, David Sterba wrote: > On Mon, Feb 05, 2018 at 05:52:52PM +0900, Wang Shilong wrote: > >These ioctl are originally introduced by Sage Weil for Ceph use? > > Not sure whether it still useful, Cc Sage just in case. > > We have checked that the ioct

[PATCH] Btrfs: fix race in WAIT_SYNC ioctl

2014-09-26 Thread Sage Weil
EINVAL to the caller. This has been observed occasionally by ceph-osd (which uses this ioctl heavily). Fix by rechecking whether the provided transid = last_trans_committed after the search fails, and if so return 0. Signed-off-by: Sage Weil s...@redhat.com --- fs/btrfs/transaction.c | 12

warn at fs/btrfs/extent-tree.c:5748 __btrfs_free_extent+0x9ce/0xa20

2014-03-11 Thread Sage Weil
Hey, Is this something you guys have seen before? This is from v3.13-rc2. kernel: [49432.696440] WARNING: CPU: 3 PID: 26411 at /srv/autobuild-ceph/gitbuilder.git/build/fs/btrfs/extent-tree.c:5748 __btrfs_free_extent+0x9ce/0xa20 [btrfs]() kernel: [49432.710128] Modules linked in: arc4(F)

Re: transaction commit deadlock on current rc

2013-10-18 Thread Sage Weil
On Fri, 18 Oct 2013, Josef Bacik wrote: On Thu, Oct 17, 2013 at 12:56:14PM -0700, Sage Weil wrote: Hey, I'm seeing the deadlock below under a ceph-osd workload. There may be a subtle problem with the async transaction sequence (since nobody but ceph uses that that I know

Re: transaction commit deadlock on current rc

2013-10-18 Thread Sage Weil
On Fri, 18 Oct 2013, Chris Mason wrote: Quoting Sage Weil (2013-10-18 11:42:28) On Fri, 18 Oct 2013, Josef Bacik wrote: On Thu, Oct 17, 2013 at 12:56:14PM -0700, Sage Weil wrote: Hey, I'm seeing the deadlock below under a ceph-osd workload. There may be a subtle

transaction commit deadlock on current rc

2013-10-17 Thread Sage Weil
Hey, I'm seeing the deadlock below under a ceph-osd workload. There may be a subtle problem with the async transaction sequence (since nobody but ceph uses that that I know of), but not obvious to me why create_pending_snapshots would get stuck on btrfs_tree_lock... [ 602.217383] INFO: task

Re: hang on 3.9, 3.10-rc5

2013-08-09 Thread Sage Weil
Hi Chris, On Thu, 20 Jun 2013, Chris Mason wrote: Quoting Sage Weil (2013-06-20 17:56:19) On Wed, 19 Jun 2013, Sage Weil wrote: Hi Chris, On Tue, 18 Jun 2013, Chris Mason wrote: [...] Very long way of saying I think we're one release_path short. Sage, I haven't tested

Re: hang on 3.9, 3.10-rc5

2013-06-20 Thread Sage Weil
On Wed, 19 Jun 2013, Sage Weil wrote: Hi Chris, On Tue, 18 Jun 2013, Chris Mason wrote: [...] Very long way of saying I think we're one release_path short. Sage, I haven't tested this at all yet, I was hoping to trigger it first. diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree

Re: hang on 3.9, 3.10-rc5

2013-06-20 Thread Sage Weil
On Thu, 20 Jun 2013, Chris Mason wrote: Quoting Sage Weil (2013-06-20 17:56:19) On Wed, 19 Jun 2013, Sage Weil wrote: Hi Chris, On Tue, 18 Jun 2013, Chris Mason wrote: [...] Very long way of saying I think we're one release_path short. Sage, I haven't tested this at all

Re: hang on 3.9, 3.10-rc5

2013-06-20 Thread Sage Weil
On Thu, 20 Jun 2013, Chris Mason wrote: Quoting Sage Weil (2013-06-20 21:00:21) On Thu, 20 Jun 2013, Chris Mason wrote: Awesome, thanks for getting the traces for us. Looks like this one has been around since v3.7, so I'm not going to try and sneak it into the 3.10 final. I'll have

Re: hang on 3.9, 3.10-rc5

2013-06-19 Thread Sage Weil
Hi Chris, On Tue, 18 Jun 2013, Chris Mason wrote: [...] Very long way of saying I think we're one release_path short. Sage, I haven't tested this at all yet, I was hoping to trigger it first. diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index c276ac9..c1954b3 100644 ---

Re: hang on 3.9, 3.10-rc5

2013-06-18 Thread Sage Weil
On Wed, 12 Jun 2013, Sage Weil wrote: On Tue, 11 Jun 2013, Chris Mason wrote: Quoting Sage Weil (2013-06-11 11:43:30) I'm also seeing this hang regularly with both 3.9 and 3.10-rc5. Is this is a known problem? In this case there is no powercycling; just a regular ceph-osd workload

hang on 3.9, 3.10-rc5

2013-06-11 Thread Sage Weil
I'm also seeing this hang regularly with both 3.9 and 3.10-rc5. Is this is a known problem? In this case there is no powercycling; just a regular ceph-osd workload. Thanks- sage [ 2885.479116] INFO: task kworker/u64:1:28713 blocked for more than 120 seconds. [ 2885.486277] echo 0

Re: v3.9 bug at /fs/btrfs/free-space-cache.c:1567 after powercycle

2013-06-05 Thread Sage Weil
On Wed, 5 Jun 2013, Josef Bacik wrote: On Tue, Jun 04, 2013 at 09:59:05PM -0600, Sage Weil wrote: Hi- I'm pretty reliably triggering the following bug after powercycling an active btrfs + ceph workload and then trying to remount. Is this a known issue? Yeah sorry it's fixed

v3.9 bug at /fs/btrfs/free-space-cache.c:1567 after powercycle

2013-06-04 Thread Sage Weil
Hi- I'm pretty reliably triggering the following bug after powercycling an active btrfs + ceph workload and then trying to remount. Is this a known issue? sage 2013-06-04T18:54:28.532988-07:00 plana71 kernel: [ 39.311120] [ cut here ]

Re: corruption of active mmapped files in btrfs snapshots

2013-03-22 Thread Sage Weil
On Fri, 22 Mar 2013, Chris Mason wrote: Quoting Alexandre Oliva (2013-03-22 10:17:30) On Mar 22, 2013, Chris Mason clma...@fusionio.com wrote: Are you using compression in btrfs or just in leveldb? btrfs lzo compression. Perfect, I'll focus on that part of things. I'd like

Re: corruption of active mmapped files in btrfs snapshots

2013-03-19 Thread Sage Weil
On Tue, 19 Mar 2013, Chris Mason wrote: Quoting Alexandre Oliva (2013-03-19 01:20:10) On Mar 18, 2013, Chris Mason chris.ma...@fusionio.com wrote: A few questions. Does leveldb use O_DIRECT and mmap together? No, it doesn't use O_DIRECT at all. Its I/O interface is very

Re: [PATCH] fs: encode_fh: return FILEID_INVALID if invalid fid_type

2013-02-11 Thread Sage Weil
Acked-by: Sage Weil s...@inktank.com On Mon, 11 Feb 2013, Namjae Jeon wrote: From: Namjae Jeon namjae.j...@samsung.com This patch is a follow up on below patch: [PATCH] exportfs: add FILEID_INVALID to indicate invalid fid_type commit: 216b6cbdcbd86b1db0754d58886b466ae31f5a63 Signed

Re: corrupted file size on inline extent conversion?

2013-02-04 Thread Sage Weil
On Wed, 30 Jan 2013, Josef Bacik wrote: On Wed, Jan 30, 2013 at 11:30:49AM -0700, Mike Lowe wrote: I've been running rsync against a rbd device backed by btrfs filesystems that are about 11% full for about 45 minutes before I checked and noticed the printk message. That was the first go

corrupted file size on inline extent conversion?

2013-01-28 Thread Sage Weil
A ceph user observed a incorrect i_size on btrfs. The pattern looks like this: - some writes at low file offsets - a write to 4185600 len 8704 (i_size should be 4MB) - more writes to low offsets - a write to 4181504 len 4096 (abutts the write above) - a bit of time goes by... - stat returns

Re: [PATCH] Btrfs: move the sb_end_intwrite until after the throttle logic

2012-09-09 Thread Sage Weil
after this logic makes the lockdep go away. Thanks, Signed-off-by: Josef Bacik jba...@fusionio.com This appears to have done the trick (at least so far). I'm running it on top of -rc5 and the 3 patches I posted on Aug 30th. Tested-by: Sage Weil s...@inktank.com Thanks! sage --- fs

[PATCH 0/3] lockdep fixups for sb_interval#2 vs async transaction commit

2012-08-30 Thread Sage Weil
These three patches (in combination with btrfs-next) fix things up for me! Sage Weil (3): Btrfs: pass lockdep rwsem metadata to async commit transaction Btrfs: set journal_info in async trans commit worker Btrfs: do not take cleanup_work_sem in btrfs_run_delayed_iputs() fs/btrfs/inode.c

[PATCH 2/3] Btrfs: set journal_info in async trans commit worker

2012-08-30 Thread Sage Weil
We expect current-journal_info to point to the trans handle we are committing. Signed-off-by: Sage Weil s...@inktank.com --- fs/btrfs/transaction.c |2 ++ 1 file changed, 2 insertions(+) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index efc41a5..f910a26 100644 --- a/fs

[PATCH 1/3] Btrfs: pass lockdep rwsem metadata to async commit transaction

2012-08-30 Thread Sage Weil
/xfs_aops.c. Signed-off-by: Sage Weil s...@inktank.com --- fs/btrfs/transaction.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 17be3de..efc41a5 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c

[PATCH 3/3] Btrfs: do not take cleanup_work_sem in btrfs_run_delayed_iputs()

2012-08-30 Thread Sage Weil
] kernel_thread_helper+0x4/0x10 [ 607.202472] [81635430] ? retint_restore_args+0x13/0x13 [ 607.235884] [81079140] ? flush_kthread_work+0x1a0/0x1a0 [ 607.268731] [8163e740] ? gs_change+0x13/0x13 Signed-off-by: Sage Weil s...@inktank.com --- fs/btrfs/inode.c |2 -- 1

[PATCH] Btrfs: pass lockdep rwsem metadata to async commit transaction

2012-08-24 Thread Sage Weil
-by: Sage Weil s...@inktank.com --- fs/btrfs/transaction.c | 16 1 files changed, 16 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c index 17be3de..efc41a5 100644 --- a/fs/btrfs/transaction.c +++ b/fs/btrfs/transaction.c @@ -1228,6

Re: [PATCH] Btrfs: pass lockdep rwsem metadata to async commit transaction

2012-08-24 Thread Sage Weil
] [8163e740] ? gs_change+0x13/0x13 On Fri, 24 Aug 2012, Sage Weil wrote: The freeze rwsem is taken by sb_start_intwrite() and dropped during the commit_ or end_transaction(). In the async case, that happens in a worker thread. Tell lockdep the calling thread is releasing ownership

Re: btrfs call trace

2012-06-25 Thread Sage Weil
Hi Stefan, I haven't seen this one. The async commit stuff is mine, but there haven't been problems with it for a year or more. This is a recent kernel, I assume? Can you dump the other tasks? Something is preventing the commit from completing. [adding linux-btrfs to cc] sage On Fri,

Re: Ceph on btrfs 3.4rc

2012-04-24 Thread Sage Weil
On Tue, 24 Apr 2012, Josef Bacik wrote: On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner wrote: After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current for-linux-min branch and big metadata is much better. The only problem (?) I'm still

[LSF/MM TOPIC] COWing writeback pages

2012-02-10 Thread Sage Weil
Hi everyone, The takeaway from the 'stable pages' discussions in the last few workshops was that pages under writeback should remain locked so that subsequent writers don't touch them while they are en route to the disk. This prevents bad checksums and DIF/DIX type failures (whereas

Re: [LSF/MM TOPIC] COWing writeback pages

2012-02-10 Thread Sage Weil
On Fri, 10 Feb 2012, Josef Bacik wrote: On Fri, Feb 10, 2012 at 11:25:27AM -0800, Sage Weil wrote: Hi everyone, The takeaway from the 'stable pages' discussions in the last few workshops was that pages under writeback should remain locked so that subsequent writers don't touch them

Re: WARNING: at fs/btrfs/inode.c:2222

2012-01-27 Thread Sage Weil
On Fri, 27 Jan 2012, Chris Mason wrote: On Fri, Jan 27, 2012 at 09:31:46AM -0800, Sage Weil wrote: Has anyone see this one before? We've been hitting it sporatically for a while now, although recently we've been able to trigger pretty easily. The result is EINVAL from the snap create

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-26 Thread Sage Weil
On Wed, 26 Oct 2011, Christian Brunner wrote: 2011/10/26 Sage Weil s...@newdream.net: On Wed, 26 Oct 2011, Christian Brunner wrote: Christian, have you tweaked those settings in your ceph.conf?  It would be something like 'journal dio = false'.  If not, can you verify

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Sage Weil
On Tue, 25 Oct 2011, Christoph Hellwig wrote: On Mon, Oct 24, 2011 at 10:06:49AM -0700, Sage Weil wrote: - When I run ceph with btrfs snaps disabled, the situation is getting slightly better. I can run an OSD for about 3 days without problems, but then again the load increases. This time

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Sage Weil
On Tue, 25 Oct 2011, Josef Bacik wrote: At this point it seems like the biggest problem with latency in ceph-osd is not related to btrfs, the latency seems to all be from the fact that ceph-osd is fsyncing a block dev for whatever reason. There is one place where we sync_file_range() on the

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Sage Weil
On Tue, 25 Oct 2011, Christian Brunner wrote: 2011/10/25 Sage Weil s...@newdream.net: On Tue, 25 Oct 2011, Josef Bacik wrote: At this point it seems like the biggest problem with latency in ceph-osd is not related to btrfs, the latency seems to all be from the fact that ceph-osd

ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-24 Thread Sage Weil
beyond just the Ceph related issues. Progress is slow, but we are working on it! sage Kind regards, Christian 2011/10/24 Sage Weil s...@newdream.net: Although running on ext4, xfs, or whatever other non-btrfs you want mostly works, there are a few important remaining issues: 1- ext4

Re: OSD: no current directory

2011-10-11 Thread Sage Weil
On Tue, 11 Oct 2011, Christian Brunner wrote: 2011/10/11 Sage Weil s...@newdream.net: On Tue, 11 Oct 2011, Christian Brunner wrote: Maybe this one is easier: One of our OSDs isn't starting, because ther is no current directory. What I have are three snap directories. total 0 -rw-r

Re: [GIT PULL] Btrfs fixes

2011-09-20 Thread Sage Weil
Hi Chris- This pull misses the clone reservation fix again... :) http://www.spinics.net/lists/linux-btrfs/msg11826.html Thanks! sage On Mon, 19 Sep 2011, Chris Mason wrote: Hi everyone, The for-linus branch of the btrfs tree on github: Head commit:

Re: [PATCH] Btrfs: fix pages truncation in btrfs_ioctl_clone()

2011-09-18 Thread Sage Weil
On Sun, 18 Sep 2011, Chris Mason wrote: Excerpts from Sage Weil's message of 2011-09-16 12:39:06 -0400: On Fri, 16 Sep 2011, Li Zefan wrote: It's a bug in commit f81c9cdc567cd3160ff9e64868d9a1a7ee226480 (Btrfs: truncate pages from clone ioctl target range) We should pass the dest

Re: [PATCH] Btrfs: fix pages truncation in btrfs_ioctl_clone()

2011-09-16 Thread Sage Weil
On Fri, 16 Sep 2011, Li Zefan wrote: It's a bug in commit f81c9cdc567cd3160ff9e64868d9a1a7ee226480 (Btrfs: truncate pages from clone ioctl target range) We should pass the dest range to the truncate function, but not the src range. Sigh... yes. Also move the function before locking extent

Re: WARNING: at fs/btrfs/inode.c:2193 btrfs_orphan_commit_root+0xb0/0xc0 [btrfs]()

2011-09-15 Thread Sage Weil
On Tue, 13 Sep 2011, Liu Bo wrote: On 09/11/2011 05:47 AM, Martin Mailand wrote: Hi I am hitting this Warning reproducible, the workload is a ceph osd, kernel ist 3.1.0-rc5. Have posted a patch for this: http://marc.info/?l=linux-btrfsm=131547325515336w=2 We're still seeing this

Re: WARNING: at fs/btrfs/inode.c:2193 btrfs_orphan_commit_root+0xb0/0xc0 [btrfs]()

2011-09-15 Thread Sage Weil
On Thu, 15 Sep 2011, David Sterba wrote: On Thu, Sep 15, 2011 at 03:50:29PM -0400, Josef Bacik wrote: We're still seeing this with -rc6, which includes 98c9942 and 65450aa. I haven't looked at the reservation code in much detail. Is there anything I can do to help track this down?

Re: [PATCH 1/2] Btrfs: don't make a file partly checksummed through file clone

2011-09-14 Thread Sage Weil
when we read the file. Disallow file clone if src and dst file have different checksum flag, so we ensure a file is completely checksummed or unchecksummed. Signed-off-by: Li Zefan l...@cn.fujitsu.com Looks good to me. Reviewed-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c |5

Re: [GIT PULL] Btrfs updates

2011-08-18 Thread Sage Weil
): Btrfs: fix uninitialized sync_pending (+1/-1) Btrfs: fix wrong free space information (+11/-5) Sage Weil (1) commits (+4/-0): Btrfs: truncate pages from clone ioctl target range Tsutomu Itoh (1) commits (+16/-10): Btrfs: forced readonly when btrfs_drop_snapshot() fails

Re: [PATCH] Btrfs: reserve sufficient space for ioctl clone

2011-08-15 Thread Sage Weil
On Tue, 16 Aug 2011, Miao Xie wrote: On tue, 9 Aug 2011 10:46:37 -0700, Sage Weil wrote: Fix a crash/BUG_ON in the clone ioctl due to insufficient reservation. We need to reserve space for: - adjusting the old extent (possibly splitting it) - adding the new extent - updating

[PATCH] Btrfs: reserve sufficient space for ioctl clone

2011-08-09 Thread Sage Weil
Fix a crash/BUG_ON in the clone ioctl due to insufficient reservation. We need to reserve space for: - adjusting the old extent (possibly splitting it) - adding the new extent - updating the inode Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c |7 ++- 1 files changed

clone ioctl bug with inline extents

2011-08-09 Thread Sage Weil
Hi all, I'm hitting a problem cloning inline extents that I haven't had much success tracking down. It's simple enough to reproduce: echo src echo 2 dst clone_range src 0 29 dst 0 cmp src dst # fails! dst is size 29 but contains 2\n\0\0\0\0... where

Re: clone ioctl bug with inline extents

2011-08-09 Thread Sage Weil
On Tue, 9 Aug 2011, Sage Weil wrote: Hi all, I'm hitting a problem cloning inline extents that I haven't had much success tracking down. It's simple enough to reproduce: echo src echo 2 dst clone_range src 0 29 dst 0 cmp src dst # fails! dst

Re: Btrfs slowdown

2011-08-08 Thread Sage Weil
Hi Christian, Are you still seeing this slowness? sage On Wed, 27 Jul 2011, Christian Brunner wrote: 2011/7/25 Chris Mason chris.ma...@oracle.com: Excerpts from Christian Brunner's message of 2011-07-25 03:54:47 -0400: Hi, we are running a ceph cluster with btrfs as it's base

Re: Btrfs slowdown

2011-07-28 Thread Sage Weil
On Thu, 28 Jul 2011, Christian Brunner wrote: When I look at the latencytop results, there is a high latency when calling btrfs_commit_transaction_async. Isn't async supposed to return immediately? It depends. That function has to block until the commit has started before returning in the

Re: 3.0-rcX BUG at fs/btrfs/ioctl.c:432 - bisected

2011-06-10 Thread Sage Weil
On Fri, 10 Jun 2011, Chris Mason wrote: Excerpts from Jim Schutt's message of 2011-06-10 13:06:22 -0400: [ two different btrfs crashes ] I think your two crashes in btrfs were from the uninit variables and those should be fixed in rc2. When I did my bisection, my criteria for

Re: 3.0-rcX BUG at fs/btrfs/ioctl.c:432 - bisected

2011-06-10 Thread Sage Weil
On Fri, 10 Jun 2011, Sage Weil wrote: On Fri, 10 Jun 2011, Chris Mason wrote: Excerpts from Jim Schutt's message of 2011-06-10 13:06:22 -0400: [ two different btrfs crashes ] I think your two crashes in btrfs were from the uninit variables and those should be fixed in rc2

Re: 3.0-rcX BUG at fs/btrfs/ioctl.c:432 - bisected

2011-06-10 Thread Sage Weil
On Fri, 10 Jun 2011, Josef Bacik wrote: On 06/10/2011 02:14 PM, Sage Weil wrote: On Fri, 10 Jun 2011, Sage Weil wrote: On Fri, 10 Jun 2011, Chris Mason wrote: Excerpts from Jim Schutt's message of 2011-06-10 13:06:22 -0400: [ two different btrfs crashes ] I think your two crashes

Re: 3.0-rcX BUG at fs/btrfs/ioctl.c:432 - bisected

2011-06-10 Thread Sage Weil
On Fri, 10 Jun 2011, Josef Bacik wrote: On 06/10/2011 02:35 PM, Sage Weil wrote: On Fri, 10 Jun 2011, Josef Bacik wrote: On 06/10/2011 02:14 PM, Sage Weil wrote: On Fri, 10 Jun 2011, Sage Weil wrote: On Fri, 10 Jun 2011, Chris Mason wrote: Excerpts from Jim Schutt's message of 2011-06

[PATCH 12/19] btrfs: remove unnecessary dentry_unhash in rmdir/rename_dir

2011-05-24 Thread Sage Weil
Btrfs has no problems with lingering references to unlinked directory inodes. CC: Chris Mason chris.ma...@oracle.com CC: linux-btrfs@vger.kernel.org Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/inode.c |5 - 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/fs

[PATCH 10/18] btrfs: remove unnecessary dentry_unhash in rmdir/rename_dir

2011-05-09 Thread Sage Weil
Btrfs has no problems with lingering references to unlinked directory inodes. CC: Chris Mason chris.ma...@oracle.com CC: linux-btrfs@vger.kernel.org Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/inode.c |5 - 1 files changed, 0 insertions(+), 5 deletions(-) diff --git a/fs

[PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Sage Weil
We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. There's ample room for further cleanup here, but this keeps the fix simple. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c | 20 +++- 1 files changed, 11

Re: [PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Sage Weil
On Thu, 9 Dec 2010, Chris Mason wrote: Excerpts from Li Zefan's message of 2010-12-09 21:11:25 -0500: Sage Weil wrote: We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. Ha, I fixed this accidentally in my patchset

Re: [PATCH 1/5] btrfs: Make async snapshot ioctl more generic

2010-11-29 Thread Sage Weil
Hi Li, On Mon, 29 Nov 2010, Li Zefan wrote: So we don't have to add new structures as we create more ioctls for snapshots. Now to create async snapshot, set BTRFS_SNAPSHOT_CREATE_ASYNC bit of vol_arg_v2-flags, and then call ioctl(BTRFS_IOCT_SNAP_CREATE_V2). Note: this changes the async

Re: [PATCH 1/2] btrfs: Check if dest_offset is block-size aligned before cloning file

2010-11-22 Thread Sage Weil
; /* do any pending delalloc/csum calc on src, one way or Looks good. Reviewed-by: Sage Weil s...@newdream.net -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: [PATCH 2/2] btrfs: Set file size correctly in file clone

2010-11-22 Thread Sage Weil
) btrfs_i_size_write(inode, endoff); Reviewed-by: Sage Weil s...@newdream.net -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

new lockdep warnings in 2.6.37-rc1

2010-11-11 Thread Sage Weil
Hey, I saw a few variations of this lockdep on about a dozen different nodes running 2.6.37-rc1+ (mainline from earlier this week). They all look to be the same issue, hit via a few different call paths. I haven't seen these before... sage [10862.369802]

Re: [PATCH 2/3] btrfs: implement 'async-snapshot' command

2010-11-02 Thread Sage Weil
On Tue, 2 Nov 2010, Goffredo Baroncelli wrote: On Monday, 01 November, 2010, Sage Weil wrote: On Sat, 30 Oct 2010, Goffredo Baroncelli wrote: On Saturday, 30 October, 2010, Sage Weil wrote: This is identical to 'snapshot', but uses the new async snapshot creation ioctl, and prints

Re: [PATCH v2] btrfs-progs: btrfs: implement 'start-sync' and 'wait-sync' commands

2010-11-02 Thread Sage Weil
On Tue, 2 Nov 2010, Goffredo Baroncelli wrote: Like the command btrfs subvol snapshot, I think that it is better to add a modifier instead of a new command. btrfs filesystem sync [--async] Sorry if I noticed this too late. But I don't see a valid reason to add another command. From a

Re: [PATCH 2/3] btrfs: implement 'async-snapshot' command

2010-11-01 Thread Sage Weil
On Sat, 30 Oct 2010, Goffredo Baroncelli wrote: On Saturday, 30 October, 2010, Sage Weil wrote: This is identical to 'snapshot', but uses the new async snapshot creation ioctl, and prints out the transid the new snapshot will be committed with. Signed-off-by: Sage Weil s

[PATCH] btrfs-progs: ioctl: add may_alias to btrfs_ioctl_search_header

2010-11-01 Thread Sage Weil
with gcc version 4.4.4 (Debian 4.4.4-6). Signed-off-by: Sage Weil s...@newdream.net --- ioctl.h |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/ioctl.h b/ioctl.h index 5a03317..4a850ff 100644 --- a/ioctl.h +++ b/ioctl.h @@ -87,7 +87,7 @@ struct btrfs_ioctl_search_header

[PATCH v2] btrfs-progs: btrfs: implement async option for snapshot command

2010-11-01 Thread Sage Weil
Add an 'async' option to the snapshot creating command, that will let you avoid waiting for a new snapshot to commit to disk. Signed-off-by: Sage Weil s...@newdream.net --- btrfs.c|9 ++--- btrfs_cmds.c | 31 +-- btrfs_cmds.h |2 +- man/btrfs

[PATCH v2] btrfs-progs: btrfs: implement 'start-sync' and 'wait-sync' commands

2010-11-01 Thread Sage Weil
the man page too. Signed-off-by: Sage Weil s...@newdream.net --- btrfs.c|9 + btrfs_cmds.c | 49 + btrfs_cmds.h |2 ++ man/btrfs.8.in | 14 ++ 4 files changed, 74 insertions(+), 0 deletions(-) diff --git

[PATCH 1/3] update ioctl.h from 2.6.37-rc1

2010-10-30 Thread Sage Weil
Signed-off-by: Sage Weil s...@newdream.net --- ioctl.h | 39 ++- 1 files changed, 30 insertions(+), 9 deletions(-) diff --git a/ioctl.h b/ioctl.h index 776d7a9..5a03317 100644 --- a/ioctl.h +++ b/ioctl.h @@ -23,13 +23,28 @@ #define BTRFS_IOCTL_MAGIC 0x94

[PATCH 2/3] btrfs: implement 'async-snapshot' command

2010-10-30 Thread Sage Weil
This is identical to 'snapshot', but uses the new async snapshot creation ioctl, and prints out the transid the new snapshot will be committed with. Signed-off-by: Sage Weil s...@newdream.net --- btrfs.c |8 +++- btrfs_cmds.c | 32 +++- btrfs_cmds.h

[PATCH 3/3] btrfs: implement 'start-sync' and 'wait-sync' commands

2010-10-30 Thread Sage Weil
-off-by: Sage Weil s...@newdream.net --- btrfs.c |9 + btrfs_cmds.c | 49 + btrfs_cmds.h |2 ++ 3 files changed, 60 insertions(+), 0 deletions(-) diff --git a/btrfs.c b/btrfs.c index c4b9a31..d45ac1f 100644 --- a/btrfs.c +++ b

Re: [PATCH 1/6] Btrfs: fix deadlock in btrfs_commit_transaction

2010-10-26 Thread Sage Weil
On Tue, 26 Oct 2010, liubo wrote: On 10/26/2010 03:07 AM, Sage Weil wrote: We calculate timeout (either 1 or MAX_SCHEDULE_TIMEOUT) based on whether num_writers 1 or should_grow at the top of the loop. Then, much much later, we wait for that timeout if either num_writers or should_grow

Re: [PATCH] Btrfs: allow subvol deletion by unprivileged user with -o user_subvol_rm_allowed

2010-10-26 Thread Sage Weil
On Tue, 26 Oct 2010, Goffredo Baroncelli wrote: inode = dentry-d_inode; + dest = BTRFS_I(inode)-root; + if (!capable(CAP_SYS_ADMIN)){ + /* +* Regular user. Only allow this with a special mount +* option, and when rmdir(2) would have been

[PATCH v2] Btrfs: fix deadlock in btrfs_commit_transaction

2010-10-26 Thread Sage Weil
()) to decrement num_writers such that we wait forever instead of for 1. Fix this by deciding how long to wait when we wait. Include a smp_mb() before checking if the waitqueue is active to ensure the num_writers is visible. Signed-off-by: Sage Weil s...@newdream.net --- v2: - add smp_mb() before

[PATCH v2] Btrfs: allow subvol deletion by unprivileged user with -o user_subvol_rm_allowed

2010-10-26 Thread Sage Weil
additionally require that the user has write+exec permission on the subvol root inode. Signed-off-by: Sage Weil s...@newdream.net --- v2: - add write+exec check on subvol root inode fs/btrfs/ctree.h |1 + fs/btrfs/ioctl.c | 115 +++-- fs/btrfs

[PATCH 0/6] Btrfs commit fixes, async subvol operations

2010-10-25 Thread Sage Weil
as I can do that, my daemon doesn't have to run as root and I'm a happy camper. :) If anybody has any questions or issues with any of this, please let me know so I can revise the patches accordingly. Thanks! sage --- Sage Weil (6): Btrfs: fix deadlock in btrfs_commit_transaction Btrfs

[PATCH 1/6] Btrfs: fix deadlock in btrfs_commit_transaction

2010-10-25 Thread Sage Weil
()) to decrement num_writers such that we wait forever instead of for 1. Fix this by deciding how long to wait when we wait. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/transaction.c | 12 1 files changed, 4 insertions(+), 8 deletions(-) diff --git a/fs/btrfs

[PATCH 2/6] Btrfs: async transaction commit

2010-10-25 Thread Sage Weil
). Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ctree.h |1 + fs/btrfs/disk-io.c |1 + fs/btrfs/transaction.c | 119 fs/btrfs/transaction.h |3 + 4 files changed, 124 insertions(+), 0 deletions(-) diff --git a/fs/btrfs

[PATCH 3/6] Btrfs: add START_SYNC, WAIT_SYNC ioctls

2010-10-25 Thread Sage Weil
before the START_SYNC reaches disk. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c | 34 +++ fs/btrfs/ioctl.h |2 + fs/btrfs/transaction.c | 52 fs/btrfs/transaction.h |1 + 4 files

[PATCH 5/6] Btrfs: make SNAP_DESTROY async

2010-10-25 Thread Sage Weil
-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 679b8a8..9cbda86 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c @@ -1365,7 +1365,7 @@ static noinline int

[PATCH 6/6] Btrfs: allow subvol deletion by owner

2010-10-25 Thread Sage Weil
is sticky. It is less strict than 'rm -rf subvol', which would require scanning the entire subvol to ensure all content is deletable. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c| 27 --- security/security.c |1 + 2 files changed, 25

Re: [PATCH 0/6] Btrfs commit fixes, async subvol operations

2010-10-25 Thread Sage Weil
On Mon, 25 Oct 2010, Chris Mason wrote: These all look good to me and I'm pulling them in. Great, thanks! The last item is a change to SNAP_DESTROY to allow deletion of a snapshot when the user owns the subvol's root inode and the parent directory permissions are such that we would have

[PATCH] Btrfs: allow subvol deletion by unprivileged user with -o user_subvol_rm_allowed

2010-10-25 Thread Sage Weil
). Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ctree.h |1 + fs/btrfs/ioctl.c | 109 +++-- fs/btrfs/super.c |6 ++- 3 files changed, 110 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index 5ac2bca

Re: [PATCH 0/6] Btrfs commit fixes, async subvol operations

2010-10-25 Thread Sage Weil
On Mon, 25 Oct 2010, Chris Mason wrote: Yes, lets duplicate the vfs checks. Christoph just sat bolt upright in whatever ski lift he's currently riding. :) This version is based on Goffredo's patch, but requires the user_subvol_rm_allowed mount option, and will proceed even if the subvol is

[PATCH 0/3] clone ioctl fixes

2010-10-19 Thread Sage Weil
Hi Chris, These are a few critical bug fixes for the clone ioctl. We've been hitting these on a number of different systems, so getting these into 2.6.37 would be much appreciated. Thanks! sage -- Sage Weil (1): Btrfs: fix lockdep warning on clone ioctl Yehuda Sadeh (2): Btrfs: fix

[PATCH 1/3] Btrfs: fix delalloc checks in clone ioctl

2010-10-19 Thread Sage Weil
From: Yehuda Sadeh yeh...@hq.newdream.net The lookup_first_ordered_extent() was done on the wrong inode, and the -delalloc_bytes test was wrong, as the following btrfs_wait_ordered_range() would only invoke a range write and wouldn't write the entire file data range. Also, a bad parameter was

[PATCH 2/3] Btrfs: fix clone ioctl where range is adjacent to extent

2010-10-19 Thread Sage Weil
From: Yehuda Sadeh yeh...@hq.newdream.net We had an edge case issue where the requested range was just following an existing extent. Instead of skipping to the next extent, we used the previous one which lead to having zero sized extents. Signed-off-by: Yehuda Sadeh yeh...@hq.newdream.net ---

[PATCH 3/3] Btrfs: fix lockdep warning on clone ioctl

2010-10-19 Thread Sage Weil
I'm no lockdep expert, but this appears to make the lockdep warning go away for the i_mutex locking in the clone ioctl. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs

Re: [RFC] Allow to exec btrfs subvolume delete by a non root user

2010-10-19 Thread Sage Weil
On Mon, 18 Oct 2010, Goffredo Baroncelli wrote: Hi all like my previous patch, this one allow to remove a subvolume by an ordinary user. Instead of adding this capability to the rmdir(2) syscall, I update the BTRFS_IOC_SNAP_DESTROY ioctl, relaxing the rules to be execute. The checks are

Re: Updating the wiki pages adding btrfs command

2010-09-15 Thread Sage Weil
On Wed, 15 Sep 2010, Chris Ball wrote: Hi, Or make --sync the default behavior. This is probably what most people are expecting anyway (similar to how standard filesystem commands like rm work). Add an --aysnc option for those that only care about knowing when the

Re: Updating the wiki pages adding btrfs command

2010-09-15 Thread Sage Weil
On Wed, 15 Sep 2010, Sage Weil wrote: For subvolume removal, racing doesn't seem like an issue. Why not just do # btrfs subvolume delete /path/to/subvolume1 # btrfs subvolume delete /path/to/subvolume2 # ... # sync ? Nevermind, read the whole thread this time. I'll defer to others

[PATCH] Btrfs: allow subvol deletion by owner

2010-08-03 Thread Sage Weil
is sticky. It is less strict than 'rm -rf subvol', which would require scanning the entire subvol to ensure all content is deletable. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c | 25 ++--- 1 files changed, 22 insertions(+), 3 deletions(-) diff --git a/fs/btrfs

Re: reflinked file size incorrect

2010-06-12 Thread Sage Weil
On Sat, 12 Jun 2010, Jim Ursetto wrote: Both `cp --reflink` and `bcp` sometimes round the file size up to the next 4k boundary, with the balance consisting of null bytes. At first glance this behavior occurs for source file size 3916 bytes. I have tried this with stock btrfs from kernel

[PATCH] Btrfs: fix CLONE ioctl destination file size expansion to block boundary

2010-06-12 Thread Sage Weil
the originally requested ending offset. This bug was introduced by a22285a6 (2.6.35-rc1). Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c | 16 +--- 1 files changed, 13 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 4cdb98c

Re: [Fwd: Re: Linking two files together][RFC]

2010-06-09 Thread Sage Weil
On Wed, 9 Jun 2010, Roberto Ragusa wrote: I hope that ideas about btrfs are not off-topic for this mailing list. The forwarded message below was written by me on fedora-users. The thread is about the ability to link two files in a manner similar to cat 1 2 3 rm 1 2 while avoiding any data

[PATCH] Btrfs: avoid BUG when dropping root and reference in same transaction

2010-05-17 Thread Sage Weil
it is not an error to drop the root reference and the root in the same transaction, just drop the BUG_ON() in btrfs_del_root(). Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/root-tree.c |3 --- 1 files changed, 0 insertions(+), 3 deletions(-) diff --git a/fs/btrfs/root-tree.c b/fs/btrfs/root-tree.c

[PATCH] Btrfs: fix lockdep warning on clone ioctl

2010-04-07 Thread Sage Weil
I'm no lockdep expert, but this appears to make the lockdep warning go away for the i_mutex locking in the clone ioctl. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c |8 1 files changed, 4 insertions(+), 4 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs

[PATCH 0/5] asynchronous commit, snapshot ponies

2010-03-22 Thread Sage Weil
proposals. It's simpler, does the job, and the primitives should be useful for other applications. Let me know what you think! I'm doing more testing this week, but so far I haven't seen any problems with these changes. Thanks- sage Sage Weil (5): Btrfs: async transaction commit Btrfs: add

[PATCH 1/5] Btrfs: async transaction commit

2010-03-22 Thread Sage Weil
). Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ctree.h |1 + fs/btrfs/disk-io.c |1 + fs/btrfs/transaction.c | 119 fs/btrfs/transaction.h |3 + 4 files changed, 124 insertions(+), 0 deletions(-) diff --git a/fs/btrfs

  1   2   >