Purpose of block reserves

2010-12-09 Thread Arne Jansen
Hi, while reading btrfs source code, I try to make sense of the various uses of block reserves. The working theory is as follows: 1. Every operation has to reserve upfront every single byte it needs to complete its operation fully. 2. If an operation cannot determine how much space it will

[PATCH v2 0/5] Btrfs: Readonly snapshots

2010-12-09 Thread Li Zefan
This patchset adds readonly-snapshots support. You can create a readonly snapshot, and you can also set a snapshot (subvolume also) readonly/writable on the fly. A few readonly checks are added in setattr, permission, remove_xattr and set_xattr callbacks, as well as in some ioctls. The patchset

[PATCH v2 1/5] Btrfs: Make async snapshot ioctl more generic

2010-12-09 Thread Li Zefan
So we don't have to add new structures as we create more ioctls for snapshots/subvolumes. Now to create async snapshot, set BTRFS_SUBVOL_CREATE_SNAP_ASYNC bit of vol_arg_v2-flags, and then call ioctl(BTRFS_IOCT_SNAP_CREATE_V2). Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/ioctl.c |

[PATCH v2 3/5] Btrfs: Add helper __setup_root_post()

2010-12-09 Thread Li Zefan
Eliminate duplicate code in btrfs_read_fs_root_no_radix() and find_and_setup_root(). Also prepare for the next patch. Signed-off-by: Li Zefan l...@cn.fujitsu.com --- fs/btrfs/disk-io.c | 31 +++ 1 files changed, 15 insertions(+), 16 deletions(-) diff --git

[PATCH v2 4/5] Btrfs: Add readonly snapshots support

2010-12-09 Thread Li Zefan
Usage: Set BTRFS_SUBVOL_RDONLY of btrfs_ioctl_vol_arg_v2-flags, and call ioctl(BTRFS_I0CTL_SNAP_CREATE_V2). Implementation: - In disk set readonly bit of btrfs_root_item-flags, and in memory set btrfs_root-readonly. - Add readonly checks in btrfs_permission (inode_permission), btrfs_setattr,

[PATCH v2 5/5] Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctl

2010-12-09 Thread Li Zefan
This allows us to set a snapshot or a subvolume readonly or writable on the fly. Usage: Set BTRFS_SUBVOL_RDONLY of btrfs_ioctl_vol_arg_v2-flags, and then call ioctl(BTRFS_IOCTL_SUBVOL_SETFLAGS); Changelog for v2: - Add _GETFLAGS ioctl. - Check if the passed fd is the root of a subvolume. -

[PATCH v2]Btrfs: pwrite blocked when writing from the mmaped buffer of the same page

2010-12-09 Thread Zhong, Xin
This problem is found in meego testing: http://bugs.meego.com/show_bug.cgi?id=6672 A file in btrfs is mmaped and the mmaped buffer is passed to pwrite to write to the same page of the same file. In btrfs_file_aio_write(), the pages is locked by prepare_pages(). So when btrfs_copy_from_user() is

Re: The value displayed by 'ls -s' command is strange.

2010-12-09 Thread Miao Xie
On wed, 08 Dec 2010 08:53:55 +0900, Tsutomu Itoh wrote: I think that the disk allocation size of each file becomes a monotone increase when the file is made. But, it sometimes return to 0. Is it correct? The # of blocks is: stat-blocks = (inode_get_bytes(inode) +

mount snapshot by object-id

2010-12-09 Thread Michael Niederle
Hi! I'm currently writing a btrfs-rescue tool and therefor began to study the btrfs-on-disk structures in detail. The root tree contains a ROOT_ITEM entry for *every* subvolume in the whole file system, but only DIR_ITEM entries for subvolumes that were created in the root directory of the

Re: mount snapshot by object-id

2010-12-09 Thread Calvin Walton
On Thu, 2010-12-09 at 14:51 +0100, Michael Niederle wrote: Hi! I'm currently writing a btrfs-rescue tool and therefor began to study the btrfs-on-disk structures in detail. The root tree contains a ROOT_ITEM entry for *every* subvolume in the whole file system, but only DIR_ITEM entries

Re: mount snapshot by object-id

2010-12-09 Thread Dipl.-Ing. Michael Niederle
Hi, Calvin! Thanks a lot for this information and for updating the wiki! The option works - on healthy disks ... I will continue writing my rescue-tool. I also wrote a btrfs_subvolumes command that displays all subvolumes of an unmounted filesystem. This helps a lot if mounting the filesystem

Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation

2010-12-09 Thread John Stoffel
Eric == Eric Paris epa...@redhat.com writes: Eric SELinux would like to implement a new labeling behavior of newly Eric created inodes. We currently label new inodes based on the Eric parent and the creating process. This new behavior would also Eric take into account the name of the new

Re: What to do about subvolumes?

2010-12-09 Thread J. Bruce Fields
On Wed, Dec 08, 2010 at 09:41:33PM -0700, Andreas Dilger wrote: On 2010-12-08, at 16:07, Neil Brown wrote: On Mon, 6 Dec 2010 11:48:45 -0500 J. Bruce Fields bfie...@redhat.com wrote: On Fri, Dec 03, 2010 at 04:01:44PM -0700, Andreas Dilger wrote: Any chance we can add a -get_fsid(sb,

Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation

2010-12-09 Thread Eric Paris
On Thu, 2010-12-09 at 10:05 -0500, John Stoffel wrote: Eric == Eric Paris epa...@redhat.com writes: So what happens when I create a file /home/john/shadow, does selinux (or LSM in general) then run extra checks because the filename is 'shadow' in your model? It's entirely a question of

Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation

2010-12-09 Thread Serge Hallyn
Quoting John Stoffel (j...@stoffel.org): Eric == Eric Paris epa...@redhat.com writes: Eric SELinux would like to implement a new labeling behavior of newly Eric created inodes. We currently label new inodes based on the Eric parent and the creating process. This new behavior would also

Re: Fsck, parent transid verify failed

2010-12-09 Thread Tom Kuther
Chris Mason chris.mason at oracle.com writes: [...] Build the latest tools, then: btrfsck -s 1 /dev/xxx btrfsck -s 2 /dev/xxx If either of these work we have an easy way to get it mounted. Just let me know. Hello, I get those parent transid verify failed errors too after a system

Re: Fsck, parent transid verify failed

2010-12-09 Thread Chris Mason
Excerpts from Tommy Jonsson's message of 2010-12-08 15:07:58 -0500: Build the latest tools, then: btrfsck -s 1 /dev/xxx btrfsck -s 2 /dev/xxx If either of these work we have an easy way to get it mounted. Just let me know. -chris $ btrfsck -s 1 /dev/sda using SB copy 1,

Permissions model for btrfs?

2010-12-09 Thread Wayne Pollock
I looked though the wiki (and searched the archives) but don't see an answer. Will btrfs support old POSIX-style ACLs and permissions, or the new NFS/NT style ACLs like ZFS? From the patch I saw, it seems old POSIX ACLs and permissions, but I'd like to know for sure. (And maybe the FAQ on the

Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation

2010-12-09 Thread John Stoffel
Eric == Eric Paris epa...@redhat.com writes: Eric On Thu, 2010-12-09 at 10:05 -0500, John Stoffel wrote: Eric == Eric Paris epa...@redhat.com writes: So what happens when I create a file /home/john/shadow, does selinux (or LSM in general) then run extra checks because the filename is

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Ted Ts'o
On Tue, Dec 07, 2010 at 09:37:20PM -0600, Jon Nelson wrote: One difference is the location of the transaction logs (pg_xlog). In my case, /var/lib/pgsql/data *is* mountpoint for the test volume (actually, it's a symlink to the mount point). In your case, that is not so. Perhaps that makes a

Re: Permissions model for btrfs?

2010-12-09 Thread Sean Bartell
On Thu, Dec 09, 2010 at 12:35:35PM -0500, Wayne Pollock wrote: I looked though the wiki (and searched the archives) but don't see an answer. Will btrfs support old POSIX-style ACLs and permissions, or the new NFS/NT style ACLs like ZFS? From the patch I saw, it seems old POSIX ACLs and

Re: [PATCH] fs/vfs/security: pass last path component to LSM on inode creation

2010-12-09 Thread Eric Paris
On Thu, 2010-12-09 at 12:48 -0500, John Stoffel wrote: Eric == Eric Paris epa...@redhat.com writes: Eric On Thu, 2010-12-09 at 10:05 -0500, John Stoffel wrote: Eric == Eric Paris epa...@redhat.com writes: Eric This patch adds a 4th piece of information, the name of the Eric object

[PATCH] Btrfs-progs: update super fields for space cache

2010-12-09 Thread Josef Bacik
This patch updates the super field to add the cache_generation member. It also makes us set it to -1 on mkfs so any new filesystem will get the space cache stuff turned on. Thanks, Signed-off-by: Josef Bacik jo...@redhat.com --- ctree.h |6 +- utils.c |1 + 2 files changed, 6

[PATCH] Btrfs-progs: add support for mixed data+metadata block groups

2010-12-09 Thread Josef Bacik
So alot of crazy people (I'm looking at you Meego) want to use btrfs on phones and such with small devices. Unfortunately the way we split out metadata/data chunks it makes space usage inefficient for volumes that are smaller than 1gigabyte. So add a -M option for mixing metadata+data, and

Re: btrfs subvolume snapshot hung in btrfs_commit_transaction

2010-12-09 Thread Ian! D. Allen
Hello developers - Last chance to get more details on this btrfs hang (below) before I reboot the machine. Anything I can do to gather more data? Linux linux 2.6.35-23-generic #40-Ubuntu SMP Wed Nov 17 22:14:33 UTC 2010 x86_64 GNU/Linux Description: Ubuntu 10.10 Package: btrfs-tools Status:

Re: [PATCH v2 5/5] Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctl

2010-12-09 Thread Goffredo Baroncelli
Hi Li, On Thursday, 09 December, 2010, Li Zefan wrote: This allows us to set a snapshot or a subvolume readonly or writable on the fly. Usage: Set BTRFS_SUBVOL_RDONLY of btrfs_ioctl_vol_arg_v2-flags, and then call ioctl(BTRFS_IOCTL_SUBVOL_SETFLAGS); Changelog for v2: - Add _GETFLAGS

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Ted Ts'o
On Thu, Dec 09, 2010 at 12:10:58PM -0600, Jon Nelson wrote: You should be OK, there. Are you using encryption or no? I had difficulty replicating the issue without encryption. Yes, I'm using encryption. LUKS with aes-xts-plain-sha256, and then LVM on top of LUKS. If you can point out how

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Jon Nelson
On Thu, Dec 9, 2010 at 2:13 PM, Ted Ts'o ty...@mit.edu wrote: On Thu, Dec 09, 2010 at 12:10:58PM -0600, Jon Nelson wrote: You should be OK, there. Are you using encryption or no? I had difficulty replicating the issue without encryption. Yes, I'm using encryption.  LUKS with

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Andi Kleen
512MB. 'free' reports 75MB, 419MB free. I originally noticed the problem on really real hardware (thinkpad T61p), however. If you can easily reproduce it could you try a git bisect? -Andi -- a...@linux.intel.com -- Speaking for myself only. -- To unsubscribe from this list: send the

[PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Sage Weil
We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. There's ample room for further cleanup here, but this keeps the fix simple. Signed-off-by: Sage Weil s...@newdream.net --- fs/btrfs/ioctl.c | 20 +++- 1 files changed, 11

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Chris Mason
Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500: 512MB. 'free' reports 75MB, 419MB free. I originally noticed the problem on really real hardware (thinkpad T61p), however. If you can easily reproduce it could you try a git bisect? Do we have a known good kernel?

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Matt
On Fri, Dec 10, 2010 at 2:38 AM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500: 512MB. 'free' reports 75MB, 419MB free. I originally noticed the problem on really real hardware (thinkpad T61p), however. If you can easily

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Mike Fedyk
On Thu, Dec 9, 2010 at 5:38 PM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500: 512MB. 'free' reports 75MB, 419MB free. I originally noticed the problem on really real hardware (thinkpad T61p), however. If you can easily

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Chris Mason
Excerpts from Mike Fedyk's message of 2010-12-09 20:58:40 -0500: On Thu, Dec 9, 2010 at 5:38 PM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Andi Kleen's message of 2010-12-09 18:16:16 -0500: 512MB. 'free' reports 75MB, 419MB free. I originally noticed the problem on

Re: [PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Li Zefan
Sage Weil wrote: We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. Ha, I fixed this accidentally in my patchset. :) There's ample room for further cleanup here, but this keeps the fix simple. Signed-off-by: Sage Weil

Re: [PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Chris Mason
Excerpts from Li Zefan's message of 2010-12-09 21:11:25 -0500: Sage Weil wrote: We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. Ha, I fixed this accidentally in my patchset. :) Thanks guys, Unless either of you object, I'll

Re: [PATCH] Btrfs: fix sync subvol/snapshot creation

2010-12-09 Thread Sage Weil
On Thu, 9 Dec 2010, Chris Mason wrote: Excerpts from Li Zefan's message of 2010-12-09 21:11:25 -0500: Sage Weil wrote: We were incorrectly taking the async path even for the sync ioctls by passing in transid unconditionally. Ha, I fixed this accidentally in my patchset. :)

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Ted Ts'o
On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote: Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179 from the tests I've done that one showed the least or no corruption if you count the empty /etc/env.d/03opengl as an artefact Yes, that's a good test. Also try commit

[PATCH v3] Btrfs: Make async snapshot ioctl more generic

2010-12-09 Thread Li Zefan
If we had reserved some bytes in struct btrfs_ioctl_vol_args, we wouldn't have to create a new structure for async snapshot creation. Here we convert async snapshot ioctl to use a more generic ABI, as we'll add more ioctls for snapshots/subvolumes in the future, readonly snapshots for example.

Re: [PATCH v2 4/5] Btrfs: Add readonly snapshots support

2010-12-09 Thread Li Zefan
+#define BTRFS_ROOT_SNAP_RDONLY (1ULL 0) + struct btrfs_root_item { struct btrfs_inode_item inode; __le64 generation; @@ -1116,6 +1118,7 @@ struct btrfs_root { int defrag_running; char *name; int in_sysfs; +bool readonly; Does make sense to store

Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective)

2010-12-09 Thread Jon Nelson
On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o ty...@mit.edu wrote: On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote: Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179 from the tests I've done that one showed the least or no corruption if you count the empty /etc/env.d/03opengl as

Re: [PATCH v2 5/5] Btrfs: Add BTRFS_IOC_SUBVOL_GETFLAGS/SETFLAGS ioctl

2010-12-09 Thread Li Zefan
Goffredo Baroncelli wrote: Hi Li, On Thursday, 09 December, 2010, Li Zefan wrote: This allows us to set a snapshot or a subvolume readonly or writable on the fly. Usage: Set BTRFS_SUBVOL_RDONLY of btrfs_ioctl_vol_arg_v2-flags, and then call ioctl(BTRFS_IOCTL_SUBVOL_SETFLAGS);

Re: [PATCH] Btrfs: do not BUG if we fail to remove the orphan item for dead snapshots

2010-12-09 Thread Li Zefan
01:25, Josef Bacik wrote: Not being able to delete an orphan item isn't a horrible thing. The worst that happens is the next time around we try and do the orphan cleanup and we can't find the referenced object and just delete the item and move on. Thanks, Would be better to add code