Re: [PATCH] btrfs: Mem leak in btrfs_get_acl()

2011-01-07 Thread Aneesh Kumar K. V
On Thu, 6 Jan 2011 22:45:21 +0100 (CET), Jesper Juhl j...@chaosbits.net wrote: It seems to me that we leak the memory allocated to 'value' in btrfs_get_acl() if the call to posix_acl_from_xattr() fails. Here's a patch that attempts to correct that problem. Signed-off-by: Jesper Juhl

Bug report: parent transid failed after heavy load

2011-01-07 Thread Arie Peterson
Dear all, During a move of some 60GB of data from an ext4 partition to a btrfs partition, both on the same disk, the following happened: - my window manager froze; - the move suspended, i.e., no more data was written to the destination or deleted from the source; - part of top output:

Re: Atomic file data replace API

2011-01-07 Thread Mike Fleetwood
On 6 January 2011 20:01, Olaf van der Spek olafvds...@gmail.com wrote: Hi, Does btrfs support atomic file data replaces? Hi Olaf, Yes btrfs does support atomic replace, since kernel 2.6.30 circa June 2009. [1] Special handling was added to ext3, ext4, btrfs (and probably other Linux FSs) for

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 2:55 PM, Mike Fleetwood mike.fleetw...@googlemail.com wrote: On 6 January 2011 20:01, Olaf van der Spek olafvds...@gmail.com wrote: Hi, Does btrfs support atomic file data replaces? Hi Olaf, Yes btrfs does support atomic replace, since kernel 2.6.30 circa June 2009.

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 3:01 PM, Olaf van der Spek olafvds...@gmail.com wrote: According to Ted, via-truncate and via-rename are unsafe. Only fsync, rename is safe. Disadvantage of rename is resetting file owner (if non-root), having issues with meta-data and other stuff. My proposal was for

Re: [PATCH v2 1/5] add metadata_incore ioctl in vfs

2011-01-07 Thread Arnd Bergmann
On Thursday 06 January 2011, Shaohua Li wrote: Subject: add metadata_incore ioctl in vfs Add an ioctl to dump filesystem's metadata in memory in vfs. Userspace collects such info and uses it to do metadata readahead. Filesystem can hook to super_operations.metadata_incore to get metadata

Re: [PATCH v2 3/5]add metadata_readahead ioctl in vfs

2011-01-07 Thread Arnd Bergmann
On Thursday 06 January 2011, Shaohua Li wrote: Subject: add metadata_readahead ioctl in vfs Add metadata readahead ioctl in vfs. Filesystem can hook to super_operations.metadata_readahead to handle filesystem specific task. Next patch will give an example how btrfs implements it.

Re: Atomic file data replace API

2011-01-07 Thread Chris Mason
Excerpts from Olaf van der Spek's message of 2011-01-06 15:01:15 -0500: Hi, Does btrfs support atomic file data replaces? Basically, the atomic variant of this: // old stage open(O_TRUNC) write() // 0+ times close() // new state Yes and no. We have a best effort mechanism where we try

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 3:58 PM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Olaf van der Spek's message of 2011-01-06 15:01:15 -0500: Hi, Does btrfs support atomic file data replaces? Basically, the atomic variant of this: // old stage open(O_TRUNC) write() // 0+ times close()

Re: Atomic file data replace API

2011-01-07 Thread Chris Mason
Excerpts from Olaf van der Spek's message of 2011-01-07 10:01:59 -0500: On Fri, Jan 7, 2011 at 3:58 PM, Chris Mason chris.ma...@oracle.com wrote: Excerpts from Olaf van der Spek's message of 2011-01-06 15:01:15 -0500: Hi, Does btrfs support atomic file data replaces? Basically, the atomic

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 4:05 PM, Chris Mason chris.ma...@oracle.com wrote: The problem is the write() // 0+ times.  The kernel has no idea what new result you want the file to contain because the application isn't telling us. Isn't it safe for the kernel to wait until the first write or

Re: Atomic file data replace API

2011-01-07 Thread Chris Mason
Excerpts from Olaf van der Spek's message of 2011-01-07 10:08:24 -0500: On Fri, Jan 7, 2011 at 4:05 PM, Chris Mason chris.ma...@oracle.com wrote: The problem is the write() // 0+ times.  The kernel has no idea what new result you want the file to contain because the application isn't

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 4:13 PM, Chris Mason chris.ma...@oracle.com wrote: That's not what I asked. ;) I asked to wait until the first write (or close). That way, you don't get unintentional empty files. One step further, you don't have to keep the data in memory, you're free to write them to

Re: Atomic file data replace API

2011-01-07 Thread Chris Mason
Excerpts from Olaf van der Spek's message of 2011-01-07 10:17:31 -0500: On Fri, Jan 7, 2011 at 4:13 PM, Chris Mason chris.ma...@oracle.com wrote: That's not what I asked. ;) I asked to wait until the first write (or close). That way, you don't get unintentional empty files. One step

Re: Synching a Backup Server

2011-01-07 Thread Hubert Kario
On Friday, January 07, 2011 00:07:37 Carl Cook wrote: On Thu 06 January 2011 14:26:30 Carl Cook wrote: According To Doyle... Er, Hoyle... I am trying to create a multi-device BTRFS system using two identical drives. I want them to be raid 0 for no redunancy, and a total of 4TB. But in

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 5:12 PM, Chris Mason chris.ma...@oracle.com wrote: I'm not sure why you would run out of memory in that case. Well, lets make sure I've got a good handle on the proposed interface: 1) fd = open(some_file, O_ATOMIC) No, O_TRUNC should be used in open. Maybe it works

Re: Synching a Backup Server

2011-01-07 Thread Hubert Kario
On Thursday, January 06, 2011 22:52:25 Freddie Cash wrote: On Thu, Jan 6, 2011 at 1:42 PM, Carl Cook cac...@quantum-sci.com wrote: On Thu 06 January 2011 11:16:49 Freddie Cash wrote: Also with this system, I'm concerned that if there is corruption on the HTPC, it could be propagated to

Re: Atomic file data replace API

2011-01-07 Thread Hubert Kario
On Friday, January 07, 2011 17:12:11 Chris Mason wrote: Excerpts from Olaf van der Spek's message of 2011-01-07 10:17:31 -0500: On Fri, Jan 7, 2011 at 4:13 PM, Chris Mason chris.ma...@oracle.com wrote: That's not what I asked. ;) I asked to wait until the first write (or close). That

Re: Atomic file data replace API

2011-01-07 Thread Massimo Maggi
Are you suggesting to do: 1)fopen with O_TRUNC, O_ATOMIC: returns fd to a temporary file 2)application writes to that fd, with one or more system calls, in a short time or in long time, at his will. 3)at fclose (or even at fsync ) atomically swap data pointer of real file with temp file, then

Re: Atomic file data replace API

2011-01-07 Thread Olaf van der Spek
On Fri, Jan 7, 2011 at 5:32 PM, Massimo Maggi mass...@.it wrote: Are you suggesting to do: 1)fopen with O_TRUNC, O_ATOMIC: returns fd to a temporary file 2)application writes to that fd, with one or more system calls, in a short time or in long time, at his will. 3)at fclose (or even at

Re: hunt for 2.6.37 dm-crypt+ext4 corruption?

2011-01-07 Thread Matt
On Thu, Jan 6, 2011 at 4:56 PM, Heinz Diehl h...@fancy-poultry.org wrote: On 05.12.2010, Milan Broz wrote: It still seems to like dmcrypt with its parallel processing is just trigger to another bug in 37-rc. To come back to this: my 3 systems (XFS filesystem) running the latest

Various Questions

2011-01-07 Thread Carl Cook
On Fri 07 January 2011 08:14:17 Hubert Kario wrote: I'd suggest at least mkfs.btrfs -m raid1 -d raid0 /dev/sdc /dev/sdd if you really want raid0 I don't fully understand -m or -d. Why would this make a truer raid0 that with no options? Is it necessary to use fdisk on new drives in

Re: Various Questions

2011-01-07 Thread C Anthony Risinger
On Fri, Jan 7, 2011 at 11:15 AM, Carl Cook cac...@quantum-sci.com wrote: On Fri 07 January 2011 08:14:17 Hubert Kario wrote: I'd suggest at least mkfs.btrfs -m raid1 -d raid0 /dev/sdc /dev/sdd if you really want raid0 I don't fully understand -m or -d.  Why would this make a truer raid0

Re: Various Questions

2011-01-07 Thread Freddie Cash
On Fri, Jan 7, 2011 at 9:15 AM, Carl Cook cac...@quantum-sci.com wrote: How do you know what options to rsync are on by default?  I can't find this anywhere.  For example, it seems to me that --perms -ogE  --hard-links and --delete-excluded should be on by default, for a true sync? Who cares

Re: Various Questions

2011-01-07 Thread Carl Cook
Wow, this rsync and backup system is pretty amazing. I've always just tarred each directory manually, but now find I can RELIABLY automate backups, and have SOLID versioning to boot. Thanks to everyone who advised, especially Freddie and Anthony. I am still waiting for hardware for my

open_ctree failed, unable to mount the fs

2011-01-07 Thread Tomasz Chmielewski
I got a power cycle, after which I'm no longer able to mount btrfs filesystem: device fsid x-y devid 1 transid 169686 /dev/vda3 device fsid x-y devid 1 transid 169686 /dev/vda3 parent transid verify failed on 3260289024 wanted 169686 found 169685 parent transid verify failed on

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread Hugo Mills
On Fri, Jan 07, 2011 at 08:01:47PM +0100, Tomasz Chmielewski wrote: I got a power cycle, after which I'm no longer able to mount btrfs filesystem: device fsid x-y devid 1 transid 169686 /dev/vda3 device fsid x-y devid 1 transid 169686 /dev/vda3 parent transid verify failed

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread Tomasz Chmielewski
The forthcoming[1] btrfsck tool should handle that particular error, I believe. I noticed a similar problem was discussed here, with a solution: http://www.spinics.net/lists/linux-btrfs/msg07572.html where a btrfs-selects-super was used: git clone

Re: Atomic file data replace API

2011-01-07 Thread Chris Mason
Excerpts from Hubert Kario's message of 2011-01-07 11:26:02 -0500: On Friday, January 07, 2011 17:12:11 Chris Mason wrote: Excerpts from Olaf van der Spek's message of 2011-01-07 10:17:31 -0500: On Fri, Jan 7, 2011 at 4:13 PM, Chris Mason chris.ma...@oracle.com wrote: That's not what I

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread cwillu
On Fri, Jan 7, 2011 at 1:25 PM, Tomasz Chmielewski man...@wpkg.org wrote: The forthcoming[1] btrfsck tool should handle that particular error, I believe. I noticed a similar problem was discussed here, with a solution: http://www.spinics.net/lists/linux-btrfs/msg07572.html where a

Re: Atomic file data replace API

2011-01-07 Thread Thomas Bellman
Olaf van der Spek wrote: On Fri, Jan 7, 2011 at 5:32 PM, Massimo Maggi mass...@.it wrote: Are you suggesting to do: 1)fopen with O_TRUNC, O_ATOMIC: returns fd to a temporary file 2)application writes to that fd, with one or more system calls, in a short time or in long time, at his will.

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread Tomasz Chmielewski
On 07.01.2011 20:46, cwillu wrote: However, I don't see the tool when I clone the latest git - am I missing something? It's not built by the makefile by default; make btrfs-select-super as stated above will make it. $ grep select Makefile $ grep super Makefile $ grep -r select-super * $

Re: 'open_ctree failed', unable to mount the fs

2011-01-07 Thread Ken D'Ambrosio
On Fri, January 7, 2011 2:09 pm, Hugo Mills wrote: On Fri, Jan 07, 2011 at 08:01:47PM +0100, Tomasz Chmielewski wrote: I got a power cycle, after which I'm no longer able to mount btrfs filesystem: [...] The forthcoming[1] btrfsck tool should handle that particular error, I believe. I tried

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread cwillu
On Fri, Jan 7, 2011 at 2:01 PM, Tomasz Chmielewski man...@wpkg.org wrote: On 07.01.2011 20:46, cwillu wrote: However, I don't see the tool when I clone the latest git - am I missing something? It's not built by the makefile by default;  make btrfs-select-super as stated above will make it.

Re: 'open_ctree failed', unable to mount the fs

2011-01-07 Thread cwillu
On Fri, Jan 7, 2011 at 2:02 PM, Ken D'Ambrosio k...@jots.org wrote: On Fri, January 7, 2011 2:09 pm, Hugo Mills wrote: On Fri, Jan 07, 2011 at 08:01:47PM +0100, Tomasz Chmielewski wrote: I got a power cycle, after which I'm no longer able to mount btrfs filesystem: [...] The forthcoming[1]

Re: open_ctree failed, unable to mount the fs

2011-01-07 Thread Tomasz Chmielewski
On 07.01.2011 21:18, cwillu wrote: You checked out the master branch into a folder called next. -b next is the option to checkout a specific branch. From your existing checkout, git checkout -t origin/next will switch to that branch. Good catch - thanks for a hint. The filesystem mounted

btrfsck segmentation fault

2011-01-07 Thread Andrew Schretter
I have a 10TB btrfs filesystem over iSCSI that is currently unmountable. I'm currently running Fedora 13 with a recent Fedora 14 kernel (2.6.35.9-64.fc14.i686.PAE) and the system hung with messages like : parent transid verify failed on 5937615339520 wanted 48547 found 48542 I've rebooted and

Re: btrfsck segmentation fault

2011-01-07 Thread cwillu
On Fri, Jan 7, 2011 at 3:15 PM, Andrew Schretter schr...@math.duke.edu wrote: I have a 10TB btrfs filesystem over iSCSI that is currently unmountable.  I'm currently running Fedora 13 with a recent Fedora 14 kernel (2.6.35.9-64.fc14.i686.PAE) and the system hung with messages like : parent

Re: Offline Deduplication for Btrfs

2011-01-07 Thread Peter A
On Thursday, January 06, 2011 01:35:15 pm Chris Mason wrote: What is the smallest granularity that the datadomain searches for in terms of dedup? Josef's current setup isn't restricted to a specific block size, but there is a min match of 4k. I talked to a few people I know and didn't get a

Re: Atomic file data replace API

2011-01-07 Thread Phillip Susi
On 01/07/2011 09:58 AM, Chris Mason wrote: Yes and no. We have a best effort mechanism where we try to guess that since you've done this truncate and the write that you want the writes to show up quickly. But its a guess. It is a pretty good guess, and one that the NT kernel has been making