[PATCH v2] Btrfs: add readahead for send_write

2014-03-03 Thread Liu Bo
Btrfs send reads data from disk and then writes to a stream via pipe or a file via flush. Currently we're going to read each page at a time, so every page results in a disk read, which is not friendly to disks, esp. HDD. Given that, the performance can be gained by adding readahead for those

[PATCH] Btrfs: avoid unnecessary utimes update in incremental send

2014-03-03 Thread Filipe David Borba Manana
When we're finishing processing of an inode, if we're dealing with a directory inode that has a pending move/rename operation, we don't need to send a utimes update instruction to the send stream, as we'll do it later after doing the move/rename operation. Therefore we save some time here building

Re: [PATCH] Btrfs: use btrfs_crc32c everywhere instead of libcrc32c

2014-03-03 Thread Filipe David Manana
On Sat, Mar 1, 2014 at 9:20 PM, WorMzy Tykashi wormzy.tyka...@gmail.com wrote: On 27 February 2014 12:43, Filipe David Manana fdman...@gmail.com wrote: On Wed, Feb 26, 2014 at 11:26 PM, WorMzy Tykashi wormzy.tyka...@gmail.com wrote: Hi, Hi Hi again, Sorry for the delay in replying,

[PATCH 2/2] Btrfs: share the same code for __record_{new,deleted}_ref

2014-03-03 Thread Liu Bo
This has no functional change, only picks out the same part of two functions, and makes it shared. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/send.c | 49 + 1 file changed, 17 insertions(+), 32 deletions(-) diff --git

[PATCH 1/2] Btrfs: remove unused code in full_send_tree

2014-03-03 Thread Liu Bo
It's unnecessary to update key's value, and remove it to keep code clean. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/send.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 3fe4d6e..a5f9626 100644 --- a/fs/btrfs/send.c +++

Re: [PATCH 1/2] Btrfs: remove unused code in full_send_tree

2014-03-03 Thread gHcAgree
On 2014年03月03日 21:31, Liu Bo wrote: It's unnecessary to update key's value, and remove it to keep code clean. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/send.c | 4 1 file changed, 4 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 3fe4d6e..a5f9626 100644

Re: [PATCH v2] Btrfs: add readahead for send_write

2014-03-03 Thread David Sterba
On Mon, Mar 03, 2014 at 05:24:43PM +0800, Liu Bo wrote: --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -3972,6 +3972,7 @@ static ssize_t fill_read_buf(struct send_ctx *sctx, u64 offset, u32 len) pgoff_t last_index; unsigned pg_offset = offset ~PAGE_CACHE_MASK; ssize_t

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Marc MERLIN
On Mon, Mar 03, 2014 at 02:50:33PM +0800, Wang Shilong wrote: Here's the log of failure: http://marc.merlins.org/tmp/btrfs_nofreeze2.txt Unfortunately, i could not reproduce this problem here. It should not be the problem that i addressed before, there is not deadlock here. try the

Filesystem extremly fragmented after scrub

2014-03-03 Thread Clemens Eisserer
Hi, I am using btrfs on a single device with crompress-force=lzo and after I scrubbed the device once (kernel-3.12), the filesystem is extremly fragmented - running defrag on all file didn't improve the situation unfourtunately. Because I have a fast SSD this is barely noticeable when booting the

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Josef Bacik
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/27/2014 02:06 PM, Marc MERLIN wrote: This does not happen consistently, but sometimes: PM: Preparing system for mem sleep Freezing user space processes ... (...) Freezing of tasks failed after 20.002 seconds (1 tasks refusing to freeze,

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Josef Bacik
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/03/2014 11:56 AM, Josef Bacik wrote: On 02/27/2014 02:06 PM, Marc MERLIN wrote: This does not happen consistently, but sometimes: PM: Preparing system for mem sleep Freezing user space processes ... (...) Freezing of tasks failed after

Re: Filesystem extremly fragmented after scrub

2014-03-03 Thread Duncan
Clemens Eisserer posted on Mon, 03 Mar 2014 16:50:35 +0100 as excerpted: I am using btrfs on a single device with crompress-force=lzo and after I scrubbed the device once (kernel-3.12), the filesystem is extremly fragmented - running defrag on all file didn't improve the situation

Re: Filesystem extremly fragmented after scrub

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 8:50 AM, Clemens Eisserer linuxhi...@gmail.com wrote: When booting fsck reports the filesystem is 26% non-continous. I don't know what this means. There is no boot time fsck for Btrfs. So could you post the messages you're receiving and state what distribution and kernel

ENOSPC errors during raid1 rebalance

2014-03-03 Thread Mike Russo
Hi guys - I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and the filesystem was previously ext4 but the conversion seemed to go just fine, and I have no snapshots. System and metadata convert, and almost all my data converts, but there are 70 stubborn GB (14 blocks of

Re: Ordering of directory operations maintained across system crashes in Btrfs?

2014-03-03 Thread thanumalayan mad
Any ideas about this? Guessed-up, not-entirely-sure answers would help too. An example application bug that would be affected by this is from LevelDB: https://code.google.com/p/leveldb/issues/detail?id=189 Thanks, Thanu On Tue, Feb 25, 2014 at 8:01 PM, thanumalayan mad madth...@gmail.com wrote:

Re: Filesystem extremly fragmented after scrub

2014-03-03 Thread Clemens Eisserer
Hi, Some general notes .. VM images ... While NOCOW will prevent COW ... As mentioned the VM is configured to access the partition where linux is installed directly. So btrfs is not used as filesystem on the host machine, but rather on the guest which writes directly to disk (so no other

Re: Ordering of directory operations maintained across system crashes in Btrfs?

2014-03-03 Thread Chris Mason
On 02/25/2014 09:01 PM, thanumalayan mad wrote: Hi all, Slightly complicated question. Assume I do two directory operations in a Btrfs partition (such as an unlink() and a rename()), one after the other, and a crash happens after the rename(). Can Btrfs (the current version) send the second

Re: [PATCH] Btrfs-progs: free path if we don't find root item

2014-03-03 Thread David Sterba
On Fri, Feb 28, 2014 at 07:27:06PM +0200, Rakesh Pandit wrote: In btrfs_find_last_root before returning with -ENOENT (if root item is not found) free path and also remove btrfs_release_path before btrfs_free_path because btrfs_free_path anyway calls it. Signed-off-by: Rakesh Pandit

Re: Ordering of directory operations maintained across system crashes in Btrfs?

2014-03-03 Thread thanumalayan mad
Chris, Great, thanks. Any guesses whether other filesystems (disk-based) do things similar to the last two examples you pointed out? Saying we think 3 normal filesystems reorder stuff seems to motivate application developers to fix bugs ... Also, just for more information, the sequence we

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Duncan
Mike Russo posted on Mon, 03 Mar 2014 17:23:43 + as excerpted: I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and the filesystem was previously ext4 but the conversion seemed to go just fine, and I have no snapshots. System and metadata convert, and almost all

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Duncan 1i5t5.duncan at cox.net writes: That allows rollback if desired, but does tie up some some space with the automatically created btrfs snapshot that contains the ext3/4 metadata and untouched data. Nope, I definitely deleted the snapshots, running btrfs sub list gives me nothing

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Hugo Mills
On Mon, Mar 03, 2014 at 05:23:43PM +, Mike Russo wrote: Hi guys - I'm trying to convert a disk from single (/dev/sdc1) to RAID1 (dev/sdd1), and the filesystem was previously ext4 but the conversion seemed to go just fine, and I have no snapshots. System and metadata convert, and almost

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 11:24 AM, Michael Russo m...@papersolve.com wrote: Duncan 1i5t5.duncan at cox.net writes: That allows rollback if desired, but does tie up some some space with the automatically created btrfs snapshot that contains the ext3/4 metadata and untouched data. Nope, I

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: It might be worth adding enospc_debug as a mount option These messages only appear when I mount with enospc_debug. I included them as examples but I can post the full output later if needed. As for whether btrfs-convert should be recommended, I

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 12:24 PM, Michael Russo m...@papersolve.com wrote: Chris Murphy lists at colorremedies.com writes: It might be worth adding enospc_debug as a mount option These messages only appear when I mount with enospc_debug. Gotcha. I think Hugo has the best next step.

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Michael Russo
Chris Murphy lists at colorremedies.com writes: Chris Murphy lists at colorremedies.com writes: Gotcha. I think Hugo has the best next step. Defragment. I think this is going to work. I cancelled a partial defrag and did another move attempt, and this time 5GB got moved! So I'm going to

Re: ENOSPC errors during raid1 rebalance

2014-03-03 Thread Chris Murphy
On Mar 3, 2014, at 1:50 PM, Michael Russo m...@papersolve.com wrote: Oh yeah, it was definitely a problem with either the drives or the external enclosure, which was converting USB to SATA and mirroring the drives internally (it was a WD MyBook Mirror Edition). There was a problem with

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Marc MERLIN
On Mon, Mar 03, 2014 at 12:09:11PM -0500, Josef Bacik wrote: Ok I lied I just went ahead and did it, please let me know if this fixes it This looked promising, but I still have the problem. PM: Syncing filesystems ... done. PM: Preparing system for mem sleep Freezing user space processes ...

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Josef Bacik
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/03/2014 05:05 PM, Marc MERLIN wrote: On Mon, Mar 03, 2014 at 12:09:11PM -0500, Josef Bacik wrote: Ok I lied I just went ahead and did it, please let me know if this fixes it This looked promising, but I still have the problem. PM:

Re: btrfsck does not fix

2014-03-03 Thread Chris Murphy
On Mar 2, 2014, at 11:39 AM, Hendrik Friedel hend...@friedels.name wrote: I am lacking space why I did the balance (to free one of the two discs). So, unless the above helps, it seems, I need to buy another HDD? It might be worth finding large files to defragment. See the ENOSPC errors

Re: 3.14.0-rc3 btrfs scrub is preventing my laptop from going to sleep

2014-03-03 Thread Marc MERLIN
On Mon, Mar 03, 2014 at 05:18:33PM -0500, Josef Bacik wrote: Maybe it will work if we cancel the scrub as opposed to pausing it, but of course it's not ideal. Is that the next step? Sigh I thought the PM stuff called freeze_fs() but I think that was just tuxonice. I don't have a quick fix

Subscription.

2014-03-03 Thread royy walls
Hi, I want to subscribe on brtfs mailing list. Regards, Chaitanya -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Subscription.

2014-03-03 Thread Justus Seifert
On 04.03.2014 01:18, royy walls wrote: Hi, I want to subscribe on brtfs mailing list. Regards, Chaitanya -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH 1/2] Btrfs: remove unused code in full_send_tree

2014-03-03 Thread Liu Bo
On Mon, Mar 03, 2014 at 09:53:45PM +0800, gHcAgree wrote: On 2014年03月03日 21:31, Liu Bo wrote: It's unnecessary to update key's value, and remove it to keep code clean. Signed-off-by: Liu Bo bo.li@oracle.com --- fs/btrfs/send.c | 4 1 file changed, 4 deletions(-) diff --git

Re: [PATCH v2] Btrfs: add readahead for send_write

2014-03-03 Thread Liu Bo
On Mon, Mar 03, 2014 at 03:25:28PM +0100, David Sterba wrote: On Mon, Mar 03, 2014 at 05:24:43PM +0800, Liu Bo wrote: --- a/fs/btrfs/send.c +++ b/fs/btrfs/send.c @@ -3972,6 +3972,7 @@ static ssize_t fill_read_buf(struct send_ctx *sctx, u64 offset, u32 len) pgoff_t last_index;

Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-03-03 Thread Miao Xie
On sat, 1 Mar 2014 20:05:01 +0200, Alex Lyakas wrote: Hi Miao, On Wed, Jan 15, 2014 at 2:00 PM, Miao Xie mi...@cn.fujitsu.com wrote: When we mounted the filesystem after the crash, we got the following message: BTRFS error (device xxx): block group 4315938816 has wrong amount of free

Re: btrfsck does not fix

2014-03-03 Thread Hendrik Friedel
Hi Chris, It might be worth finding large files to defragment. See the ENOSPC errors during raid1 rebalance thread. It sounds like it might be possible for some fragmented files to be stuck across multiple chunks, preventing conversion. I moved 400Gb from my other (but full) disc to the