Re: Question about a specific error.

On 2016-02-03 16:27, Hugo Mills wrote: On Wed, Feb 03, 2016 at 02:17:06PM -0700, Chris Murphy wrote: On Tue, Feb 2, 2016 at 6:00 AM, Austin S. Hemmelgarn wrote: On 2016-02-01 15:21, Hugo Mills wrote: On Mon, Feb 01, 2016 at 02:44:24PM -0500, Austin S. Hemmelgarn wrote:

Re: btrfs-progs and btrfs(8) inconsistencies

On 2016-02-04 05:14, Martin Steigerwald wrote: Am Donnerstag, 4. Februar 2016, 09:57:54 CET schrieb Moviuro: Although personally I like to let all the backward compatibility things go hell, but that's definitely not how things work. :( 2) End-user taste. Some end-users like such info as

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 4, 2016 at 11:14 AM, Martin Steigerwald wrote: > Am Donnerstag, 4. Februar 2016, 09:57:54 CET schrieb Moviuro: >> > Although personally I like to let all the backward compatibility >> > things go hell, but that's definitely not how things work. :( >> > >> > 2)

Re: btrfs-progs and btrfs(8) inconsistencies

Am Donnerstag, 4. Februar 2016, 09:57:54 CET schrieb Moviuro: > > Although personally I like to let all the backward compatibility > > things go hell, but that's definitely not how things work. :( > > > > 2) End-user taste. > > Some end-users like such info as feedback of success. > > Of course

Re: Question about a specific error.

Chris Murphy posted on Wed, 03 Feb 2016 14:17:06 -0700 as excerpted: > The problems look different. You're reporting errors 2000. I'm seeing > errors 2001. I'm not sure what the distinction is; but in my case, > cancelling btrfs check and just rerunning it gives different results. >

Re: btrfs partition spontaneously corrupted - No recovery options. Kernel oops / "Kernel Bug"?

Dion Gullotta posted on Thu, 04 Feb 2016 12:28:09 +1100 as excerpted: > root@odin:/var/readynasd# uname -a > Linux odin 3.0.101.RN2120.3 #1 SMP > Wed Apr 1 16:09:30 PDT 2015 armv7l GNU/Linux Qu's helping you on the practical side, as a dev, far better than I could as a user, but here's a bit of

Re: [PATCH] fstests: btrfs, test for send with clone operations

On Thu, Feb 04, 2016 at 12:11:28AM +, fdman...@kernel.org wrote: > From: Filipe Manana > > Test that an incremental send operation which issues clone operations > works for files that have a full path containing more than one parent > directory component. > > This used to

Re: [PATCH] Btrfs: fix hang on extent buffer lock caused by the inode_paths ioctl

On Wed, Feb 03, 2016 at 08:26:50PM +, fdman...@kernel.org wrote: > From: Filipe Manana > > While doing some tests I ran into an hang on an extent buffer's rwlock > that produced the following trace: > > [39389.800012] NMI watchdog: BUG: soft lockup - CPU#15 stuck for 22s!

[PATCH v6 18/19] btrfs: dedup: add per-file online dedup control

From: Wang Xiaoguang Introduce inode_need_dedup() to implement per-file online dedup control. Signed-off-by: Wang Xiaoguang --- fs/btrfs/inode.c | 15 ++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git

[PATCH v6 10/19] btrfs: dedup: Add basic tree structure for on-disk dedup method

Introduce a new tree, dedup tree to record on-disk dedup hash. As a persist hash storage instead of in-memeory only implement. Unlike Liu Bo's implement, in this version we won't do hack for bytenr -> hash search, but add a new type, DEDUP_BYTENR_ITEM for such search case, just like in-memory

[PATCH v6 02/19] btrfs: dedup: Introduce function to initialize dedup info

From: Wang Xiaoguang Add generic function to initialize dedup info. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang --- fs/btrfs/Makefile | 2 +- fs/btrfs/dedup.c | 97

[PATCH v5 3/8] btrfs-progs: dedup: Add disable support for inband deduplication

Add disable subcommand for dedup command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 5 + cmds-dedup.c | 42 ++ 2 files changed, 47 insertions(+) diff --git

[PATCH v6 09/19] btrfs: dedup: Inband in-memory only de-duplication implement

Core implement for inband de-duplication. It reuse the async_cow_start() facility to do the calculate dedup hash. And use dedup hash to do inband de-duplication at extent level. The work flow is as below: 1) Run delalloc range for an inode 2) Calculate hash for the delalloc range at the unit of

[PATCH v6 07/19] btrfs: dedup: Implement btrfs_dedup_calc_hash interface

From: Wang Xiaoguang Unlike in-memory or on-disk dedup method, only SHA256 hash method is supported yet, so implement btrfs_dedup_calc_hash() interface using SHA256. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v6 06/19] btrfs: dedup: Introduce function to search for an existing hash

From: Wang Xiaoguang Introduce static function inmem_search() to handle the job for in-memory hash tree. The trick is, we must ensure the delayed ref head is not being run at the time we search the for the hash. With inmem_search(), we can implement the

[PATCH v6 00/19][For 4.6] Btrfs: Add inband (write time) de-duplication framework

The patchset can also be fetched from github: https://github.com/adam900710/linux.git wang_dedup_20160205 This updated version of inband de-duplication has the following features: 1) ONE unified dedup framework. Most of its code is hidden quietly in dedup.c and export the minimal interfaces

[PATCH v6 11/19] btrfs: dedup: Introduce interfaces to resume and cleanup dedup info

Since we will introduce a new on-disk based dedup method, introduce new interfaces to resume previous dedup setup. And since we introduce a new tree for status, also add disable handler for it. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo

[PATCH v6 05/19] btrfs: delayed-ref: Add support for increasing data ref under spinlock

For in-band dedup, btrfs needs to increase data ref with delayed_ref locked, so add a new function btrfs_add_delayed_data_ref_lock() to increase extent ref with delayed_refs already locked. Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 1 +

[PATCH v5 4/8] btrfs-progs: dedup: Add status subcommand

Add status subcommand for dedup command group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 3 ++ cmds-dedup.c | 76 ++ 2 files changed, 79 insertions(+) diff --git

[PATCH v5 1/8] btrfs-progs: Basic framework for dedup command group

Add basic ioctl header and command group framework for later use. Alone with basic man page doc. Signed-off-by: Qu Wenruo --- Documentation/Makefile.in | 1 + Documentation/btrfs-dedup.asciidoc | 39 +++ Documentation/btrfs.asciidoc

[PATCH v5 7/8] btrfs-progs: debug-tree: Add dedup tree support

Add dedup tree support for btrfs-debug-tree. Signed-off-by: Qu Wenruo --- v2: Add support to print hex objectid/offset for dedup hash. Add support to print hex hash. --- btrfs-debug-tree.c | 4 +++ ctree.h| 7 print-tree.c | 94

[PATCH v5 0/8] btrfs-progs: Support in-band de-duplication

Preparation patchset for incoming (aimed for 4.6) kernel in-band de-duplication patchset. New kernel dedup will has 2 different dedup backends and a ioctl interface to enable/disable dedup. The ioctl interface and on-disk format (mostly) is determined, so submit this patchset first before

[PATCH v6 01/19] btrfs: dedup: Introduce dedup framework and its header

From: Wang Xiaoguang Introduce the header for btrfs online(write time) de-duplication framework and needed header. The new de-duplication framework is going to support 2 different dedup method and 1 dedup hash. Signed-off-by: Qu Wenruo

[PATCH v6 12/19] btrfs: dedup: Add support for on-disk hash search

Now on-disk backend should be able to search hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 133 --- fs/btrfs/dedup.h | 1 + 2 files changed, 118

[PATCH v6 08/19] btrfs: ordered-extent: Add support for dedup

From: Wang Xiaoguang Add ordered-extent support for dedup. Note, current ordered-extent support only supports non-compressed source extent. Support for compressed source extent will be added later. Signed-off-by: Qu Wenruo Signed-off-by:

[PATCH v6 19/19] btrfs: try more times to alloc metadata reserve space

From: Wang Xiaoguang In btrfs_delalloc_reserve_metadata(), the number of metadata bytes we try to reserve is calculated by the difference between outstanding_extents and reserved_extents. When reserve_metadata_bytes() fails to reserve desited metadata space, it has

[PATCH v6 03/19] btrfs: dedup: Introduce function to add hash into in-memory tree

From: Wang Xiaoguang Introduce static function inmem_add() to add hash into in-memory tree. And now we can implement the btrfs_dedup_add() interface. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang ---

[PATCH v5 8/8] btrfs-progs: property: add a dedup property

From: Wang Xiaoguang Normally if we enable online dedup for a fs, it's filesystem wide de-duplication. With this property, we can explicitly disable data de-duplication for specified files. Signed-off-by: Wang Xiaoguang ---

[PATCH v6 17/19] btrfs: dedup: add a property handler for online dedup

From: Wang Xiaoguang We use btrfs extended attribute "btrfs.dedup" to record per-file online dedup status, so add a dedup property handler. Signed-off-by: Wang Xiaoguang --- fs/btrfs/props.c | 40

[PATCH v6 04/19] btrfs: dedup: Introduce function to remove hash from in-memory tree

From: Wang Xiaoguang Introduce static function inmem_del() to remove hash from in-memory dedup tree. And implement btrfs_dedup_del() and btrfs_dedup_destroy() interfaces. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v6 13/19] btrfs: dedup: Add support to delete hash for on-disk backend

Now on-disk backend can delete hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 100 +++ 1 file changed, 100 insertions(+) diff --git

[PATCH v6 16/19] btrfs: dedup: add an inode nodedup flag

From: Wang Xiaoguang Introduce BTRFS_INODE_NODEDUP flag, then we can explicitly disable online data deduplication for specified files. Signed-off-by: Wang Xiaoguang --- fs/btrfs/ctree.h | 1 + fs/btrfs/ioctl.c | 6 +- 2 files

[PATCH v5 6/8] btrfs-progs: Add show-super support for new DEDUP flag

Now btrfs-show-super can handle DEDUP ro compat flag. Signed-off-by: Qu Wenruo --- btrfs-show-super.c | 17 + 1 file changed, 17 insertions(+) diff --git a/btrfs-show-super.c b/btrfs-show-super.c index 051bd11..0bc0b1f 100644 --- a/btrfs-show-super.c

Re: btrfs-progs and btrfs(8) inconsistencies

If you look critically we have been using UI/CLI as API, IMO these two class of interfaces be distinct clearly. Btrfs needs library functions/APIs which is callable in popular scripting language like python. Thanks, Anand On 02/04/2016 05:54 AM, Moviuro wrote: Hi all, I should have

[PATCH v6 14/19] btrfs: dedup: Add support for adding hash for on-disk backend

Now on-disk backend can add hash now. Signed-off-by: Wang Xiaoguang Signed-off-by: Qu Wenruo --- fs/btrfs/dedup.c | 83 1 file changed, 83 insertions(+) diff --git a/fs/btrfs/dedup.c

[PATCH v6 15/19] btrfs: dedup: Add ioctl for inband deduplication

From: Wang Xiaoguang Add ioctl interface for inband deduplication, which includes: 1) enable 2) disable 3) status We will later add ioctl to disable inband dedup for given file/dir. Signed-off-by: Qu Wenruo Signed-off-by: Wang Xiaoguang

[PATCH v5 2/8] btrfs-progs: dedup: Add enable command for dedup command group

Add enable subcommand for dedup commmand group. Signed-off-by: Qu Wenruo --- Documentation/btrfs-dedup.asciidoc | 81 +- cmds-dedup.c | 138 + ioctl.h| 2 + 3

[PATCH v5 5/8] btrfs-progs: Add dedup feature for mkfs and convert

Add new DEDUP ro compat flag and corresponding mkfs/convert flag 'dedup'. Since dedup tree is completely isolated from fs tree, so even old kernel could do read mount. So add it to RO compat flag instead of common incompat flags Signed-off-by: Qu Wenruo ---

Re: [PATCH] btrfs-progs: describe btrfs-send requires read-only subvolume

On Thu, Feb 04, 2016 at 02:02:14PM +0900, Satoru Takeuchi wrote: > Both man btrfs-send(8) and usage message don't describe > btrfs-send needs read-only snapshot as its argument. > > Signed-off-by: Satoru Takeuchi Applied, thanks. -- To unsubscribe from this list:

Re: [PATCH] btrfs-progs: Fix self-reference of man btrfs-subvolume

On Thu, Feb 04, 2016 at 10:51:23AM +0900, Satoru Takeuchi wrote: > btrfs-subvolume(8) is mentioned at "SEE ALSO" section of itself. > > Signed-off-by: Satoru Takeuchi Aplied, thanks. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the

Re: [PATCH 10/10] fs: btrfs: Replace CURRENT_TIME by current_fs_time()

On Tue, Feb 02, 2016 at 10:07:50PM -0800, Deepa Dinamani wrote: > CURRENT_TIME macro is not appropriate for filesystems as it > doesn't use the right granularity for filesystem timestamps. > Use current_fs_time() instead. > > Signed-off-by: Deepa Dinamani > Cc: Chris

Re: [PATCH 2/2] Btrfs: fix lockdep deadlock warning due to dev_replace

On Thu, Feb 04, 2016 at 04:44:41PM +0800, Qu Wenruo wrote: > Hi > > Will this patch get merged in next merge windows? > (Although it should be merged in 4.5 merge window). > > I also encountered the same warning, but with fsstress + some sync calls. Besides a ping, you can help to get this

Re: btrfs-progs and btrfs(8) inconsistencies

Moviuro wrote on 2016/02/04 09:57 +0100: On Thu, Feb 4, 2016 at 2:33 AM, Qu Wenruo wrote: Moviuro wrote on 2016/02/03 22:54 +0100: Hi all, ... # btrfs subvolume create foo # should be silent # btrfs subvolume create -v foo /absolute/path/foo # btrfs subvolume

Re: RAID1 disk upgrade method

On 30 January 2016 at 15:50, Patrik Lundquist wrote: > On 29 January 2016 at 13:14, Austin S. Hemmelgarn > wrote: >> >> Last I checked, Seagate's 'NAS' drives and whatever they've re-branded their >> other enterprise line as, as well as WD's

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 4, 2016 at 5:53 AM, Austin S. Hemmelgarn wrote: > 4) Possibly get rid of the message on subvolume delete (It provides no > useful information at all, and it has no option to not error out on non > existence of a subvolume. In addition, the same arguments as

Re: btrfs-progs and btrfs(8) inconsistencies

On 2016-02-04 02:33, Qu Wenruo wrote: > The idea itself makes a lot of sense. > But I have at least two things to worry about: > > 1) Old scripts backward compatibility > Especially xfstests. Maintainer will hate it a lot. > As we have changed it several times and broken existing test

Re: btrfs-progs and btrfs(8) inconsistencies

On Wed, Feb 3, 2016 at 2:54 PM, Moviuro wrote: > Hi all, > > I should have done this a long time ago, so here goes. > > Btrfs is certainly the best FS out there but btrfs(8) is a pain to use in > scripts. That is, it talks way too much, exposes unparseable outputs and > has

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 4, 2016 at 10:17 AM, Goffredo Baroncelli wrote: > On 2016-02-04 02:33, Qu Wenruo wrote: >> The idea itself makes a lot of sense. >> But I have at least two things to worry about: >> >> 1) Old scripts backward compatibility >> Especially xfstests. Maintainer

Re: btrfs-progs and btrfs(8) inconsistencies

On 2016-02-04 14:40, Chris Murphy wrote: On Thu, Feb 4, 2016 at 5:53 AM, Austin S. Hemmelgarn wrote: 4) Possibly get rid of the message on subvolume delete (It provides no useful information at all, and it has no option to not error out on non existence of a subvolume.

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 04, 2016 at 06:17:58PM +0100, Goffredo Baroncelli wrote: > On 2016-02-04 02:33, Qu Wenruo wrote: > > The idea itself makes a lot of sense. > > But I have at least two things to worry about: > > > > 1) Old scripts backward compatibility > > Especially xfstests. Maintainer will hate

Re: Crash, boot mount failure: "corrupt leaf, slot offset bad"

I had another btrfs crash with identical symptoms. This time I managed to reproduce the crash and identify the root cause. It turns out that the Apple boot firmware is buggy and leaves wifi DMA turned on, which randomly corrupts memory after Linux is running.

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 4, 2016 at 9:28 PM Austin S. Hemmelgarn wrote: > > On 2016-02-04 14:40, Chris Murphy wrote: > > On Thu, Feb 4, 2016 at 5:53 AM, Austin S. Hemmelgarn > > wrote: > > > > > >> 4) Possibly get rid of the message on subvolume delete (It provides

Re: btrfs-progs and btrfs(8) inconsistencies

On Thu, Feb 4, 2016 at 2:33 AM, Qu Wenruo wrote: > > > Moviuro wrote on 2016/02/03 22:54 +0100: >> >> Hi all, >> >> ... >> >> # btrfs subvolume create foo # should be silent >> # btrfs subvolume create -v foo >> /absolute/path/foo >> # btrfs subvolume create

Re: [PATCH 2/2] Btrfs: fix lockdep deadlock warning due to dev_replace

Hi Will this patch get merged in next merge windows? (Although it should be merged in 4.5 merge window). I also encountered the same warning, but with fsstress + some sync calls. Thanks, Qu Liu Bo wrote on 2015/07/17 16:49 +0800: Xfstests btrfs/011 complains about a deadlock warning, [