btrfs-progs-4.13: cmds-inspect-tree-stats.h: No such file or directory

2017-09-08 Thread Uli Heller
I dowloaded the btrfs-progs-v4.13.tar.xz tarball and tried to build it. Unfortunately, I get an error about cmds-inspect-tree-stats.h. The file seems to be missing from the tarball, it used to exist in v4.12. ... true -g -Os -fstack-protector --param=ssp-buffer-size=4 -Wformat

Re: btrfs-progs task tracking on github

2017-09-08 Thread Lakshmipathi.G
Existing project list is good. Reg: "Project tests: Build and test coverage: pre-built images on docker hub" If we don't have pre-built images on docker hub, I'll pick up this task and start working on it. thanks. Cheers, Lakshmipathi.G On Sat, Sep 9, 2017 at 5:43 AM, Qu Wenruo

Re: [GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Herbert Xu
On Sat, Sep 09, 2017 at 02:48:55AM +, Nick Terrell wrote: > > There is a patch [1] floating around adding zstd support to compressed RAM > that uses the crypto API. > > [1] > https://lkml.kernel.org/r/20170824014936.4738-1-sergey.senozhatsky%40gmail.com Yes but I think it would make more

Re: [GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Nick Terrell
On 9/8/17, 6:36 PM, "Herbert Xu" wrote: > On Fri, Sep 08, 2017 at 03:33:05PM -0400, Chris Mason wrote: > > > > crypto/Kconfig |9 + > > crypto/Makefile|1 + > > crypto/testmgr.c | 10 + > > crypto/testmgr.h | 71

Re: [GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Chris Mason
On Sat, Sep 09, 2017 at 09:35:59AM +0800, Herbert Xu wrote: On Fri, Sep 08, 2017 at 03:33:05PM -0400, Chris Mason wrote: crypto/Kconfig |9 + crypto/Makefile|1 + crypto/testmgr.c | 10 + crypto/testmgr.h | 71 + crypto/zstd.c

Re: [GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Herbert Xu
On Fri, Sep 08, 2017 at 03:33:05PM -0400, Chris Mason wrote: > > crypto/Kconfig |9 + > > crypto/Makefile|1 +

Re: [GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Chris Mason
On 09/08/2017 03:33 PM, Chris Mason wrote: Hi Linus, Nick Terrell's patch series to add zstd support to the kernel has been floating around for a while. After talking with Dave Sterba, Herbert and Phillip, we decided to send the whole thing in as one pull request. I have it in my zstd

Re: btrfs-progs task tracking on github

2017-09-08 Thread Qu Wenruo
On 2017年09月09日 01:36, David Sterba wrote: Hi, in order to make more visible which tasks for btrfs-progs are in progress or desired, I've started to populate the github Projects [1] some time ago. I haven't fleshed out the workflow so this hasn't been announced yet. The aim is to help

[PATCH RFC] Btrfs: fix confusing worker helper info

2017-09-08 Thread Liu Bo
We've seen the following backtrace stack in ftrace or dmesg log, kworker/u16:10-4244 [000] 241942.480955: function: btrfs_put_ordered_extent kworker/u16:10-4244 [000] 241942.480956: kernel_stack: => finish_ordered_fn (a0384475) => btrfs_scrubparity_helper

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Tomasz Kłoczko
On 8 September 2017 at 20:06, Austin S. Hemmelgarn wrote: [..] >> If you don't like awk you can use jq, sed, perl, python, ruby or >> whatever you have/like/want. > > And which command is more readable? Something like the above, or: > btrfs device stats --format=json / How

[PATCH v2] btrfs-progs: Add zstd support

2017-09-08 Thread Nick Terrell
Adds zstd support to the btrfs program. An optional dependency on libzstd >= 1.0.0 is added. Autoconf accepts `--enable-zstd' or `--disable-zstd' and defaults to detecting if libzstd is present using `pkg-config'. Adds tests for the new features based on a prebuilt btrfs image with a zstd

Re: 4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-08 Thread Josef Bacik
On Fri, Sep 08, 2017 at 01:33:43PM +0900, Tomasz Chmielewski wrote: > Just got this one in dmesg with btrfs RAID-1 on top of Linux software > RAID-5. > > Why does it say "No space left" if we have 9 TB free there? > We've exhausted the global reserve. I have a patch that I'm pretty sure fixes

btrfs check --clear-space-cache v1 fails if not --clear-space-cache v2 first

2017-09-08 Thread A L
Seems to be some inconsistency with btrfs check --clear-space-cache v1. It only worked after clearing v2 space cache. See below result. Background. Since some time I was running with space_cache mount option. I recently changed to space_cache=v2 and used "clear_cache" mount option to clear

[GIT PULL] zstd support (lib, btrfs, squashfs)

2017-09-08 Thread Chris Mason
Hi Linus, Nick Terrell's patch series to add zstd support to the kernel has been floating around for a while. After talking with Dave Sterba, Herbert and Phillip, we decided to send the whole thing in as one pull request. I have it in my zstd branch:

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Austin S. Hemmelgarn
On 2017-09-08 14:09, Tomasz Kłoczko wrote: On 8 September 2017 at 17:39, David Sterba wrote: [..] My plan is to introduce a global options to set various this, also the output format, eg. $ btrfs -t bell be om -format=json subvolume list that would dump the list in json

Re: test if a subvolume is a snapshot?

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 08:41:57PM +0200, Ulli Horlacher wrote: > On Fri 2017-09-08 (15:10), David Sterba wrote: > > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > > > > > How can I test if a subvolume is a snapshot? > > > > The inode number is 256 on a btrfs filesystem: > >

Re: test if a subvolume is a snapshot?

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 07:09:55PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 17:39, David Sterba wrote: > [..] > > My plan is to introduce a global options to set various this, also the > > output format, eg. > > > > $ btrfs -t bell be om -format=json subvolume list

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Ulli Horlacher
On Fri 2017-09-08 (15:10), David Sterba wrote: > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > > > How can I test if a subvolume is a snapshot? > > The inode number is 256 on a btrfs filesystem: > > if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ];

[GIT PULL] Btrfs for 4.14

2017-09-08 Thread David Sterba
Hi, please pull the following btrfs branch to 4.14. The changes range through all types: cleanups, core chagnes, sanity checks, fixes, other user visible changes, detailed list below. Merging notes: there's a minor conflict with the blk_status_t fix that went to 4.13-rc7 while this pull is on

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Tomasz Kłoczko
On 8 September 2017 at 17:39, David Sterba wrote: [..] > My plan is to introduce a global options to set various this, also the > output format, eg. > > $ btrfs -t bell be om -format=json subvolume list > > that would dump the list in json obviously, more formats could follow. >

btrfs-progs task tracking on github

2017-09-08 Thread David Sterba
Hi, in order to make more visible which tasks for btrfs-progs are in progress or desired, I've started to populate the github Projects [1] some time ago. I haven't fleshed out the workflow so this hasn't been announced yet. The aim is to help contributors to join progs development and start from

Btrfs progs release 4.13

2017-09-08 Thread David Sterba
Hi, btrfs-progs version 4.13 have been released. Changes: * convert: reiserfs support * check: new option --force to allow check of a mounted filesystem (no repair) * mkfs: --rootdir will now copy special files * dump-tree: minor output changes * inspect rootid: accept file as arugment

Re: test if a subvolume is a snapshot?

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 03:38:16PM +, Hugo Mills wrote: > > sometimes I'm really thinking about start rewrite btrfs-progs to make > > btrfs basic tools syntax as similar as it is only possible to ZFS zfs, > > zpool and zdb commands on using which in +90% cases you can guess how > > necessary

Re: test if a subvolume is a snapshot?

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 04:25:55PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 14:10, David Sterba wrote: > > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > > > How can I test if a subvolume is a snapshot? > > > > The inode number is 256 on a btrfs

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
On Fri, Sep 08, 2017 at 05:12:11PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 16:38, Hugo Mills wrote: > [..] > >> sometimes I'm really thinking about start rewrite btrfs-progs to make > >> btrfs basic tools syntax as similar as it is only possible to ZFS zfs, > >>

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Tomasz Kłoczko
On 8 September 2017 at 16:38, Hugo Mills wrote: [..] >> sometimes I'm really thinking about start rewrite btrfs-progs to make >> btrfs basic tools syntax as similar as it is only possible to ZFS zfs, >> zpool and zdb commands on using which in +90% cases you can guess how >>

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Hugo Mills
On Fri, Sep 08, 2017 at 04:25:55PM +0100, Tomasz Kłoczko wrote: > On 8 September 2017 at 14:10, David Sterba wrote: > > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > > > How can I test if a subvolume is a snapshot? > > > > The inode number is 256 on a btrfs

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Tomasz Kłoczko
On 8 September 2017 at 14:10, David Sterba wrote: > On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > > How can I test if a subvolume is a snapshot? > > The inode number is 256 on a btrfs filesystem: > > if [ stat -f --format=%T $path = btrfs -a stat --format=%i

Re: [PATCH 1/2] btrfs: clear ordered flag on cleaning up ordered extents

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 05:10:04PM +0900, Naohiro Aota wrote: > On 2017年09月08日 03:25, David Sterba wrote: > > On Fri, Sep 01, 2017 at 06:59:49PM +0800, Qu Wenruo wrote: > >> On 2017年09月01日 16:58, Naohiro Aota wrote: > >>> commit 524272607e88 ("btrfs: Handle delalloc error correctly to avoid > >>>

Re: test if a subvolume is a snapshot?

2017-09-08 Thread David Sterba
On Fri, Sep 08, 2017 at 10:54:46AM +0200, Ulli Horlacher wrote: > How can I test if a subvolume is a snapshot? The inode number is 256 on a btrfs filesystem: if [ stat -f --format=%T $path = btrfs -a stat --format=%i $path = 256 ]; ... The directory that's result of snapshotting a subvolume,

Re: 4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-08 Thread Tomasz Chmielewski
So the numbers that matter are: Data,single: Size:12.84TiB, Used:7.13TiB /dev/md2 12.84TiB Metadata,DUP: Size:79.00GiB, Used:77.87GiB /dev/md2 158.00GiB Unallocated: /dev/md23.31TiB * If you are using the 'space_cache' it has a known issue:

Re: test if a subvolume is a snapshot?

2017-09-08 Thread Peter Grandi
> How can I test if a subvolume is a snapshot? [ ... ] This question is based on the assumption that "snapshot" is a distinct type of subvolume and not just an operation that creates a subvolume with reflinked contents. Unfortunately Btrfs does indeed make snapshots a distinct type of

Re: 4.13: No space left with plenty of free space (/home/kernel/COD/linux/fs/btrfs/extent-tree.c:6989 __btrfs_free_extent.isra.62+0xc2c/0xdb0)

2017-09-08 Thread Peter Grandi
[ ... ] > [233787.921018] Call Trace: > [233787.921031] ? btrfs_merge_delayed_refs+0x62/0x550 [btrfs] > [233787.921039] __btrfs_run_delayed_refs+0x6f0/0x1380 [btrfs] > [233787.921047] btrfs_run_delayed_refs+0x6b/0x250 [btrfs] > [233787.921054] btrfs_write_dirty_block_groups+0x158/0x390 [btrfs]

test if a subvolume is a snapshot?

2017-09-08 Thread Ulli Horlacher
How can I test if a subvolume is a snapshot? Example: /test/.snapshot/2017-09-08_1037.single is a snapshot (of /test) /test/data is a regular subvolume I know this, because I have created them with suitable names :-) But how can I see/test it? root@fex:~/bin# btrfs subvol show

[PATCH] btrfs: propagate error to btrfs_cmp_data_prepare caller

2017-09-08 Thread Naohiro Aota
btrfs_cmp_data_prepare() (almost) always returns 0 i.e. ignoring errors from gather_extent_pages(). While the pages are freed by btrfs_cmp_data_free(), cmp->num_pages still has > 0. Then, btrfs_extent_same() try to access the already freed pages causing faults (or violates PageLocked assertion).

Re: [PATCH 1/2] btrfs: clear ordered flag on cleaning up ordered extents

2017-09-08 Thread Naohiro Aota
On 2017年09月08日 03:25, David Sterba wrote: > On Fri, Sep 01, 2017 at 06:59:49PM +0800, Qu Wenruo wrote: >> On 2017年09月01日 16:58, Naohiro Aota wrote: >>> commit 524272607e88 ("btrfs: Handle delalloc error correctly to avoid >>> ordered extent hang") introduced btrfs_cleanup_ordered_extents() to