Re: [PATCH v2 7/7] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On 2017年09月15日 23:48, David Sterba wrote: On Fri, Sep 15, 2017 at 09:24:19PM +0800, Qu Wenruo wrote: I'm going to review & merge this series to devel. Tests and documentation should be updated to make the usecase clear. I'm happy to address any comment, both code and doc/test. For the

Re: A user cannot remove his readonly snapshots?!

Am Sat, 16 Sep 2017 00:02:01 +0200 schrieb Ulli Horlacher : > On Fri 2017-09-15 (23:44), Ulli Horlacher wrote: > > On Fri 2017-09-15 (22:07), Peter Grandi wrote: > > > [...] > > > > > > Ordinary permissions still apply both to 'create' and 'delete': > > >

Re: [PATCH] Btrfs: do not backup tree roots when fsync

On Thu, Sep 14, 2017 at 02:49:03PM +0200, David Sterba wrote: > On Thu, Sep 14, 2017 at 09:55:48AM +0800, Qu Wenruo wrote: > > > > > > On 2017年09月14日 02:25, Liu Bo wrote: > > > It doens't make sense to backup tree roots when doing fsync, since > > > during fsync those tree roots have not been

[PATCH] Btrfs: fix unexpected result when dio reading corrupted blocks

commit 4246a0b63bd8 ("block: add a bi_error field to struct bio") changed the logic of how dio read endio reports errors. For single stripe dio read, %bio->bi_status reflects the error before verifying checksum, and now we're updating it when data block matches with its checksum, while in the

Re: A user cannot remove his readonly snapshots?!

On Fri 2017-09-15 (23:44), Ulli Horlacher wrote: > On Fri 2017-09-15 (22:07), Peter Grandi wrote: > > > > [ ... ] mounted with option user_subvol_rm_allowed [ ... ] > > > root can delete this snapshot, but not the user. Why? [ ... ] > > > > Ordinary permissions still apply both to 'create' and

Re: A user cannot remove his readonly snapshots?!

On Fri 2017-09-15 (22:07), Peter Grandi wrote: > > [ ... ] mounted with option user_subvol_rm_allowed [ ... ] > > root can delete this snapshot, but not the user. Why? [ ... ] > > Ordinary permissions still apply both to 'create' and 'delete': My user tux is the owner of the snapshot directory,

Re: A user cannot remove his readonly snapshots?!

> [ ... ] mounted with option user_subvol_rm_allowed [ ... ] > root can delete this snapshot, but not the user. Why? [ ... ] Ordinary permissions still apply both to 'create' and 'delete': tree$ sudo mkdir /fs/sda7/dir tree$ btrfs sub create /fs/sda7/dir/sub ERROR: cannot access

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On Fri, Sep 15, 2017 at 08:57:41PM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 07:01 PM, Liu Bo wrote: > >> Conclusion: even if the file is corrupted and normally BTRFS prevent to > >> access it, using O_DIRECT > >> a) no error is returned to the caller > >> b) instead of the page stored on

Re: snapshots of encrypted directories?

On Fri 2017-09-15 (13:16), Austin S. Hemmelgarn wrote: > >> And then mount enryptfs: > >> > >> mount.ecryptfs / / > > > > This only possible by root. > > For a user it is not possible to have access for his own snapshots. > > Bad. > > Which is why you use EncFS (which is a FUSE module that runs

Re: A user cannot remove his readonly snapshots?!

On Fri 2017-09-15 (13:08), Austin S. Hemmelgarn wrote: > On 2017-09-15 12:37, Ulli Horlacher wrote: > > > I have my btrfs filesystem mounted with option user_subvol_rm_allowed > > tux@xerus: btrfs subvolume delete > > /test/tux/zz/.snapshot/2017-09-15_1824.test > > Delete subvolume (no-commit):

Re: defragmenting best practice?

On 15 September 2017 at 18:08, Kai Krakow wrote: [..] > According to Tomasz, your tests should not run at vastly different > speeds because fragmentation has no impact on performance, quod est > demonstrandum... I think we will not get to the "erat" part. No. This is not

Re: \o/ compsize

On Mon, Sep 04, 2017 at 08:42:29PM +0200, Adam Borowski wrote: > On Mon, Sep 04, 2017 at 07:07:25PM +0300, Timofey Titovets wrote: > > 2017-09-04 18:11 GMT+03:00 Adam Borowski : > > > Here's an utility to measure used compression type + ratio on a set of > > > files > > > or

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On 09/15/2017 07:01 PM, Liu Bo wrote: >> Conclusion: even if the file is corrupted and normally BTRFS prevent to >> access it, using O_DIRECT >> a) no error is returned to the caller >> b) instead of the page stored on the disk, it is returned a page filled with >> 0x01 (according also with the

[PATCH 2/4 v3] btrfs-progs: add a parameter to btrfs_mksubvol

From: yingyil A convert parameter is added as a flag to indicate if btrfs_mksubvol() is used for btrfs-convert. The change cascades down to the callchain. Signed-off-by: yingyil --- v3: changed the convert flag from int type to bool type.

Re: [PATCH preview] btrfs: allow to set compression level for zlib

On 9/15/17, 7:53 AM, "David Sterba" wrote: > On Sun, Aug 20, 2017 at 06:38:50PM -0600, Chris Murphy wrote: > > On Fri, Aug 18, 2017 at 10:08 AM, David Sterba wrote: > > > > > That's quite a lot, in kernel. IIRC zlib and lzo use less than 200kb, > > > zstd wants

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On Fri, Sep 15, 2017 at 12:00:19AM +0200, Goffredo Baroncelli wrote: > Hi all, > > I discovered two bugs when O_DIRECT is used... > > 1) a corrupted file doesn't return -EIO when O_DIRECT is used > > Normally BTRFS prevents to access the contents of a corrupted file; however I > was able read

[PATCH 1/4 v3] btrfs-progs: convert: move link_subvol out of main

From: yingyil link_subvol() is moved to inode.c and renamed as btrfs_mksubvol(). The change cascades down to the callchain. Signed-off-by: yingyil --- v3: moved link_subvol to inode.c and put its header in ctree.h. The name is changed to btrfs_mksubvol.

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On 09/15/2017 11:50 AM, Marat Khalili wrote: > May I state my user's point of view: > > I know one applications that uses O_DIRECT, and it is subtly broken > on BTRFS. I know no applications that use O_DIRECT and are not > broken. (Really more statistics would help here, probably some exist >

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On 09/15/2017 07:01 PM, Andrei Borzenkov wrote: > 15.09.2017 08:50, Goffredo Baroncelli пишет: >> On 09/15/2017 05:55 AM, Andrei Borzenkov wrote: >>> 15.09.2017 01:00, Goffredo Baroncelli пишет: 2) The second bug, is a more severe bug. If during a writing of a buffer with O_DIRECT,

Re: snapshots of encrypted directories?

15.09.2017 15:35, Austin S. Hemmelgarn пишет: > On 2017-09-14 23:45, Andrei Borzenkov wrote: >> 14.09.2017 18:32, Hugo Mills пишет: >>> On Thu, Sep 14, 2017 at 04:57:39PM +0200, Ulli Horlacher wrote: I use encfs on top of btrfs. I can create btrfs snapshots, but I have no suggestive

Re: snapshots of encrypted directories?

On 2017-09-15 12:28, Ulli Horlacher wrote: On Fri 2017-09-15 (12:15), Peter Becker wrote: 2017-09-15 12:01 GMT+02:00 Ulli Horlacher : On Fri 2017-09-15 (06:45), Andrei Borzenkov wrote: The actual question is - do you need to mount each individual btrfs

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On 09/15/2017 10:26 AM, Hugo Mills wrote: > On Fri, Sep 15, 2017 at 08:04:35AM +0200, Goffredo Baroncelli wrote: >> On 09/15/2017 12:18 AM, Hugo Mills wrote: >>>As far as I know, both of these are basically known issues, with no >>> good solution, other than not using O_DIRECT. Certainly the

Re: [RFC 0/3]: settable compression level for zstd

On 2017-09-15 11:34, Adam Borowski wrote: Hi! Here's a patch set that allows changing the compression level for zstd, currently at mount time only. I've played with it for a month, so despite being a quick hack, it's reasonably well tested. Tested on 4.13 + btrfs-for-4.14 only, though -- I've

Re: A user cannot remove his readonly snapshots?!

On 2017-09-15 12:37, Ulli Horlacher wrote: I have my btrfs filesystem mounted with option user_subvol_rm_allowed tux@xerus: btrfs --version btrfs-progs v4.4 tux@xerus: uname -a Linux xerus 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux tux@xerus:

Re: defragmenting best practice?

Am Fri, 15 Sep 2017 16:11:50 +0200 schrieb Michał Sokołowski : > On 09/15/2017 03:07 PM, Tomasz Kłoczko wrote: > > [...] > > Case #1 > > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu cow2 > > storage -> guest BTRFS filesystem > > SQL table row insertions per

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

15.09.2017 08:50, Goffredo Baroncelli пишет: > On 09/15/2017 05:55 AM, Andrei Borzenkov wrote: >> 15.09.2017 01:00, Goffredo Baroncelli пишет: >>> >>> 2) The second bug, is a more severe bug. If during a writing of a buffer >>> with O_DIRECT, the buffer is updated at the same time by a second

A user cannot remove his readonly snapshots?!

I have my btrfs filesystem mounted with option user_subvol_rm_allowed tux@xerus: btrfs --version btrfs-progs v4.4 tux@xerus: uname -a Linux xerus 4.4.0-93-generic #116-Ubuntu SMP Fri Aug 11 21:17:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux tux@xerus: id uid=1000(tux) gid=100(users)

Re: defragmenting best practice?

[ ... ] Case #1 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu cow2 storage -> guest BTRFS filesystem SQL table row insertions per second: 1-2 Case #2 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu raw storage -> guest EXT4 filesystem

Re: snapshots of encrypted directories?

On Fri 2017-09-15 (12:15), Peter Becker wrote: > 2017-09-15 12:01 GMT+02:00 Ulli Horlacher : > > > On Fri 2017-09-15 (06:45), Andrei Borzenkov wrote: > > > >> The actual question is - do you need to mount each individual btrfs > >> subvolume when using encfs? > > >

Re: [PATCH v2 7/7] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On Fri, Sep 15, 2017 at 09:24:19PM +0800, Qu Wenruo wrote: > > I'm going to review & merge this series to devel. Tests and > > documentation should be updated to make the usecase clear. > > I'm happy to address any comment, both code and doc/test. For the tests I'd like to see: * with file

[RFC PATCH 1/3] btrfs: allow to set compression level for zlib

From: David Sterba Preliminary support for setting compression level for zlib, the following works: $ mount -o compess=zlib # default $ mount -o compess=zlib0# same $ mount -o compess=zlib9# level 9, slower sync, less data $

[RFC PATCH 2/3] btrfs: allow setting zlib compression level via :9

This is bikeshedding, but it seems people are drastically more likely to understand "zlib:9" as compression level rather than an algorithm version compared to "zlib9". Signed-off-by: Adam Borowski --- fs/btrfs/compression.c | 2 ++ fs/btrfs/super.c | 2 +- 2 files

[RFC PATCH 3/3] btrfs: allow setting zstd level

Capped at 15 because of currently used encoding, which is also a reasonable limit because highest levels shine only on blocks much bigger than btrfs' 128KB. Memory is allocated for the biggest supported level rather than for what is actually used. Signed-off-by: Adam Borowski

[RFC 0/3]: settable compression level for zstd

Hi! Here's a patch set that allows changing the compression level for zstd, currently at mount time only. I've played with it for a month, so despite being a quick hack, it's reasonably well tested. Tested on 4.13 + btrfs-for-4.14 only, though -- I've booted 11th-day-of-merge-window only an hour

Re: [PATCH preview] btrfs: allow to set compression level for zlib

On Sun, Aug 20, 2017 at 06:38:50PM -0600, Chris Murphy wrote: > On Fri, Aug 18, 2017 at 10:08 AM, David Sterba wrote: > > > That's quite a lot, in kernel. IIRC zlib and lzo use less than 200kb, > > zstd wants 800kb for level 1. And this needs to be contiguous memory, so > > if

Re: defragmenting best practice?

On 09/15/2017 03:07 PM, Tomasz Kłoczko wrote: > [...] > Case #1 > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu cow2 storage > -> guest BTRFS filesystem > SQL table row insertions per second: 1-2 > > Case #2 > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu raw storage -> >

Re: [PATCH v2 7/7] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On 2017年09月15日 20:56, David Sterba wrote: On Wed, Sep 13, 2017 at 08:32:50AM +0800, Qu Wenruo wrote: On 2017年09月13日 02:07, David Sterba wrote: On Tue, Sep 12, 2017 at 07:50:19PM +0200, Goffredo Baroncelli wrote: On 09/12/2017 07:03 PM, David Sterba wrote: Say I want to prepare a minimal

Re: defragmenting best practice?

On 15 September 2017 at 11:54, Michał Sokołowski wrote: [..] >> Just please some example which I can try to replay which ill be >> showing that we have similar results. > > Case #1 > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu cow2 storage > -> guest BTRFS

Re: [PATCH v2 7/7] btrfs-progs: Doc/mkfs: Add extra condition for rootdir option

On Wed, Sep 13, 2017 at 08:32:50AM +0800, Qu Wenruo wrote: > On 2017年09月13日 02:07, David Sterba wrote: > > On Tue, Sep 12, 2017 at 07:50:19PM +0200, Goffredo Baroncelli wrote: > >> On 09/12/2017 07:03 PM, David Sterba wrote: > >>> Say I want to prepare a minimal image but will provide a large file

Re: snapshots of encrypted directories?

On 2017-09-14 23:45, Andrei Borzenkov wrote: 14.09.2017 18:32, Hugo Mills пишет: On Thu, Sep 14, 2017 at 04:57:39PM +0200, Ulli Horlacher wrote: I use encfs on top of btrfs. I can create btrfs snapshots, but I have no suggestive access to the files in these snaspshots, because they look like:

Re: defragmenting best practice?

On 2017-09-14 22:26, Tomasz Kłoczko wrote: On 14 September 2017 at 19:53, Austin S. Hemmelgarn wrote: [..] While it's not for BTRFS< a tool called e4rat might be of interest to you regarding this. It reorganizes files on an ext4 filesystem so that stuff used by the boot

Re: defragmenting best practice?

> Case #1 > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs -> qemu cow2 storage > -> guest BTRFS filesystem > SQL table row insertions per second: 1-2 "Doctor, if I stab my hand with a fork it hurts a lot: can you cure that?" > Case #2 > 2x 7200 rpm HDD -> md raid 1 -> host BTRFS rootfs ->

Re: defragmenting best practice?

On 09/14/2017 07:48 PM, Tomasz Kłoczko wrote: > On 14 September 2017 at 16:24, Kai Krakow wrote: > [..] >> > Getting e.g. boot files into read order or at least nearby improves >> > boot time a lot. Similar for loading applications. > [...] > Just please some example which I

Re: snapshots of encrypted directories?

2017-09-15 12:01 GMT+02:00 Ulli Horlacher : > On Fri 2017-09-15 (06:45), Andrei Borzenkov wrote: > >> The actual question is - do you need to mount each individual btrfs >> subvolume when using encfs? > > And even worse it goes with ecryptfs: I do not know at all how

RE: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs- > ow...@vger.kernel.org] On Behalf Of Marat Khalili > Sent: Friday, 15 September 2017 7:50 PM > To: Hugo Mills ; Goffredo Baroncelli > ; linux-btrfs

Re: snapshots of encrypted directories?

On Fri 2017-09-15 (06:45), Andrei Borzenkov wrote: > The actual question is - do you need to mount each individual btrfs > subvolume when using encfs? And even worse it goes with ecryptfs: I do not know at all how to mount a snapshot, so that the user has access to it. It seems snapshots are

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

May I state my user's point of view: I know one applications that uses O_DIRECT, and it is subtly broken on BTRFS. I know no applications that use O_DIRECT and are not broken. (Really more statistics would help here, probably some exist that provably work.) According to developers making

Re: [PATCH] Btrfs: incremental send, apply asynchronous page cache readahead

Much appreciate your suggestion. I've modified the patch based on your advice and sent out a new patch with new subject "Btrfs: send, apply asynchronous page cache readahead to enhance page read". Filipe Manana 於 2017-09-13 18:45 寫到: On Wed, Sep 13, 2017 at 7:38 AM, peterh

[PATCH] Btrfs: send, apply asynchronous page cache readahead to enhance page read

From: Kuanling Huang By analyzing the perf on btrfs send, we found it take large amount of cpu time on page_cache_sync_readahead. This effort can be reduced after switching to asynchronous one. Overall performance gain on HDD and SSD were 9 and 15 percent if simply send a

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On Fri, Sep 15, 2017 at 08:04:35AM +0200, Goffredo Baroncelli wrote: > On 09/15/2017 12:18 AM, Hugo Mills wrote: > >As far as I know, both of these are basically known issues, with no > > good solution, other than not using O_DIRECT. Certainly the first > > issue is one I recognise. The second

Re: BUG: BTRFS and O_DIRECT could lead to wrong checksum and wrong data

On 09/15/2017 12:18 AM, Hugo Mills wrote: >As far as I know, both of these are basically known issues, with no > good solution, other than not using O_DIRECT. Certainly the first > issue is one I recognise. The second isn't one I recognise directly, > but is unsurprising to me. > >There