Re: SLES 11 SP4: can't mount btrfs

2017-10-19 Thread Andrei Borzenkov
19.10.2017 23:04, Chris Murphy пишет: > Btrfs > is not just supported by SUSE, it's the default file system. > It is default choice for root starting with SLES12, not in SLES11. But yes, it should still be supported. I do not hold my breath though. For all I can tell transid errors are usually

[PATCH 5/7] btrfs-progs: mkfs/rootdir: Shrink fs for rootdir option

2017-10-19 Thread Qu Wenruo
Use the new dev extent based shrink method for rootdir option. Signed-off-by: Qu Wenruo --- mkfs/main.c| 5 +++ mkfs/rootdir.c | 111 + mkfs/rootdir.h | 1 + 3 files changed, 117 insertions(+) diff --git

[PATCH 6/7] btrfs-progs: mkfs: Update allocation info before verbose output

2017-10-19 Thread Qu Wenruo
Since new --rootdir can allocate chunk, it will modify the chunk allocation result. This patch will update allocation info before verbose output to reflect such info. Signed-off-by: Qu Wenruo --- mkfs/main.c | 33 + 1 file changed, 33

[PATCH 3/7] btrfs-progs: mkfs: Only zero out the first 1M for rootdir

2017-10-19 Thread Qu Wenruo
It's a waste of IO to fill the whole image before creating btrfs on it, just wiping the first 1M, and then write 1 byte to the last position to create a sparse file. Signed-off-by: Qu Wenruo --- mkfs/main.c | 19 +-- 1 file changed, 13 insertions(+), 6

[PATCH 7/7] btrfs-progs: mkfs: Separate shrink from rootdir

2017-10-19 Thread Qu Wenruo
Make --shrink a separate option for --rootdir, and make it default to off. So this will cause less confusion. Signed-off-by: Qu Wenruo --- Documentation/mkfs.btrfs.asciidoc | 11 +++ mkfs/main.c | 27 +-- mkfs/rootdir.c

[PATCH 4/7] btrfs-progs: mkfs/rootdir: Introduce function to get end position of last device extent

2017-10-19 Thread Qu Wenruo
Useful for later 'mkfs.btrfs --rootdir' shrink support. Signed-off-by: Qu Wenruo --- mkfs/rootdir.c | 43 +++ 1 file changed, 43 insertions(+) diff --git a/mkfs/rootdir.c b/mkfs/rootdir.c index 99022afaa030..1ca37996a3b3 100644 ---

[PATCH 2/7] btrfs-progs: mkfs/rootdir: Use over-reserve method to make size estimate easier

2017-10-19 Thread Qu Wenruo
Use an easier method to calculate the estimate device for mkfs.btrfs --rootdir. The new method will over-estimate, but should ensure we won't encounter ENOSPC. It relies on the following data to estimate: 1) number of inodes for metadata chunk size 2) rounded up data size of each regular

[PATCH 1/7] btrfs-progs: mkfs: Don't use custom chunk allocator for rootdir

2017-10-19 Thread Qu Wenruo
Remove these custom chunk allocator for mkfs. Use generic btrfs chunk allocator instead. Signed-off-by: Qu Wenruo --- mkfs/main.c | 75 ++--- 1 file changed, 7 insertions(+), 68 deletions(-) diff --git a/mkfs/main.c

[PATCH 0/7] btrfs-progs: mkfs: Reword --rootdir

2017-10-19 Thread Qu Wenruo
Can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/mkfs_rootdir_rework And fetching from github is preferred method to test, as this patchset has 2 prerequisite: 1) Minimal device size patchset The image size estimate algorithm heavily relies on the minimal device

[PATCH v4 2/2] btrfs-progs: doc: add description of missing and example, of device remove

2017-10-19 Thread Misono, Tomohiro
This patch updates help/document of "btrfs device remove" in two points: 1. Add explanation of 'missing' for 'device remove'. This is only written in wikipage currently. (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices) 2. Add example of device removal in the man

[PATCH v4 0/2] btrfs-progs: doc: update btrfs device remove

2017-10-19 Thread Misono, Tomohiro
This updates help/doc of "btrfs device remove". First patch adds the explanation that delete is the alias of remove to help message. Second patch adds the description of "remove missing", which is currently only written in wikipage, and example of device removal. v1->v2: split the patch and

[PATCH v4 1/2] btrfs-progs: device: add description of alias to help message

2017-10-19 Thread Misono, Tomohiro
State that the 'delete' is the alias of 'remove' as the man page says. Signed-off-by: Tomohiro Misono Reviewed-by: Satoru Takeuchi --- cmds-device.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cmds-device.c

Re: [PATCH v3 11/49] btrfs: avoid access to .bi_vcnt directly

2017-10-19 Thread Ming Lei
On Thu, Aug 10, 2017 at 04:29:59AM -0700, Christoph Hellwig wrote: > > +static unsigned int get_bio_pages(struct bio *bio) > > +{ > > + unsigned i; > > + struct bio_vec *bv; > > + > > + bio_for_each_segment_all(bv, bio, i) > > + ; > > + > > + return i; > > +} > >

Re: [PATCH v8 0/6] Btrfs: populate heuristic with code

2017-10-19 Thread Timofey Titovets
2017-10-19 18:39 GMT+03:00 David Sterba : > On Fri, Sep 29, 2017 at 06:22:00PM +0200, David Sterba wrote: >> On Thu, Sep 28, 2017 at 05:33:35PM +0300, Timofey Titovets wrote: >> > Compile tested, hand tested on live system >> > >> > Change v7 -> v8 >> > - All code moved to

Re: SLES 11 SP4: can't mount btrfs

2017-10-19 Thread Chris Murphy
On Thu, Oct 19, 2017 at 6:43 PM, Lentes, Bernd wrote: > Hi, > > this is the continuation of a thread i started on a SLES forum > (https://forums.suse.com/showthread.php?10109-lv-with-btrfs-corrupt-some-tips-please), > but i think this is the more appropriate

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Peter Grandi
[ ... ] >> are USB drives really that unreliable [ ... ] [ ... ] > There are similar SATA chips too (occasionally JMicron and > Marvell for example are somewhat less awesome than they could > be), and practically all Firewire bridge chips of old "lied" a > lot [ ... ] > That plus Btrfs is

RE: SLES 11 SP4: can't mount btrfs

2017-10-19 Thread Lentes, Bernd
> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs- > ow...@vger.kernel.org] On Behalf Of Lentes, Bernd > Sent: Thursday, October 19, 2017 7:44 PM > To: Btrfs ML > Subject: SLES 11 SP4: can't mount btrfs > > Hi, > > this is the

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Peter Grandi
[ ... ] >>> Oh please, please a bit less silliness would be welcome here. >>> In a previous comment on this tedious thread I had written: > If the block device abstraction layer and lower layers work > correctly, Btrfs does not have problems of that sort when > adding new devices;

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Peter Grandi
> [ ... ] when writes to a USB device fail due to a temporary > disconnection, the kernel can actually recognize that a write > error happened. [ ... ] Usually, but who knows? Maybe half transfer gets written; maybe the data gets written to the wrong address; maybe stuff gets written but failure

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Peter Grandi
> [ ... ] However, the disappearance of the device doesn't get > propagated up to the filesystem correctly, Indeed, sometimes it does, sometimes it does not, in part because of chipset bugs, in part because the USB protocol signaling side does not handle errors well even if the chipset were bug

[PATCH 8/8] btrfs: move btrfs_truncate_block out of trans handle

2017-10-19 Thread Josef Bacik
Since we do a delalloc reserve in btrfs_truncate_block we can deadlock with freeze. If somebody else is trying to allocate metadata for this inode and it gets stuck in start_delalloc_inodes because of freeze we will deadlock. Be safe and move this outside of a trans handle. This also has a

[PATCH 4/8] btrfs: switch args for comp_*_refs

2017-10-19 Thread Josef Bacik
Make it more consistent, we want the inserted ref to be compared against what's already in there. This will make the order go from lowest seq -> highest seq, which will make us more likely to make forward progress if there's a seqlock currently held. Signed-off-by: Josef Bacik

[PATCH 3/8] btrfs: make the delalloc block rsv per inode

2017-10-19 Thread Josef Bacik
The way we handle delalloc metadata reservations has gotten progressively more complicated over the years. There is so much cruft and weirdness around keeping the reserved count and outstanding counters consistent and handling the error cases that it's impossible to understand. Fix this by

[PATCH 6/8] btrfs: track refs in a rb_tree instead of a list

2017-10-19 Thread Josef Bacik
If we get a significant amount of delayed refs for a single block (think modifying multiple snapshots) we can end up spending an ungodly amount of time looping through all of the entries trying to see if they can be merged. This is because we only add them to a list, so we have O(2n) for every

[PATCH 5/8] btrfs: add a comp_refs() helper

2017-10-19 Thread Josef Bacik
Instead of open-coding the delayed ref comparisons, add a helper to do the comparisons generically and use that everywhere. We compare sequence numbers last for following patches. Signed-off-by: Josef Bacik --- fs/btrfs/delayed-ref.c | 54

[PATCH 7/8] btrfs: don't call btrfs_start_delalloc_roots in flushoncommit

2017-10-19 Thread Josef Bacik
We're holding the sb_start_intwrite lock at this point, and doing async filemap_flush of the inodes will result in a deadlock if we freeze the fs during this operation. This is because we could do a btrfs_join_transaction() in the thread we are waiting on which would block at sb_start_intwrite,

[PATCH 2/8] btrfs: add tracepoints for outstanding extents mods

2017-10-19 Thread Josef Bacik
This is handy for tracing problems with modifying the outstanding extents counters. Signed-off-by: Josef Bacik --- fs/btrfs/btrfs_inode.h | 2 ++ include/trace/events/btrfs.h | 21 + 2 files changed, 23 insertions(+) diff --git a/fs/btrfs/btrfs_inode.h

[PATCH 1/8] Btrfs: rework outstanding_extents

2017-10-19 Thread Josef Bacik
Right now we do a lot of weird hoops around outstanding_extents in order to keep the extent count consistent. This is because we logically transfer the outstanding_extent count from the initial reservation through the set_delalloc_bits. This makes it pretty difficult to get a handle on how and

[PATCH 0/8] Remaining queue

2017-10-19 Thread Josef Bacik
Here's the updated batch of the remaining queue of patches from me. I've addressed all of the outstanding review feedback for everything and they've been pretty thoroughly tested. Most of the changes are around changelogs and adding comments, as well as switching to lockdep_assert_held from

Re: [PATCH 01/21] Btrfs: rework outstanding_extents

2017-10-19 Thread Josef Bacik
On Fri, Oct 13, 2017 at 04:55:58PM +0300, Nikolay Borisov wrote: > > > > > The outstanding_extents accounting is consistent with only the items needed > > to > > handle the outstanding extent items. However since changing the inode > > requires > > updating the inode item as well we have to

Re: [PATCH] btrfs: Remove WARN_ON for unaligned device created before v4.13 and adds more user friendly output

2017-10-19 Thread David Sterba
On Sat, Sep 23, 2017 at 03:22:36PM +0800, Qu Wenruo wrote: > >>> --- a/fs/btrfs/volumes.c > >>> +++ b/fs/btrfs/volumes.c > >>> @@ -6472,15 +6472,23 @@ static int read_one_chunk(struct btrfs_fs_info > >>> *fs_info, struct btrfs_key *key, > >>> return 0; > >>> } > >>> > >>> -static

SLES 11 SP4: can't mount btrfs

2017-10-19 Thread Lentes, Bernd
Hi, this is the continuation of a thread i started on a SLES forum (https://forums.suse.com/showthread.php?10109-lv-with-btrfs-corrupt-some-tips-please), but i think this is the more appropriate place. I have a SLES 11 SP4 with a btrfs on top of a logical volume i can't mount anymore. The host

Re: [PATCH 3/3] btrfs: increase output size for LOGICAL_INO_V2 ioctl

2017-10-19 Thread David Sterba
Hi, On Sat, Sep 23, 2017 at 11:06:42PM +0200, Hans van Kranenburg wrote: > Reviewed-by: Hans van Kranenburg > Tested-by: Hans van Kranenburg the patches look good to me and the usecase and testing coverage seem sufficient to take

Re: [PATCH 1/3] btrfs: add a flag to iterate_inodes_from_logical to find all extent refs for uncompressed extents

2017-10-19 Thread David Sterba
On Fri, Sep 22, 2017 at 01:58:45PM -0400, Zygo Blaxell wrote: > The LOGICAL_INO ioctl provides a backward mapping from extent bytenr and > offset (encoded as a single logical address) to a list of extent refs. > LOGICAL_INO complements TREE_SEARCH, which provides the forward mapping > (extent ref

Re: [PATCH] btrfs-progs: qgroup: show subvol path when qgroup show

2017-10-19 Thread David Sterba
On Wed, Oct 18, 2017 at 11:36:21AM +0800, Lu Fengqi wrote: > >> @@ -1140,7 +1249,8 @@ static int __qgroups_search(int fd, struct > >> qgroup_lookup *qgroup_lookup) > >>goto skip; > >>add_qgroup(qgroup_lookup, > >>

[no subject]

2017-10-19 Thread Denis 'GNUtoo' Carikli
subscribe linux-btrfs -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v8 0/6] Btrfs: populate heuristic with code

2017-10-19 Thread David Sterba
On Fri, Sep 29, 2017 at 06:22:00PM +0200, David Sterba wrote: > On Thu, Sep 28, 2017 at 05:33:35PM +0300, Timofey Titovets wrote: > > Compile tested, hand tested on live system > > > > Change v7 -> v8 > > - All code moved to compression.c (again) > > - Heuristic workspaces inmplemented

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Austin S. Hemmelgarn
On 2017-10-19 10:42, Zoltan wrote: On Thu, Oct 19, 2017 at 4:27 PM, Austin S. Hemmelgarn wrote: and thus when the same device reappears (as it will when the disconnect was due to a transient bus error, which happens a lot), it shows up as a different device node, which

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Zoltan
On Thu, Oct 19, 2017 at 4:27 PM, Austin S. Hemmelgarn wrote: > and thus when the same device reappears (as it will when the disconnect was > due to a transient bus error, which happens a lot), it shows up as a > different device node, which gets scanned for filesystems by

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Austin S. Hemmelgarn
On 2017-10-19 09:48, Zoltan wrote: Hi, On Thu, Oct 19, 2017 at 1:01 PM, Peter Grandi wrote: What the OP was doing was using "unreliable" both for the case where the device "lies" and the case where the device does not "lie" but reports a failure. Both of these

Re: [RFC 0/3]: settable compression level for zstd

2017-10-19 Thread David Sterba
On Fri, Sep 15, 2017 at 05:34:57PM +0200, Adam Borowski wrote: > Hi! > Here's a patch set that allows changing the compression level for zstd, > currently at mount time only. I've played with it for a month, so despite > being a quick hack, it's reasonably well tested. Tested on 4.13 + >

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Zoltan
Hi, On Thu, Oct 19, 2017 at 1:01 PM, Peter Grandi wrote: > What the OP was doing was using "unreliable" both for the case > where the device "lies" and the case where the device does not > "lie" but reports a failure. Both of these are malfunctions in a > wide

Re: [PATCH v2 2/2] btrfs: Fix transaction abort during failure in btrfs_rm_dev_item

2017-10-19 Thread Nikolay Borisov
On 19.10.2017 14:54, David Sterba wrote: > On Thu, Sep 28, 2017 at 11:45:27AM +0300, Nikolay Borisov wrote: >> btrfs_rm_dev_item calls several function under an activa transaction, however >> it fails to abort it if an error happens. Fix this by adding explicit >>

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Austin S. Hemmelgarn
On 2017-10-19 07:01, Peter Grandi wrote: [ ... ] Oh please, please a bit less silliness would be welcome here. In a previous comment on this tedious thread I had written: If the block device abstraction layer and lower layers work correctly, Btrfs does not have problems of that sort when

Re: 4.13: "error in btrfs_run_delayed_refs:3009: errno=-28 No space left" with 1.3TB unallocated / 737G free?

2017-10-19 Thread Martin Raiber
On 19.10.2017 10:16 Vladimir Panteleev wrote: > On Tue, 17 Oct 2017 16:21:04 -0700, Duncan wrote: >> * try the balance on 4.14-rc5+, where the known bug should be fixed > > Thanks! However, I'm getting the same error on > 4.14.0-rc5-g9aa0d2dde6eb. The stack trace is different, though: > > Aside

Re: [PATCH v2 2/2] btrfs: Fix transaction abort during failure in btrfs_rm_dev_item

2017-10-19 Thread David Sterba
On Thu, Sep 28, 2017 at 11:45:27AM +0300, Nikolay Borisov wrote: > btrfs_rm_dev_item calls several function under an activa transaction, however > it fails to abort it if an error happens. Fix this by adding explicit > btrfs_abort_transaction/btrfs_end_transaction calls > > Signed-off-by: Nikolay

Re: Is it safe to use btrfs on top of different types of devices?

2017-10-19 Thread Peter Grandi
[ ... ] >> Oh please, please a bit less silliness would be welcome here. >> In a previous comment on this tedious thread I had written: >> > If the block device abstraction layer and lower layers work >> > correctly, Btrfs does not have problems of that sort when >> > adding new devices;

Re: [PATCH] btrfs-progs: change mans to describe the third copy of superblock

2017-10-19 Thread Satoru Takeuchi
At Thu, 19 Oct 2017 17:05:18 +0800, Qu Wenruo wrote: > > > > On 2017年10月19日 16:34, Misono, Tomohiro wrote: > > On 2017/10/19 16:45, Satoru Takeuchi wrote: > >> Some tools can select which superblock these commands use by "-s > >> " > >> option. Although this option says the valid values are

Re: [PATCH] btrfs-progs: change mans to describe the third copy of superblock

2017-10-19 Thread Qu Wenruo
On 2017年10月19日 16:34, Misono, Tomohiro wrote: > On 2017/10/19 16:45, Satoru Takeuchi wrote: >> Some tools can select which superblock these commands use by "-s >> " >> option. Although this option says the valid values are 0-2, we can set 3 >> if filesystem is very large. >> > > Hello, > Wiki

Re: check: "warning line 4144"

2017-10-19 Thread Qu Wenruo
On 2017年10月19日 16:53, Tom Hale wrote: > In running btrfs check, I got the following message: > > warning line 4144 > > Could this be a little more descriptive? > > * Does it mean I should rebuild my FS from scratch? > * Is there anything I can do to remove this warning? > > Complete

check: "warning line 4144"

2017-10-19 Thread Tom Hale
In running btrfs check, I got the following message: warning line 4144 Could this be a little more descriptive? * Does it mean I should rebuild my FS from scratch? * Is there anything I can do to remove this warning? Complete output below:

Re: [PATCH] btrfs-progs: change mans to describe the third copy of superblock

2017-10-19 Thread Misono, Tomohiro
On 2017/10/19 16:45, Satoru Takeuchi wrote: > Some tools can select which superblock these commands use by "-s " > option. Although this option says the valid values are 0-2, we can set 3 > if filesystem is very large. > Hello, Wiki says there are 4 superblocks. However in the implementation

Re: 4.13: "error in btrfs_run_delayed_refs:3009: errno=-28 No space left" with 1.3TB unallocated / 737G free?

2017-10-19 Thread Vladimir Panteleev
On Tue, 17 Oct 2017 16:21:04 -0700, Duncan wrote: * try the balance on 4.14-rc5+, where the known bug should be fixed Thanks! However, I'm getting the same error on 4.14.0-rc5-g9aa0d2dde6eb. The stack trace is different, though: [25886.024757] BTRFS: Transaction aborted (error -28)

Re: Unmountable fs - missing generation?

2017-10-19 Thread Satoru Takeuchi
At Thu, 19 Oct 2017 12:03:08 +0900, satoru takeuchi wrote: > > Resend it since I forgot to CC linux-btrfs ML >Larkin > > On Oct 17, 2017, at 0:16, Larkin Lowrey > wrote: > > I am unable to mount one my my filesystems. The superblock thinks > the latest

[PATCH] btrfs-progs: change mans to describe the third copy of superblock

2017-10-19 Thread Satoru Takeuchi
Some tools can select which superblock these commands use by "-s " option. Although this option says the valid values are 0-2, we can set 3 if filesystem is very large. Signed-off-by: Satoru Takeuchi --- Documentation/btrfs-check.asciidoc| 2 +-