Re: [PATCH] generic/159: record error value correctly

hello, On 08/30/2016 12:45 PM, Darrick J. Wong wrote: On Tue, Aug 30, 2016 at 12:00:44PM +0800, Wang Xiaoguang wrote: Hi, On 08/30/2016 11:59 AM, Eryu Guan wrote: On Tue, Aug 30, 2016 at 11:51:14AM +0800, Wang Xiaoguang wrote: This case failed in btrfs, it's because when trying to open an

Re: [PATCH] generic/159: record error value correctly

On Tue, Aug 30, 2016 at 12:00:44PM +0800, Wang Xiaoguang wrote: > Hi, > > On 08/30/2016 11:59 AM, Eryu Guan wrote: > >On Tue, Aug 30, 2016 at 11:51:14AM +0800, Wang Xiaoguang wrote: > >>This case failed in btrfs, it's because when trying to open an immutable > >>file, vfs returns EPERM, not

Re: [PATCH] generic/159: record error value correctly

Hi, On 08/30/2016 11:59 AM, Eryu Guan wrote: On Tue, Aug 30, 2016 at 11:51:14AM +0800, Wang Xiaoguang wrote: This case failed in btrfs, it's because when trying to open an immutable file, vfs returns EPERM, not EACCESS, fix this. Signed-off-by: Wang Xiaoguang ---

Re: [PATCH] generic/159: record error value correctly

On Tue, Aug 30, 2016 at 11:51:14AM +0800, Wang Xiaoguang wrote: > This case failed in btrfs, it's because when trying to open an immutable > file, vfs returns EPERM, not EACCESS, fix this. > > Signed-off-by: Wang Xiaoguang > --- > tests/generic/159.out | 2 +- > 1

[PATCH] generic/159: record error value correctly

This case failed in btrfs, it's because when trying to open an immutable file, vfs returns EPERM, not EACCESS, fix this. Signed-off-by: Wang Xiaoguang --- tests/generic/159.out | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/generic/159.out

[PATCH 2/2] btrfs-progs: fuzz-test: Add image for unaligned tree block ptr

From: Lukas Lueg Add test case image for unaligned tree block ptr. It should lead to BUG_ON in free_extent_buffer(). Signed-off-by: Lukas Lueg Signed-off-by: Qu Wenruo --- .../images/unaligned-tree-block-bytenr.raw.txt

[PATCH 1/2] btrfs-progs: fsck: Do early check for read_tree_block

Although we have enhanced read_tree_block() from a lot of different aspects, it lacks the early bytenr/blocksize alignment check. And the lack of such check can lead to strange use-after-free bugs, due to the fact that alloc_extent_buffer() will free overlapping extent buffers, and allocate new

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

hello, On 08/29/2016 11:52 PM, Ronan Arraes Jardim Chagas wrote: Hi guys, I just have the problem again. Now, it happens during the lunch time when the machine was idle. Only the system processes were running. It was not the first time that I saw this problem just after lunch when the machine

[PATCH] btrfs-progs: fuzz-test: Add image for wrong chunk item in root tree

From: Lukas Lueg Reported by Lukas and the same image from him. DATA_RELOC tree's key type is modifed to CHUNK_ITEM, causing btrfsck interpret it as CHUNK_ITEM and cause 0 num_stripes. Add the image to fuzz-test. Signed-off-by: Lukas Lueg

Re: [PATCH] btrfs-progs: check: skip shared node or leaf check for low_memory mode

At 08/29/2016 11:20 PM, David Sterba wrote: On Thu, Aug 25, 2016 at 01:30:09PM +0800, Wang Xiaoguang wrote: Hi, On 08/24/2016 08:44 PM, David Sterba wrote: On Fri, Aug 19, 2016 at 05:59:46PM +0800, Wang Xiaoguang wrote: The basic idea is simple. Assume a middle tree node A is shared and

Re: [PATCH 0/4] Fix chunk num_stripes FPE error found by fuzzed image

At 08/30/2016 12:28 AM, David Sterba wrote: On Mon, Aug 29, 2016 at 04:08:58PM +0800, Qu Wenruo wrote: This patchset will fix chunk num_stripes FPE error, by introducing another 2 layers of check. 1) Check key type validation against leaf owner(for original mode) This will prevent invalid

Re: btrfs send extremely slow (almost stuck)

At 08/29/2016 06:02 PM, Oliver Freyermuth wrote: Am 29.08.2016 um 04:11 schrieb Qu Wenruo: Unknown bug, while unfortunately no good idea to solve yet. I sent a RFC patch to completely disable shared extent detection, while got strong objection. I also submitted some other ideas on fixing

Re: Recommendation on raid5 drive error resolution

When I can get this stupid box to boot from an external drive I'll have some idea of what is going on -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

On 8/29/16 11:52 AM, Ronan Arraes Jardim Chagas wrote: > Hi guys, > > I just have the problem again. Now, it happens during the lunch time > when the machine was idle. Only the system processes were running. It > was not the first time that I saw this problem just after lunch when > the machine

Re: `btrfs dev del` fails with `No space left on device`

On Mon, Aug 29, 2016 at 10:04 AM, ojab // wrote: > Hi, > [and I hope that this message will not be sent during compose] > I've had BTRFS filesystem with two 1Tb drives (sdb1 & sdc1), data > raid0 & metadata raid1. I need to replace one drive with 2Tb drive, so > I've done `btrfs dev

Re: Multiple bugs found by fuzzing BTRFS

I'll report new issues to bz as they turn up from the current round only if they represent a yet unreported kind of problem (e.g. there are stack-based buffer over- and underruns lurking, I lost them due to a bug in my setup, though). The next round will be much faster as I've now vastly improved

Re: btrfs send extremely slow (almost stuck)

Am Sun, 28 Aug 2016 17:41:22 -0400 schrieb james harvey : > On Sun, Aug 28, 2016 at 12:15 PM, Oliver Freyermuth > wrote: > > For me, this means I have to stay with rsync backups, which are > > sadly incomplete since special FS attrs like "C"

Re: Carefully crafted BTRFS-image causes kernel to crash

On Tue, Apr 21, 2015 at 07:44:12AM -0400, Austin S Hemmelgarn wrote: > On 2015-04-21 05:38, Russell Coker wrote: > > On Tue, 21 Apr 2015, Qu Wenruo wrote: > >> Although we may add extra check for such problem to improve robustness, > >> but IMHO it's not a real world

Re: Multiple bugs found by fuzzing BTRFS

Hi, thanks for the testing and reports. On Mon, Aug 29, 2016 at 08:06:24AM +0200, Lukas Lueg wrote: > I've now spent around 160 hours of fuzzing BTRFS, here are the crashes > I found so far. Every type of crash is reported only once although > there are usually multiple locations where they show

Re: [PATCH 0/4] Fix chunk num_stripes FPE error found by fuzzed image

On Mon, Aug 29, 2016 at 04:08:58PM +0800, Qu Wenruo wrote: > This patchset will fix chunk num_stripes FPE error, by introducing another 2 > layers of check. > > 1) Check key type validation against leaf owner(for original mode) >This will prevent invalid key, like CHUNK_ITEM key in root tree,

Re: [PATCH 1/3] btrfs-progs: check: remove unused found_key variable in walk_down_tree()

On Thu, Aug 25, 2016 at 01:20:59PM +0800, Wang Xiaoguang wrote: > Signed-off-by: Wang Xiaoguang > --- > cmds-check.c | 5 - > 1 file changed, 5 deletions(-) > > diff --git a/cmds-check.c b/cmds-check.c > index 0ddfd24..1cd0421 100644 > --- a/cmds-check.c > +++

`btrfs dev del` fails with `No space left on device`

Hi, [and I hope that this message will not be sent during compose] I've had BTRFS filesystem with two 1Tb drives (sdb1 & sdc1), data raid0 & metadata raid1. I need to replace one drive with 2Tb drive, so I've done `btrfs dev add /dev/sdd /mnt/xxx` and now trying to do `btrfs dev del /dev/sdc1

`btrfs dev del` fails with `No space left on device`

Hi, I've had BTRFS filesystem with two 1Tb drives (sdb1 & sdc1), data raid0 & metadata raid1. I need to replace one drive with 2Tb drive, so I've done `btrfs dev add /dev/sdd /mnt/xxx` and now trying to do `btrfs dev del /dev/sdc1 /mnt/xxx` which fails due to `ERROR: error removing device

`btrfs dev delz fails with `No space left on device`

Hi, I've had BTRFS filesystem with two 1Tb drives (sdb1 & sdc1), data raid0 & metadata raid1. I need to replace one drive with 2Tb drive, so I've done `btrfs dev add /dev/sdd /mnt/xxx` and now trying to do `btrfs dev del /dev/sdc1 /mnt/xxx` which fails due to `ERROR: error removing device

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

Hi guys, I just have the problem again. Now, it happens during the lunch time when the machine was idle. Only the system processes were running. It was not the first time that I saw this problem just after lunch when the machine stayed idle for a long period (+- 1h).  Here is the information

Re: [PATCH] btrfs-progs: check: skip shared node or leaf check for low_memory mode

On Thu, Aug 25, 2016 at 01:30:09PM +0800, Wang Xiaoguang wrote: > Hi, > > On 08/24/2016 08:44 PM, David Sterba wrote: > > On Fri, Aug 19, 2016 at 05:59:46PM +0800, Wang Xiaoguang wrote: > >> The basic idea is simple. Assume a middle tree node A is shared and > >> its referenceing fs/file tree

Re: memory overflow or undeflow in free space tree / space_info?

Hi Josef, this still hapens with current 4.8-rc* releases. Anything i can do to debug this? May be insert some code to check for an under or overflow in the code? Stefan Am 14.08.2016 um 17:22 schrieb Stefan Priebe - Profihost AG: > Hi Josef, > > anything i could do or test? Results with a

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

Hi! Em Seg, 2016-08-29 às 20:12 +0800, Wang Xiaoguang escreveu: > When strange ENOSPC errors occur, I think "btrfs fi usage" > or "btrfs di df" do not help too much. Their output do not > reflect btrfs kernel current status :) > > Would you please provide attribute files' values in  >

Re: btrfs and systemd

> Why not just create a Systemd unit (or whatever the proper term is) that > runs on boot and runs the mount command manually and doesn't wait for it to > return? Seems easier than messing with init systems. Exactly: Never "mess" with inits.. -- To unsubscribe from this list: send the line

Re: linux 4.7.2 & btrfs & rsync & OOM gone crazy

Didn't go so well unfortunately, system ended up panicking: Out of memory and no killable processes. So I guess I'll be staying on 4.6 for a bit longer. On Sat, Aug 27, 2016 at 9:57 AM, E V wrote: > OOM killer still killed rsync with swappiness 0, so rebuilt and > rebooted

Re: BTRFS constantly reports "No space left on device" even with a huge unallocated space

hello, On 08/13/2016 01:36 AM, Ronan Arraes Jardim Chagas wrote: Hi guys, I'm facing a daily problem with BTRFS. Almost everyday, I get the message "No space left on device". Sometimes I can recover by balancing the system but sometimes even balancing does not work due to the lack of space. In

[PATCH] btrfs: add ENOSPC check for temporarily allocated btrfs_block_rsv

Signed-off-by: Wang Xiaoguang --- fs/btrfs/extent-tree.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 56a9c59..602e410 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -5552,6 +5552,9

Re: Raid 5 to raid 1: balance hangs and scrub aborts. Is this salvageable?

I converted my significantly smaller raid 5 array to raid 1 a little less than a year ago now and I encountered some similar issues. What i ended up doing was starting balance again and again with slightly different arguments (usually thresholds for what blocks to move) and eventually (a week or

Re: btrfs and systemd

On 2016-08-29 07:18, Imran Geriskovan wrote: I can't find any fstab setting for systemd to higher this timeout. There's just the x-systemd.device-timeout but this controls how long to wait for the device and not for the mount command. Is there any solution for big btrfs volumes and systemd?

Re: btrfs and systemd

Do you try: nofail,noauto,x-systemd.automount ? 2016-08-29 9:28 GMT+03:00 Stefan Priebe - Profihost AG : > Hi Qu, > > Am 29.08.2016 um 03:48 schrieb Qu Wenruo: >> >> >> At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote: >>> Hi, >>> >>> i'm trying to get my 60TB

RE: btrfs and systemd

> -Original Message- > From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs- > ow...@vger.kernel.org] On Behalf Of Imran Geriskovan > Sent: Monday, 29 August 2016 9:19 PM > To: Stefan Priebe - Profihost AG > Cc: Qu Wenruo ;

Re: btrfs and systemd

>>> I can't find any fstab setting for systemd to higher this timeout. >>> There's just the x-systemd.device-timeout but this controls how long to >>> wait for the device and not for the mount command. >>> Is there any solution for big btrfs volumes and systemd? >>> Stefan Switch to Runit.

[PATCH v2] btrfs: should block unused block groups deletion work when allocating data space

cleaner_kthread() may run at any time, in which it'll call btrfs_delete_unused_bgs() to delete unused block groups. Because this work is asynchronous, it may also result in false ENOSPC error. Please see below race window: CPU1 | CPU2

Re: btrfs send extremely slow (almost stuck)

Am 29.08.2016 um 04:11 schrieb Qu Wenruo: > Unknown bug, while unfortunately no good idea to solve yet. > > I sent a RFC patch to completely disable shared extent detection, while > got strong objection. > > I also submitted some other ideas on fixing it, while still got strong > objection.

Raid 5 to raid 1: balance hangs and scrub aborts. Is this salvageable?

Following the recent posts on the mailing list I'm trying to convert a running raid5 system to raid1. This conversion fails to complete with checksum verify failures. Running a scrub does not fix these checksum failures and moreover scrub itself aborts after ~9TB (despite repeated tries). All

Re: does btrfs-receive use/compare the checksums from the btrfs-send side?

At 08/28/2016 11:46 AM, Christoph Anton Mitterer wrote: Hey. I've often wondered: When I do a send/receive, does the receiving side use the checksums from the sending side (either by directly storing them or by comparing them with calculated checksums and failing if they don't match after the

[PATCH 0/4] Fix chunk num_stripes FPE error found by fuzzed image

This patchset will fix chunk num_stripes FPE error, by introducing another 2 layers of check. 1) Check key type validation against leaf owner(for original mode) This will prevent invalid key, like CHUNK_ITEM key in root tree, to be passed to later check. For original mode, it uses

[PATCH 4/4] btrfs-progs: Do extra chunk check before processing chunk item

Current we only do chunk validation check at mount time. It's good for most case, but for fuzzed or manually crafted images, we can insert a CHUNK_ITEM key into root tree. Since mount time check will only check chunk tree, it will not check CHUNK_ITEM in root tree. Even with previous key type

[PATCH 1/4] btrfs-progs: Enhance and export print_objectid function

This function is quite useful for a lot of error report. Enhance it to support custom output other than stdout. And export it for later btrfsck enhancement. Signed-off-by: Qu Wenruo --- print-tree.c | 59 ++-

[PATCH 3/4] btrfs-progs: Ignore invalid key in invalid root

Btrfs tree implies a lot of restriction on which key types are allowed in specific roots. Like CHUNK_ITEM keys are only valid in chunk root. This patch will add such check at run_next_block() for original mode. Reported-by: Lukas Lueg Signed-off-by: Qu Wenruo

[PATCH 2/4] btrfs-progs: Enhance and export print_key_type function

Just the same thing done for print_objectid(). Signed-off-by: Qu Wenruo --- print-tree.c | 86 ++-- print-tree.h | 1 + 2 files changed, 44 insertions(+), 43 deletions(-) diff --git a/print-tree.c b/print-tree.c

Re: Crash in btrfs_uuid_tree_iterate during mount

On 08/08/2016 05:24 PM, Chris Mason wrote: > On 08/08/2016 10:21 AM, Nikolay Borisov wrote: >> >> >> On 08/08/2016 05:16 PM, Chris Mason wrote: >>> >>> Fantastic, thanks again for digging through it. Making the patch is >>> much easier than testing the patch in this case. If you can trigger

Re: Multiple bugs found by fuzzing BTRFS

I will let the current setup run for another 200 hours and deal with low memory mode then. Having had a quick glance at it, at least some of the bugs mentioned above show up and should get fix beforehand. 2016-08-29 8:20 GMT+02:00 Qu Wenruo : > Thanks for your fuzzing

Re: btrfs and systemd

Hi Qu, Am 29.08.2016 um 03:48 schrieb Qu Wenruo: > > > At 08/29/2016 04:15 AM, Stefan Priebe - Profihost AG wrote: >> Hi, >> >> i'm trying to get my 60TB btrfs volume to mount with systemd at boot. >> But this always fails with: "mounting timed out. Stopping." after 90s. > > 60TB is quite

Re: Multiple bugs found by fuzzing BTRFS

Thanks for your fuzzing images. Quite helpful. At 08/29/2016 02:06 PM, Lukas Lueg wrote: Hi, I've now spent around 160 hours of fuzzing BTRFS, here are the crashes I found so far. Every type of crash is reported only once although there are usually multiple locations where they show up

Multiple bugs found by fuzzing BTRFS

Hi, I've now spent around 160 hours of fuzzing BTRFS, here are the crashes I found so far. Every type of crash is reported only once although there are usually multiple locations where they show up (especially heap-use-after-free and calls to abort()). The following bug reports have attached to