[PATCH 2/2] btrfs-progs: corrupt-block: Add node corruption without transaction.

2014-10-27 Thread Qu Wenruo
Just like leaf corruption without using transaction, this patch will corrupt node, doing much more damage to the filesystem, for later leaf/node repair patches. Signed-off-by: Qu Wenruo --- btrfs-corrupt-block.c | 45 - 1 file changed, 36 insertions(+)

[PATCH 1/2] btrfs-progs: corrupt-block: Add leaf corruption without transaction.

2014-10-27 Thread Qu Wenruo
In fact, a lot of btrfs corruption like byte corruption will cause the tree block csum mismatch. However most of the corruption provided by btrfs-corrupt-block will use btrfs transaction and recalculate the csum, which is somewhat far away from the real world corruption. This patch will add the le

Re: Does btrfs-restore report missing/corrupt files?

2014-10-27 Thread Duncan
Robert White posted on Mon, 27 Oct 2014 17:39:13 -0700 as excerpted: > On 10/26/2014 12:59 AM, Christian Tschabuschnig wrote: >> >> Hello, >> >> currently I am trying to recover a btrfs filesystem which had a few >> subvolumes. When running # btrfs restore -sx /dev/xxx . >> one subvolume gets rest

Re: Btrfs-progs release 3.17

2014-10-27 Thread Gui Hecheng
On Thu, 2014-10-23 at 21:36 +0800, Anand Jain wrote: > > there is no point in re-creating so many btrfs kernel's logic in user > space. its just unnecessary, when kernel is already doing it. use > some interface to get info from kernel after device is registered, > (not necessarily mounted

Re: Btrfs-progs release 3.17

2014-10-27 Thread Gui Hecheng
On Thu, 2014-10-23 at 15:23 +0200, Petr Janecek wrote: > Hello Gui, > > > Oh, it seems that there are btrfs with missing devs that are bringing > > troubles to the @open_ctree_... function. > > what do you mean by "missing devs"? I have no degraded fs. Ah, sorry, I'm too focused on the problem

Re: [PATCH] Btrfs: fix race that makes btrfs_lookup_extent_info miss skinny extent items

2014-10-27 Thread Miao Xie
On Mon, 27 Oct 2014 13:44:22 +, Filipe David Manana wrote: > On Mon, Oct 27, 2014 at 12:11 PM, Filipe David Manana > wrote: >> On Mon, Oct 27, 2014 at 11:08 AM, Miao Xie wrote: >>> On Mon, 27 Oct 2014 09:19:52 +, Filipe Manana wrote: We have a race that can lead us to miss skinny ext

Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root

2014-10-27 Thread Qu Wenruo
Original Message Subject: Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root From: Qu Wenruo To: Ansgar Hockmann-Stolle , Date: 2014年10月28日 09:05 Original Message Subject: Re: btrfs unmountable: read block failed check_tr

Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root

2014-10-27 Thread Qu Wenruo
Original Message Subject: Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root From: Ansgar Hockmann-Stolle To: Date: 2014年10月28日 07:03 Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: Hi! My btrfs system partition went readonly. After re

Re: Does btrfs-restore report missing/corrupt files?

2014-10-27 Thread Robert White
On 10/26/2014 12:59 AM, Christian Tschabuschnig wrote: Hello, currently I am trying to recover a btrfs filesystem which had a few subvolumes. When running # btrfs restore -sx /dev/xxx . one subvolume gets restored. Important Aside: The one time I had to resort to btrfs restore I didn't get

Re: BTRFS balance segfault, where to go from here

2014-10-27 Thread Duncan
Chris Murphy posted on Mon, 27 Oct 2014 10:51:16 -0600 as excerpted: > On Oct 27, 2014, at 3:26 AM, Stephan Alz wrote: >> >> My question is where to go from here? What I going to do right now is >> to copy the most important data to another separated XFS drive. >> What I planning to do is: >> >

Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root

2014-10-27 Thread Duncan
Ansgar Hockmann-Stolle posted on Mon, 27 Oct 2014 14:23:19 +0100 as excerpted: > Hi! > > My btrfs system partition went readonly. After reboot it doesnt mount > anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm > on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the

Re: btrfs unmountable: read block failed check_tree_block; Couldn't read tree root

2014-10-27 Thread Ansgar Hockmann-Stolle
Am 27.10.14 um 14:23 schrieb Ansgar Hockmann-Stolle: Hi! My btrfs system partition went readonly. After reboot it doesnt mount anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 GB SSD to some USB file a

Re: [PATCH] Btrfs: fix snapshot inconsistency after a file write followed by truncate

2014-10-27 Thread Chris Mason
On Tue, Oct 21, 2014 at 6:12 AM, Filipe Manana wrote: If right after starting the snapshot creation ioctl we perform a write against a file followed by a truncate, with both operations increasing the file's size, we can get a snapshot tree that reflects a state of the source subvolume's tree w

RAID1 fails to recover chunk tree

2014-10-27 Thread Zack Coffey
Revisit of a previous issue. Setup a single 640GB drive with BTRFS and compression. This was not a system drive, just a place to put random junk. Made a RAID1 with another drive of just the metadata. Was in that state for less than 12 hours-ish, removed the second drive and now cannot get to any d

Re: suspicious number of devices: 72057594037927936

2014-10-27 Thread Hugo Mills
On Mon, Oct 27, 2014 at 11:21:13AM -0700, Christian Kujau wrote: > On Mon, 27 Oct 2014 at 16:35, David Sterba wrote: > > Yeah sorry, I sent the v2 too late, here's an incremental that applies > > on top of current 3.18-rc > > > > https://patchwork.kernel.org/patch/5160651/ > > Yup, that fixes it.

Re: suspicious number of devices: 72057594037927936

2014-10-27 Thread Christian Kujau
On Mon, 27 Oct 2014 at 16:35, David Sterba wrote: > Yeah sorry, I sent the v2 too late, here's an incremental that applies > on top of current 3.18-rc > > https://patchwork.kernel.org/patch/5160651/ Yup, that fixes it. Thank you! If it's needed: Tested-by: Christian Kujau @Filipe: and thanks

Re: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Jasper Verberk
I already tried balancing with dusage=1. This removed all the allocated space. It still doesn't allow me to do the conversion though, as it then reallocates the space. > On 27 okt. 2014, at 18:28, Chris Murphy wrote: > > >> On Oct 27, 2014, at 10:54 AM, Jasper Verberk wrote: >> >> I actua

Re: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Chris Murphy
On Oct 27, 2014, at 10:54 AM, Jasper Verberk wrote: > I actually tried going to raid1 with kernel 3.10 beforethe reason I > updated to 3.17 was to see if this would fix the error. > > It still remained though…. If you have btrfs-progs you could do a btrfs check without repair and see if

RE: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Jasper Verberk
I actually tried going to raid1 with kernel 3.10 beforethe reason I updated to 3.17 was to see if this would fix the error.  It still remained though > Subject: Re: Problem converting data raid0 to raid1: enospc errors during > balance > From: l

Re: BTRFS balance segfault, where to go from here

2014-10-27 Thread Chris Murphy
On Oct 27, 2014, at 3:26 AM, Stephan Alz wrote: > > My question is where to go from here? What I going to do right now is to copy > the most important data to another separated XFS drive. > What I planning to do is: > > 1, Upgrade the kernel > 2, Upgrade BTRFS > 3, Continue the balancing. Def

Re: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Chris Murphy
On Oct 27, 2014, at 9:56 AM, Jasper Verberk wrote: > These are the results to a normal df: > > http://paste.debian.net/128932/ > > The mountpoint is /data. OK so this is with the new computation in kernel 3.17 (which I think contains a bug by counting free space twice); so now it shows avail

RE: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Jasper Verberk
Hej guys! Thanks for your input on the issue this far. Too my knowledge raid1 in btrfs means 2 copies of each piece of data independent of the amount of disks used. So 4 x 2,73tb would result in a totaal storage of roughly 5,5tb right? Shouldn't this be more then enough? btw, here is the outp

Re: suspicious number of devices: 72057594037927936

2014-10-27 Thread David Sterba
On Mon, Oct 27, 2014 at 10:57:59AM +, Filipe David Manana wrote: > > The only thing fancy may be the machine: PowerBook G4 (powerpc 32 bit), > > running Debian/Linux (stable). > > > > The message comes from the newly added fs/btrfs/disk-io.c: > > > > if (sb->num_devices > (1UL << 31)) > >

Re: Problem converting data raid0 to raid1: enospc errors during balance

2014-10-27 Thread Chris Murphy
> > > On Oct 26, 2014, at 7:40 PM, Qu Wenruo wrote: > > BTW what's the output of 'df' command? Jasper, What do you get for the conventional df command when this btrfs volume is mounted? Thanks. Chris Murphy-- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body o

btrfs unmountable: read block failed check_tree_block; Couldn't read tree root

2014-10-27 Thread Ansgar Hockmann-Stolle
Hi! My btrfs system partition went readonly. After reboot it doesnt mount anymore. System was openSUSE 13.1 Tumbleweed (kernel 3.17.??). Now I'm on openSUSE 13.2-RC1 rescue (kernel 3.16.3). I dumped (dd) the whole 250 GB SSD to some USB file and tried some btrfs tools on another copy per loop

Re: [PATCH] Btrfs: fix race that makes btrfs_lookup_extent_info miss skinny extent items

2014-10-27 Thread Filipe David Manana
On Mon, Oct 27, 2014 at 12:11 PM, Filipe David Manana wrote: > On Mon, Oct 27, 2014 at 11:08 AM, Miao Xie wrote: >> On Mon, 27 Oct 2014 09:19:52 +, Filipe Manana wrote: >>> We have a race that can lead us to miss skinny extent items in the function >>> btrfs_lookup_extent_info() when the skin

Re: nocow and compression

2014-10-27 Thread Swâmi Petaramesh
As far as I understood, "NOCOW" means that modified parts of files be rewritten into place, whereas compression causes compressed blocks of variable sizes to be created (depending upon their compression ratio). Changing a block in a file will most probably change its compressed size, and then yo

Re: nocow and compression

2014-10-27 Thread Marc Dietrich
Am Montag, 27. Oktober 2014, 13:59:24 schrieb Swâmi Petaramesh: > Le lundi 27 octobre 2014, 13:56:07 Marc Dietrich a écrit : > > oops, no compression. > > Is this intended? > > « Compression does not work for NOCOW files » is clearly stated in > > https://btrfs.wiki.kernel.org/index.php/Compressi

Re: nocow and compression

2014-10-27 Thread Swâmi Petaramesh
Le lundi 27 octobre 2014, 13:56:07 Marc Dietrich a écrit : > oops, no compression. > Is this intended? « Compression does not work for NOCOW files » is clearly stated in https://btrfs.wiki.kernel.org/index.php/Compression#How_does_compression_interact_with_direct_IO_or_COW.3F Kind regards. --

nocow and compression

2014-10-27 Thread Marc Dietrich
Hi, I created a filesystem and mounted it with compress-force=lzo. Then I did: # df -h . Filesystem Size Used Avail Use% Mounted on /dev/loop0 100M 4.1M 96M 5% /mnt # yes "Hello World" | dd of=/mnt/test iflag=fullblock bs=1M count=20 status=none yes: standard output: Broken pipe

[PATCH] btrfs: use macro accessors in superblock validation checks

2014-10-27 Thread David Sterba
The initial patch c926093ec516f5d316 (btrfs: add more superblock checks) did not properly use the macro accessors that wrap endianness and the code would not work correctly on big endian machines. Reported-by: Qu Wenruo Signed-off-by: David Sterba --- fs/btrfs/disk-io.c | 43 +++

[PATCH] btrfs: get the accurate value of used_bytes in btrfs_get_block_group_info().

2014-10-27 Thread Dongsheng Yang
Reproducer: # mkfs.btrfs -f -b 20G /dev/sdb # mount /dev/sdb /mnt/test # fallocate -l 17G /mnt/test/largefile # btrfs fi df /mnt/test Data, single: total=17.49GiB, used=6.00GiB <- only 6G, but actually it should be 17G. System, DUP: total=8.00MiB, u

Re: [PATCH] Btrfs: fix race that makes btrfs_lookup_extent_info miss skinny extent items

2014-10-27 Thread Filipe David Manana
On Mon, Oct 27, 2014 at 11:08 AM, Miao Xie wrote: > On Mon, 27 Oct 2014 09:19:52 +, Filipe Manana wrote: >> We have a race that can lead us to miss skinny extent items in the function >> btrfs_lookup_extent_info() when the skinny metadata feature is enabled. >> So basically the sequence of ste

Re: Heavy nocow'd VM image fragmentation

2014-10-27 Thread Austin S Hemmelgarn
On 2014-10-26 13:20, Larkin Lowrey wrote: On 10/24/2014 10:28 PM, Duncan wrote: Robert White posted on Fri, 24 Oct 2014 19:41:32 -0700 as excerpted: On 10/24/2014 04:49 AM, Marc MERLIN wrote: On Thu, Oct 23, 2014 at 06:04:43PM -0500, Larkin Lowrey wrote: I have a 240GB VirtualBox vdi image t

Re: [PATCH] Btrfs: fix race that makes btrfs_lookup_extent_info miss skinny extent items

2014-10-27 Thread Miao Xie
On Mon, 27 Oct 2014 09:19:52 +, Filipe Manana wrote: > We have a race that can lead us to miss skinny extent items in the function > btrfs_lookup_extent_info() when the skinny metadata feature is enabled. > So basically the sequence of steps is: > > 1) We search in the extent tree for the skin

Re: suspicious number of devices: 72057594037927936

2014-10-27 Thread Filipe David Manana
On Mon, Oct 27, 2014 at 10:34 AM, Christian Kujau wrote: > (somehow this message did not make it to the list) > > Hi, > > After upgrading from linux 3.17.0 to 3.18.0-rc2, I cannot mount my btrfs > partition any more. It's just one btrfs partition, no raid, no > compression, no fancy mount options:

[PATCH v2] Btrfs: fix invalid leaf slot access in btrfs_lookup_extent()

2014-10-27 Thread Filipe Manana
If we couldn't find our extent item, we accessed the current slot (path->slots[0]) to check if it corresponds to an equivalent skinny metadata item. However this slot could be beyond our last item in the leaf (i.e. path->slots[0] >= btrfs_header_nritems(leaf)), in which case we shouldn't process it

suspicious number of devices: 72057594037927936

2014-10-27 Thread Christian Kujau
(somehow this message did not make it to the list) Hi, After upgrading from linux 3.17.0 to 3.18.0-rc2, I cannot mount my btrfs partition any more. It's just one btrfs partition, no raid, no compression, no fancy mount options: # mount -t btrfs -o ro /dev/sda6 /usr/local/ mount: wrong fs type,

Re: [PATCH] Btrfs: fix invalid leaf slot access in btrfs_lookup_extent()

2014-10-27 Thread Miao Xie
On Mon, 27 Oct 2014 09:16:55 +, Filipe Manana wrote: > If we couldn't find our extent item, we accessed the current slot > (path->slots[0]) to check if it corresponds to an equivalent skinny > metadata item. However this slot could be beyond our last item in the > leaf (i.e. path->slots[0] >= b

BTRFS balance segfault, where to go from here

2014-10-27 Thread Stephan Alz
Hello Folks, I used to have an array of 4x4TB drives with BTRFS in raid10. The kernel version is: 3.13-0.bpo.1-amd64 BTRFS version is: v3.14.1 When it was reaching 80% in space I added another 4TB drive to the array with: > btrfs device add /dev/sdf /mnt/backup And started the balancing to the

[PATCH] Btrfs: fix race that makes btrfs_lookup_extent_info miss skinny extent items

2014-10-27 Thread Filipe Manana
We have a race that can lead us to miss skinny extent items in the function btrfs_lookup_extent_info() when the skinny metadata feature is enabled. So basically the sequence of steps is: 1) We search in the extent tree for the skinny extent, which returns > 0 (not found); 2) We check the previ

[PATCH] Btrfs: fix invalid leaf slot access in btrfs_lookup_extent()

2014-10-27 Thread Filipe Manana
If we couldn't find our extent item, we accessed the current slot (path->slots[0]) to check if it corresponds to an equivalent skinny metadata item. However this slot could be beyond our last item in the leaf (i.e. path->slots[0] >= btrfs_header_nritems(leaf)), in which case we shouldn't process it

Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation.

2014-10-27 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation. From: Liu Bo To: Qu Wenruo Date: 2014年10月27日 16:14 On Mon, Oct 27, 2014 at 08:18:12AM +0800, Qu Wenruo wrote: Orig

Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm to reduce ENOSPC caused by unbalanced data/metadata allocation.

2014-10-27 Thread Liu Bo
On Mon, Oct 27, 2014 at 08:18:12AM +0800, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH] btrfs: Enhance btrfs chunk allocation algorithm > to reduce ENOSPC caused by unbalanced data/metadata allocation. > From: Liu Bo > To: Qu Wenruo > Date: 2014年10月24日 19:06 > >O

Re: Poll: time to switch skinny-metadata on by default?

2014-10-27 Thread Duncan
Marc Joliet posted on Mon, 27 Oct 2014 02:24:15 +0100 as excerpted: > Am Sat, 25 Oct 2014 14:35:33 -0600 schrieb Chris Murphy > : > > >> On Oct 25, 2014, at 2:33 PM, Chris Murphy >> wrote: >> >> >> > On Oct 25, 2014, at 6:24 AM, Marc Joliet wrote: >> >> >> >> First of all: does grub2 suppor

Re: Poll: time to switch skinny-metadata on by default?

2014-10-27 Thread Duncan
Zygo Blaxell posted on Mon, 27 Oct 2014 00:39:25 -0400 as excerpted: > One thing that may be significant is _when_ those 3 hanging filesystems > are hanging: when using rsync to update local files. These machines > are using the traditional rsync copy-then-rename method rather than > --inplace u