Re: Wrong device?

2017-09-27 Thread Andrei Borzenkov 
26.09.2017 10:31, Lukas Pirl пишет:
> On 09/25/2017 06:11 PM, linux-bt...@oh3mqu.pp.hyper.fi wrote as excerpted:
>> After a long googling (about more complex situations) I suddenly
>> noticed "device sdb" WTF???  Filesystem is mounted from /dev/md3 (sdb
>> is part of that mdraid) so btrfs should not even know anything about
>> that /dev/sdb.
> 
> I would be interested in explanations regarding this too. It happened
> to me as well, that I was confused by /dev/sd* device paths being
> printed by btrfs in the logs, even though it runs on /dev/md-*
> (/dev/mapper/*) devices exclusively.
> 

Could be related:

https://forums.opensuse.org/showthread.php/526696-Upgrade-destroys-dmraid?p=2838207#post2838207




signature.asc
Description: OpenPGP digital signature

Re: Wrong device?

2017-09-27 Thread Ari Saastamoinen 

On Tue, 26 Sep 2017, Duncan wrote:


consider it worth having.  No backups means you are defining the data to
be of less value than the time/trouble/resources to make the backup, so
loss is never a big deal, because either you either have a backup if you


I know importance of backups, and this problematic machine are backups  (I 
still have originals if this crashes)



That's why btrfs is listing one of the md components as part of the
filesystem -- it obviously has the same btrfs UUID as the md device that
you actually created the filesystem on.


# for I in c d e f g h i j k l n o p q r ; do blkid /dev/sd$I ; done ; blkid 
/dev/md3
/dev/sdc: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="cffdd56f-f8d6-5191-bd7b-d528018e1106" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdd: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="5d586780-d22f-a68b-769a-5c1bfe9491fb" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sde: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="5777fb6a-7156-3a70-5384-789ac8dd5656" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdf: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="f017f1c7-df26-9279-ccb9-002b8ce1edd9" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdg: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="0de626d7-f754-e53b-792b-3217f6cb63d2" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdh: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="036e8679-2c6f-4a48-601e-1379cdd0a3eb" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdi: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="f9cb6c12-dc73-92b9-363c-fafb5b0efc81" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdj: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="7f3f09d8-d09d-c263-b292-a9d5d004c7ce" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdk: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="07bde9df-8f5b-518d-78fa-3cff3a950eb4" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdl: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="1895c727-e8b2-463a-4f67-e260ee6e98ff" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdn: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="72a6d25c-a7c0-3329-5f15-8abb7c3edbdb" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdo: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="3580f74a-1605-3e5e-9396-9e929eaeb6fc" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdp: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="006883cb-76f6-2967-1a5a-225d258d72b0" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdq: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="b72b2bc3-d983-ed27-f64a-fea9b27e42bf" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/sdr: UUID="d68e26ee-baeb-f40d-c579-2d506e18d378" 
UUID_SUB="bd7b687d-2882-3709-abe4-03f1a1010a67" LABEL="dyn-50:3" TYPE="linux_raid_member"
/dev/md3: UUID="84a31ba6-d22d-4653-b071-5525dbdfe561" 
UUID_SUB="36cf435b-e589-45dd-99a5-5efdb1439913" TYPE="btrfs"

Those sd* devices are member of md, and seems they aren't equal with that
md-device

--
Ari Saastamoinen
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Wrong device?

2017-09-26 Thread Duncan 
n mind that btrfs, unlike most other filesystems, can be multi-
device.  As such, it needs a way to track which devices are part of each 
filesystem, and it uses the filesystem UUID for that purpose.

Meanwhile, btrfs device scan, which is auto-run by udev after a device 
appears, is what lets the kernel know about all those btrfs containing 
devices and the UUIDs associated with them.

That's why btrfs is listing one of the md components as part of the 
filesystem -- it obviously has the same btrfs UUID as the md device that 
you actually created the filesystem on.

That can cause btrfs to write to the wrong device in some instances, tho 
obviously it doesn't do it all the time or things wouldn't work as well 
as they do.

It's thus recommended that when you're using btrfs on device layering 
such as mdraid or LVM, that you ensure that the lower layer devices 
aren't exposed to btrfs so it doesn't get confused.  I believe LVM can be 
configured to hide the lower layer devices in at least some instances, 
but I'm not sure about mdraid, altho this is the first time I've seen 
that particular issue with it (tho it may simply be because I've not been 
watching for it, Chris Murphy or Hugo are likely to have more information 
about that as they're more active with user support than I am, and are 
more technically skilled too, I believe).

> # btrfs --version btrfs-progs v4.4

As with the kernel version, likewise with the btrfs-progs userspace, tho 
in operation it's less important than the kernel, because for most normal 
operating commands, userspace simply calls the real code in the kernel 
anyway.  But once things begin to go wrong, the userspace version becomes 
more important, because it's the userspace code that handles btrfs check, 
btrfs restore, etc.

So while userspace 4.4 is fine for normal operations, you might want to 
be sure you have a current 4.12 or so available for recovery if needed, 
since it'll have the latest fixes and thus should give you the best 
chance at recovery, if it /is/ needed.

> # btrfs fi usage /data2
> Overall:
>  Device size:  70.95TiB
>  Device allocated:  4.95TiB
>  Device unallocated:   66.01TiB
>  Device missing:  0.00B
>  Used:  4.94TiB
>  Free (estimated): 66.01TiB  (min: 33.01TiB)
>  Data ratio:   1.00
>  Metadata ratio:   2.00
>  Global reserve:  512.00MiB  (used: 0.00B)
> 
> Data,single: Size:4.77TiB, Used:4.76TiB
> /dev/md34.77TiB
> 
> Metadata,DUP: Size:92.00GiB, Used:90.79GiB
> /dev/md3  184.00GiB
> 
> System,DUP: Size:32.00MiB, Used:592.00KiB
> /dev/md3   64.00MiB
> 
> Unallocated:
> /dev/md3   66.01TiB

This usage looks healthy.  No problems here. =:^)

That's the easy to see and address stuff, anyway.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Wrong device?

2017-09-26 Thread Lukas Pirl 
On 09/25/2017 06:11 PM, linux-bt...@oh3mqu.pp.hyper.fi wrote as excerpted:
> After a long googling (about more complex situations) I suddenly
> noticed "device sdb" WTF???  Filesystem is mounted from /dev/md3 (sdb
> is part of that mdraid) so btrfs should not even know anything about
> that /dev/sdb.

I would be interested in explanations regarding this too. It happened
to me as well, that I was confused by /dev/sd* device paths being
printed by btrfs in the logs, even though it runs on /dev/md-*
(/dev/mapper/*) devices exclusively.

> PS. I have noticed another bug too, but I haven't tested it with
> lastest kernels after I noticed that it happens only with
> compression=lzo.  So maybe it is already fixed.  With gzip or none
> compression probem does not happens.  I have email server with about
> 0.5 TB volume. It is using Maildir so it contains huge amount of
> files.  Sometimes some files goes unreadable.  After server reset
> problematic file could be readable again (but not always)...
> 
> But weird thing is that unreadable file always seems to be
> dovecot.index.log.

Confirm this (non-reproducible) behavior on a VPS running Debian
4.5.4-1~bpo8+1.

Lukas

-- 
+49 174 940 74 71
GPG key available via key servers



signature.asc
Description: OpenPGP digital signature

Wrong device?

2017-09-25 Thread linux-btrfs 


Hi,

I have 15 x 6 TB disks in md-raid (Because btrfs's raid6 was marked as 
not-for-real-use when I first time installed this machine)


Now I have got both problem twice.

At this last time I have 233 subvolums, and millions of files (all 
together)


Then filesystem went to read only with this dmesg:

[Sat Sep 23 07:25:28 2017] [ cut here ]
[Sat Sep 23 07:25:28 2017] WARNING: CPU: 5 PID: 5431 at 
/build/linux-hwe-edge-CrMNv8/linux-hwe-edge-4.10.0/fs/btrfs/extent-tree.c:6947 
__btrfs_free_extent.isra.61+0x2cb/0xeb0 [btrfs]
[Sat Sep 23 07:25:28 2017] BTRFS: Transaction aborted (error -28)
[Sat Sep 23 07:25:28 2017] Modules linked in: 8021q garp mrp stp llc bonding 
ip6table_filter ip6_tables ipt_REJECT nf_reject_ipv4 nf_log_ipv4 nf_log_common 
xt_LOG xt_limit xt_tcpudp xt_multiport iptable_filter ip_tables x_tables 
intel_rapl sb_edac edac_core x86_pkg_temp_thermal intel_powerclamp coretemp 
ipmi_ssif kvm_intel kvm irqbypass intel_cstate intel_rapl_perf joydev 
input_leds mei_me mei lpc_ich shpchp ioatdma ipmi_si ipmi_devintf 
ipmi_msghandler acpi_pad acpi_power_meter mac_hid ib_iser rdma_cm iw_cm ib_cm 
ib_core configfs iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi autofs4 
btrfs raid10 raid0 multipath linear raid456 async_raid6_recov async_memcpy 
async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 ses enclosure ixgbe 
ast crct10dif_pclmul i2c_algo_bit crc32_pclmul ttm drm_kms_helper 
ghash_clmulni_intel
[Sat Sep 23 07:25:28 2017]  dca syscopyarea pcbc sysfillrect aesni_intel 
aes_x86_64 hid_generic sysimgblt crypto_simd mpt3sas fb_sys_fops ptp r8169 
raid_class ahci glue_helper usbhid pps_core mxm_wmi cryptd drm hid libahci mdio 
scsi_transport_sas mii fjes wmi
[Sat Sep 23 07:25:28 2017] CPU: 5 PID: 5431 Comm: btrfs-transacti Tainted: G
W   4.10.0-26-generic #30~16.04.1-Ubuntu
[Sat Sep 23 07:25:28 2017] Hardware name: Supermicro X10DRi/X10DRI-T, BIOS 2.1 
09/13/2016
[Sat Sep 23 07:25:28 2017] Call Trace:
[Sat Sep 23 07:25:28 2017]  dump_stack+0x63/0x90
[Sat Sep 23 07:25:28 2017]  __warn+0xcb/0xf0
[Sat Sep 23 07:25:28 2017]  warn_slowpath_fmt+0x5f/0x80
[Sat Sep 23 07:25:28 2017]  __btrfs_free_extent.isra.61+0x2cb/0xeb0 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? btrfs_merge_delayed_refs+0x64/0x640 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? btrfs_search_slot+0x981/0x9c0 [btrfs]
[Sat Sep 23 07:25:28 2017]  __btrfs_run_delayed_refs+0xb44/0x13a0 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? btree_set_page_dirty+0xe/0x10 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? free_extent_buffer+0x4b/0xa0 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? igrab+0x1e/0x60
[Sat Sep 23 07:25:28 2017]  btrfs_run_delayed_refs+0x7f/0x2a0 [btrfs]
[Sat Sep 23 07:25:28 2017]  btrfs_write_dirty_block_groups+0x169/0x3c0 [btrfs]
[Sat Sep 23 07:25:28 2017]  commit_cowonly_roots+0x221/0x2c0 [btrfs]
[Sat Sep 23 07:25:28 2017]  btrfs_commit_transaction+0x542/0x950 [btrfs]
[Sat Sep 23 07:25:28 2017]  transaction_kthread+0x1a6/0x1c0 [btrfs]
[Sat Sep 23 07:25:28 2017]  kthread+0x109/0x140
[Sat Sep 23 07:25:28 2017]  ? btrfs_cleanup_transaction+0x540/0x540 [btrfs]
[Sat Sep 23 07:25:28 2017]  ? kthread_create_on_node+0x60/0x60
[Sat Sep 23 07:25:28 2017]  ret_from_fork+0x2c/0x40
[Sat Sep 23 07:25:28 2017] ---[ end trace 76fd3c22e75c2b85 ]---
[Sat Sep 23 07:25:28 2017] BTRFS: error (device sdb) in 
__btrfs_free_extent:6947: errno=-28 No space left
[Sat Sep 23 07:25:28 2017] BTRFS: error (device sdb) in 
btrfs_drop_snapshot:9193: errno=-28 No space left
[Sat Sep 23 07:25:28 2017] BTRFS info (device sdb): forced readonly


After a long googling (about more complex situations) I suddenly noticed 
"device sdb" WTF???  Filesystem is mounted from /dev/md3 (sdb is part of 
that mdraid) so btrfs should not even know anything about that /dev/sdb.


So maybe the error is more simple that I thought.  It tries to allocate 
more metadata from physical drive (not from /dev/md3 as it was supposed to 
do), so that "no space left on device" could be the REAL error...


But why it is doing so?  Does it help if I add some extra virtualization 
layer, like LVM?



# grep data /etc/fstab
/dev/md3 /data btrfs   compress=zlib,space_cache,noatime 0 0

# btrfs fi df /data
Data, single: total=4.77TiB, used=4.77TiB
System, DUP: total=32.00MiB, used=592.00KiB
Metadata, DUP: total=92.00GiB, used=90.87GiB
GlobalReserve, single: total=512.00MiB, used=172.50MiB

# btrfs fi show /data
Label: none  uuid: 84a31ba6-d22d-4653-b071-5525dbdfe561
Total devices 1 FS bytes used 4.85TiB
devid2 size 70.95TiB used 4.95TiB path /dev/md3

# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4] [linear] [multipath] [raid0] 
[raid10]
md3 : active raid6 sdl[17](S) sdb[16] sdq[15] sdp[14] sdm[11] sdc[1] sdo[13] 
sde[3] sdj[8] sdn[12] sdh[6] sdi[7] sdk[9] sdf[4] sdd[2] sdg[5]
  76185088512 blocks super 1.2 level 6, 512k chunk, algorithm 2 [15/15] 
[UUU]
  bitmap: 0/44 pages [0KB], 65536KB chunk

# btrfs --version
btrfs-progs v4.4

Re: degraded raid scribbling upon wrong device

2017-07-22 Thread Adam Borowski 
On Thu, Jul 13, 2017 at 08:40:12AM +0200, Adam Borowski wrote:
> Here's a set of test cases, two of them in some cases seem to scribble upon
> the wrong device:
> 
> * deg-mid-missing
> * deg-last-replaced (not on the innocent "re")
> * but never deg-last-missing
> 
> When all goes ok, there are no errors other than wrong generation on the
> re-added disk (expected).   When it goes bad, there's a lot of corruption.
> In all cases, though, the "Device missing:" field is wrong.

I did not explore this adequately yet, in a good part because of ENOSPC
triggering a lot of time for an unrelated reason that Omar just fixed
(thanks!).  So, here's what I know so far:

* copying in, say, 2.2GB /usr/share is a lot more likely to trigger than
  dd-ing 2.2GB of /dev/null
* no "real" degrading is needed: in the original scripts, the missing device
  is empty so all blocks are doubled anyway.  It's not about degraded chunks
  but because of a bogus device.
* bogus output of "btrfs f u" is a sure predictor that, with enough tries,
  you'll get corruption -- if it shows something when it should say
  "missing", shit is likely to happen


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

degraded raid scribbling upon wrong device

2017-07-13 Thread Adam Borowski 
Hi!
Here's a set of test cases, two of them in some cases seem to scribble upon
the wrong device:

* deg-mid-missing
* deg-last-replaced (not on the innocent "re")
* but never deg-last-missing

When all goes ok, there are no errors other than wrong generation on the
re-added disk (expected).   When it goes bad, there's a lot of corruption.
In all cases, though, the "Device missing:" field is wrong.

I'm not yet sure how to trigger this, perhaps someone would have a clue?

8:30am, hitting the sack, will try again todorrow.


Meow!
-- 
⢀⣴⠾⠻⢶⣦⠀ 
⣾⠁⢠⠒⠀⣿⡁ A dumb species has no way to open a tuna can.
⢿⡄⠘⠷⠚⠋⠀ A smart species invents a can opener.
⠈⠳⣄ A master species delegates.
#!/bin/sh
set -e
set -x

umount /mnt/vol1 ||:
losetup -D

dd if=/dev/zero bs=1048576 count=1 seek=4095 of=ra
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rb
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rc
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rd

mkfs.btrfs -draid1 -mraid1 ra rb rc rd

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
cp -pr /bin /mnt/vol1
btrfs fi sync /mnt/vol1
btrfs fi us /mnt/vol1
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rd
sleep 1
mount -odegraded /dev/loop0 /mnt/vol1
btrfs fi us /mnt/vol1
dd if=/dev/zero of=/mnt/vol1/foo bs=1048576 count=
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
btrfs scrub start -B /mnt/vol1
#!/bin/sh
set -e
set -x

umount /mnt/vol1 ||:
losetup -D

dd if=/dev/zero bs=1048576 count=1 seek=4095 of=ra
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rb
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rc
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rd

mkfs.btrfs -draid1 -mraid1 ra rb rc rd

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
cp -pr /bin /mnt/vol1
btrfs fi sync /mnt/vol1
btrfs fi us /mnt/vol1
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
sleep 1
mount -odegraded /dev/loop0 /mnt/vol1
btrfs fi us /mnt/vol1
dd if=/dev/zero of=/mnt/vol1/foo bs=1048576 count=
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
btrfs scrub start -B /mnt/vol1
#!/bin/sh
set -e
set -x

umount /mnt/vol1 ||:
losetup -D

dd if=/dev/zero bs=1048576 count=1 seek=4095 of=ra
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rb
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rc
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=rd
dd if=/dev/zero bs=1048576 count=1 seek=4095 of=re

mkfs.btrfs -draid1 -mraid1 ra rb rc rd

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
cp -pr /bin /mnt/vol1
btrfs fi sync /mnt/vol1
btrfs fi us /mnt/vol1
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f re
sleep 1
mount -odegraded /dev/loop0 /mnt/vol1
btrfs fi us /mnt/vol1
dd if=/dev/zero of=/mnt/vol1/foo bs=1048576 count=
umount /mnt/vol1

losetup -D
losetup -f ra
losetup -f rb
losetup -f rc
losetup -f rd
sleep 1
mount /dev/loop0 /mnt/vol1
btrfs scrub start -B /mnt/vol1

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-20 Thread Maximilian Böhm 
Hey, I want you to know that it was impossible to recover the filesystem and 
that I have recreated the partition. Lost ~1.5 TiB unredundant data but it's 
just an annoyance, no catastrophe – I can recreate my collection and it wasn't 
any critical data.

For others with related problems: My latest attempt was to recover the 
superblock and metadata:

$ btrfs rescue super-recover -v /dev/sdc1
All Devices:
Device: id = 1, name = /dev/sdc1

Before Recovering:
[All good supers]:
device name = /dev/sdc1
superblock bytenr = 274877906944

[All bad supers]:
device name = /dev/sdc1
superblock bytenr = 65536

device name = /dev/sdc1
superblock bytenr = 67108864


Make sure this is a btrfs disk otherwise the tool will destroy other fs, Are 
you sure? [y/N]: y
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
bytenr mismatch, want=21004288, have=0
ERROR: cannot read chunk root
Failed to recover bad superblocks
*** Error in `btrfs': double free or corruption (fasttop): 0x00946010 
***
=== Backtrace: =
/usr/lib/libc.so.6(+0x6ed4b)[0x7fa620c8cd4b]
/usr/lib/libc.so.6(+0x74546)[0x7fa620c92546]
/usr/lib/libc.so.6(+0x74d1e)[0x7fa620c92d1e]
btrfs(btrfs_close_devices+0xf7)[0x456227]
btrfs(btrfs_recover_superblocks+0x459)[0x437729]
btrfs(main+0x7b)[0x40ac3b]
/usr/lib/libc.so.6(__libc_start_main+0xf1)[0x7fa620c3e741]
btrfs(_start+0x29)[0x40ad39]
=== Memory map: 
0040-00495000 r-xp  00:13 4097939
/usr/bin/btrfs
00694000-00698000 r--p 00094000 00:13 4097939
/usr/bin/btrfs
00698000-0069a000 rw-p 00098000 00:13 4097939
/usr/bin/btrfs
0069a000-0069f000 rw-p  00:00 0
00946000-00967000 rw-p  00:00 0  [heap]
7fa61c00-7fa61c021000 rw-p  00:00 0
7fa61c021000-7fa62000 ---p  00:00 0
7fa620a06000-7fa620a1c000 r-xp  00:13 3704352
/usr/lib/libgcc_s.so.1
7fa620a1c000-7fa620c1b000 ---p 00016000 00:13 3704352
/usr/lib/libgcc_s.so.1
7fa620c1b000-7fa620c1c000 rw-p 00015000 00:13 3704352
/usr/lib/libgcc_s.so.1
7fa620c1e000-7fa620db5000 r-xp  00:13 3867364
/usr/lib/libc-2.23.so
7fa620db5000-7fa620fb5000 ---p 00197000 00:13 3867364
/usr/lib/libc-2.23.so
7fa620fb5000-7fa620fb9000 r--p 00197000 00:13 3867364
/usr/lib/libc-2.23.so
7fa620fb9000-7fa620fbb000 rw-p 0019b000 00:13 3867364
/usr/lib/libc-2.23.so
7fa620fbb000-7fa620fbf000 rw-p  00:00 0
7fa620fc6000-7fa620fde000 r-xp  00:13 3867345
/usr/lib/libpthread-2.23.so
7fa620fde000-7fa6211dd000 ---p 00018000 00:13 3867345
/usr/lib/libpthread-2.23.so
7fa6211dd000-7fa6211de000 r--p 00017000 00:13 3867345
/usr/lib/libpthread-2.23.so
7fa6211de000-7fa6211df000 rw-p 00018000 00:13 3867345
/usr/lib/libpthread-2.23.so
7fa6211df000-7fa6211e3000 rw-p  00:00 0
7fa6211e6000-7fa621207000 r-xp  00:13 6889   
/usr/lib/liblzo2.so.2.0.0
7fa621207000-7fa621406000 ---p 00021000 00:13 6889   
/usr/lib/liblzo2.so.2.0.0
7fa621406000-7fa621407000 r--p 0002 00:13 6889   
/usr/lib/liblzo2.so.2.0.0
7fa621407000-7fa621408000 rw-p 00021000 00:13 6889   
/usr/lib/liblzo2.so.2.0.0
7fa62140e000-7fa621423000 r-xp  00:13 7051   
/usr/lib/libz.so.1.2.8
7fa621423000-7fa621622000 ---p 00015000 00:13 7051   
/usr/lib/libz.so.1.2.8
7fa621622000-7fa621623000 r--p 00014000 00:13 7051   
/usr/lib/libz.so.1.2.8
7fa621623000-7fa621624000 rw-p 00015000 00:13 7051   
/usr/lib/libz.so.1.2.8
7fa621626000-7fa621664000 r-xp  00:13 3436728
/usr/lib/libblkid.so.1.1.0
7fa621664000-7fa621863000 ---p 0003e000 00:13 3436728
/usr/lib/libblkid.so.1.1.0
7fa621863000-7fa621867000 r--p 0003d000 00:13 3436728
/usr/lib/libblkid.so.1.1.0
7fa621867000-7fa621868000 rw-p 00041000 00:13 3436728
/usr/lib/libblkid.so.1.1.0
7fa621868000-7fa621869000 rw-p  00:00 0
7fa62186e000-7fa621872000 r-xp  00:13 3436727
/usr/lib/libuuid.so.1.3.0
7fa621872000-7fa621a71000 ---p 4000 00:13 3436727
/usr/lib/libuuid.so.1.3.0
7fa621a71000-7fa621a72000 r--p 3000 00:13 3436727
/usr/lib/libuuid.so.1.3.0
7fa621a72000-7fa621a73000 rw-p 4000 00:13 3436727

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-14 Thread Maximilian Böhm 
Wow, I'm overwhelmed, thank you very much for your help!

So, firstly I did a
dd if=/dev/zero of=nullz.raw bs=1 count=0 seek=2028060672
and overwrote the ISO on the HDD.

Then I was able to restore the GPT using gdisk with "use backup GPT
header (rebuilding main)". Now I have an intact GPT, a displayed
partition with original name, and an unknown filesystem.

btrfs check --repair /dev/sdc1
and
btrfs-show-super /dev/sdc1 --all

still don't work.

But I btrfs-show-super now finds the third superblock (see at the
bottom of this mail).

Then I tried btrfs restore with -u (superblock 3, because it's 0, 1, 2):

$ losetup /dev/loop1 -o 1M /dev/sdc
$ btrfs restore /dev/loop1 -l -u 2
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
checksum verify failed on 21004288 found E4E3BDB6 wanted 
bytenr mismatch, want=21004288, have=0
ERROR: cannot read chunk root
Could not open root, trying backup super

Hm, doesn't "-u 2" defines the backup superblock 3?


$ btrfsck /dev/loop1
No valid Btrfs found on /dev/loop1
Couldn't open file system

BTW, Photorecs finds lots of files but e.g. MKV video are corrupted in
some form that they don't want to play but KDE's Dolphin is able to
generate previews in some cases.
Any idea about how I should proceed further?




$ btrfs-show-super /dev/sdc1 --all
superblock: bytenr=65536, device=/dev/sdc1
-
ERROR: bad magic on superblock on /dev/sdc1 at 65536

superblock: bytenr=67108864, device=/dev/sdc1
-
ERROR: bad magic on superblock on /dev/sdc1 at 67108864

superblock: bytenr=274877906944, device=/dev/sdc1
-
csum0x615e669a [match]
bytenr  274877906944
flags   0x1
( WRITTEN )
magic   _BHRfS_M [match]
fsid446c7a9c-fcad-42f2-b093-ee495ca8f5be
label   Speicherschatz
generation  5503
root511320200
sys_array_size  129
chunk_root_generation   5485
root_level  1
chunk_root  21004288
chunk_root_level1
log_root0
log_root_transid0
log_root_level  0
total_bytes 8001561821184
bytes_used  6272827662336
sectorsize  4096
nodesize16384
leafsize16384
stripesize  4096
root_dir6
num_devices 1
compat_flags0x0
compat_ro_flags 0x0
incompat_flags  0x169
( MIXED_BACKREF |
  COMPRESS_LZO |
  BIG_METADATA |
  EXTENDED_IREF |
  SKINNY_METADATA )
csum_type   0
csum_size   4
cache_generation5503
uuid_tree_generation5503
dev_item.uuid   0471bf89-89d1-424a-9fc1-d48241ff453b
dev_item.fsid   446c7a9c-fcad-42f2-b093-ee495ca8f5be [match]
dev_item.type   0
dev_item.total_bytes8001561821184
dev_item.bytes_used 6283562319872
dev_item.io_align   4096
dev_item.io_width   4096
dev_item.sector_size4096
dev_item.devid  1
dev_item.dev_group  0
dev_item.seek_speed 0
dev_item.bandwidth  0
dev_item.generation 0
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Andrei Borzenkov 
13.06.2016 01:49, Henk Slager пишет:
> On Sun, Jun 12, 2016 at 11:22 PM, Maximilian Böhm  wrote:
>> Hi there, I did something terribly wrong, all blame on me. I wanted to
>> write to an USB stick but /dev/sdc wasn't the stick in this case but
>> an attached HDD with GPT and an 8 TB btrfs partition…
> 
> GPT has a secondary copy at the end of the device, so maybe gdisk can
> reconstruct in first main one at the beginning of the disk, I don't
> know all gdisk commands. 

kernel should automatically fall back to secondary GPT if booted with
gpt=1 parameter. Otherwise it is 'x' for expert mode and 'b' to rebuild
primary GPT from secondary copy. Also 'c' to load partition information
from secondary (without writing anything). But note that kernel will
also check for valid PMBR (unless gpt is force with parameter mentioned
above), so you will need yet another 'x' to enter second expert mode and
'n' to create new protective MBR.


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Chris Murphy 
On Sun, Jun 12, 2016 at 3:22 PM, Maximilian Böhm  wrote:
> Hi there, I did something terribly wrong, all blame on me. I wanted to
> write to an USB stick but /dev/sdc wasn't the stick in this case but
> an attached HDD with GPT and an 8 TB btrfs partition…
>
> $ sudo dd bs=4M if=manjaro-kde-16.06.1-x86_64.iso of=/dev/sdc
> 483+1 Datensätze ein
> 483+1 Datensätze aus
> 2028060672 bytes (2,0 GB, 1,9 GiB) copied, 16,89 s, 120 MB/s
>
> So, shit.
>
> $ sudo btrfs check --repair /dev/sdc
> enabling repair mode
> No valid Btrfs found on /dev/sdc
> Couldn't open file system
>
> $ sudo btrfs-find-root /dev/sdc
> No valid Btrfs found on /dev/sdc
> ERROR: open ctree failed
>
> $ sudo btrfs-show-super /dev/sdc --all
> superblock: bytenr=65536, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 65536
>
> superblock: bytenr=67108864, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 67108864
>
> superblock: bytenr=274877906944, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 274877906944
>
>

OK but none of these can work anyway because the tools work based on
fixed offsets. By pointing the tool to /dev/sdc the wrong offset is
always being used, because originally the drive had a partition. By
default GPT disks offset the 1st partition 1MiB, or 2048 using 512
byte sectors assuming the drive uses 512 byte logical sectors (and
most still do).

What you should do is run gdisk /dev/sdc and it'll find the backup GPT
at the end of the drive, and offer to fix up the primary one. And now
you can point the tools to /dev/sdc1 (or whatever partition is for
Btrfs).

2GIB of Btrfs metadata is a lot of missing metadata though. So even
though btrfs-show-super will find the 3rd superblock, chances are it's
going to point to metadata somewhere in those first 2GiB that were
overwritten, but maybe not. It might be possible to get a -o ro mount
at least and start copying off some data.

If it's ro mountable, it might be possible to put /dev/sdc1 on an
overlay, and then fix the overlay rather than the original with
btrfsck - maybe it can fix up enough such that you'll just get piles
of read errors for the data that's missing rather than hitting some
brick wall and stopping.


-- 
Chris Murphy
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Henk Slager 
On Sun, Jun 12, 2016 at 11:22 PM, Maximilian Böhm  wrote:
> Hi there, I did something terribly wrong, all blame on me. I wanted to
> write to an USB stick but /dev/sdc wasn't the stick in this case but
> an attached HDD with GPT and an 8 TB btrfs partition…

GPT has a secondary copy at the end of the device, so maybe gdisk can
reconstruct in first main one at the beginning of the disk, I don't
know all gdisk commands. But is you once created just 1 partition max
size for the whole disk with modern tools, your btrfs fs starts at
sector 2048, (with logical sectorsize 512).


> $ sudo dd bs=4M if=manjaro-kde-16.06.1-x86_64.iso of=/dev/sdc
> 483+1 Datensätze ein
> 483+1 Datensätze aus
> 2028060672 bytes (2,0 GB, 1,9 GiB) copied, 16,89 s, 120 MB/s

To confuse btrfs+tools as little as possible, I would first overwrite
/dev/sdc again from the start with the same amount of bytes byte then
from /dev/zero.
Then create / 'newly overlay' a the original partition offset 1M, till
the end. Alternatively:
$ losetup /dev/loopX -o 1M /dev/sdc
then your broken fs will be on /dev/sdc1 or /dev/loopX

Or overlay it with dm, snapshot, set the original ro and then work on
the rw flavor, so you keep the current broken HDD/fs as is and then do
the above.

> $ sudo btrfs check --repair /dev/sdc
> enabling repair mode
> No valid Btrfs found on /dev/sdc
> Couldn't open file system

Forget --repair I would say, hopefully btrfs restore can still find /
copy most of the data

> $ sudo btrfs-find-root /dev/sdc
> No valid Btrfs found on /dev/sdc
> ERROR: open ctree failed
>
> $ sudo btrfs-show-super /dev/sdc --all
> superblock: bytenr=65536, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 65536
>
> superblock: bytenr=67108864, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 67108864
>
> superblock: bytenr=274877906944, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 274877906944

run
$ sudo btrfs-show-super /dev/sdc1 --all
or
$ sudo btrfs-show-super /dev/loopX --all



> System infos:
>
> $ uname -a
> Linux Mongo 4.6.2-1-MANJARO #1 SMP PREEMPT Wed Jun 8 11:00:08 UTC 2016
> x86_64 GNU/Linux
>
> $ btrfs --version
> btrfs-progs v4.5.3
>
> Don't think dmesg is necessary here.
>
>
> OK, the btrfs wiki says there is a second superblock at 64 MiB
> (overwritten too in my case) and a third at 256 GiB ("0x40").
> But how to restore it? And how to restore the general btrfs header
> metadata? How to restore GPT without doing something terrible again?

See above, maybe you have lookup various sizes etc GPT etc first, but
I think your 3rd SB should be there.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Martin Steigerwald 
Hi Maximilian,

On Sonntag, 12. Juni 2016 23:22:11 CEST Maximilian Böhm wrote:
> Hi there, I did something terribly wrong, all blame on me. I wanted to
> write to an USB stick but /dev/sdc wasn't the stick in this case but
> an attached HDD with GPT and an 8 TB btrfs partition…
> 
> $ sudo dd bs=4M if=manjaro-kde-16.06.1-x86_64.iso of=/dev/sdc
> 483+1 Datensätze ein
> 483+1 Datensätze aus
> 2028060672 bytes (2,0 GB, 1,9 GiB) copied, 16,89 s, 120 MB/s
> 
> So, shit.
> 
> $ sudo btrfs check --repair /dev/sdc
> enabling repair mode
> No valid Btrfs found on /dev/sdc
> Couldn't open file system
> 
> $ sudo btrfs-find-root /dev/sdc
> No valid Btrfs found on /dev/sdc
> ERROR: open ctree failed
> 
> $ sudo btrfs-show-super /dev/sdc --all
> superblock: bytenr=65536, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 65536
> 
> superblock: bytenr=67108864, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 67108864
> 
> superblock: bytenr=274877906944, device=/dev/sdc
> -
> ERROR: bad magic on superblock on /dev/sdc at 274877906944
> 
> 
> System infos:
> 
> $ uname -a
> Linux Mongo 4.6.2-1-MANJARO #1 SMP PREEMPT Wed Jun 8 11:00:08 UTC 2016
> x86_64 GNU/Linux
> 
> $ btrfs --version
> btrfs-progs v4.5.3
> 
> Don't think dmesg is necessary here.
> 
> 
> OK, the btrfs wiki says there is a second superblock at 64 MiB
> (overwritten too in my case) and a third at 256 GiB ("0x40").
> But how to restore it? And how to restore the general btrfs header
> metadata? How to restore GPT without doing something terrible again?
> Would be glad for any help!

But it says bad magic on that one as well.

Well, no idea if there is any chance to fix BTRFS in this situation.

Does btrfs restore do anything useful to copy of what if can find from this 
device? It does not work in place, so you need an additional space to let it 
restore to.

If BTRFS cannot be salvaged… you can still have a go with photorec, but it 
will not recover filenames and directory structure, just the data of any file 
of a file in a known format that it finds in one piece.

I suspect you have no backup.

So *good* luck.


I do think tough that dd should just bail out or warn for a BTRFS filesystem 
that is still mounted, or wasn´t it mounted at that time?

I also think it would be good to add an existing filesystem check just like in 
mkfs.btrfs, mkfs.xfs and so on. I´d like that, but that would be a suggestions 
for the coreutils people.

Yes, Unix is for people who know what they are doing… unless they don´t. And 
in the end even one of the most experienced admin could make such a mistake.

Goodnight,
-- 
Martin
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

dd on wrong device, 1.9 GiB from the beginning has been overwritten, how to restore partition?

2016-06-12 Thread Maximilian Böhm 
Hi there, I did something terribly wrong, all blame on me. I wanted to
write to an USB stick but /dev/sdc wasn't the stick in this case but
an attached HDD with GPT and an 8 TB btrfs partition…

$ sudo dd bs=4M if=manjaro-kde-16.06.1-x86_64.iso of=/dev/sdc
483+1 Datensätze ein
483+1 Datensätze aus
2028060672 bytes (2,0 GB, 1,9 GiB) copied, 16,89 s, 120 MB/s

So, shit.

$ sudo btrfs check --repair /dev/sdc
enabling repair mode
No valid Btrfs found on /dev/sdc
Couldn't open file system

$ sudo btrfs-find-root /dev/sdc
No valid Btrfs found on /dev/sdc
ERROR: open ctree failed

$ sudo btrfs-show-super /dev/sdc --all
superblock: bytenr=65536, device=/dev/sdc
-
ERROR: bad magic on superblock on /dev/sdc at 65536

superblock: bytenr=67108864, device=/dev/sdc
-
ERROR: bad magic on superblock on /dev/sdc at 67108864

superblock: bytenr=274877906944, device=/dev/sdc
-
ERROR: bad magic on superblock on /dev/sdc at 274877906944


System infos:

$ uname -a
Linux Mongo 4.6.2-1-MANJARO #1 SMP PREEMPT Wed Jun 8 11:00:08 UTC 2016
x86_64 GNU/Linux

$ btrfs --version
btrfs-progs v4.5.3

Don't think dmesg is necessary here.


OK, the btrfs wiki says there is a second superblock at 64 MiB
(overwritten too in my case) and a third at 256 GiB ("0x40").
But how to restore it? And how to restore the general btrfs header
metadata? How to restore GPT without doing something terrible again?
Would be glad for any help!
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

csum error shows wrong device

2015-02-14 Thread Paul Jones 
Hi,

I have a failing SSD I need to replace so I added another disk to my array, but 
didn't get around to removing the faulty one. A few days later I went to remove 
the faulty one and there were no more errors, I presume because the bad portion 
of the SSD was now not in use. Because I didn't know what disk to remove (2 are 
identical) I removed the new one in the hope to trigger the error with the old 
one so I could note it's serial number.
This was successful, but btrfs is claiming the error is on the disk that I 
removed. I presume this is a bug?
This is the command I used to remove the new disks:
btrfs dev del /dev/sda1 /dev/sdb1 /media/fast/

I note in dmesg there was no mention of /dev/sda1

'Fast' is the filesystem in question.

..
[46474.872519] BTRFS info (device sdb1): relocating block group 382315003904 
flags 17
[46477.090985] BTRFS info (device sdb1): found 8 extents
[46479.747538] BTRFS info (device sdb1): found 8 extents
[46479.921532] BTRFS info (device sdb1): relocating block group 383388745728 
flags 17
[46482.140193] BTRFS info (device sdb1): found 8 extents
[46484.880514] BTRFS info (device sdb1): found 8 extents
[46485.024671] BTRFS info (device sdb1): relocating block group 384462487552 
flags 17
[46487.291166] BTRFS info (device sdb1): found 8 extents
[46490.073738] BTRFS info (device sdb1): found 8 extents
[46490.235649] BTRFS info (device sdb1): relocating block group 385536229376 
flags 17
[46492.364366] BTRFS info (device sdb1): found 3903 extents
[46495.326886] BTRFS info (device sdb1): found 3902 extents
[46495.446426] BTRFS info (device sdb1): disk deleted /dev/sdb1
[59516.406169] BTRFS info (device sdb1): csum failed ino 40605 off 54317748224 
csum 3288447127 expected csum 2629848265
[59516.406175] BTRFS info (device sdb1): csum failed ino 40605 off 54412632064 
csum 4245055996 expected csum 3450832795
[59516.406359] BTRFS: read error corrected: ino 40605 off 54317748224 (dev 
/dev/sde1 sector 431254408)
[59516.406449] BTRFS: read error corrected: ino 40605 off 54412632064 (dev 
/dev/sde1 sector 431439728)
[59516.406548] BTRFS info (device sdb1): csum failed ino 40605 off 55994003456 
csum 42039089 expected csum 2302529568
[59516.406727] BTRFS: read error corrected: ino 40605 off 55994003456 (dev 
/dev/sdf1 sector 434440280)
[59516.408562] BTRFS info (device sdb1): csum failed ino 40605 off 14022791168 
csum 3859837505 expected csum 3337046449
[59516.408573] BTRFS info (device sdb1): csum failed ino 40605 off 55445471232 
csum 1108896639 expected csum 3859837505
[59516.408683] BTRFS info (device sdb1): csum failed ino 40605 off 56895586304 
csum 2469353283 expected csum 761457267
[59516.408691] BTRFS info (device sdb1): csum failed ino 40605 off 61744017408 
csum 1385882588 expected csum 2439148438
[59516.408702] BTRFS info (device sdb1): csum failed ino 40605 off 63789928448 
csum 3337046449 expected csum 2586270899
[59516.408717] BTRFS info (device sdb1): csum failed ino 40605 off 14022791168 
csum 1108896639 expected csum 3337046449
[59516.408785] BTRFS info (device sdb1): csum failed ino 40605 off 55445471232 
csum 3867769240 expected csum 3859837505
[59516.408894] BTRFS: read error corrected: ino 40605 off 56895586304 (dev 
/dev/sde1 sector 434192096)
[59516.409129] BTRFS: read error corrected: ino 40605 off 63789928448 (dev 
/dev/sde1 sector 574273160)
[59516.409230] BTRFS: read error corrected: ino 40605 off 61744017408 (dev 
/dev/sde1 sector 505265528)
[59516.417206] BTRFS: read error corrected: ino 40605 off 62792196096 (dev 
/dev/sdf1 sector 441426544)
[59516.417241] BTRFS: read error corrected: ino 40605 off 59971190784 (dev 
/dev/sdf1 sector 579309536)
[59517.423114] BTRFS: read error corrected: ino 40605 off 59917271040 (dev 
/dev/sde1 sector 505891968)
[59518.423088] BTRFS: read error corrected: ino 40605 off 22458654720 (dev 
/dev/sde1 sector 400486896)

Linux vm-server 3.19.0-gentoo #1 SMP PREEMPT Wed Feb 11 16:48:13 AEDT 2015 
x86_64 Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz GenuineIntel GNU/Linux

Label: 'Root'  uuid: 58d27dbd-7c1e-4ef7-8d43-e93df1537b08
    Total devices 3 FS bytes used 14.85GiB
    devid    3 size 40.00GiB used 33.03GiB path /dev/sdf3
    devid    4 size 40.00GiB used 33.03GiB path /dev/sde3
    devid    6 size 40.00GiB used 0.00B path /dev/sdb2

Label: 'Storage'  uuid: 63ec312b-5f7a-4137-a8f0-e877d5a85902
    Total devices 2 FS bytes used 1.07TiB
    devid    1 size 3.64TiB used 1.08TiB path /dev/sdd1
    devid    2 size 3.64TiB used 1.08TiB path /dev/sdj1

Label: 'Backup'  uuid: 8149e719-022b-4a7a-8465-704e24ba7898
    Total devices 4 FS bytes used 1.11TiB
    devid    1 size 2.73TiB used 1.12TiB path /dev/sdc1
    devid    2 size 931.51GiB used 567.00GiB path /dev/sdi1
    devid    3 size 931.51GiB used 569.03GiB path /dev/sdh1
    devid    4 size 2.73TiB used 7.00GiB path /dev/sdg1

Label: 'Fast'  uuid: e15bab07-1fcf-48b1-92d5-a2609f0fe469
    Total devices 2 FS bytes used

[PATCH 05/18] Btrfs: fix wrong device bytes_used in the super block

2014-09-03 Thread Miao Xie 
device-bytes_used will be changed when allocating a new chunk, and
disk_total_size will be changed if resizing is successful.
Meanwhile, the on-disk super blocks of the previous transaction
might not be updated. Considering the consistency of the metadata
in the previous transaction, We should use the size in the previous
transaction to check if the super block is beyond the boundary
of the device.

Though it is not big problem because we don't use it now, but anyway
it is better that we make it be consistent with the common metadata,
maybe we will use it in the future.

Signed-off-by: Miao Xie mi...@cn.fujitsu.com
---
 fs/btrfs/dev-replace.c |  3 +++
 fs/btrfs/disk-io.c |  3 ++-
 fs/btrfs/transaction.c |  1 +
 fs/btrfs/volumes.c | 27 +++
 fs/btrfs/volumes.h |  4 
 5 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c
index 7877b0f..1be03d8 100644
--- a/fs/btrfs/dev-replace.c
+++ b/fs/btrfs/dev-replace.c
@@ -172,6 +172,8 @@ no_valid_dev_replace_entry_found:
dev_replace-srcdev-commit_total_bytes;
dev_replace-tgtdev-bytes_used =
dev_replace-srcdev-bytes_used;
+   dev_replace-tgtdev-commit_bytes_used =
+   dev_replace-srcdev-commit_bytes_used;
}
dev_replace-tgtdev-is_tgtdev_for_dev_replace = 1;
btrfs_init_dev_replace_tgtdev_for_resume(fs_info,
@@ -558,6 +560,7 @@ static int btrfs_dev_replace_finishing(struct btrfs_fs_info 
*fs_info,
ASSERT(list_empty(src_device-resized_list));
tgt_device-commit_total_bytes = src_device-commit_total_bytes;
tgt_device-bytes_used = src_device-bytes_used;
+   tgt_device-commit_bytes_used = src_device-bytes_used;
if (fs_info-sb-s_bdev == src_device-bdev)
fs_info-sb-s_bdev = tgt_device-bdev;
if (fs_info-fs_devices-latest_bdev == src_device-bdev)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 0c7ae0e..ff3ee22 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3450,7 +3450,8 @@ static int write_all_supers(struct btrfs_root *root, int 
max_mirrors)
btrfs_set_stack_device_id(dev_item, dev-devid);
btrfs_set_stack_device_total_bytes(dev_item,
   dev-commit_total_bytes);
-   btrfs_set_stack_device_bytes_used(dev_item, dev-bytes_used);
+   btrfs_set_stack_device_bytes_used(dev_item,
+ dev-commit_bytes_used);
btrfs_set_stack_device_io_align(dev_item, dev-io_align);
btrfs_set_stack_device_io_width(dev_item, dev-io_width);
btrfs_set_stack_device_sector_size(dev_item, dev-sector_size);
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 2f7c0be..16d0c1b 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -1869,6 +1869,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle 
*trans,
   sizeof(*root-fs_info-super_copy));
 
btrfs_update_commit_device_size(root-fs_info);
+   btrfs_update_commit_device_bytes_used(root, cur_trans);
 
spin_lock(root-fs_info-trans_lock);
cur_trans-state = TRANS_STATE_UNBLOCKED;
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 7b5c042..f8273bb 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -2370,6 +2370,7 @@ int btrfs_init_dev_replace_tgtdev(struct btrfs_root 
*root, char *device_path,
ASSERT(list_empty(srcdev-resized_list));
device-commit_total_bytes = srcdev-commit_total_bytes;
device-bytes_used = srcdev-bytes_used;
+   device-commit_bytes_used = device-bytes_used;
device-dev_root = fs_info-dev_root;
device-bdev = bdev;
device-in_fs_metadata = 1;
@@ -6009,6 +6010,7 @@ static void fill_device_from_item(struct extent_buffer 
*leaf,
device-total_bytes = device-disk_total_bytes;
device-commit_total_bytes = device-disk_total_bytes;
device-bytes_used = btrfs_device_bytes_used(leaf, dev_item);
+   device-commit_bytes_used = device-bytes_used;
device-type = btrfs_device_type(leaf, dev_item);
device-io_align = btrfs_device_io_align(leaf, dev_item);
device-io_width = btrfs_device_io_width(leaf, dev_item);
@@ -6558,3 +6560,28 @@ void btrfs_update_commit_device_size(struct 
btrfs_fs_info *fs_info)
unlock_chunks(fs_info-dev_root);
mutex_unlock(fs_devices-device_list_mutex);
 }
+
+/* Must be invoked during the transaction commit */
+void btrfs_update_commit_device_bytes_used(struct btrfs_root *root,
+   struct btrfs_transaction *transaction)
+{
+   struct extent_map *em;
+   struct map_lookup *map;
+

[PATCH 08/10] Btrfs: Fix wrong device size when we are resizing the device

2014-07-23 Thread Miao Xie 
total_bytes of device is just a in-memory variant which is used to record
the size of the device, and it might be changed before we resize a device,
if the resize operation fails, it will be fallbacked. But some code used it
to update on-disk metadata of the device, it would cause the problem that
on-disk metadata of the devices was not consistent. We should use the other
variant named disk_total_bytes to update the on-disk metadata of device,
because that variant is updated only when the resize operation is successful.
Fix it.

Signed-off-by: Miao Xie mi...@cn.fujitsu.com
---
 fs/btrfs/disk-io.c | 3 ++-
 fs/btrfs/volumes.c | 2 +-
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 08e65e9..2a9560c1 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -3452,7 +3452,8 @@ static int write_all_supers(struct btrfs_root *root, int 
max_mirrors)
btrfs_set_stack_device_generation(dev_item, 0);
btrfs_set_stack_device_type(dev_item, dev-type);
btrfs_set_stack_device_id(dev_item, dev-devid);
-   btrfs_set_stack_device_total_bytes(dev_item, dev-total_bytes);
+   btrfs_set_stack_device_total_bytes(dev_item,
+  dev-disk_total_bytes);
btrfs_set_stack_device_bytes_used(dev_item, dev-bytes_used);
btrfs_set_stack_device_io_align(dev_item, dev-io_align);
btrfs_set_stack_device_io_width(dev_item, dev-io_width);
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 0d37746..0140bff 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -1489,7 +1489,7 @@ static int btrfs_add_device(struct btrfs_trans_handle 
*trans,
btrfs_set_device_io_align(leaf, dev_item, device-io_align);
btrfs_set_device_io_width(leaf, dev_item, device-io_width);
btrfs_set_device_sector_size(leaf, dev_item, device-sector_size);
-   btrfs_set_device_total_bytes(leaf, dev_item, device-total_bytes);
+   btrfs_set_device_total_bytes(leaf, dev_item, device-disk_total_bytes);
btrfs_set_device_bytes_used(leaf, dev_item, device-bytes_used);
btrfs_set_device_group(leaf, dev_item, 0);
btrfs_set_device_seek_speed(leaf, dev_item, 0);
-- 
1.9.3

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

df shows wrong device while waiting for umount

2010-02-19 Thread Mike Fedyk 
Hi,

Kernel 2.6.33-0.46.rc8.git1.fc13.x86_64

I think I ran into the issue that triggers when you write the a btrfs
filesystem and then umount it and it takes a long time while writing
out the data.  It ends up writing at about 1MiB/second according to
dstat.  My understanding this issue is already fixed in the latest
code.

But that is not the issue I am reporting.  While waiting for the
umount to complete, df shows the wrong device.

# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda2 9.8G  7.5G  2.3G  77% /
tmpfs 1.9G  536K  1.9G   1% /dev/shm
/dev/sda6 485M   62M  398M  14% /boot
/dev/sda7 111G  464M  111G   1% /mnt/t

# umount /mnt/t

# df -h
FilesystemSize  Used Avail Use% Mounted on
/dev/sda2 9.8G  7.5G  2.3G  77% /
tmpfs 1.9G  536K  1.9G   1% /dev/shm
/dev/sda6 485M   62M  398M  14% /boot
/dev/sda7 9.8G  7.5G  2.3G  77% /mnt/t

The second df was run while waiting for the 111GB partition to unmount.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html