09.02.2019 16:45, Jakob Schöttl пишет:
> Thanks Qu,
>
> Am 09.02.19 um 13:16 schrieb Qu Wenruo:
>> On 2019/2/9 下午6:36, Jakob Schöttl wrote:
>>> Hi,
>>>
>>> I've setup a RAID1 with two disks (disk1 and disk2) and I'm testing the
>>> btrfs replace command.
>>>
>>> After replacing disk2 with disk3, I can only mount
>>> (a) disk1 or disk3 (if both disk are plugged) and
>>> (b) the original disk1 (degraded, if disk3 is unplugged).
>>>
>>> I cannot mount the replacement disk3 if disk1 is unplugged.
>> Sounds like there is one single chunk on disk1, which caused the problem.
>>
>>>> mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop3,
>>> missing codepage or helper program, or other error.
>> dmesg please.
> [60456.856883] BTRFS: device label datavol devid 1 transid 5 /dev/loop1
> [60456.940785] BTRFS: device label datavol devid 2 transid 5 /dev/loop2
> [60525.211389] BTRFS info (device loop1): allowing degraded mounts
> [60525.211395] BTRFS info (device loop1): disk space caching is enabled
> [60525.211398] BTRFS info (device loop1): has skinny extents
> [60525.211401] BTRFS info (device loop1): flagging fs with big metadata
> feature
> [60525.213854] BTRFS warning (device loop1): devid 2 uuid
> 0b4e0b31-e2b1-40a0-8360-09978f58a2e4 is missing
> [60525.214639] BTRFS info (device loop1): checking UUID tree
> [60525.386695] BTRFS info (device loop1): dev_replace from <missing
> disk> (devid 2) to /dev/loop3 started
> [60525.394403] BTRFS info (device loop1): dev_replace from <missing
> disk> (devid 2) to /dev/loop3 finished
> [60533.721841] BTRFS info (device loop3): allowing degraded mounts
> [60533.721846] BTRFS info (device loop3): disk space caching is enabled
> [60533.721850] BTRFS info (device loop3): has skinny extents
> [60533.723703] BTRFS error (device loop3): failed to read chunk root
> [60533.773553] BTRFS error (device loop3): open_ctree failed
>
>> And btrfs-progs version please.
> $ pacman -Q btrfs-progs
> btrfs-progs 4.20.1-2
> $ uname -a
> Linux jathink 4.20.7-arch1-1-ARCH #1 SMP PREEMPT Wed Feb 6 18:42:40 UTC
> 2019 x86_64 GNU/Linux
>
>> Maybe mkfs is too old to leave SINGLE profile chunks on the original fs.
>>
>> And you could verify the chunk mapping by executing 'btrfs ins dump-tree
>> -t chunk <device>' and paste the output.
>
> When only /dev/loop3 is plugged:
>
> # btrfs inspect-internal dump-tree -t chunk /dev/loop3
> btrfs-progs v4.20.1
> warning, device 1 is missing
> warning, device 1 is missing
> warning, device 1 is missing
> warning, device 1 is missing
> bad tree block 198180864, bytenr mismatch, want=198180864, have=0
> ERROR: cannot read chunk root
> ERROR: unable to open /dev/loop3
>
> When only /dev/loop1 is plugged:
>
> # btrfs inspect-internal dump-tree -t chunk /dev/loop1
> btrfs-progs v4.20.1
> warning, device 2 is missing
> chunk tree
> leaf 198180864 items 10 free space 15005 generation 8 owner CHUNK_TREE
> leaf 198180864 flags 0x1(WRITTEN) backref revision 1
> fs uuid 005a8d59-a561-4371-869e-b0ccc4a4862b
> chunk uuid b3e609f1-a7fe-4add-bc51-6231f0bbf320
> item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
> devid 1 total_bytes 307200000 bytes_used 306053120
> io_align 4096 io_width 4096 sector_size 4096 type 0
> generation 0 start_offset 0 dev_group 0
> seek_speed 0 bandwidth 0
> uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> fsid 005a8d59-a561-4371-869e-b0ccc4a4862b
> item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
> devid 2 total_bytes 307200000 bytes_used 109051904
> io_align 4096 io_width 4096 sector_size 4096 type 0
> generation 0 start_offset 0 dev_group 0
> seek_speed 0 bandwidth 0
> uuid 0b4e0b31-e2b1-40a0-8360-09978f58a2e4
> fsid 005a8d59-a561-4371-869e-b0ccc4a4862b
> item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975
> itemsize 112
> length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 0
> stripe 0 devid 2 offset 1048576
> dev_uuid 0b4e0b31-e2b1-40a0-8360-09978f58a2e4
> stripe 1 devid 1 offset 22020096
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863
> itemsize 112
> length 33554432 owner 2 stripe_len 65536 type METADATA|RAID1
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 0
> stripe 0 devid 2 offset 9437184
> dev_uuid 0b4e0b31-e2b1-40a0-8360-09978f58a2e4
> stripe 1 devid 1 offset 30408704
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 63963136) itemoff 15751
> itemsize 112
> length 67108864 owner 2 stripe_len 65536 type DATA|RAID1
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 0
> stripe 0 devid 2 offset 42991616
> dev_uuid 0b4e0b31-e2b1-40a0-8360-09978f58a2e4
> stripe 1 devid 1 offset 63963136
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 131072000) itemoff 15671
> itemsize 80
> length 33554432 owner 2 stripe_len 65536 type METADATA
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 1 sub_stripes 1
> stripe 0 devid 1 offset 131072000
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 164626432) itemoff 15591
> itemsize 80
> length 33554432 owner 2 stripe_len 65536 type SYSTEM
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 1 sub_stripes 1
> stripe 0 devid 1 offset 164626432
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 198180864) itemoff 15479
> itemsize 112
> length 33554432 owner 2 stripe_len 65536 type SYSTEM|DUP
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 1
> stripe 0 devid 1 offset 198180864
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> stripe 1 devid 1 offset 231735296
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 8 key (FIRST_CHUNK_TREE CHUNK_ITEM 231735296) itemoff 15367
> itemsize 112
> length 20905984 owner 2 stripe_len 65536 type METADATA|DUP
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 1
> stripe 0 devid 1 offset 265289728
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> stripe 1 devid 1 offset 286195712
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> item 9 key (FIRST_CHUNK_TREE CHUNK_ITEM 252641280) itemoff 15255
> itemsize 112
> length 10485760 owner 2 stripe_len 65536 type DATA|DUP
> io_align 65536 io_width 65536 sector_size 4096
> num_stripes 2 sub_stripes 1
> stripe 0 devid 1 offset 1048576
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
> stripe 1 devid 1 offset 11534336
> dev_uuid 043443c7-ac91-4085-a5e4-983b59dd0803
>
>
Yes, I can trivially reproduce it using commands in original post.
Immediately after mkfs:
10:/home/bor # btrfs inspect-internal dump-tree -t chunk /dev/sdb1
btrfs-progs v4.15
warning, device 2 is missing
chunk tree
leaf 22036480 items 5 free space 15626 generation 5 owner 3
leaf 22036480 flags 0x1(WRITTEN) backref revision 1
fs uuid d33e6bff-5903-449d-b49c-f2d992676e30
chunk uuid ed55b2ad-66c4-4764-acd5-7fa3a9fe619e
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
devid 1 total_bytes 5367640064 bytes_used 1081868288
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
fsid d33e6bff-5903-449d-b49c-f2d992676e30
item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
devid 2 total_bytes 5367640064 bytes_used 1081868288
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
fsid d33e6bff-5903-449d-b49c-f2d992676e30
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975
itemsize 112
length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 1048576
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 22020096
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863
itemsize 112
length 536739840 owner 2 stripe_len 65536 type METADATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 9437184
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 30408704
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 567148544) itemoff 15751
itemsize 112
length 536739840 owner 2 stripe_len 65536 type DATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 546177024
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 567148544
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
Immediately after degraded mount
btrfs-progs v4.15
warning, device 2 is missing
chunk tree
leaf 22020096 items 8 free space 15311 generation 6 owner 3
leaf 22020096 flags 0x1(WRITTEN) backref revision 1
fs uuid e69b93ef-d852-4afa-8c74-50fa07d33a01
chunk uuid 74317bdd-f011-44a8-9c62-0cce665f1b52
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
devid 1 total_bytes 5368709120 bytes_used 2994733056
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
fsid e69b93ef-d852-4afa-8c74-50fa07d33a01
item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
devid 2 total_bytes 5368709120 bytes_used 2155872256
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 1989d9cd-9f82-4d3a-a577-c7e583157ed3
fsid e69b93ef-d852-4afa-8c74-50fa07d33a01
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975
itemsize 112
length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 1048576
dev_uuid 1989d9cd-9f82-4d3a-a577-c7e583157ed3
stripe 1 devid 1 offset 22020096
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863
itemsize 112
length 1073741824 owner 2 stripe_len 65536 type METADATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 9437184
dev_uuid 1989d9cd-9f82-4d3a-a577-c7e583157ed3
stripe 1 devid 1 offset 30408704
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 1104150528) itemoff 15751
itemsize 112
length 1073741824 owner 2 stripe_len 65536 type DATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 1083179008
dev_uuid 1989d9cd-9f82-4d3a-a577-c7e583157ed3
stripe 1 devid 1 offset 1104150528
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 2177892352) itemoff 15671
itemsize 80
length 268435456 owner 2 stripe_len 65536 type METADATA
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 2177892352
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 2446327808) itemoff 15591
itemsize 80
length 33554432 owner 2 stripe_len 65536 type SYSTEM
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 2446327808
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 2479882240) itemoff 15511
itemsize 80
length 536870912 owner 2 stripe_len 65536 type DATA
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 2479882240
dev_uuid 85365dff-ceb8-46bb-a4c5-78ce422a32f8
After device replacement completed
10:/home/bor # btrfs replace status /mnt
Started on 9.Feb 18:07:18, finished on 9.Feb 18:07:19, 0 write errs, 0
uncorr. read errs
10:/home/bor # btrfs inspect-internal dump-tree -t chunk /dev/sdb1
btrfs-progs v4.15
chunk tree
leaf 1942749184 items 11 free space 14900 generation 9 owner 3
leaf 1942749184 flags 0x1(WRITTEN) backref revision 1
fs uuid d33e6bff-5903-449d-b49c-f2d992676e30
chunk uuid ed55b2ad-66c4-4764-acd5-7fa3a9fe619e
item 0 key (DEV_ITEMS DEV_ITEM 1) itemoff 16185 itemsize 98
devid 1 total_bytes 5367640064 bytes_used 2793144320
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
fsid d33e6bff-5903-449d-b49c-f2d992676e30
item 1 key (DEV_ITEMS DEV_ITEM 2) itemoff 16087 itemsize 98
devid 2 total_bytes 5367640064 bytes_used 1081868288
io_align 4096 io_width 4096 sector_size 4096 type 0
generation 0 start_offset 0 dev_group 0
seek_speed 0 bandwidth 0
uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
fsid d33e6bff-5903-449d-b49c-f2d992676e30
item 2 key (FIRST_CHUNK_TREE CHUNK_ITEM 22020096) itemoff 15975
itemsize 112
length 8388608 owner 2 stripe_len 65536 type SYSTEM|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 1048576
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 22020096
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 3 key (FIRST_CHUNK_TREE CHUNK_ITEM 30408704) itemoff 15863
itemsize 112
length 536739840 owner 2 stripe_len 65536 type METADATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 9437184
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 30408704
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 4 key (FIRST_CHUNK_TREE CHUNK_ITEM 567148544) itemoff 15751
itemsize 112
length 536739840 owner 2 stripe_len 65536 type DATA|RAID1
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 0
stripe 0 devid 2 offset 546177024
dev_uuid 1e91c346-eba9-4a41-aeb2-d7efef64a5ea
stripe 1 devid 1 offset 567148544
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 5 key (FIRST_CHUNK_TREE CHUNK_ITEM 1103888384) itemoff 15671
itemsize 80
length 268435456 owner 2 stripe_len 65536 type METADATA
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 1103888384
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 6 key (FIRST_CHUNK_TREE CHUNK_ITEM 1372323840) itemoff 15591
itemsize 80
length 33554432 owner 2 stripe_len 65536 type SYSTEM
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 1372323840
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 7 key (FIRST_CHUNK_TREE CHUNK_ITEM 1405878272) itemoff 15511
itemsize 80
length 536870912 owner 2 stripe_len 65536 type DATA
io_align 65536 io_width 65536 sector_size 4096
num_stripes 1 sub_stripes 1
stripe 0 devid 1 offset 1405878272
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 8 key (FIRST_CHUNK_TREE CHUNK_ITEM 1942749184) itemoff 15399
itemsize 112
length 33554432 owner 2 stripe_len 65536 type SYSTEM|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 1942749184
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
stripe 1 devid 1 offset 1976303616
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 9 key (FIRST_CHUNK_TREE CHUNK_ITEM 1976303616) itemoff 15287
itemsize 112
length 134217728 owner 2 stripe_len 65536 type METADATA|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 2009858048
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
stripe 1 devid 1 offset 2144075776
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
item 10 key (FIRST_CHUNK_TREE CHUNK_ITEM 2110521344) itemoff 15175
itemsize 112
length 268435456 owner 2 stripe_len 65536 type DATA|DUP
io_align 65536 io_width 65536 sector_size 4096
num_stripes 2 sub_stripes 1
stripe 0 devid 1 offset 2278293504
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
stripe 1 devid 1 offset 2546728960
dev_uuid 653ea52e-f95f-417b-af6f-feb82ca49cd6
10:/home/bor #
Running
btrfs balance start -f -ssoft,convert=raid1 -dsoft,convert=raid1
-msoft,convert=raid1 /mnt
fixes it. I am not sure whether btrfs is expected to do it automatically
as of now.
