I/O Error Test 4
================
commit "bcache: add io_disable to struct cached_dev"
Problem: in case of the backing device hits I/o errors or is
disconected, the I/O can still be accepted to the bcache device.
Original kernel: dd writes in writeback mode to failed backing device complete.
Modified kernel: the bcache0 device is removed after some I/O errors in backing
device.
Original
--------
# uname -rv
4.15.0-55-generic #60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019
# ./setup.sh >/dev/null 2>&1
[ 24.820401] bcache: register_bdev() registered backing device dm-0
[ 24.833268] bcache: run_cache_set() invalidating existing data
[ 24.848314] bcache: register_cache() registered cache device dm-1
[ 26.824645] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set
dd465fa7-4e85-484b-89dd-353c24c6b041
# echo writeback > /sys/block/bcache0/bcache/cache_mode
# cat /sys/block/bcache0/bcache/cache_mode
writethrough [writeback] writearound none
# ./dm_fake_dev.sh /dev/loop0 bad
[ 41.439684] Buffer I/O error on dev dm-0, logical block 262128, async page
read
[ 41.445284] Buffer I/O error on dev dm-0, logical block 262128, async page
read
[ 41.451846] bcache: register_bcache() error /dev/dm-0: device already
registered (emitting change event)
[ 41.454704] Buffer I/O error on dev bcache0, logical block 262112, async
page read
[ 41.457685] Buffer I/O error on dev bcache0, logical block 262112, async
page read
[ 41.457743] Buffer I/O error on dev bcache0, logical block 1, async page read
# dd if=/dev/zero of=/dev/bcache0 bs=4k
[ 49.048036] Buffer I/O error on dev bcache0, logical block 0, lost async
page write
[ 49.051702] Buffer I/O error on dev bcache0, logical block 1, lost async
page write
[ 49.054062] Buffer I/O error on dev bcache0, logical block 2, lost async
page write
[ 49.056466] Buffer I/O error on dev bcache0, logical block 3, lost async
page write
[ 49.058867] Buffer I/O error on dev bcache0, logical block 4, lost async
page write
[ 49.072020] Buffer I/O error on dev bcache0, logical block 5, lost async
page write
[ 49.074440] Buffer I/O error on dev bcache0, logical block 6, lost async
page write
[ 49.078658] Buffer I/O error on dev bcache0, logical block 7, lost async
page write
[ 49.079008] Buffer I/O error on dev bcache0, logical block 6834, lost async
page write
[ 49.079022] Buffer I/O error on dev bcache0, logical block 6835, lost async
page write
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.58342 s, 416 MB/s
# dd if=/dev/zero of=/dev/bcache0 bs=4k
[ 62.696034] buffer_io_error: 260992 callbacks suppressed
[ 62.696037] Buffer I/O error on dev bcache0, logical block 0, lost async
page write
[ 62.701996] Buffer I/O error on dev bcache0, logical block 1, lost async
page write
[ 62.704394] Buffer I/O error on dev bcache0, logical block 2, lost async
page write
[ 62.706763] Buffer I/O error on dev bcache0, logical block 3, lost async
page write
[ 62.716025] Buffer I/O error on dev bcache0, logical block 4, lost async
page write
[ 62.718421] Buffer I/O error on dev bcache0, logical block 5, lost async
page write
[ 62.720821] Buffer I/O error on dev bcache0, logical block 6, lost async
page write
[ 62.723193] Buffer I/O error on dev bcache0, logical block 7, lost async
page write
[ 62.725584] Buffer I/O error on dev bcache0, logical block 8, lost async
page write
[ 62.725763] Buffer I/O error on dev bcache0, logical block 5405, lost async
page write
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.88915 s, 372 MB/s
# dd if=/dev/zero of=/dev/bcache0 bs=4k
[ 67.700114] buffer_io_error: 290043 callbacks suppressed
[ 67.700117] Buffer I/O error on dev bcache0, logical block 40750, lost async
page write
[ 67.706230] Buffer I/O error on dev bcache0, logical block 40751, lost async
page write
[ 67.709846] Buffer I/O error on dev bcache0, logical block 40752, lost async
page write
[ 67.713503] Buffer I/O error on dev bcache0, logical block 40753, lost async
page write
[ 67.717241] Buffer I/O error on dev bcache0, logical block 40754, lost async
page write
[ 67.720938] Buffer I/O error on dev bcache0, logical block 40755, lost async
page write
[ 67.741395] Buffer I/O error on dev bcache0, logical block 40756, lost async
page write
dd: error writing '/dev/bcache0': No space left on device
[ 67.748145] Buffer I/O error on dev bcache0, logical block 40757, lost async
page write
[ 67.752352] Buffer I/O error on dev bcache0, logical block 41038, lost async
page write
[ 67.756642] Buffer I/O error on dev bcache0, logical block 41313, lost async
page write
262142+0 records in
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.99741 s, 358 MB/s
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
└─bcache0 251:0 0 1024M 0 disk
fake-loop0 253:0 0 1G 0 dm
└─bcache0 251:0 0 1024M 0 disk
Modified
--------
# uname -rv
4.15.0-55-generic #60+test20190703build1bcache1-Ubuntu SMP Wed Jul 3 21:41:37
UTC
# ./setup.sh >/dev/null 2>&1
[ 22.202972] bcache: run_cache_set() invalidating existing data
[ 22.213346] bcache: register_cache() registered cache device dm-1
[ 22.226165] bcache: register_bdev() registered backing device dm-0
[ 24.198940] bcache: bch_cached_dev_attach() Caching dm-0 as bcache0 on set
cabfdb60-4301-46e0-940a-eb96e801c816
# echo writeback > /sys/block/bcache0/bcache/cache_mode
# cat /sys/block/bcache0/bcache/cache_mode
writethrough [writeback] writearound none
# ./dm_fake_dev.sh /dev/loop0 bad
[ 40.025536] Buffer I/O error on dev dm-0, logical block 262128, async page
read
[ 40.030156] Buffer I/O error on dev dm-0, logical block 262128, async page
read
[ 40.035808] bcache: register_bcache() error /dev/dm-0: device already
registered (emitting change event)
[ 40.038534] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 40.038567] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 40.038574] Buffer I/O error on dev bcache0, logical block 262112, async
page read
[ 40.041268] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 40.041284] Buffer I/O error on dev bcache0, logical block 262112, async
page read
[ 40.041319] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 40.041341] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 40.041346] Buffer I/O error on dev bcache0, logical block 1, async page
read
# dd if=/dev/zero of=/dev/bcache0 bs=4k
[ 48.178854] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.181495] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.183988] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.186469] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.188962] Buffer I/O error on dev bcache0, logical block 0, lost async
page write
[ 48.191117] Buffer I/O error on dev bcache0, logical block 1, lost async
page write
[ 48.193295] Buffer I/O error on dev bcache0, logical block 2, lost async
page write
[ 48.195457] Buffer I/O error on dev bcache0, logical block 3, lost async
page write
[ 48.197607] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.200116] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.202597] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.205087] Buffer I/O error on dev bcache0, logical block 4, lost async
page write
[ 48.207229] Buffer I/O error on dev bcache0, logical block 5, lost async
page write
[ 48.209377] Buffer I/O error on dev bcache0, logical block 6, lost async
page write
...
[ 48.362824] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.365085] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.367294] bcache: bch_count_backing_io_errors() dm-0: IO error on backing
device, unrecoverable
[ 48.369457] bcache: bch_cached_dev_error() stop bcache0: too many IO errors
on backing device dm-0
[ 48.369457]
dd: error writing '/dev/bcache0': No space left on device
262142+0 records in[ 48.866726] bcache: bcache_device_free() bcache0 stopped
262141+0 records out
1073729536 bytes (1.1 GB, 1.0 GiB) copied, 2.27785 s, 471 MB/s
# lsblk -e 252
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
loop0 7:0 0 1G 0 loop
loop1 7:1 0 1G 0 loop
└─fake-loop1 253:1 0 1024M 0 dm
fake-loop0 253:0 0 1G 0 dm
# ls /dev/bcache0
ls: cannot access '/dev/bcache0': No such file or directory
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1829563
Title:
bcache: risk of data loss on I/O errors in backing or caching devices
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1829563/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
