Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
> I suspect the cache itself is fine and we're just missing a way to clear > that flag in the cache metadata. I'm looking for a workaround for that > until we can sort out a complete fix. The proper fix is here https://sourceware.org/git/?p=lvm2.git;a=commit;h=9fe7aba251ff24c14277dfdf9be2d861a7699230 (A test for that using actual flagged metadata is still needed.) Dave ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
On Wed, Dec 09, 2020 at 08:57:07PM +0100, Dennis Schridde wrote: > On Mittwoch, 9. Dezember 2020 20:30:51 CET David Teigland wrote: > > On Wed, Dec 09, 2020 at 08:06:41PM +0100, Dennis Schridde wrote: > > > How would you proceed to repair this cache? I can recreate it as a > > > cachepool, but first I need to gain back access to the data, i.e. get > > > into a state where I can --uncache. > > > > I know that cachevols do not yet work with lvconvert --repair, which uses > > cache_repair, but this must be another case where cache_repair is still > > needed. If this is writethough mode you should be able to simply drop the > > cache with lvconvert --splitcache and then recreate it. > > Sadly the cache is in writeback mode. > > Is my understanding correct that the cache itself is fine? It is just the > inability to clear the needs_check flag that is preventing me from using the > device? I suspect the cache itself is fine and we're just missing a way to clear that flag in the cache metadata. I'm looking for a workaround for that until we can sort out a complete fix. Dave ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
On Mittwoch, 9. Dezember 2020 20:30:51 CET David Teigland wrote: > On Wed, Dec 09, 2020 at 08:06:41PM +0100, Dennis Schridde wrote: > > How would you proceed to repair this cache? I can recreate it as a > > cachepool, but first I need to gain back access to the data, i.e. get > > into a state where I can --uncache. > > I know that cachevols do not yet work with lvconvert --repair, which uses > cache_repair, but this must be another case where cache_repair is still > needed. If this is writethough mode you should be able to simply drop the > cache with lvconvert --splitcache and then recreate it. Would this be safe if cache_dump showed all blocks are clean (dirty=false), even if the cache was in writeback mode? --Dennis signature.asc Description: This is a digitally signed message part. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
On Mittwoch, 9. Dezember 2020 20:30:51 CET David Teigland wrote: > On Wed, Dec 09, 2020 at 08:06:41PM +0100, Dennis Schridde wrote: > > How would you proceed to repair this cache? I can recreate it as a > > cachepool, but first I need to gain back access to the data, i.e. get > > into a state where I can --uncache. > > I know that cachevols do not yet work with lvconvert --repair, which uses > cache_repair, but this must be another case where cache_repair is still > needed. If this is writethough mode you should be able to simply drop the > cache with lvconvert --splitcache and then recreate it. Sadly the cache is in writeback mode. Is my understanding correct that the cache itself is fine? It is just the inability to clear the needs_check flag that is preventing me from using the device? Or did I miss to run some checks that could reveal a problem / did I misinterpret the output of cache_check on lv_cache? --Dennis signature.asc Description: This is a digitally signed message part. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
On Wed, Dec 09, 2020 at 08:06:41PM +0100, Dennis Schridde wrote: > How would you proceed to repair this cache? I can recreate it as a cachepool, > but first I need to gain back access to the data, i.e. get into a state where > I can --uncache. I know that cachevols do not yet work with lvconvert --repair, which uses cache_repair, but this must be another case where cache_repair is still needed. If this is writethough mode you should be able to simply drop the cache with lvconvert --splitcache and then recreate it. Dave ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
Hello Zdenek! Thank you for your reply! On Mittwoch, 9. Dezember 2020 12:05:13 CET Zdenek Kabelac wrote: > Dne 06. 12. 20 v 22:01 Dennis Schridde napsal(a): > > The live system I am running these commands from is a Fedora 33: > > > > $ uname -a > > Linux localhost-live 5.8.15-301.fc33.x86_64 #1 SMP Thu Oct 15 16:58:06 UTC > > 2020 x86_64 x86_64 x86_64 GNU/Linux > > > > $ sudo lvm version > > > > LVM version: 2.03.10(2) (2020-08-09) > > Library version: 1.02.173 (2020-08-09) > > Driver version: 4.42.0 > > It seems you are using 'cvol' instead of 'cpool' solution which has been > evolving and you will need a newer lvm2 version. > > If you want to using caching with the version of lvm2 you have - you will > need to use cpools (which are a bit faster anyway). 2.03.10 is the latest released version of lvm2. So I would have to use the Git main branch instead? A bit confusing is that the lvmcache(7) manpage of lvm2 2.03.10 suggests to create a cachevol as the first option: $ lvconvert --type cache --cachevol fast vg/main Should this be adjusted, if cachevols are not yet fully supported in 2.03.10? How would you proceed to repair this cache? I can recreate it as a cachepool, but first I need to gain back access to the data, i.e. get into a state where I can --uncache. --Dennis signature.asc Description: This is a digitally signed message part. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
Dne 06. 12. 20 v 22:01 Dennis Schridde napsal(a): Hello! A cached logical volume of mine cannot be activated anymore: $ sudo vgchange -ay device-mapper: reload ioctl on (253:6) failed: Invalid argument 0 logical volume(s) in volume group "vg_ernie" now active dmesg logs: device-mapper: cache: 253:6: unable to switch cache to write mode until repaired. device-mapper: cache: 253:6: switching cache to read-only mode device-mapper: table: 253:6: cache: Unable to get write access to metadata, please check/repair metadata. device-mapper: ioctl: error adding target to table The code in question seems to be: https://github.com/torvalds/linux/blob/v5.8/ drivers/md/dm-cache-target.c#L957-L964 Hence I set out to check the cache and, if it is clean, clear the needs_check flag: $ sudo lvchange -ay vg_ernie/lv_cache Do you want to activate component LV in read-only mode? [y/n]: y Allowing activation of component LV. As said - by component activation you will get 'read-only' volume thus you cannot do 'in-place' changes this way. A bit puzzling is that the status of the needs_check flag appears to be "unknown": $ sudo lvs -a -o +lv_check_needed LVVG Attr LSizePool Origin Data% Meta% Move Log Cpy%Sync Convert CheckNeeded [lv_cache]vg_ernie CRi-a-C--- 232.88g unknown lv_system vg_ernie Cwi---C--- <1.82t [lv_cache] [lv_system_corig] unknown [lv_system_corig] vg_ernie owi---C--- <1.82t unknown The live system I am running these commands from is a Fedora 33: $ uname -a Linux localhost-live 5.8.15-301.fc33.x86_64 #1 SMP Thu Oct 15 16:58:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux $ sudo lvm version LVM version: 2.03.10(2) (2020-08-09) Library version: 1.02.173 (2020-08-09) Driver version: 4.42.0 It seems you are using 'cvol' instead of 'cpool' solution which has been evolving and you will need a newer lvm2 version. If you want to using caching with the version of lvm2 you have - you will need to use cpools (which are a bit faster anyway). Regards Zdenek ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
On Sonntag, 6. Dezember 2020 22:01:37 CET Dennis Schridde wrote: > [...] > $ sudo lvchange -ay vg_ernie/lv_cache > Do you want to activate component LV in read-only mode? [y/n]: y >Allowing activation of component LV. > [...] > Is it possible that `cache_check --clear-needs-check-flag does not clear > needs_check flag` does not actually clear the needs_check flag in my case? I suspect that this has something to do with the read only status of the LV... What if the superblock gets changed, but is then not written back to the LV, because it is read only, and failure to write it back was not treated as an error? I dd-ed lv_cache into a file, ran cache_check --clear-needs-check-flag on it, and then tried to dd it back to the LV. Sadly that results in a permission denied error. I was not able to figure out how to force writing it back. --Dennis signature.asc Description: This is a digitally signed message part. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
[linux-lvm] cache_check --clear-needs-check-flag does not clear needs_check flag?
Hello! A cached logical volume of mine cannot be activated anymore: $ sudo vgchange -ay device-mapper: reload ioctl on (253:6) failed: Invalid argument 0 logical volume(s) in volume group "vg_ernie" now active dmesg logs: device-mapper: cache: 253:6: unable to switch cache to write mode until repaired. device-mapper: cache: 253:6: switching cache to read-only mode device-mapper: table: 253:6: cache: Unable to get write access to metadata, please check/repair metadata. device-mapper: ioctl: error adding target to table The code in question seems to be: https://github.com/torvalds/linux/blob/v5.8/ drivers/md/dm-cache-target.c#L957-L964 Hence I set out to check the cache and, if it is clean, clear the needs_check flag: $ sudo lvchange -ay vg_ernie/lv_cache Do you want to activate component LV in read-only mode? [y/n]: y Allowing activation of component LV. $ sudo cache_check /dev/vg_ernie/lv_cache examining superblock examining mapping array examining hint array examining discard bitset $ sudo cache_check --clear-needs-check-flag /dev/vg_ernie/lv_cache examining superblock examining mapping array examining hint array examining discard bitset $ sudo lvchange -an vg_ernie/lv_cache But the problem persists: $ sudo vgchange -ay device-mapper: reload ioctl on (253:6) failed: Invalid argument 0 logical volume(s) in volume group "vg_ernie" now active I tried again in read/write mode, in case that would make a difference / silently fail: $ sudo lvchange -ay vg_ernie/lv_cache Do you want to activate component LV in read-only mode? [y/n]: n $ sudo cache_check --clear-needs-check-flag /dev/vg_ernie/lv_cache examining superblock examining mapping array examining hint array examining discard bitset $ sudo lvchange -an vg_ernie/lv_cache With the same results: $ sudo vgchange -ay device-mapper: reload ioctl on (253:6) failed: Invalid argument 0 logical volume(s) in volume group "vg_ernie" now active A bit puzzling is that the status of the needs_check flag appears to be "unknown": $ sudo lvs -a -o +lv_check_needed LVVG Attr LSizePool Origin Data% Meta% Move Log Cpy%Sync Convert CheckNeeded [lv_cache]vg_ernie CRi-a-C--- 232.88g unknown lv_system vg_ernie Cwi---C--- <1.82t [lv_cache] [lv_system_corig] unknown [lv_system_corig] vg_ernie owi---C--- <1.82t unknown The live system I am running these commands from is a Fedora 33: $ uname -a Linux localhost-live 5.8.15-301.fc33.x86_64 #1 SMP Thu Oct 15 16:58:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux $ sudo lvm version LVM version: 2.03.10(2) (2020-08-09) Library version: 1.02.173 (2020-08-09) Driver version: 4.42.0 Configuration:./configure --build=x86_64-redhat-linux-gnu -- host=x86_64-redhat-linux-gnu --program-prefix= --disable-dependency-tracking --prefix=/usr --exec-prefix=/usr --bindir=/usr/bin --sbindir=/usr/sbin -- sysconfdir=/etc --datadir=/usr/share --includedir=/usr/include --libdir=/usr/ lib64 --libexecdir=/usr/libexec --localstatedir=/var --sharedstatedir=/var/lib --mandir=/usr/share/man --infodir=/usr/share/info --with-default-dm-run-dir=/ run --with-default-run-dir=/run/lvm --with-default-pid-dir=/run --with- default-locking-dir=/run/lock/lvm --with-usrlibdir=/usr/lib64 --enable-fsadm --enable-write_install --with-user= --with-group= --with-device-uid=0 --with- device-gid=6 --with-device-mode=0660 --enable-pkgconfig --enable-cmdlib -- enable-dmeventd --enable-blkid_wiping --with-cluster=internal --enable- cmirrord --with-udevdir=/usr/lib/udev/rules.d --enable-udev_sync --with- thin=internal --with-cache=internal --enable-lvmpolld --enable-lvmlockd-dlm -- enable-lvmlockd-dlmcontrol --enable-lvmlockd-sanlock --enable-dbus-service -- enable-notify-dbus --enable-dmfilemapd --with-writecache=internal --with- vdo=internal --with-vdo-format=/usr/bin/vdoformat --disable-silent-rules Is it possible that `cache_check --clear-needs-check-flag does not clear needs_check flag` does not actually clear the needs_check flag in my case? Any help to get the system back online (without data loss, if possible) is appreciated! Best regards, Dennis signature.asc Description: This is a digitally signed message part. ___ linux-lvm mailing list linux-lvm@redhat.com https://www.redhat.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/