On 2019/10/20 下午9:04, Ferry Toth wrote: > Op 20-10-2019 om 02:51 schreef Qu Wenruo: >> >> >> On 2019/10/20 上午8:26, Qu Wenruo wrote: >>> >>> >>> On 2019/10/20 上午12:24, Ferry Toth wrote: >>>> Hi, >>>> >>>> Op 19-10-2019 om 01:50 schreef Qu WenRuo: >>>>> >>>>> >>>>> On 2019/10/19 上午4:32, Ferry Toth wrote: >>>>>> Op 24-09-2019 om 10:11 schreef Qu Wenruo: >>>>>>> We have at least two user reports about bad inode generation makes >>>>>>> kernel reject the fs. >>>>>> >>>>>> May I add my report? I just upgraded Ubuntu from 19.04 -> 19.10 so >>>>>> kernel went from 5.0 -> 5.3 (but I was using 4.15 too). >>>>>> >>>>>> Booting 5.3 leaves me in initramfs as I have /boot on @boot and / >>>>>> on /@ >>>>>> >>>>>> In initramfs I can try to mount but get something like >>>>>> btrfs critical corrupt leaf invalid inode generation open_ctree >>>>>> failed >>>>>> >>>>>> Booting old kernel works just as before, no errors. >>>>>> >>>>>>> According to the creation time, the inode is created by some 2014 >>>>>>> kernel. >>>>>> >>>>>> How do I get the creation time? >>>>> >>>>> # btrfs ins dump-tree -b <the bytenr reported by kernel> <your device> >>>> >>>> I just went back to the office to reboot to 5.3 and check the creation >>>> times and found they were 2013 - 2014. >>>> >>>>>> >>>>>>> And the generation member of INODE_ITEM is not updated (unlike the >>>>>>> transid member) so the error persists until latest tree-checker >>>>>>> detects. >>>>>>> >>>>>>> Even the situation can be fixed by reverting back to older kernel >>>>>>> and >>>>>>> copying the offending dir/file to another inode and delete the >>>>>>> offending >>>>>>> one, it still should be done by btrfs-progs. >>>>>>> >>>>>> How to find the offending dir/file from the command line manually? >>>>> >>>>> # find <mount point> -inum <inode number> >>>> >>>> This works, thanks. >>>> >>>> But appears unpractical. After fix 2 files and reboot, I found 4 more, >>>> then 16, then I gave up. >> >> Another solution is use "find" to locate the files with creation time >> before 2015, and copy them to a new file, then replace the old file with >> the new file. > > Hmm. But how do I "find" by creation time (otime)? Do you have a > suggestion for this?
$ touch -t 201501010000 /tmp/sample $ find <mnt> -not -cnewer /tmp/sample If you want, you can add -exec to that find, but I'd only add that after confirming the execution load is verified. Thanks, Qu > >> It would be much safer than btrfs check --repair. >> >> Thanks, >> Qu >> >> >>>> >>>>> Thanks, >>>>> Qu >>>>> >>>>>> >>>>>>> This patchset adds such check and repair ability to btrfs-check, >>>>>>> with a >>>>>>> simple test image. >>>>>>> >>>>>>> Qu Wenruo (3): >>>>>>> btrfs-progs: check/lowmem: Add check and repair for invalid >>>>>>> inode >>>>>>> generation >>>>>>> btrfs-progs: check/original: Add check and repair for >>>>>>> invalid inode >>>>>>> generation >>>>>>> btrfs-progs: fsck-tests: Add test image for invalid inode >>>>>>> generation >>>>>>> repair >>>>>>> >>>>>>> check/main.c | 50 +++++++++++- >>>>>>> check/mode-lowmem.c | 76 >>>>>>> ++++++++++++++++++ >>>>>>> check/mode-original.h | 1 + >>>>>>> .../.lowmem_repairable | 0 >>>>>>> .../bad_inode_geneartion.img.xz | Bin 0 -> 2012 >>>>>>> bytes >>>>>>> 5 files changed, 126 insertions(+), 1 deletion(-) >>>>>>> create mode 100644 >>>>>>> tests/fsck-tests/043-bad-inode-generation/.lowmem_repairable >>>>>>> create mode 100644 >>>>>>> tests/fsck-tests/043-bad-inode-generation/bad_inode_geneartion.img.xz >>>>>>> >>>>>>> >>>>>> >>>> >>>> I checked out and built v5.3-rc1 of btrfs-progs. Then ran it on my >>>> mounted rootfs with linux 5.0 and captured the log (~1800 lines 209 >>>> errors). >>> >>> It's really not recommended to run btrfs check, especially repair on the >>> mounted fs, unless it's RO. >>> >>> A new transaction from kernel can easily screw up the repaired fs. >>>> >>>> I'm not sure if using the v5.0 kernel and/or checking mounted distorts >>>> the results? Else I'm going to need a live usb with a v5.3 kernel and >>>> v5.3 btrfs-progs. >>>> >>>> If you like I can share the log. Let me know. >>>> >>>> This issue can potentially cause a lot of grief. Our company server >>>> runs >>>> Ubuntu LTS (18.04.02) with a 4.15 kernel on a btrfs boot/rootfs with >>>> ~100 snapshots. I guess the problematic inodes need to be fixed on each >>>> snapshot prior to upgrading to 20.04 LTS (which might be on kernel >>>> ~5.6)? >>> >>> Yes. >>> >>>> >>>> Do I understand correctly that this FTB is caused by more strict >>>> checking of the fs by the kernel, while the tools to fix the detected >>>> corruptions are not yet released? >>> >>> Yes. >>> >>> Thanks, >>> Qu >>> >> >
