Re: Need help with a weird kernel update panic.

2020-05-12 Thread Roger Heflin
All it takes is one systemd bug, or any number of other conditions that prevent a proper umount. I have seen various kernel bugs cause it, I have seen the hw operating the disks stop working. In general using ext4+journal and/or xfs under the exact wrong conditions can result in an unbootable

Re: Need help with a weird kernel update panic.

2020-05-12 Thread Samuel Sieb
On 5/12/20 4:38 PM, John Mellor wrote: Hi Samuel, But that's the point.  A default install uses LVM, and therefore constructs a separate /boot partition running EXT4 with a journal.  You may of course decide to do your own partitions, but that setup is highly unusual and rare.  Most people

Re: Need help with a weird kernel update panic.

2020-05-12 Thread John Mellor
Hi Samuel, But that's the point. A default install uses LVM, and therefore constructs a separate /boot partition running EXT4 with a journal. You may of course decide to do your own partitions, but that setup is highly unusual and rare. Most people will have the default partitioning, and

Re: Need help with a weird kernel update panic.

2020-05-12 Thread Samuel Sieb
On 5/12/20 5:15 AM, John Mellor wrote: On 2020-05-09 5:53 p.m., Ed Greshko wrote: On 2020-05-09 21:01, John Mellor wrote: Now I need to wait for the next kernel upgrade, and see if that change fixes my crash/panic issue. Of course you could boot to the previous kernel, remove the latest one,

Re: Need help with a weird kernel update panic.

2020-05-12 Thread John Mellor
On 2020-05-09 5:53 p.m., Ed Greshko wrote: On 2020-05-09 21:01, John Mellor wrote: Now I need to wait for the next kernel upgrade, and see if that change fixes my  crash/panic issue. Of course you could boot to the previous kernel, remove the latest one, reboot again to the previous kernel

Re: Need help with a weird kernel update panic.

2020-05-09 Thread Ed Greshko
On 2020-05-09 21:01, John Mellor wrote: > Now I need to wait for the next kernel upgrade, and see if that change fixes  > my crash/panic issue. Of course you could boot to the previous kernel, remove the latest one, reboot again to the previous kernel and allow the upgrade to test. -- The key

Re: Need help with a weird kernel update panic.

2020-05-09 Thread Sam Varshavchik
Roger Heflin writes: I never said that it does not get unmounted, but unmounting does not mean the journal got replayed. Note the first thing ext4/xfs does on a mount/fsck is replays the journal, the read-only mount would replay the journal, but indications are that that fails often enough

Re: Need help with a weird kernel update panic.

2020-05-09 Thread Roger Heflin
I never said that it does not get unmounted, but unmounting does not mean the journal got replayed. Note the first thing ext4/xfs does on a mount/fsck is replays the journal, the read-only mount would replay the journal, but indications are that that fails often enough that the replay does not

Re: Need help with a weird kernel update panic.

2020-05-09 Thread John Mellor
Ok, I think I've finally managed to remove the journal on /boot. I did it by rebooting into an old F30 boot media in troubleshooting mode using 'tune2fs -O ^has_journal'.  I was unable to do this on the disk even in single user mode, as it always marked the /boot partition as unclean, and

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Samuel Sieb
On 5/8/20 8:59 PM, Sam Varshavchik wrote: I have used ext4 for /boot for 20+ years on Fedora, and can't say that I ever had this happen. And I always reboot after installing updates. I've never had this happen over all the very many computers I've worked with. Long term the solution is to

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Ed Greshko
On 2020-05-09 11:59, Sam Varshavchik wrote: > I would be very much surprised if a regular reboot does not unmount /boot, > prior to rebooting. I would think that all mounted filesystems except / will > get explicitly unmounted just before a halt or a reboot. You mean like this output from a

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Sam Varshavchik
Roger Heflin writes: What you are saying does not exactly match what I have previously seen, but there is a known feature with using a journaling filesystem (ext4-journal, or xfs) for /boot, if only the journal is updated and if it is not yet replayed into the non-journal then grub will not be

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Roger Heflin
Did you do a lazy unmount? Any other umount will fail if there are open files, and lazy does not actually unmount it until whatever has the files open closes them, which generally means it never gets unmounted. On Fri, May 8, 2020 at 5:53 PM John Mellor wrote: > > I like the idea of just

Re: Need help with a weird kernel update panic.

2020-05-08 Thread John Mellor
I like the idea of just removing the journal tag, but I don't think that I can modify the /boot filesystem.  Doing a umount works, but tune2fs claims that an e2fsck is required.  Running e2fsck says that the filesystem is still mounted, even though it is not.  Doing a fuser /dev/sda1 shows a

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Roger Heflin
You have to hit the timing right. ie install the kernel package and as quickly as possible reboot (automated, or very efficient). And if the update is more than just kernel, that may slowdown the process enough that the immediate reboot won't be quick enough. I have seen it 3-5 times and that

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Mauricio Tavares
On Fri, May 8, 2020 at 12:12 PM Roger Heflin wrote: > > A sync will flush the writes to the journal were the data is safe. It > will not force a replay of the journal. > > Nothing except removing the journal from the ext4 filesystem will fix it. > > This is not a fedora bug, this is a long

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Roger Heflin
A sync will flush the writes to the journal were the data is safe. It will not force a replay of the journal. Nothing except removing the journal from the ext4 filesystem will fix it. This is not a fedora bug, this is a long standing kernel/grub/filesystem interaction bug (all who use a

Re: Need help with a weird kernel update panic.

2020-05-08 Thread John Mellor
Interesting!  This machine does reboot in about 5secs and the other machines take longer, so it makes sense.  My /boot is mounted just like /home and / as follows:    /dev/sda1 on /boot type ext4 (rw,relatime,seclabel) I assume that a symple sync would flush the journal.  Its pretty easy to

Re: Need help with a weird kernel update panic.

2020-05-08 Thread Roger Heflin
What you are saying does not exactly match what I have previously seen, but there is a known feature with using a journaling filesystem (ext4-journal, or xfs) for /boot, if only the journal is updated and if it is not yet replayed into the non-journal then grub will not be able to find the new

Need help with a weird kernel update panic.

2020-05-08 Thread John Mellor
I have one completely stock workstation F32 machine where kernel updates almost always cause a multiple-reboot panic problem.  This problem also occurred on F31, but not on releases before that. I'm stumped and need some help in figuring it out. The symptoms vary in the number of reboots and