Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
On Wed, Jan 26, 2011 at 08:44:14PM -0800, H. Peter Anvin wrote: > On 01/26/2011 06:13 PM, Vivek Goyal wrote: >>> >>> - Use kexec/kdump. I wonder though, can kexec work on early panics? The >>>kind where the disk itself might not yet be initialized? >> >> As of today kexec/kdump does not work with early panics as kdump kernel >> needs to be loaded in reserved memory and that action happens once the >> first kernel has booted. >> >> There is so much to it that I really don't think that modifying bootloaders >> to >> do all that might not trivial and more importantly I think it probably is not >> worth the effort. >> > > It's really not all that hard, actually. > Nice, I'll try prototyping this using syslinux and kexec: this seems the safest path so far. Further details are in my first reply to Vivek. > -hpa thanks, -- Darwish http://darwish.07.googlepages.com ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
On 01/26/2011 06:13 PM, Vivek Goyal wrote: - Use kexec/kdump. I wonder though, can kexec work on early panics? The kind where the disk itself might not yet be initialized? As of today kexec/kdump does not work with early panics as kdump kernel needs to be loaded in reserved memory and that action happens once the first kernel has booted. There is so much to it that I really don't think that modifying bootloaders to do all that might not trivial and more importantly I think it probably is not worth the effort. It's really not all that hard, actually. -hpa ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
(Very timely! was just exploring kexec-tools and friends) On Wed, Jan 26, 2011 at 09:13:38PM -0500, Vivek Goyal wrote: > On Wed, Jan 26, 2011 at 02:49:54PM +0200, Ahmed S. Darwish wrote: > > > > If not, can kexec be modified to do so? That possibly needs the help > > of a a bootloader extension. > > - Generally kdump uses same kernel as first one. So if first kernel panics > then kdump kernel will also panic. Though one can keep a known good kdump > kernel. > I was thinking of a 'special' kdump kernel: the older kernel will just pass the physical RAM address of its ring buffer to the newer kernel. The newer kernel will be __so specialized__ that it will just: - display the log in a friendly manner - write it to the swap partition - .. and halt This way, I completely toss the BIOS while accomplishing similar goals. > > - Currently kexec-tools take care of loading the kdump kernel, and then > preparing memory map for second kernel and pass to it and also pass > along some information about vmcore in ELF headers to second kernel. > > There is so much to it that I really don't think that modifying bootloaders to > do all that might not trivial and more importantly I think it probably is not > worth the effort. > IMHO if done in a user-friendly and stable manner, it can really be worth the effort. But quite honestly, since I did not read all the setup work done by kexec-tools yet, I might be too optimistic. > Thanks > Vivek thanks, -- Darwish http://darwish.07.googlepages.com ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
On Wed, Jan 26, 2011 at 03:07:44PM -0800, Luck, Tony wrote: > >- The latest approach (proposed by Linus) is to forget the disk: jump to > > real-mode, but display the kernel log in a fancy format (with scroll > > ups and downs) instead. > > A while ago (first Plumbers conference?) someone was talking about > using a 2-d barcode to display the tail of the kernel log & oops > register data - with the plan that you could capture the image with > a cell phone camera, and then get all the oops data without worrying > about transcription errors as you wrote down & re-typed all the hex. > Yes, I added a link to the video of this presentation in PATCH #0/2. -- Darwish ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
On Wed, Jan 26, 2011 at 03:07:44PM -0800, Luck, Tony wrote: > >- The latest approach (proposed by Linus) is to forget the disk: jump to > > real-mode, but display the kernel log in a fancy format (with scroll > > ups and downs) instead. > > A while ago (first Plumbers conference?) someone was talking about > using a 2-d barcode to display the tail of the kernel log & oops > register data - with the plan that you could capture the image with > a cell phone camera, and then get all the oops data without worrying > about transcription errors as you wrote down & re-typed all the hex. > > Anyone know what happened to that plan? How is this more useful than a photograph of the backtrace ? Dave ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
>- The latest approach (proposed by Linus) is to forget the disk: jump to > real-mode, but display the kernel log in a fancy format (with scroll > ups and downs) instead. A while ago (first Plumbers conference?) someone was talking about using a 2-d barcode to display the tail of the kernel log & oops register data - with the plan that you could capture the image with a cell phone camera, and then get all the oops data without worrying about transcription errors as you wrote down & re-typed all the hex. Anyone know what happened to that plan? -Tony ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
RE: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
> How is this more useful than a photograph of the backtrace ? You can fit a lot more data into the 2-d barcode that will fit on the screen. You can also automate the recovery of the data (e.g. for posting to kerneloops.org). -Tony ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
* Ahmed S. Darwish wrote: > > Also, have you tried BIOS warm reset vector, which is supposed to reboot > > without > > clearing RAM contents - how well does it work in practice on typical > > laptops? If > > on crash we could reboot without memory getting cleared that would open up > > a > > vast area of space to store the kernel log into (RAM). > > AFAIK, the lmode->rmode transition is more forward-compatible. > > It seems the only place warm boot was documented is in the Intel MP spec, a > 12-years old document long obsoleted by ACPI. Meanwhile, the real-mode > transition > is rigorously documented in the current Intel and AMD manuals, albeit in kind > of a > holier-than-thou approach. I mean, use the warm reset vector to truly reset the box. Then, once a stable known-good kernel boots, *that* kernel can then recover all the log data which is sitting in a well-known place in RAM, automatically and transparently. Basically a bit like kexec, just more convenient and it also goes through the BIOS warm reset, so it might work better than kexec ... Thanks, Ingo ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
On Wed, Jan 26, 2011 at 01:58:13PM +0100, Ingo Molnar wrote: > > * Ahmed S. Darwish wrote: > > > - The latest approach (proposed by Linus) is to forget the disk: jump to > > real-mode, but display the kernel log in a fancy format (with scroll > > ups and downs) instead. > > > > Will re-initializing the VGA registers to their POST state be possible? > > If not, what about a "fail-safe" VGA driver? > > > > I'm most likely going to implement either the second or the third point, > > so I'd really appreciate some input. > > The third one suggested by Linus is the most useful and most direct one IMO. > Such a > 'oops mode' screen would be useful for regular kernel crashes as well. > Yes, it is the safest approach. If I can make kexec and kdump work on early panics using some help from bootloaders though, that can possibly give us the feature of saving kernel logs to disk upon panics _without_ danger. I did not look at the kexec and kdump code bases yet, so this might be a pipe dream (respective developers are CCed -- input appreciated). > > Also, have you tried BIOS warm reset vector, which is supposed to reboot > without > clearing RAM contents - how well does it work in practice on typical laptops? > If on > crash we could reboot without memory getting cleared that would open up a > vast area > of space to store the kernel log into (RAM). > AFAIK, the lmode->rmode transition is more forward-compatible. It seems the only place warm boot was documented is in the Intel MP spec, a 12-years old document long obsoleted by ACPI. Meanwhile, the real-mode transition is rigorously documented in the current Intel and AMD manuals, albeit in kind of a holier-than-thou approach. I'll investigate this further though. thanks, -- Darwish http://darwish.07.googlepages.com ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
Hi, On Wed, Jan 26, 2011 at 06:25:19AM +1000, Linus Torvalds wrote: > On Tue, Jan 25, 2011 at 11:47 PM, Ahmed S. Darwish > wrote: > > > > As a possible solution, below patches prototypes the idea of persistently > > storing the kernel log ring to a hard disk partition using the enhanced BIOS > > 0x13 services. > > > > Quite frankly, I'm not likely to _ever_ merge anything like this. > > Over the years, many people have tried to write things to disk on > oops. I refuse to take it. No way in hell do I want the situation of > "the system is screwed, so let's overwrite the disk" to be something > the kernel I release might do. It's crazy. That disk is a lot more > important than the kernel, and overwriting it when we might have > serious memory corruption issues or something is not a thing I feel is > appropriate. > OK, I've carefully read all the criticism posted here and in the other replies. A short summary: - It will be _too_ much to restore the PIC, IO-APIC, PCI configuration, and disk controllers to the BIOS Power-on self-test (POST) state. The solution will not be guaranteed to work on all machines. - The write path of the BIOS INT 0x13 services is not tested enough by contemporary x86 kernels (Windows and Linux). Thus, that part of the BIOS is probably not as stable as I originally claimed. Possible solutions: - Having a minimal "fail-safe" disk controller driver to access the disk. This can't be made "perfect" enough to work on all combinations of machines (Though it can work on a small subset, e.g. the usual lot of x86 laptops with Intel chipsets). Linus is not OK with the _whole_ idea of doing _any_ disk write on panic. - Use kexec/kdump. I wonder though, can kexec work on early panics? The kind where the disk itself might not yet be initialized? If not, can kexec be modified to do so? That possibly needs the help of a a bootloader extension. - The latest approach (proposed by Linus) is to forget the disk: jump to real-mode, but display the kernel log in a fancy format (with scroll ups and downs) instead. Will re-initializing the VGA registers to their POST state be possible? If not, what about a "fail-safe" VGA driver? I'm most likely going to implement either the second or the third point, so I'd really appreciate some input. thanks, > > If you want to do the BIOS services thing, do it for video: copy the > oops to low RAM, return to real mode, re-run the graphics card POST > routines to initialize text-mode, and use the BIOS to print out the > oops. That is WAY less scary than writing to disk. > > Linus -- Darwish http://darwish.07.googlepages.com ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
Re: [PATCH 0/2][concept RFC] x86: BIOS-save kernel log to disk upon panic
* Ahmed S. Darwish wrote: > - The latest approach (proposed by Linus) is to forget the disk: jump to > real-mode, but display the kernel log in a fancy format (with scroll > ups and downs) instead. > > Will re-initializing the VGA registers to their POST state be possible? > If not, what about a "fail-safe" VGA driver? > > I'm most likely going to implement either the second or the third point, > so I'd really appreciate some input. The third one suggested by Linus is the most useful and most direct one IMO. Such a 'oops mode' screen would be useful for regular kernel crashes as well. Also, have you tried BIOS warm reset vector, which is supposed to reboot without clearing RAM contents - how well does it work in practice on typical laptops? If on crash we could reboot without memory getting cleared that would open up a vast area of space to store the kernel log into (RAM). Thanks, Ingo ___ kexec mailing list kexec@lists.infradead.org http://lists.infradead.org/mailman/listinfo/kexec
