On 10/13/06, Vivek Goyal <[EMAIL PROTECTED]> wrote: > On Fri, Oct 13, 2006 at 01:43:54PM +0900, Magnus Damm wrote: > > Hi Vivek, > > > > Thanks for the quick reply! > > > > On 10/12/06, Vivek Goyal <[EMAIL PROTECTED]> wrote: > > >On Thu, Oct 12, 2006 at 06:47:08PM +0900, Magnus Damm wrote: > > >> Hi Vivek, > > >> > > >> This patch breaks kdump under Xen/dom0. This is because "Kernel code" > > >> isn't listed in /proc/iomem. The reason behind this is probably that > > >> the address where the kernel is loaded is in pseudo physical address > > >> space, and this space is not contiguously mapped 1:1 to machine > > >> addresses (real physical addresses in the Linux world). > > >> > > >> If we really want to pass physical addresses under Xen we need to pass > > >> the address of each and every page that maps the kernel. This sounds > > >> like a pretty redundant move IMO because the tool that parses Xen > > >> crash dumps needs to be (and is already) aware of this mapping anyhow. > > >> > > >> So the following (mangled) patch fixes this by warning instead of > > >> returning error: > > >> > > > > > >Magnus, I think this patch is problematic. This bypasses the > > >error checking mechanism in non-virtualized env. So due to some reason if > > >"Kernel Code" is not exported from /proc/iomem, kexec-tools will never > > >detect it and set the phyaddr to zero. And a kernel header will be created > > >with kernel virtual address X pointing to physical addr 0 which is wrong. > > >It should have been kernel virtual addr X pointing to physical addr 2MB > > >(Assuming kernel is loaded physically at 2MB addr). > > > > Yeah, I agree that the broken elf header is far from good. It was > > mostly a quick fix to be able to continue to use kexec-tools to kdump > > on x86_64. > > > > >I think somehow we shall have to make kexec-tools Xen aware to take care > > >of Xen special cases. > > > > So far we've been able to use an unmodified / generic version of > > kexec-tools under Xen/dom0. This has been very nice as it has given us > > more time to focus on the kernel side. > > > > Maybe now is a good time to put in some Xen knowledge into > > kexec-tools. I would like to keep things as simple as possible - > > especially the kernel code. I believe that the right solution is to > > put as much knowledge into the tool that analyzes the crash dump > > instead of adding things to the kernel. > > > > I agree.
Good! > > A good example on how things get complicated in the Xen case is the > > crash notes. The crash notes are per physical cpu, but the cpu:s that > > linux sees are all virtual cpus under Xen/dom0. So we could either > > create/use some interface between dom0 and the hypervisor to get that > > mapping and export that interface to userspace and kexec-tools, OR we > > can just simply scrap the crash notes in the Xen case. I'm all in > > favor for that solution, and instead we keep a per cpu structure in > > the hypervisor visible to the analysis tool as a global symbol in > > hypervisor space. > > > > In case of bare metal linux, creating Elf notes for cpu register states in > advance helps as we don't have to run any utility post crash to prepare > the core dump for analysis. For Xen its tricky. Primarily fot the reason > because kdump kernel captures state of the whole of the system and not > just the state belonging to one particular partition. So I guess I can > not afford to create ELF notes just for the virtual cpus for dom0. > We better capture the physical cpu register state at the time of crash > and recreate the the virtual view for dom0 back using dumpread utility. > (I think that's what you are already doing.) Yes, exactly! I agree with you. > So to me it makes perfect sense to make kexec-tools Xen aware, and take > action specific to Xen or specific to crash dumps in virtual environments. > > > Please bear in mind that our perspective here is the _entire_ system > > with both the hypervisor and domains. It seems to me like kexec-tools > > is kind of linux-centric and adds various pieces of information to the > > elf header depending on the running kernel. That's great for Linux - > > but in the Xen case there is no point in trying to fill in a physical > > address range for the kernel that doesn't even exist. Especially since > > the dump if for the entire system. Maybe it would make sense to have a > > physical address range for the hypervisor, but until Xen is > > relocatable I see no point in adding it. > > > > Does that mean that currently you are creating ELF headers for only the > physical memory used by dom0 and Xen hypervisor only? Well, we are not doing anything special - just using the standard version of kexec-tool. This probably means that the ELF headers are wrong right now. > > >I got few basic questions. > > > > > >- What is exported through /proc/iomem in Xen dom0. Chunks of RAM, are > > >these > > > pseudo physical addresses or actual physical addresses? > > > > Actual physical addresses. But no kernel addresses because they are > > not actual physical addresses. Under Xen I think that we shouldn't > > create such a header at all. > > > > These are the physical addresses for memory used by dom0 or physical > address for all the memory present in the system? The entire system. > Given the nature of crash dump in Xen, I think it makes sense to just > create ELF headers for all the physical memory present in the system > and don't create ELF notes for the cpu register states. Let a user space > utility later re-create the system state based on what partition dump > user is looking for. That sounds like a good and simple. I must confess that I'm a bit confused by all types of ELF headers, do you have any recommendation which headers that should be present? > > >- How do you create ELF headers for Xen dom0? I am assuming that you are > > > dumping whole of the physical memory when dom0 or hypervisor crashes. > > > From where do you get the info about all the physical memory? > > > > I suppose kexec-tools sets up enough information to make /proc/vmcore > > point out the correct amount of physical memory in the secondary crash > > kernel. Not sure if it is correct, but we are doing things as similar > > to standard Linux as possible. Not sure if it is such a good idea that > > if now kexec-tools sets up the amount of memory - what about memory > > hotplug? > > > > Yes, kexec-tools sets up ELF headers for the physical memory visible > through /proc/iomem. So if dom0 /proc/iomem is exporting all the physical > memory present in the system not just the memory being used by dom0 then > ELF headers will be set up for all the RAM present in the system. This is what happens today if I'm not mistaken. > For memory hot plug, kdump kernel needs to be reloaded. I think some > hotplug memory agent in user space can do that. Last time I checked, I > could not find any hotplug memory agent yet. Somebody said it is yet to > be written. Yeah, reloading the kernel should solve the problem. With memory hotplug as example I wanted to highlight some problem the regular Linux port may run into on bare metal because of the current kexec / kdump architecture. Compare the current memory solution with say, parsing e820 tables in-kernel and the memory-world-view-out-of-sync problem would go away. I feel that the current architecture with kexec-tools, purgatory, kernel and crash tools may be complicated for no apparent reason. But this feeling may just be because I'm trying to cram Xen into the mix... =) Anyway, exporting hardware information only in the Xen case sounds like the way to go, right? Thanks for the help! / magnus _______________________________________________ fastboot mailing list [email protected] https://lists.osdl.org/mailman/listinfo/fastboot
