Re: [Xen-devel] kdump with xen-unstable on efi machine
On 27/11/14 09:49, Ian Campbell wrote: On Wed, 2014-11-26 at 14:26 +, Andrew Cooper wrote: libxc (or some new alternative) should suck it up and gain some notion of a stable API or ABI (like the rest of the world appears to be able to manage), such that it is possible to compile with an older header and use a newer .so at runtime. Retrofitting a stable API/ABI to the melting pot which is libxc simply isn't going to work in practice. IMO the most likely to succeed approach would be to split off the bits of libxc which 3rd party's can/should/need to rely on into one of more libraries, probably by functional area. So far I'm aware of plans (or at least desires) to do that for: * Interfaces used by device-models/qemu. * The bits which are useful inside a guest (i.e. the various /dev/xen/* related helpers). So it sounds like libxenkexec should be added to that list. Ian. Agreed. For a domU, I think we need libxenevt, libxengnt and libxenstore with stable API and ABIs. This in turn will permit libvchan to work without needing libxenctrl. For dom0, each of the libraries is going to need basic hypercall functionality. It might be worth considering making libxenbasic (name looking for improvement) which is more along the lines of a privcmd driver, providing do_hypercall() and bounce buffering. libxenctrl and others can then avoid reimplementing the wheel many times. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] kdump with xen-unstable on efi machine
On Thu, 2014-11-27 at 11:05 +, Andrew Cooper wrote: On 27/11/14 09:49, Ian Campbell wrote: On Wed, 2014-11-26 at 14:26 +, Andrew Cooper wrote: libxc (or some new alternative) should suck it up and gain some notion of a stable API or ABI (like the rest of the world appears to be able to manage), such that it is possible to compile with an older header and use a newer .so at runtime. Retrofitting a stable API/ABI to the melting pot which is libxc simply isn't going to work in practice. IMO the most likely to succeed approach would be to split off the bits of libxc which 3rd party's can/should/need to rely on into one of more libraries, probably by functional area. So far I'm aware of plans (or at least desires) to do that for: * Interfaces used by device-models/qemu. * The bits which are useful inside a guest (i.e. the various /dev/xen/* related helpers). So it sounds like libxenkexec should be added to that list. Ian. Agreed. For a domU, I think we need libxenevt, libxengnt and libxenstore with stable API and ABIs. This in turn will permit libvchan to work without needing libxenctrl. For dom0, each of the libraries is going to need basic hypercall functionality. It might be worth considering making libxenbasic (name looking for improvement) which is more along the lines of a privcmd driver, providing do_hypercall() and bounce buffering. libxenctrl and others can then avoid reimplementing the wheel many times. NB that if such a library contains the h/call wrappers for any non-stable hypercall interface (I'm not sure if that was what you were suggesting or not) then it would itself remain ABI/API unstable. But anyway there's nothing stopping libxen* using interfaces exposed by an unstable libxenctrl or libxenbasic and remaining interface stable themselves, so you can avoid reimplementing the wheel that way. Those libraries just need to do the impedance matching and link the correct SONAME for the unstable library, the linker will take care of the rest. I don't think it is going to be realistic in practice to support keeping the exact same libxenfoo.so.N binary working across multiple releases, but rather each release will provide its own libxenfoo.so.N with the same N and supporting the same ABI, such that it can just be swapped in under the feet of the application. The alternative is to do dlopen stuff on the unstable backend library, which is going to be a nightmare, from the test matrix PoV if nothing else. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] kdump with xen-unstable on efi machine
On 11/26/2014 01:41 PM, Andrew Cooper wrote: On 26/11/14 12:15, Juergen Gross wrote: Hi, I tried to enable kdump on my test-machine with actual xen-unstable booting via EFI. The kdump kernel is not being loaded. I'm seeing the memory being reserved: (XEN) EFI RAM map: (XEN) - 000a (usable) (XEN) 0010 - 4bc0 (usable) (XEN) 4bc0 - 5bc0 (reserved) (XEN) 5bc0 - 5bfec000 (usable) (XEN) 5bfec000 - 5c00 (ACPI NVS) (XEN) 5c00 - 6a429000 (usable) (XEN) 6a429000 - 6a42c000 (reserved) (XEN) 6a42c000 - 6a7a2000 (usable) (XEN) 6a7a2000 - 6a7a8000 (reserved) (XEN) 6a7a8000 - 6a987000 (usable) (XEN) 6a987000 - 6a98d000 (reserved) (XEN) 6a98d000 - 6aa63000 (usable) (XEN) 6aa63000 - 6aa73000 (reserved) (XEN) 6aa73000 - 6ac6 (usable) (XEN) 6ac6 - 6ac61000 (reserved) (XEN) 6ac61000 - 6ac9b000 (ACPI data) (XEN) 6ac9b000 - 6acac000 (reserved) (XEN) 6acac000 - 6acad000 (usable) (XEN) 6acad000 - 6acae000 (reserved) (XEN) 6acae000 - 7189c000 (usable) (XEN) 7189c000 - 71946000 (reserved) (XEN) 71946000 - 72d76000 (ACPI NVS) (XEN) 72d76000 - 72db2000 (ACPI data) (XEN) 72db2000 - 72edc000 (usable) (XEN) 8000 - 9000 (reserved) (XEN) 0001 - 00208000 (usable) (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 I'd expect this area being visible in the efi or e820 map presented to dom0, but I can't see anything: This is expected. The dom0 kernel now has nothing at all do with loading crash kernel. Loading happens via hypercalls straight from the kexec utility. You need kexec-tools 2.0.4 (I think) or later, compiled with Xen support, but it should JustWork. Should. I have kexec 2.0.5 with Xen support. Doesn't work: Excerpt form strace: sysctl operation failed -- need to rebuild the user-space tool set?\n My personal translation: kexec is tightly coupled to the Xen version (this one was built against Xen 4.4.1 AFAIK). Perhaps we should add kexec to the tools directory? Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] kdump with xen-unstable on efi machine
On 26/11/14 14:01, Juergen Gross wrote: On 11/26/2014 01:41 PM, Andrew Cooper wrote: On 26/11/14 12:15, Juergen Gross wrote: Hi, I tried to enable kdump on my test-machine with actual xen-unstable booting via EFI. The kdump kernel is not being loaded. I'm seeing the memory being reserved: (XEN) EFI RAM map: (XEN) - 000a (usable) (XEN) 0010 - 4bc0 (usable) (XEN) 4bc0 - 5bc0 (reserved) (XEN) 5bc0 - 5bfec000 (usable) (XEN) 5bfec000 - 5c00 (ACPI NVS) (XEN) 5c00 - 6a429000 (usable) (XEN) 6a429000 - 6a42c000 (reserved) (XEN) 6a42c000 - 6a7a2000 (usable) (XEN) 6a7a2000 - 6a7a8000 (reserved) (XEN) 6a7a8000 - 6a987000 (usable) (XEN) 6a987000 - 6a98d000 (reserved) (XEN) 6a98d000 - 6aa63000 (usable) (XEN) 6aa63000 - 6aa73000 (reserved) (XEN) 6aa73000 - 6ac6 (usable) (XEN) 6ac6 - 6ac61000 (reserved) (XEN) 6ac61000 - 6ac9b000 (ACPI data) (XEN) 6ac9b000 - 6acac000 (reserved) (XEN) 6acac000 - 6acad000 (usable) (XEN) 6acad000 - 6acae000 (reserved) (XEN) 6acae000 - 7189c000 (usable) (XEN) 7189c000 - 71946000 (reserved) (XEN) 71946000 - 72d76000 (ACPI NVS) (XEN) 72d76000 - 72db2000 (ACPI data) (XEN) 72db2000 - 72edc000 (usable) (XEN) 8000 - 9000 (reserved) (XEN) 0001 - 00208000 (usable) (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 I'd expect this area being visible in the efi or e820 map presented to dom0, but I can't see anything: This is expected. The dom0 kernel now has nothing at all do with loading crash kernel. Loading happens via hypercalls straight from the kexec utility. You need kexec-tools 2.0.4 (I think) or later, compiled with Xen support, but it should JustWork. Should. I have kexec 2.0.5 with Xen support. Doesn't work: Excerpt form strace: sysctl operation failed -- need to rebuild the user-space tool set?\n My personal translation: kexec is tightly coupled to the Xen version (this one was built against Xen 4.4.1 AFAIK). It uses libxc, so needs to be built from the same source. Perhaps we should add kexec to the tools directory? The tools directory, and this willingness to fork other projects and keep a local copy, is the primary driver behind this situation being as dire as it is. libxc (or some new alternative) should suck it up and gain some notion of a stable API or ABI (like the rest of the world appears to be able to manage), such that it is possible to compile with an older header and use a newer .so at runtime. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] kdump with xen-unstable on efi machine
On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: On 11/26/2014 01:41 PM, Andrew Cooper wrote: On 26/11/14 12:15, Juergen Gross wrote: Hi, I tried to enable kdump on my test-machine with actual xen-unstable booting via EFI. The kdump kernel is not being loaded. I'm seeing the memory being reserved: (XEN) EFI RAM map: (XEN) - 000a (usable) (XEN) 0010 - 4bc0 (usable) (XEN) 4bc0 - 5bc0 (reserved) (XEN) 5bc0 - 5bfec000 (usable) (XEN) 5bfec000 - 5c00 (ACPI NVS) (XEN) 5c00 - 6a429000 (usable) (XEN) 6a429000 - 6a42c000 (reserved) (XEN) 6a42c000 - 6a7a2000 (usable) (XEN) 6a7a2000 - 6a7a8000 (reserved) (XEN) 6a7a8000 - 6a987000 (usable) (XEN) 6a987000 - 6a98d000 (reserved) (XEN) 6a98d000 - 6aa63000 (usable) (XEN) 6aa63000 - 6aa73000 (reserved) (XEN) 6aa73000 - 6ac6 (usable) (XEN) 6ac6 - 6ac61000 (reserved) (XEN) 6ac61000 - 6ac9b000 (ACPI data) (XEN) 6ac9b000 - 6acac000 (reserved) (XEN) 6acac000 - 6acad000 (usable) (XEN) 6acad000 - 6acae000 (reserved) (XEN) 6acae000 - 7189c000 (usable) (XEN) 7189c000 - 71946000 (reserved) (XEN) 71946000 - 72d76000 (ACPI NVS) (XEN) 72d76000 - 72db2000 (ACPI data) (XEN) 72db2000 - 72edc000 (usable) (XEN) 8000 - 9000 (reserved) (XEN) 0001 - 00208000 (usable) (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 I'd expect this area being visible in the efi or e820 map presented to dom0, but I can't see anything: This is expected. The dom0 kernel now has nothing at all do with loading crash kernel. Loading happens via hypercalls straight from the kexec utility. You need kexec-tools 2.0.4 (I think) or later, compiled with Xen support, but it should JustWork. Should. I have kexec 2.0.5 with Xen support. Doesn't work: Excerpt form strace: sysctl operation failed -- need to rebuild the user-space tool set?\n My personal translation: kexec is tightly coupled to the Xen version (this one was built against Xen 4.4.1 AFAIK). Odd, the hypercall interface did not change in Xen 4.5 for kexec? Perhaps it is making some other hypercalls that are tied in to the version of Xen (like sysctl ones?). I presume with recompiling it works? Perhaps we should add kexec to the tools directory? Gosh no. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] kdump with xen-unstable on efi machine
On 11/27/2014 01:19 AM, Daniel Kiper wrote: On Wed, Nov 26, 2014 at 03:49:54PM +0100, Juergen Gross wrote: On 11/26/2014 03:30 PM, Konrad Rzeszutek Wilk wrote: On Wed, Nov 26, 2014 at 03:01:51PM +0100, Juergen Gross wrote: On 11/26/2014 01:41 PM, Andrew Cooper wrote: On 26/11/14 12:15, Juergen Gross wrote: Hi, I tried to enable kdump on my test-machine with actual xen-unstable booting via EFI. The kdump kernel is not being loaded. I'm seeing the memory being reserved: (XEN) EFI RAM map: (XEN) - 000a (usable) (XEN) 0010 - 4bc0 (usable) (XEN) 4bc0 - 5bc0 (reserved) (XEN) 5bc0 - 5bfec000 (usable) (XEN) 5bfec000 - 5c00 (ACPI NVS) (XEN) 5c00 - 6a429000 (usable) (XEN) 6a429000 - 6a42c000 (reserved) (XEN) 6a42c000 - 6a7a2000 (usable) (XEN) 6a7a2000 - 6a7a8000 (reserved) (XEN) 6a7a8000 - 6a987000 (usable) (XEN) 6a987000 - 6a98d000 (reserved) (XEN) 6a98d000 - 6aa63000 (usable) (XEN) 6aa63000 - 6aa73000 (reserved) (XEN) 6aa73000 - 6ac6 (usable) (XEN) 6ac6 - 6ac61000 (reserved) (XEN) 6ac61000 - 6ac9b000 (ACPI data) (XEN) 6ac9b000 - 6acac000 (reserved) (XEN) 6acac000 - 6acad000 (usable) (XEN) 6acad000 - 6acae000 (reserved) (XEN) 6acae000 - 7189c000 (usable) (XEN) 7189c000 - 71946000 (reserved) (XEN) 71946000 - 72d76000 (ACPI NVS) (XEN) 72d76000 - 72db2000 (ACPI data) (XEN) 72db2000 - 72edc000 (usable) (XEN) 8000 - 9000 (reserved) (XEN) 0001 - 00208000 (usable) (XEN) Kdump: 256MB (262144kB) at 0x206dff4000 I'd expect this area being visible in the efi or e820 map presented to dom0, but I can't see anything: This is expected. The dom0 kernel now has nothing at all do with loading crash kernel. Loading happens via hypercalls straight from the kexec utility. You need kexec-tools 2.0.4 (I think) or later, compiled with Xen support, but it should JustWork. Should. I have kexec 2.0.5 with Xen support. Doesn't work: Excerpt form strace: sysctl operation failed -- need to rebuild the user-space tool set?\n My personal translation: kexec is tightly coupled to the Xen version (this one was built against Xen 4.4.1 AFAIK). Odd, the hypercall interface did not change in Xen 4.5 for kexec? Perhaps it is making some other hypercalls that are tied in to the version of Xen (like sysctl ones?). The error message above suggests that, yes. :-) Grepping for xc_ in kexec sources finds e.g. xc_get_max_cpus() which in turn calls xc_physinfo() doing a sysctl. I presume with recompiling it works? Didn't check up to now, but I think it should. Are you sure that kexec-tools configure script discovered Xen headers and development libraries? Please check that. ldd kexec is your friend. Aah, here is the problem: The kexec installed on my test system has been linked statically with libxenctrl to be usable on non-Xen systems, too. The proper solution, however, would have been to check for running on Xen dynamically and then load the current library via dlopen(). For now I'll just rebuild kexec. I put above on my todo list. As I expect to modify kexec in the future for support of the linear mapped p2m list I'll do the dlopen() stuff then. Do not forget to use kexec-tools version 2.0.5 or newer. Perhaps we should add kexec to the tools directory? Gosh no. Oops, did I forget the smiley? ;-) I think we should look what kexec is really needing and put this in a stable interface set (perhaps an own library?). This might require some David did the work. Okay. new sub functions of e.g. the KEXEC hypercall, but this is better than making kexec depending on the Xen version. Maybe we need some things which are specific for EFI platforms. I am going to investigate that after finishing EFI + multiboot2 work. Probably it will happen at the beginning of next year. I'll come back to you when I'm going to change kexec. Juergen ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
