Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 09/02/2014 09:22 PM, Konrad Rzeszutek Wilk wrote: On Mon, Sep 01, 2014 at 06:03:06AM +0200, Juergen Gross wrote: On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). I have a patch which should do the job. It is based on the classic kernel patch Jan mentioned above. The system is coming up with it, I haven't tested it with a huge initrd up to now. My plan was to post the patch together with the rest of the >500GB support, but I can send it on it's own if required. Oooh goodies! I think it makes sense to post it whenever you think it is in the right state to be posted. Now that your pvSCSI drivers are in, you have tons of free time I suspect :-) Oh yeah. Only one or two lines missing in xl to support it. :-) I hope to have the >500GB patch ready for testing soon. I'd prefer to combine this and the large initrd patch in one series, as both need the same headers to be synced with Xen. In case I'm meeting some serious issues I'll post the large initrd patch on Friday. Juergen -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On Mon, Sep 01, 2014 at 06:03:06AM +0200, Juergen Gross wrote: > On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: > >On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: > >On 29.08.14 at 16:27, wrote: > >>>Sure. Btw, someone also contacted me saying they have the same problem > >>>without > >>>changing the layout but having really big initrd (500M). While that feels > >>>like > >>>it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M > >>>kernel image size area then). But if it can happen, then surely it does > >>>cause > >>>mappings to be where the module space starts then. > >> > >>Since the initrd doesn't really need to be mapped into the (limited) > >>virtual address space a pv guest starts with, we specifically got > >> > >>/* > >> * Whether or not the guest can deal with being passed an initrd not > >> * mapped through its initial page tables. > >> */ > >>#define XEN_ELFNOTE_MOD_START_PFN 16 > >> > >>to deal with that situation. The hypervisor side for Dom0 is in place, > >>and the kernel side works in our (classic) kernels. Whether it got > >>implemented for DomU meanwhile I don't know; I'm pretty certain > >>pv-ops kernels don't support it so far. > > > >Correct - Not implemented. Here is what I had mentioned in the past: > >(see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) > > > > > >XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking > > at that but I can't figure out a nice way of implementing this > > without the usage of SPARSEMAP_VMAP virtual addresses - which is how > > the classic Xen does it. But then - I don't know who is using huge PV > > guests - as the PVHVM does a fine job? But then with PVH, now you can > > boot with large amount of memory (1TB?) - so some of these issues > > would go away? Except the 'large ramdisk' as that would eat in the > > MODULES_VADDR I think? Needs more thinking. > > > >.. and then I left it and to my suprise saw on Luis's slides that > >Jurgen is going to take a look at that (500GB support). > > I have a patch which should do the job. It is based on the classic > kernel patch Jan mentioned above. The system is coming up with it, I > haven't tested it with a huge initrd up to now. My plan was to post the > patch together with the rest of the >500GB support, but I can send it > on it's own if required. Oooh goodies! I think it makes sense to post it whenever you think it is in the right state to be posted. Now that your pvSCSI drivers are in, you have tons of free time I suspect :-) > > Juergen > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On Mon, Sep 01, 2014 at 06:03:06AM +0200, Juergen Gross wrote: On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, stefan.ba...@canonical.com wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). I have a patch which should do the job. It is based on the classic kernel patch Jan mentioned above. The system is coming up with it, I haven't tested it with a huge initrd up to now. My plan was to post the patch together with the rest of the 500GB support, but I can send it on it's own if required. Oooh goodies! I think it makes sense to post it whenever you think it is in the right state to be posted. Now that your pvSCSI drivers are in, you have tons of free time I suspect :-) Juergen -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 09/02/2014 09:22 PM, Konrad Rzeszutek Wilk wrote: On Mon, Sep 01, 2014 at 06:03:06AM +0200, Juergen Gross wrote: On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, stefan.ba...@canonical.com wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). I have a patch which should do the job. It is based on the classic kernel patch Jan mentioned above. The system is coming up with it, I haven't tested it with a huge initrd up to now. My plan was to post the patch together with the rest of the 500GB support, but I can send it on it's own if required. Oooh goodies! I think it makes sense to post it whenever you think it is in the right state to be posted. Now that your pvSCSI drivers are in, you have tons of free time I suspect :-) Oh yeah. Only one or two lines missing in xl to support it. :-) I hope to have the 500GB patch ready for testing soon. I'd prefer to combine this and the large initrd patch in one series, as both need the same headers to be synced with Xen. In case I'm meeting some serious issues I'll post the large initrd patch on Friday. Juergen -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). I have a patch which should do the job. It is based on the classic kernel patch Jan mentioned above. The system is coming up with it, I haven't tested it with a huge initrd up to now. My plan was to post the patch together with the rest of the >500GB support, but I can send it on it's own if required. Juergen -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 08/29/2014 04:55 PM, Konrad Rzeszutek Wilk wrote: On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, stefan.ba...@canonical.com wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). I have a patch which should do the job. It is based on the classic kernel patch Jan mentioned above. The system is coming up with it, I haven't tested it with a huge initrd up to now. My plan was to post the patch together with the rest of the 500GB support, but I can send it on it's own if required. Juergen -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: > >>> On 29.08.14 at 16:27, wrote: > > Sure. Btw, someone also contacted me saying they have the same problem > > without > > changing the layout but having really big initrd (500M). While that feels > > like > > it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M > > kernel image size area then). But if it can happen, then surely it does > > cause > > mappings to be where the module space starts then. > > Since the initrd doesn't really need to be mapped into the (limited) > virtual address space a pv guest starts with, we specifically got > > /* > * Whether or not the guest can deal with being passed an initrd not > * mapped through its initial page tables. > */ > #define XEN_ELFNOTE_MOD_START_PFN 16 > > to deal with that situation. The hypervisor side for Dom0 is in place, > and the kernel side works in our (classic) kernels. Whether it got > implemented for DomU meanwhile I don't know; I'm pretty certain > pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). > > Jan > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
>>> On 29.08.14 at 16:27, wrote: > Sure. Btw, someone also contacted me saying they have the same problem > without > changing the layout but having really big initrd (500M). While that feels > like > it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M > kernel image size area then). But if it can happen, then surely it does > cause > mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Jan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29/08/14 15:32, Stefan Bader wrote: > On 29.08.2014 16:19, Andrew Cooper wrote: >> On 29/08/14 09:37, Stefan Bader wrote: >>> On 29.08.2014 00:42, Andrew Cooper wrote: On 28/08/2014 19:01, Stefan Bader wrote: >>> So not much further... but then I think I know what I do next. Probably >>> should >>> have done before. I'll replace the WARN_ON in vmalloc that triggers by >>> a panic >>> and at least get a crash dump of that situation when it occurs. Then I >>> can dig >>> in there with crash (really should have thought of that before)... >> I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing >> there >> that screams at me, so I fear I will have to wait until you get the crash >> and get some clues from that. > Ok, what a journey. So after long hours of painful staring at the code... > (and btw, if someone could tell me how the heck one can do a mfn_to_pfn > in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, "MACH2PHYS_VIRT_START[(unsigned long)pfn]" ought to do in a pinch. >>> machine_to_phys_mapping is set to that address but its not mapped inside the >>> crash dump. Somehow vtop in crash handles translations. I need to have a >>> look at >>> their code, I guess. >>> >>> Thanks, >>> Stefan >> What context is the crash dump? If it is a Xen+dom0 kexec()d to native >> linux, then the m2p should still be accessible given dom0's cr3. If it >> is some state copied off-host then you will need to adjust the copy to >> include that virtual range. > No its a domU dump of a PV guest taken with "xl dump-core" (or actually the > result of on-crash trigger). Ah - I believe the m2p lives in one of the Xen elf notes for a domain coredump. See what readelf -n shows. ~Andrew -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29.08.2014 16:19, Andrew Cooper wrote: > On 29/08/14 09:37, Stefan Bader wrote: >> On 29.08.2014 00:42, Andrew Cooper wrote: >>> On 28/08/2014 19:01, Stefan Bader wrote: >> So not much further... but then I think I know what I do next. Probably >> should >> have done before. I'll replace the WARN_ON in vmalloc that triggers by a >> panic >> and at least get a crash dump of that situation when it occurs. Then I >> can dig >> in there with crash (really should have thought of that before)... > I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing > there > that screams at me, so I fear I will have to wait until you get the crash > and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) >>> The M2P map lives in the Xen reserved virtual address space in each PV >>> guest, and forms part of the PV ABI. It is mapped read-only, in the >>> native width of the guest. >>> >>> 32bit PV (PAE) at 0xF580 >>> 64bit PV at 0x8000 >>> >>> This is represented by the MACH2PHYS_VIRT_START symbol from the Xen >>> public header files. You should be able to blindly construct a pointer >>> to it (if you have nothing better to hand), as it will be hooked into >>> the guests pagetables before execution starts. Therefore, >>> "MACH2PHYS_VIRT_START[(unsigned long)pfn]" ought to do in a pinch. >> machine_to_phys_mapping is set to that address but its not mapped inside the >> crash dump. Somehow vtop in crash handles translations. I need to have a >> look at >> their code, I guess. >> >> Thanks, >> Stefan > > What context is the crash dump? If it is a Xen+dom0 kexec()d to native > linux, then the m2p should still be accessible given dom0's cr3. If it > is some state copied off-host then you will need to adjust the copy to > include that virtual range. No its a domU dump of a PV guest taken with "xl dump-core" (or actually the result of on-crash trigger). > > ~Andrew > signature.asc Description: OpenPGP digital signature
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29/08/14 09:37, Stefan Bader wrote: > On 29.08.2014 00:42, Andrew Cooper wrote: >> On 28/08/2014 19:01, Stefan Bader wrote: > So not much further... but then I think I know what I do next. Probably > should > have done before. I'll replace the WARN_ON in vmalloc that triggers by a > panic > and at least get a crash dump of that situation when it occurs. Then I > can dig > in there with crash (really should have thought of that before)... I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. >>> Ok, what a journey. So after long hours of painful staring at the code... >>> (and btw, if someone could tell me how the heck one can do a mfn_to_pfn >>> in crash, I really would appreaciate :-P) >> The M2P map lives in the Xen reserved virtual address space in each PV >> guest, and forms part of the PV ABI. It is mapped read-only, in the >> native width of the guest. >> >> 32bit PV (PAE) at 0xF580 >> 64bit PV at 0x8000 >> >> This is represented by the MACH2PHYS_VIRT_START symbol from the Xen >> public header files. You should be able to blindly construct a pointer >> to it (if you have nothing better to hand), as it will be hooked into >> the guests pagetables before execution starts. Therefore, >> "MACH2PHYS_VIRT_START[(unsigned long)pfn]" ought to do in a pinch. > machine_to_phys_mapping is set to that address but its not mapped inside the > crash dump. Somehow vtop in crash handles translations. I need to have a look > at > their code, I guess. > > Thanks, > Stefan What context is the crash dump? If it is a Xen+dom0 kexec()d to native linux, then the m2p should still be accessible given dom0's cr3. If it is some state copied off-host then you will need to adjust the copy to include that virtual range. ~Andrew -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29.08.2014 00:42, Andrew Cooper wrote: > On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... >>> I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing >>> there >>> that screams at me, so I fear I will have to wait until you get the crash >>> and get some clues from that. >> Ok, what a journey. So after long hours of painful staring at the code... >> (and btw, if someone could tell me how the heck one can do a mfn_to_pfn >> in crash, I really would appreaciate :-P) > > The M2P map lives in the Xen reserved virtual address space in each PV > guest, and forms part of the PV ABI. It is mapped read-only, in the > native width of the guest. > > 32bit PV (PAE) at 0xF580 > 64bit PV at 0x8000 > > This is represented by the MACH2PHYS_VIRT_START symbol from the Xen > public header files. You should be able to blindly construct a pointer > to it (if you have nothing better to hand), as it will be hooked into > the guests pagetables before execution starts. Therefore, > "MACH2PHYS_VIRT_START[(unsigned long)pfn]" ought to do in a pinch. machine_to_phys_mapping is set to that address but its not mapped inside the crash dump. Somehow vtop in crash handles translations. I need to have a look at their code, I guess. Thanks, Stefan > > ~Andrew > signature.asc Description: OpenPGP digital signature
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29.08.2014 00:42, Andrew Cooper wrote: On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... nods I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, MACH2PHYS_VIRT_START[(unsigned long)pfn] ought to do in a pinch. machine_to_phys_mapping is set to that address but its not mapped inside the crash dump. Somehow vtop in crash handles translations. I need to have a look at their code, I guess. Thanks, Stefan ~Andrew signature.asc Description: OpenPGP digital signature
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29/08/14 09:37, Stefan Bader wrote: On 29.08.2014 00:42, Andrew Cooper wrote: On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... nods I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, MACH2PHYS_VIRT_START[(unsigned long)pfn] ought to do in a pinch. machine_to_phys_mapping is set to that address but its not mapped inside the crash dump. Somehow vtop in crash handles translations. I need to have a look at their code, I guess. Thanks, Stefan What context is the crash dump? If it is a Xen+dom0 kexec()d to native linux, then the m2p should still be accessible given dom0's cr3. If it is some state copied off-host then you will need to adjust the copy to include that virtual range. ~Andrew -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29.08.2014 16:19, Andrew Cooper wrote: On 29/08/14 09:37, Stefan Bader wrote: On 29.08.2014 00:42, Andrew Cooper wrote: On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... nods I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, MACH2PHYS_VIRT_START[(unsigned long)pfn] ought to do in a pinch. machine_to_phys_mapping is set to that address but its not mapped inside the crash dump. Somehow vtop in crash handles translations. I need to have a look at their code, I guess. Thanks, Stefan What context is the crash dump? If it is a Xen+dom0 kexec()d to native linux, then the m2p should still be accessible given dom0's cr3. If it is some state copied off-host then you will need to adjust the copy to include that virtual range. No its a domU dump of a PV guest taken with xl dump-core (or actually the result of on-crash trigger). ~Andrew signature.asc Description: OpenPGP digital signature
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29/08/14 15:32, Stefan Bader wrote: On 29.08.2014 16:19, Andrew Cooper wrote: On 29/08/14 09:37, Stefan Bader wrote: On 29.08.2014 00:42, Andrew Cooper wrote: On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... nods I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, MACH2PHYS_VIRT_START[(unsigned long)pfn] ought to do in a pinch. machine_to_phys_mapping is set to that address but its not mapped inside the crash dump. Somehow vtop in crash handles translations. I need to have a look at their code, I guess. Thanks, Stefan What context is the crash dump? If it is a Xen+dom0 kexec()d to native linux, then the m2p should still be accessible given dom0's cr3. If it is some state copied off-host then you will need to adjust the copy to include that virtual range. No its a domU dump of a PV guest taken with xl dump-core (or actually the result of on-crash trigger). Ah - I believe the m2p lives in one of the Xen elf notes for a domain coredump. See what readelf -n shows. ~Andrew -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 29.08.14 at 16:27, stefan.ba...@canonical.com wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On Fri, Aug 29, 2014 at 03:44:06PM +0100, Jan Beulich wrote: On 29.08.14 at 16:27, stefan.ba...@canonical.com wrote: Sure. Btw, someone also contacted me saying they have the same problem without changing the layout but having really big initrd (500M). While that feels like it should be impossible (if the kernel+initrd+xen stuff has to fix the 512M kernel image size area then). But if it can happen, then surely it does cause mappings to be where the module space starts then. Since the initrd doesn't really need to be mapped into the (limited) virtual address space a pv guest starts with, we specifically got /* * Whether or not the guest can deal with being passed an initrd not * mapped through its initial page tables. */ #define XEN_ELFNOTE_MOD_START_PFN 16 to deal with that situation. The hypervisor side for Dom0 is in place, and the kernel side works in our (classic) kernels. Whether it got implemented for DomU meanwhile I don't know; I'm pretty certain pv-ops kernels don't support it so far. Correct - Not implemented. Here is what I had mentioned in the past: (see http://lists.xen.org/archives/html/xen-devel/2014-03/msg00580.html) XEN_ELFNOTE_INIT_P2M, XEN_ELFNOTE_MOD_START_PFN - I had been looking at that but I can't figure out a nice way of implementing this without the usage of SPARSEMAP_VMAP virtual addresses - which is how the classic Xen does it. But then - I don't know who is using huge PV guests - as the PVHVM does a fine job? But then with PVH, now you can boot with large amount of memory (1TB?) - so some of these issues would go away? Except the 'large ramdisk' as that would eat in the MODULES_VADDR I think? Needs more thinking. .. and then I left it and to my suprise saw on Luis's slides that Jurgen is going to take a look at that (500GB support). Jan -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 28/08/2014 19:01, Stefan Bader wrote: >>> So not much further... but then I think I know what I do next. Probably >>> should >>> have done before. I'll replace the WARN_ON in vmalloc that triggers by a >>> panic >>> and at least get a crash dump of that situation when it occurs. Then I can >>> dig >>> in there with crash (really should have thought of that before)... >> I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing >> there >> that screams at me, so I fear I will have to wait until you get the crash >> and get some clues from that. > Ok, what a journey. So after long hours of painful staring at the code... > (and btw, if someone could tell me how the heck one can do a mfn_to_pfn > in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, "MACH2PHYS_VIRT_START[(unsigned long)pfn]" ought to do in a pinch. ~Andrew -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [Xen-devel] [PATCH] Solved the Xen PV/KASLR riddle
On 28/08/2014 19:01, Stefan Bader wrote: So not much further... but then I think I know what I do next. Probably should have done before. I'll replace the WARN_ON in vmalloc that triggers by a panic and at least get a crash dump of that situation when it occurs. Then I can dig in there with crash (really should have thought of that before)... nods I dug a bit in the code (arch/x86/xen/mmu.c) but there is nothing there that screams at me, so I fear I will have to wait until you get the crash and get some clues from that. Ok, what a journey. So after long hours of painful staring at the code... (and btw, if someone could tell me how the heck one can do a mfn_to_pfn in crash, I really would appreaciate :-P) The M2P map lives in the Xen reserved virtual address space in each PV guest, and forms part of the PV ABI. It is mapped read-only, in the native width of the guest. 32bit PV (PAE) at 0xF580 64bit PV at 0x8000 This is represented by the MACH2PHYS_VIRT_START symbol from the Xen public header files. You should be able to blindly construct a pointer to it (if you have nothing better to hand), as it will be hooked into the guests pagetables before execution starts. Therefore, MACH2PHYS_VIRT_START[(unsigned long)pfn] ought to do in a pinch. ~Andrew -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
