Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Tue, May 31, 2016 at 05:04:42PM +0300, Oleksandr Dmytryshyn wrote: > On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias > wrote: > > Hi, > > > > We have similar needs (not exactly the same) in some of our setups. > > We need to map certain OCMs (On Chip Memories) to dom0. Among other things, > > these are used to communicate with remote accelerators/CPUs that have > > "hardcoded" addresses to these RAMs. > > > > Our approach is more along the lines of Juliens second suggestion. We're > > trying to use the mmio-sram DTS bindings to bring in these memories into > > dom0. > > > > IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR. > > > > Another possible solution: > > I think you could reserve the memory area by simply not mentioning it > > in the main memory node (these nodes support multiple ranges so you can > > introduce gaps). Then you could for example create an mmio-sram node to > > get the memory explicitely mapped 1:1 into dom0. > > > > Just a moment ago, I posted an RFC for the mmio-sram support to the list. > Hi, Edgar. > > How do You access to the mapped OCMs in dom0? > Are genalloc-related functions (gen_pool_get/_alloc/_virt_to_phys) > only way to work with mmio-sram memory? Hi Oleksandr, I'm not familiar enough with the Linux APIs to give a good answer on which APIs are considered OK and which not. There are examples in the tree on other ways of using the srams though. Look for example at this (search for smp-sram): arch/arm/mach-rockchip/platsmp.c Documentation/devicetree/bindings/sram/rockchip-smp-sram.txt The allwinner,sun4i-a10-emac is another example. Best regards, Edgar > > > Cheers, > > Edgar > > > > > >> > >> Regards, > >> > >> [1] > >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html > >> [2] > >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html > >> > >> -- > >> Julien Grall > >> > >> ___ > >> Xen-devel mailing list > >> Xen-devel@lists.xen.org > >> http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias wrote: > Hi, > > We have similar needs (not exactly the same) in some of our setups. > We need to map certain OCMs (On Chip Memories) to dom0. Among other things, > these are used to communicate with remote accelerators/CPUs that have > "hardcoded" addresses to these RAMs. > > Our approach is more along the lines of Juliens second suggestion. We're > trying to use the mmio-sram DTS bindings to bring in these memories into > dom0. > > IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR. > > Another possible solution: > I think you could reserve the memory area by simply not mentioning it > in the main memory node (these nodes support multiple ranges so you can > introduce gaps). Then you could for example create an mmio-sram node to > get the memory explicitely mapped 1:1 into dom0. > > Just a moment ago, I posted an RFC for the mmio-sram support to the list. Hi, Edgar. How do You access to the mapped OCMs in dom0? Are genalloc-related functions (gen_pool_get/_alloc/_virt_to_phys) only way to work with mmio-sram memory? > Cheers, > Edgar > > >> >> Regards, >> >> [1] >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html >> [2] >> http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html >> >> -- >> Julien Grall >> >> ___ >> Xen-devel mailing list >> Xen-devel@lists.xen.org >> http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Fri, 20 May 2016, Edgar E. Iglesias wrote: > On Fri, May 20, 2016 at 04:04:43PM +0100, Julien Grall wrote: > > Hello Oleksandr, > > > > On 20/05/16 15:19, Oleksandr Dmytryshyn wrote: > > >On Fri, May 20, 2016 at 12:59 PM, Jan Beulich wrote: > > >On 20.05.16 at 10:45, wrote: > > >>>On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: > > >>>On 19.05.16 at 15:58, wrote: > > >Case 1: Dom0 is driver domain: > > >There is a Ducati firmware which runs on dedicated M4 core and decodes > > >video. This firmware uses hardcoded physical addresses for graphics > > >buffers. Those addresses should be inside address-space of the driver > > >domain (Dom0). Ducati firmware is proprietary and we have no ability > > >to rework it. So Dom0 kernel should be placed to the configured > > >address (to the DOM0 RAM bank with specific address). > > > > > >Case 2: Dom0 is Thin and DomD is driver domain. > > >All is the same: Ducati firmware requires special (hardcoded) > > >addresses. > > > > For both of these cases I would then wonder whether such > > environments are actually suitable for doing virtualization on. > > >>>Currently we use Jacinto 6 evaluation board with DRA74X processor. > > >>>We have both configurations (Thin Dom0 and Thich Dom0). > > >> > > >>Which says nothing about their suitability for virtualization. > > >Our solution is based on Jacinto 6 evaluation board with DRA74X > > >processor. We need video-playback. Ducati firmware decodes video and > > >it works only with hardcoded addresses so we need this patch. > > > > This patch is a way to solve the problem and may not be the only one. > > I would like to explore all the possibilities before taking an approach that > > requires to modify the memory allocator in Xen. > > > > In my previous mails, I suggested a different solution (see [1] and [2]). If > > you think it is not suitable, please share more details or explain why you > > think your patch is the only way to solve it. > > Hi, > > We have similar needs (not exactly the same) in some of our setups. > We need to map certain OCMs (On Chip Memories) to dom0. Among other things, > these are used to communicate with remote accelerators/CPUs that have > "hardcoded" addresses to these RAMs. > > Our approach is more along the lines of Juliens second suggestion. We're > trying to use the mmio-sram DTS bindings to bring in these memories into > dom0. > > IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR. > > Another possible solution: > I think you could reserve the memory area by simply not mentioning it > in the main memory node (these nodes support multiple ranges so you can > introduce gaps). Then you could for example create an mmio-sram node to > get the memory explicitely mapped 1:1 into dom0. > > Just a moment ago, I posted an RFC for the mmio-sram support to the list. This sounds much nicer that the solution proposed here. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Hi, Edgar. On Fri, May 20, 2016 at 7:05 PM, Edgar E. Iglesias wrote: > Hi, > > We have similar needs (not exactly the same) in some of our setups. > We need to map certain OCMs (On Chip Memories) to dom0. Among other things, > these are used to communicate with remote accelerators/CPUs that have > "hardcoded" addresses to these RAMs. > > Our approach is more along the lines of Juliens second suggestion. We're > trying to use the mmio-sram DTS bindings to bring in these memories into > dom0. > > IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR. > > Another possible solution: > I think you could reserve the memory area by simply not mentioning it > in the main memory node (these nodes support multiple ranges so you can > introduce gaps). Then you could for example create an mmio-sram node to > get the memory explicitely mapped 1:1 into dom0. > > Just a moment ago, I posted an RFC for the mmio-sram support to the list. Thank You for advice. I'll try to use Your solution. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Fri, May 20, 2016 at 04:04:43PM +0100, Julien Grall wrote: > Hello Oleksandr, > > On 20/05/16 15:19, Oleksandr Dmytryshyn wrote: > >On Fri, May 20, 2016 at 12:59 PM, Jan Beulich wrote: > >On 20.05.16 at 10:45, wrote: > >>>On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: > >>>On 19.05.16 at 15:58, wrote: > >Case 1: Dom0 is driver domain: > >There is a Ducati firmware which runs on dedicated M4 core and decodes > >video. This firmware uses hardcoded physical addresses for graphics > >buffers. Those addresses should be inside address-space of the driver > >domain (Dom0). Ducati firmware is proprietary and we have no ability > >to rework it. So Dom0 kernel should be placed to the configured > >address (to the DOM0 RAM bank with specific address). > > > >Case 2: Dom0 is Thin and DomD is driver domain. > >All is the same: Ducati firmware requires special (hardcoded) addresses. > > For both of these cases I would then wonder whether such > environments are actually suitable for doing virtualization on. > >>>Currently we use Jacinto 6 evaluation board with DRA74X processor. > >>>We have both configurations (Thin Dom0 and Thich Dom0). > >> > >>Which says nothing about their suitability for virtualization. > >Our solution is based on Jacinto 6 evaluation board with DRA74X > >processor. We need video-playback. Ducati firmware decodes video and > >it works only with hardcoded addresses so we need this patch. > > This patch is a way to solve the problem and may not be the only one. > I would like to explore all the possibilities before taking an approach that > requires to modify the memory allocator in Xen. > > In my previous mails, I suggested a different solution (see [1] and [2]). If > you think it is not suitable, please share more details or explain why you > think your patch is the only way to solve it. Hi, We have similar needs (not exactly the same) in some of our setups. We need to map certain OCMs (On Chip Memories) to dom0. Among other things, these are used to communicate with remote accelerators/CPUs that have "hardcoded" addresses to these RAMs. Our approach is more along the lines of Juliens second suggestion. We're trying to use the mmio-sram DTS bindings to bring in these memories into dom0. IIUC the Ducati FW issue correctly, you need to allocate a chunk of DDR. Another possible solution: I think you could reserve the memory area by simply not mentioning it in the main memory node (these nodes support multiple ranges so you can introduce gaps). Then you could for example create an mmio-sram node to get the memory explicitely mapped 1:1 into dom0. Just a moment ago, I posted an RFC for the mmio-sram support to the list. Cheers, Edgar > > Regards, > > [1] > http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html > [2] > http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html > > -- > Julien Grall > > ___ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Hello Oleksandr, On 20/05/16 15:19, Oleksandr Dmytryshyn wrote: On Fri, May 20, 2016 at 12:59 PM, Jan Beulich wrote: On 20.05.16 at 10:45, wrote: On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: On 19.05.16 at 15:58, wrote: Case 1: Dom0 is driver domain: There is a Ducati firmware which runs on dedicated M4 core and decodes video. This firmware uses hardcoded physical addresses for graphics buffers. Those addresses should be inside address-space of the driver domain (Dom0). Ducati firmware is proprietary and we have no ability to rework it. So Dom0 kernel should be placed to the configured address (to the DOM0 RAM bank with specific address). Case 2: Dom0 is Thin and DomD is driver domain. All is the same: Ducati firmware requires special (hardcoded) addresses. For both of these cases I would then wonder whether such environments are actually suitable for doing virtualization on. Currently we use Jacinto 6 evaluation board with DRA74X processor. We have both configurations (Thin Dom0 and Thich Dom0). Which says nothing about their suitability for virtualization. Our solution is based on Jacinto 6 evaluation board with DRA74X processor. We need video-playback. Ducati firmware decodes video and it works only with hardcoded addresses so we need this patch. This patch is a way to solve the problem and may not be the only one. I would like to explore all the possibilities before taking an approach that requires to modify the memory allocator in Xen. In my previous mails, I suggested a different solution (see [1] and [2]). If you think it is not suitable, please share more details or explain why you think your patch is the only way to solve it. Regards, [1] http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01879.html [2] http://lists.xenproject.org/archives/html/xen-devel/2016-05/msg01894.html -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Fri, May 20, 2016 at 12:59 PM, Jan Beulich wrote: On 20.05.16 at 10:45, wrote: >> On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: >> On 19.05.16 at 15:58, wrote: Case 1: Dom0 is driver domain: There is a Ducati firmware which runs on dedicated M4 core and decodes video. This firmware uses hardcoded physical addresses for graphics buffers. Those addresses should be inside address-space of the driver domain (Dom0). Ducati firmware is proprietary and we have no ability to rework it. So Dom0 kernel should be placed to the configured address (to the DOM0 RAM bank with specific address). Case 2: Dom0 is Thin and DomD is driver domain. All is the same: Ducati firmware requires special (hardcoded) addresses. >>> >>> For both of these cases I would then wonder whether such >>> environments are actually suitable for doing virtualization on. >> Currently we use Jacinto 6 evaluation board with DRA74X processor. >> We have both configurations (Thin Dom0 and Thich Dom0). > > Which says nothing about their suitability for virtualization. Our solution is based on Jacinto 6 evaluation board with DRA74X processor. We need video-playback. Ducati firmware decodes video and it works only with hardcoded addresses so we need this patch. > Jan > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
>>> On 20.05.16 at 10:45, wrote: > On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: > On 19.05.16 at 15:58, wrote: >>> Case 1: Dom0 is driver domain: >>> There is a Ducati firmware which runs on dedicated M4 core and decodes >>> video. This firmware uses hardcoded physical addresses for graphics >>> buffers. Those addresses should be inside address-space of the driver >>> domain (Dom0). Ducati firmware is proprietary and we have no ability >>> to rework it. So Dom0 kernel should be placed to the configured >>> address (to the DOM0 RAM bank with specific address). >>> >>> Case 2: Dom0 is Thin and DomD is driver domain. >>> All is the same: Ducati firmware requires special (hardcoded) addresses. >> >> For both of these cases I would then wonder whether such >> environments are actually suitable for doing virtualization on. > Currently we use Jacinto 6 evaluation board with DRA74X processor. > We have both configurations (Thin Dom0 and Thich Dom0). Which says nothing about their suitability for virtualization. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
On Thu, May 19, 2016 at 5:36 PM, Jan Beulich wrote: On 19.05.16 at 15:58, wrote: >> Case 1: Dom0 is driver domain: >> There is a Ducati firmware which runs on dedicated M4 core and decodes >> video. This firmware uses hardcoded physical addresses for graphics >> buffers. Those addresses should be inside address-space of the driver >> domain (Dom0). Ducati firmware is proprietary and we have no ability >> to rework it. So Dom0 kernel should be placed to the configured >> address (to the DOM0 RAM bank with specific address). >> >> Case 2: Dom0 is Thin and DomD is driver domain. >> All is the same: Ducati firmware requires special (hardcoded) addresses. > > For both of these cases I would then wonder whether such > environments are actually suitable for doing virtualization on. Currently we use Jacinto 6 evaluation board with DRA74X processor. We have both configurations (Thin Dom0 and Thich Dom0). > Jan > ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Oleksandr Dmytryshyn | Product Engineering and Development GlobalLogic M +38.067.382.2525 www.globallogic.com http://www.globallogic.com/email_disclaimer.txt On Thu, May 19, 2016 at 5:34 PM, Julien Grall wrote: > Hello Oleksandr, > > > On 19/05/16 14:58, Oleksandr Dmytryshyn wrote: >>> >>> Why would a user want to allocate DOM0 RAM bank to a specific address? >>> >>> If I understand correctly your patch, DOM0 will only able to allocate one >>> bank of the given size at the specific address. You also add this >>> possibility for guest domain (see patch #4) and try to control where the >>> guest memory will be allocated. This will increase a lot the chance of the >>> memory allocation to fail. >>> >>> For instance, the RAM region requested for DOM0 may have been used to >>> allocate memory for Xen internal. So you need a way to reserve memory in >>> order to avoid Xen using it. >>> >>> I expect most of the users who want to use direct memory mapped guest to >>> know the number of guests which will use this feature. >>> >>> A such feature is only useful when pass-through a device to the guest on >>> platfom without SMMU, so it is by default insecure. >>> >>> So I would suggest to create a new device-tree binding (or re-use an >>> actual one) to reserve memory region to be used for direct memory mapped >>> domain. >>> >>> Those regions could have an identifier to be used later during the >>> allocation. This would avoid memory fragmentation, allow multiple RAM bank >>> for DOM0,... >>> >>> Any opinions? >> >> >> Case 1: Dom0 is driver domain: >> There is a Ducati firmware which runs on dedicated M4 core and decodes >> video. This firmware uses hardcoded physical addresses for graphics >> buffers. Those addresses should be inside address-space of the driver >> domain (Dom0). Ducati firmware is proprietary and we have no ability >> to rework it. So Dom0 kernel should be placed to the configured >> address (to the DOM0 RAM bank with specific address). >> >> Case 2: Dom0 is Thin and DomD is driver domain. >> All is the same: Ducati firmware requires special (hardcoded) addresses. > > > So if I understand correctly, patches #4, #13, #16 are only here to > workaround a firmware which does not do the right thing? > > IHMO, modifying the memory allocator in Xen to make a firmware happy is just > overkill. We need to explore all the possible solutions before going > forward. Yes, You are right. This patch written to make a firmware happy. > From your description, it looks like to me that the device-tree does not > correctly describe the platform. The graphic buffers should be reserved > using /memreserve or via a specific binding. > > This would be used later by Xen to map the buffer into dom0 or allow dom0 to > map it to a guest. > > Regards, > > -- > Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
>>> On 19.05.16 at 15:58, wrote: > Case 1: Dom0 is driver domain: > There is a Ducati firmware which runs on dedicated M4 core and decodes > video. This firmware uses hardcoded physical addresses for graphics > buffers. Those addresses should be inside address-space of the driver > domain (Dom0). Ducati firmware is proprietary and we have no ability > to rework it. So Dom0 kernel should be placed to the configured > address (to the DOM0 RAM bank with specific address). > > Case 2: Dom0 is Thin and DomD is driver domain. > All is the same: Ducati firmware requires special (hardcoded) addresses. For both of these cases I would then wonder whether such environments are actually suitable for doing virtualization on. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Hello Oleksandr, On 19/05/16 14:58, Oleksandr Dmytryshyn wrote: Why would a user want to allocate DOM0 RAM bank to a specific address? If I understand correctly your patch, DOM0 will only able to allocate one bank of the given size at the specific address. You also add this possibility for guest domain (see patch #4) and try to control where the guest memory will be allocated. This will increase a lot the chance of the memory allocation to fail. For instance, the RAM region requested for DOM0 may have been used to allocate memory for Xen internal. So you need a way to reserve memory in order to avoid Xen using it. I expect most of the users who want to use direct memory mapped guest to know the number of guests which will use this feature. A such feature is only useful when pass-through a device to the guest on platfom without SMMU, so it is by default insecure. So I would suggest to create a new device-tree binding (or re-use an actual one) to reserve memory region to be used for direct memory mapped domain. Those regions could have an identifier to be used later during the allocation. This would avoid memory fragmentation, allow multiple RAM bank for DOM0,... Any opinions? Case 1: Dom0 is driver domain: There is a Ducati firmware which runs on dedicated M4 core and decodes video. This firmware uses hardcoded physical addresses for graphics buffers. Those addresses should be inside address-space of the driver domain (Dom0). Ducati firmware is proprietary and we have no ability to rework it. So Dom0 kernel should be placed to the configured address (to the DOM0 RAM bank with specific address). Case 2: Dom0 is Thin and DomD is driver domain. All is the same: Ducati firmware requires special (hardcoded) addresses. So if I understand correctly, patches #4, #13, #16 are only here to workaround a firmware which does not do the right thing? IHMO, modifying the memory allocator in Xen to make a firmware happy is just overkill. We need to explore all the possible solutions before going forward. From your description, it looks like to me that the device-tree does not correctly describe the platform. The graphic buffers should be reserved using /memreserve or via a specific binding. This would be used later by Xen to map the buffer into dom0 or allow dom0 to map it to a guest. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
> Why would a user want to allocate DOM0 RAM bank to a specific address? > > If I understand correctly your patch, DOM0 will only able to allocate one > bank of the given size at the specific address. You also add this possibility > for guest domain (see patch #4) and try to control where the guest memory > will be allocated. This will increase a lot the chance of the memory > allocation to fail. > > For instance, the RAM region requested for DOM0 may have been used to > allocate memory for Xen internal. So you need a way to reserve memory in > order to avoid Xen using it. > > I expect most of the users who want to use direct memory mapped guest to know > the number of guests which will use this feature. > > A such feature is only useful when pass-through a device to the guest on > platfom without SMMU, so it is by default insecure. > > So I would suggest to create a new device-tree binding (or re-use an actual > one) to reserve memory region to be used for direct memory mapped domain. > > Those regions could have an identifier to be used later during the > allocation. This would avoid memory fragmentation, allow multiple RAM bank > for DOM0,... > > Any opinions? Case 1: Dom0 is driver domain: There is a Ducati firmware which runs on dedicated M4 core and decodes video. This firmware uses hardcoded physical addresses for graphics buffers. Those addresses should be inside address-space of the driver domain (Dom0). Ducati firmware is proprietary and we have no ability to rework it. So Dom0 kernel should be placed to the configured address (to the DOM0 RAM bank with specific address). Case 2: Dom0 is Thin and DomD is driver domain. All is the same: Ducati firmware requires special (hardcoded) addresses. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Hello, On 18/05/16 17:32, Andrii Anisov wrote: From: Oleksandr Dmytryshyn This setting is used to adjust starting memory address allocated for kernel Dom0. To use 'rambase_pfn' setting just add for example 'dom0_rambase_pfn=0x8' to the hypervisor command line. Note that 'dom0_rambase_pfn' should be aligned with the smallest memory chunk which use xen memory allocator. Why would a user want to allocate DOM0 RAM bank to a specific address? If I understand correctly your patch, DOM0 will only able to allocate one bank of the given size at the specific address. You also add this possibility for guest domain (see patch #4) and try to control where the guest memory will be allocated. This will increase a lot the chance of the memory allocation to fail. For instance, the RAM region requested for DOM0 may have been used to allocate memory for Xen internal. So you need a way to reserve memory in order to avoid Xen using it. I expect most of the users who want to use direct memory mapped guest to know the number of guests which will use this feature. A such feature is only useful when pass-through a device to the guest on platfom without SMMU, so it is by default insecure. So I would suggest to create a new device-tree binding (or re-use an actual one) to reserve memory region to be used for direct memory mapped domain. Those regions could have an identifier to be used later during the allocation. This would avoid memory fragmentation, allow multiple RAM bank for DOM0,... Any opinions? Signed-off-by: Oleksandr Dmytryshyn --- xen/arch/arm/domain_build.c | 24 +--- xen/common/page_alloc.c | 68 +++-- xen/include/xen/mm.h| 2 ++ 3 files changed, 75 insertions(+), 19 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 2937ff7..b48718d 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -27,6 +27,9 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static u64 __initdata opt_dom0_rambase_pfn = 0; +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn); + int dom0_11_mapping = 1; #define DOM0_MEM_DEFAULT 0x800 /* 128 MiB */ @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) const unsigned int min_order = get_order_from_bytes(MB(4)); struct page_info *pg; unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); +u64 rambase_pfn = opt_dom0_rambase_pfn; +paddr_t mem_size = kinfo->unassigned_mem; int i; bool_t lowmem = is_32bit_domain(d); @@ -267,7 +272,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) { for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ ) { -pg = alloc_domheap_pages(d, order, MEMF_bits(bits)); +pg = alloc_domheap_pages_pfn(d, order, MEMF_bits(bits), rambase_pfn); if ( pg != NULL ) goto got_bank0; } @@ -284,16 +289,21 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) /* Now allocate more memory and fill in additional banks */ order = get_11_allocation_size(kinfo->unassigned_mem); +if ( opt_dom0_rambase_pfn ) +rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT; + while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS ) { -pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0); +pg = alloc_domheap_pages_pfn(d, order, lowmem ? MEMF_bits(32) : 0, + rambase_pfn); From my understanding, when rambase_pfn is not 0, the memory must be allocated contiguously at this specific address. So if the first call of alloc_domheap_pages (see a bit above) as failed, then this one will always fail because it means that someone has allocated some page in this region. if ( !pg ) { order --; if ( lowmem && order < min_low_order) { -D11PRINT("Failed at min_low_order, allow high allocations\n"); +if ( !opt_dom0_rambase_pfn ) +D11PRINT("Failed at min_low_order, allow high allocations\n"); order = get_11_allocation_size(kinfo->unassigned_mem); lowmem = false; continue; @@ -313,7 +323,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) if ( lowmem ) { -D11PRINT("Allocation below bank 0, allow high allocations\n"); +if ( !opt_dom0_rambase_pfn ) +D11PRINT("Allocation below bank 0, allow high allocations\n"); order = get_11_allocation_size(kinfo->unassigned_mem); lowmem = false; con
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
>>> On 19.05.16 at 14:26, wrote: > On 19/05/16 10:41, Jan Beulich wrote: > On 18.05.16 at 18:32, wrote: >>> --- a/xen/arch/arm/domain_build.c >>> +++ b/xen/arch/arm/domain_build.c >>> @@ -27,6 +27,9 @@ >>> static unsigned int __initdata opt_dom0_max_vcpus; >>> integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); >>> >>> +static u64 __initdata opt_dom0_rambase_pfn = 0; >>> +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn); >> >> Any addition of a command line option needs to be accompanied by >> an entry in the command line doc. >> >>> @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct >>> kernel_info *kinfo) >>> const unsigned int min_order = get_order_from_bytes(MB(4)); >>> struct page_info *pg; >>> unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); >>> +u64 rambase_pfn = opt_dom0_rambase_pfn; >> >> Use of __initdata in a non-__init function. > > All the functions within domain_build.c should have the attributes > __init. However, it has been forgotten for half of them. > > I am planning to send a patch to enforce it using the Makefile rules. I have such a work item (low) on my todo list too, actually. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
Hi Jan, On 19/05/16 10:41, Jan Beulich wrote: On 18.05.16 at 18:32, wrote: --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -27,6 +27,9 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static u64 __initdata opt_dom0_rambase_pfn = 0; +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn); Any addition of a command line option needs to be accompanied by an entry in the command line doc. @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) const unsigned int min_order = get_order_from_bytes(MB(4)); struct page_info *pg; unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); +u64 rambase_pfn = opt_dom0_rambase_pfn; Use of __initdata in a non-__init function. All the functions within domain_build.c should have the attributes __init. However, it has been forgotten for half of them. I am planning to send a patch to enforce it using the Makefile rules. Regards, -- Julien Grall ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
>>> On 18.05.16 at 18:32, wrote: > --- a/xen/arch/arm/domain_build.c > +++ b/xen/arch/arm/domain_build.c > @@ -27,6 +27,9 @@ > static unsigned int __initdata opt_dom0_max_vcpus; > integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); > > +static u64 __initdata opt_dom0_rambase_pfn = 0; > +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn); Any addition of a command line option needs to be accompanied by an entry in the command line doc. > @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct > kernel_info *kinfo) > const unsigned int min_order = get_order_from_bytes(MB(4)); > struct page_info *pg; > unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); > +u64 rambase_pfn = opt_dom0_rambase_pfn; Use of __initdata in a non-__init function. > --- a/xen/common/page_alloc.c > +++ b/xen/common/page_alloc.c > @@ -583,16 +583,17 @@ static void check_low_mem_virq(void) > } > } > > -/* Allocate 2^@order contiguous pages. */ > -static struct page_info *alloc_heap_pages( > +/* Allocate 2^@order contiguous pages at given pfn. */ > +static struct page_info *alloc_heap_pages_pfn( > unsigned int zone_lo, unsigned int zone_hi, > unsigned int order, unsigned int memflags, > -struct domain *d) > +struct domain *d, xen_pfn_t pfn) Altering generic allocation interfaces like this, for a boot time only purpose, doesn't seem warranted. Please reconsider the entire approach. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH RFC 13/18] xen: introduce and use 'dom0_rambase_pfn' setting for kernel Dom0
From: Oleksandr Dmytryshyn This setting is used to adjust starting memory address allocated for kernel Dom0. To use 'rambase_pfn' setting just add for example 'dom0_rambase_pfn=0x8' to the hypervisor command line. Note that 'dom0_rambase_pfn' should be aligned with the smallest memory chunk which use xen memory allocator. Signed-off-by: Oleksandr Dmytryshyn --- xen/arch/arm/domain_build.c | 24 +--- xen/common/page_alloc.c | 68 +++-- xen/include/xen/mm.h| 2 ++ 3 files changed, 75 insertions(+), 19 deletions(-) diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c index 2937ff7..b48718d 100644 --- a/xen/arch/arm/domain_build.c +++ b/xen/arch/arm/domain_build.c @@ -27,6 +27,9 @@ static unsigned int __initdata opt_dom0_max_vcpus; integer_param("dom0_max_vcpus", opt_dom0_max_vcpus); +static u64 __initdata opt_dom0_rambase_pfn = 0; +integer_param("dom0_rambase_pfn", opt_dom0_rambase_pfn); + int dom0_11_mapping = 1; #define DOM0_MEM_DEFAULT 0x800 /* 128 MiB */ @@ -248,6 +251,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) const unsigned int min_order = get_order_from_bytes(MB(4)); struct page_info *pg; unsigned int order = get_11_allocation_size(kinfo->unassigned_mem); +u64 rambase_pfn = opt_dom0_rambase_pfn; +paddr_t mem_size = kinfo->unassigned_mem; int i; bool_t lowmem = is_32bit_domain(d); @@ -267,7 +272,7 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) { for ( bits = order ; bits <= (lowmem ? 32 : PADDR_BITS); bits++ ) { -pg = alloc_domheap_pages(d, order, MEMF_bits(bits)); +pg = alloc_domheap_pages_pfn(d, order, MEMF_bits(bits), rambase_pfn); if ( pg != NULL ) goto got_bank0; } @@ -284,16 +289,21 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) /* Now allocate more memory and fill in additional banks */ order = get_11_allocation_size(kinfo->unassigned_mem); +if ( opt_dom0_rambase_pfn ) +rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT; + while ( kinfo->unassigned_mem && kinfo->mem.nr_banks < NR_MEM_BANKS ) { -pg = alloc_domheap_pages(d, order, lowmem ? MEMF_bits(32) : 0); +pg = alloc_domheap_pages_pfn(d, order, lowmem ? MEMF_bits(32) : 0, + rambase_pfn); if ( !pg ) { order --; if ( lowmem && order < min_low_order) { -D11PRINT("Failed at min_low_order, allow high allocations\n"); +if ( !opt_dom0_rambase_pfn ) +D11PRINT("Failed at min_low_order, allow high allocations\n"); order = get_11_allocation_size(kinfo->unassigned_mem); lowmem = false; continue; @@ -313,7 +323,8 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) if ( lowmem ) { -D11PRINT("Allocation below bank 0, allow high allocations\n"); +if ( !opt_dom0_rambase_pfn ) +D11PRINT("Allocation below bank 0, allow high allocations\n"); order = get_11_allocation_size(kinfo->unassigned_mem); lowmem = false; continue; @@ -330,6 +341,11 @@ static void allocate_memory_11(struct domain *d, struct kernel_info *kinfo) * allocation possible. */ order = get_11_allocation_size(kinfo->unassigned_mem); +if ( opt_dom0_rambase_pfn ) +{ +rambase_pfn += (mem_size - kinfo->unassigned_mem) >> PAGE_SHIFT; +mem_size = kinfo->unassigned_mem; +} } if ( kinfo->unassigned_mem ) diff --git a/xen/common/page_alloc.c b/xen/common/page_alloc.c index 74fc1de..d0c0fbb 100644 --- a/xen/common/page_alloc.c +++ b/xen/common/page_alloc.c @@ -583,16 +583,17 @@ static void check_low_mem_virq(void) } } -/* Allocate 2^@order contiguous pages. */ -static struct page_info *alloc_heap_pages( +/* Allocate 2^@order contiguous pages at given pfn. */ +static struct page_info *alloc_heap_pages_pfn( unsigned int zone_lo, unsigned int zone_hi, unsigned int order, unsigned int memflags, -struct domain *d) +struct domain *d, xen_pfn_t pfn) { unsigned int i, j, zone = 0, nodemask_retry = 0; nodeid_t first_node, node = MEMF_get_node(memflags), req_node = node; unsigned long request = 1UL << order; -struct page_info *pg; +struct page_info *pg, *tmp_pg; +struct page_list_head *pg_list; nodemask_t nodemask = (d != NULL ) ? d->node_affinity : node_online_map; bool_t need_tlbflush = 0; uint32_t tlbflush_timestamp = 0; @@ -657,9 +658,25 @@ static struct page_info *alloc_heap_pages( continue;