Re: [Xen-devel] [RFC v4]Proposal to allow setting up shared memory areas between VMs from xl config file

[Edgar E. Iglesias]

On Wed, Aug 02, 2017 at 07:10:25PM +0100, Julien Grall wrote:
> Hi,
> 
> On 01/08/17 00:53, Stefano Stabellini wrote:
> >On Mon, 31 Jul 2017, Edgar E. Iglesias wrote:
> >>>>>  @role Can only be 'master' or 'slave', it defaults to 
> >>>>> 'slave'.
> >>>>>
> >>>>>  @prot When @role = master, this means the largest set 
> >>>>> of
> >>>>>stage-2 permission flags that can be granted to 
> >>>>> the
> >>>>>slave domains.
> >>>>>When @role = slave, this means the stage-2 permission
> >>>>>flags of the shared memory area.
> >>>>>Currently only 'rw' is supported. If not set. it
> >>>>>defaults to 'rw'.
> >>>>>
> >>>>>  @cache_policy The stage-2 cacheability/shareability attributes 
> >>>>> of the
> >>>>>shared memory area. Currently, only two policies 
> >>>>> are
> >>>>>supported:
> >>>>>  * ARM_normal: Only applicable to ARM guests. 
> >>>>> This
> >>>>>would mean Inner and Outer 
> >>>>> Write-Back
> >>>>>Cacheable, and Inner Shareable.
> >>>>
> >>>>
> >>>>Is there a reason not to set this to Outer Shareable?
> >>>>Again, mainly useful when these pages get shared with devs as well.
> >>>>
> >>>>The guest can always lower it to Inner Shareable via S1 tables if needed.
> >>>
> >>>I don't think we can support memory sharing with devices in this version
> >>>of the document (see above about GSoC timelines). Normal memory is inner
> >>>shareable in Xen today, it makes sense to default to that.
> >>
> >>I thought we mapped RAM as Outer shareable to guests but you seem to be 
> >>right.
> >>I think we should be mapping all RAM as Outer Shareable and then let the
> >>guest decide what is Inner and what is Outer via it's S1 tables.
> >>Right now it would be impossible to be Coherent with a DMA device outside
> >>of the Inner domain...
> >>
> >>Perhaps we should fix that and then ARM_normal would by itself become Outer.
> >>If there's agreement I can test it and send a patch.
> >
> >Today, only device memory is mapped Outer Shareable, while normal memory
> >is mapped Inner Shareable. I am OK with changing the default in
> >mfn_to_p2m_entry to Outer Shareable for normal RAM if the change would
> >make it possible to do coherent DMA with more devices on the platform.
> >Julien?
> 
> Per Table D4-44 in ARM DDI 0487B.a, outer-shareable takes precedence on
> inner-shareable. So if the guest configures the stage-1 page table with
> outer-shareable, the resultant will be outer-shareable.

You're right, I had gotten this backwards.

Thanks,
Edgar



> 
> As I pointed out in a previous version, this option is a bit confusing
> because the value set in stage-2 does not mean this will be the resultant
> value. I would strongly recommend any user wanting to share memory between
> guests to read D4.5.4 "Combining the stage 1 and stage 2 attributes" before
> using this option. Otherwise, he/she will likely be confused and end up to
> mis-configure the attribute.
> 
> The current configuration (i.e "ARM_normal") gives the most flexibility to
> the guest in term of memory attribute but prevent non-shareable mapping.
> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Wed, Aug 02, 2017 at 02:07:17PM +0300, Andrii Anisov wrote:
> Hello Edgar,

Hi Andrii,

> 
> 
> On 01.08.17 20:13, Edgar E. Iglesias wrote:
> >>Are master ports behind IOMMU?
> >Yes, they are.
> What IOMMU IP is used?

Today it's an SMMUv2.


> 
> >>>It's possible to reprogram the configuration of the PL and swap 
> >>>accelerators in
> >>>and out on the fly. It's probably going to be too slow for trying to
> >>>context switch between guests
> >>So let us assume it is a FW-less resource we need to share between domains.
> >>Context switch will be stripped to mapping its mmio (or passing mmio
> >>accesses) next domain to serve and IOMMU configuration switching.
> >>Yep, IOMMU matters.
> >OK. I think the PL is more like a firmware enabled resource.
> >The PL configuration needs to be loaded much like firmware
> >otherwise the accelerators can't change shape and all guests
> >must use the same kind.
> I understand this.
> But I got your words like you are going to give a try to the same kind for
> all domains first. Because you assumed that reconfiguring would be too slow,
> what is actually discussable.

Aha, OK. What I meant was that it may be to slow for context-switching
at a micro-level. But with an allocate/release interface for batch
processing, I don't think it's to slow to reprogram the PL between
guests.

I agree that we need hard numbers on the PL programming before we rule
things out. I'll try to dig internally for some.


> >>>  so I think primarily we would be looking at
> >>>a way to lets say, "allocate" and "release" the resources for batch use.
> >>Kind of voluntary preemption?
> >Right. That could be a start.
> >In the future perhaps it makes sense to context-switch.
> We still need the context switch to be done. The difference is that now it
> could be done only when the accelerator is not busy.

I don't necessarily think so. The context-switch would involve saving and
restoring accelerator state aswell as re-programming the PL.
With allocate/release, we only need to re-program the PL.

Saving the state of the PL might be tricky since we don't know how to
(we don't know the details of the acceelerator ahead of time).
I guess we could somehow let the guest that owns the accellerator
save/restore the state somehow but perhaps that brings us back
in the direction of allocate/release semantics...


> >>>If a guest cannot allocate an accelerator, it could fall back to emulation
> >>>or just to using SW libraries until an accelerator slot is available.
> >>What about the thing I called "an access emulation" [1]? From the domain's
> >>point of view it would be reflected in a delayed response (via IRQ or
> >>register polling) from an accelerator.
> >>
> >>I guess the concept described above is feasible even with current SCF code
> >>and will not take too much efforts.
> >I'll have a look, thanks!
> Do not hesitate to contact us in case you need any help or clarification.

Thanks!
Edgar

> 
> 
> -- 
> 
> *Andrii Anisov*
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 10:02:45PM +0100, Julien Grall wrote:
> Hi,
> 
> On 01/08/2017 21:40, Stefano Stabellini wrote:
> >On Tue, 1 Aug 2017, Edgar E. Iglesias wrote:
> >>On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:
> >>>(+ Edgar, Mark, Dave)
> >>>
> >>>Hi,
> >>
> >>Hi Julien,
> >>
> >>I'll share some thoughts based on our platforms.
> >>
> >>
> >>>On 14/06/17 15:10, Volodymyr Babchuk wrote:
> >>>>SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
> >>>>SMCCC states that both HVC and SMC are valid conduits to call to a 
> >>>>different
> >>>>firmware functions. Thus, for example PSCI calls can be made both by
> >>>>SMC or HVC. Also SMCCC defines function number coding for such calls.
> >>>>Besides functional calls there are query calls, which allows underling
> >>>>OS determine version, UID and number of functions provided by service
> >>>>provider.
> >>>>
> >>>>This patch adds new file `smccc.c`, which handles both generic SMCs
> >>>>and HVC according to SMC. At this moment it implements only one
> >>>>service: Standard Hypervisor Service.
> >>>>
> >>>>Standard Hypervisor Service only supports query calls, so caller can
> >>>>ask about hypervisor UID and determine that it is XEN running.
> >>>>
> >>>>This change allows more generic handling for SMCs and HVCs and it can
> >>>>be easily extended to support new services and functions.
> >>>
> >>>I have already reviewed the code and one thing I missed is how a domain 
> >>>will
> >>>know that Xen supports SMCCC.
> >>>
> >>>Currently, Xen will:
> >>>   - inject an undefined instruction for any SMC issued by a guest
> >>>   - crash the guest (quite bad) for any unknown HCV #0
> >>>
> >>>So a guest needs to be aware whether Xen supports SMCCC convention or not. 
> >>>I
> >>>am not aware of any bindings in the device-tree for doing that.
> >>
> >>On our platforms, SW probes the DT for specific service classes and then
> >>probes for specific versions via SMC calls using the standard Version FIDs.
> >>If the DT does not specify the firmware node, I don't think any SMCs will be
> >>issued but the guest may not be functional (as the firmware interface is
> >>mandatory).
> >>
> >>I don't know of a generic DT node/compat that tells guests if SMCC probing
> >>is allowed or not. Perhaps there should be one, or there should be yet
> >>another service specific one for Hypervisors. I don't know.
> >>
> >>For example, these are the nodes we've got (PSCI and EEMI/SIP):
> >>psci {
> >>compatible = "arm,psci-0.2";
> >>method = "smc";
> >>};
> >>
> >>pmufw: firmware {
> >>compatible = "xlnx,zynqmp-pm";
> >>method = "smc";
> >>interrupt-parent = <>;
> >>interrupts = <0 35 4>;
> >>};
> >>
> >>SW that does not have DT support, will either directly probe the SMC
> >>interface or in some cases just assume it's there and use it.
> >>
> >>ZynqMP-wise, Xen is in a little bit of an akward position by messing
> >>guests up if they probe for non-existing SMC services but I don't think
> >>it's that bad. Mainly because there's really very little ZynqMP guests
> >>that need the firmware would be able todo without it.
> >>
> >>For other platforms and services, I guess FW may very well be optional
> >>and probing makes more sence. So getting support for gracefully returning
> >>Unknown FID still important...
> >
> >Indeed, but unfortunately older versions of Xen don't do that. That's
> >why I think we'll have to introduce a feature flag under the Xen
> >hypervisor node on device tree. The presence of the flag could mean "it
> >is safe to probe via SMC/HVC". I think it makes sense for the flag to be
> >Xen specific, because we are talking about Xen behavior. Applications
> >that know they are running on a new enough Xen can skip the check (this
> >is going to be the case in most embedded scenarios).
> 
> This is not Xen specific behavior. Per ARM ARM, SMC will be undefined when
> EL3 is not present. Today Xen is behaving the same way as EL3 was not
> existing on the platform.
> 
> So we need a generic way to say "SMC is supported and is SMCCC capable".

Would it be unthinkable to treat the lack of "Unknown FID" returns as a bug
and backport the "fix" and leave it at that?

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 08:04:09PM +0300, Andrii Anisov wrote:
> Hello Edgar,
> 
> 
> On 01.08.17 17:56, Edgar E. Iglesias wrote:
> >On the PL, there's a chunk of programmable logic that allows you to
> >create your own custom accellerators or devices.
> >Some devices are tied to specific boards (e.g when they depend on specific 
> >IO)
> >but others are not (for example memory to memory computational accelerators).
> >To communicate with these devices, they have memory slave and master ports
> >(for register accesses and for DMA). They also have interrupts both ways.
> Are master ports behind IOMMU?

Yes, they are.

> 
> >It's possible to reprogram the configuration of the PL and swap accelerators 
> >in
> >and out on the fly. It's probably going to be too slow for trying to
> >context switch between guests
> So let us assume it is a FW-less resource we need to share between domains.
> Context switch will be stripped to mapping its mmio (or passing mmio
> accesses) next domain to serve and IOMMU configuration switching.
> Yep, IOMMU matters.

OK. I think the PL is more like a firmware enabled resource.
The PL configuration needs to be loaded much like firmware
otherwise the accelerators can't change shape and all guests
must use the same kind.

For example, one guest might want a crypto accelerator while
another might want some kind of machine-learning accelerator.

I think each guest may want to provide it's own accelerator "config".


> >  so I think primarily we would be looking at
> >a way to lets say, "allocate" and "release" the resources for batch use.
> 
> Kind of voluntary preemption?

Right. That could be a start.
In the future perhaps it makes sense to context-switch.

> 
> >If a guest cannot allocate an accelerator, it could fall back to emulation
> >or just to using SW libraries until an accelerator slot is available.
> What about the thing I called "an access emulation" [1]? From the domain's
> point of view it would be reflected in a delayed response (via IRQ or
> register polling) from an accelerator.
> 
> I guess the concept described above is feasible even with current SCF code
> and will not take too much efforts.

I'll have a look, thanks!

Cheers,
Edgar

> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg01935.html
> 
> -- 
> 
> *Andrii Anisov*
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Next Xen ARM community call - Wednesday 2nd August 2017

[Edgar E. Iglesias]

On Wed, Jul 26, 2017 at 04:59:43PM +0100, Julien Grall wrote:
> Hi all,
> 
> The next Xen ARM community call will be Wednesday 2nd August 2017 5pm BST.
> 
> Do you have any specific topic you would like to discuss?

CC: Davorin and Mirella from Aggios

Hi Julien,

I was talking with the Aggios folks today and they were wondering if
it's possible to share screens to present slides during the call?
I'm guessing not, since the info below only has dial in info.

We can also email out slides before the call instead (reply to this email).

Thanks,
Edgar


> 
> Call+44 1223 406065 (Local dial in)
> and enter the access code below followed by # key.
> Participant code: 4915191
> 
> Mobile Auto Dial:
> VoIP: voip://+441223406065;4915191#
> iOS devices: +44 1223 406065,4915191 and press #
> Other devices: +44 1223 406065x4915191#
> 
> Additional Calling Information:
> 
> UK +44 1142828002
> US CA +1 4085761502
> US TX +1 5123141073
> JP +81 453455355
> DE +49 8945604050
> NO +47 73187518
> SE +46 46313131
> FR +33 497235101
> TW +886 35657119
> HU +36 13275600
> IE +353 91337900
> 
> Toll Free
> 
> UK 0800 1412084
> US +1 8668801148
> CN +86 4006782367
> IN 0008009868365
> IN +918049282778
> TW 08000 22065
> HU 0680981587
> IE 1800800022
> KF +972732558877
> 
> -- 
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 02:52:22PM +0300, Andrii Anisov wrote:
> Dear Edgar,
> 
> 
> On 31.07.17 23:42, Edgar E. Iglesias wrote:
> >Yes I'm interested in this.
> It's good to hear at least one vote for the stuff :)
> 
> >  I'm not sure how much time I'll be able to contribute but at least I can 
> > review proposals and hopefully look at implementing a driver/backend that 
> > may be useful for our FPGA platforms.
> I really hope we can start from small things:
> - Could you please describe use-cases you have in your mind. What
> functionality do you expect? It is really important for us, we need some
> side view on the framework.
> - Also it would be good for us to have some view on the coprocessor (fpga?)
> you are going to share using SCF. How is it exposed to the system? Does it
> have mmio, ram, irq, iommu connection, whatever?

Hi,

I don't really have a defined list of requirements but I can do some initial
hand-waiving :-)

On the ZynqMP we have to classes of HW (on the chip) that I think may benefit.
1. The Cortex-R5s
2. The Programmable Logic (FPGA)

The Cortex-R5s are two real-time co-processors that can be programmed.
They have local RAMs and control registers to reset, halt etc.
I'll leave these out for now, they are probably more similar to the
use-cases you had in mind.

On the PL, there's a chunk of programmable logic that allows you to
create your own custom accellerators or devices.
Some devices are tied to specific boards (e.g when they depend on specific IO)
but others are not (for example memory to memory computational accelerators).
To communicate with these devices, they have memory slave and master ports
(for register accesses and for DMA). They also have interrupts both ways.

It's possible to reprogram the configuration of the PL and swap accelerators in
and out on the fly. It's probably going to be too slow for trying to
context switch between guests so I think primarily we would be looking at
a way to lets say, "allocate" and "release" the resources for batch use.

If a guest cannot allocate an accelerator, it could fall back to emulation
or just to using SW libraries until an accelerator slot is available.

Best regards,
Edgar


> - Please comment on SCF configuration followup letter [1]
> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2017-07/msg02124.html
> 
> -- 
> 
> *Andrii Anisov*
> 
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] arm: smccc: handle SMCs/HVCs according to SMCCC

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 11:59:00AM +0100, Julien Grall wrote:
> (+ Edgar, Mark, Dave)
> 
> Hi,

Hi Julien,

I'll share some thoughts based on our platforms.


> On 14/06/17 15:10, Volodymyr Babchuk wrote:
> >SMCCC (SMC Call Convention) describes how to handle both HVCs and SMCs.
> >SMCCC states that both HVC and SMC are valid conduits to call to a different
> >firmware functions. Thus, for example PSCI calls can be made both by
> >SMC or HVC. Also SMCCC defines function number coding for such calls.
> >Besides functional calls there are query calls, which allows underling
> >OS determine version, UID and number of functions provided by service
> >provider.
> >
> >This patch adds new file `smccc.c`, which handles both generic SMCs
> >and HVC according to SMC. At this moment it implements only one
> >service: Standard Hypervisor Service.
> >
> >Standard Hypervisor Service only supports query calls, so caller can
> >ask about hypervisor UID and determine that it is XEN running.
> >
> >This change allows more generic handling for SMCs and HVCs and it can
> >be easily extended to support new services and functions.
> 
> I have already reviewed the code and one thing I missed is how a domain will
> know that Xen supports SMCCC.
> 
> Currently, Xen will:
>   - inject an undefined instruction for any SMC issued by a guest
>   - crash the guest (quite bad) for any unknown HCV #0
> 
> So a guest needs to be aware whether Xen supports SMCCC convention or not. I
> am not aware of any bindings in the device-tree for doing that.

On our platforms, SW probes the DT for specific service classes and then
probes for specific versions via SMC calls using the standard Version FIDs.
If the DT does not specify the firmware node, I don't think any SMCs will be
issued but the guest may not be functional (as the firmware interface is
mandatory).

I don't know of a generic DT node/compat that tells guests if SMCC probing
is allowed or not. Perhaps there should be one, or there should be yet
another service specific one for Hypervisors. I don't know.

For example, these are the nodes we've got (PSCI and EEMI/SIP):
psci {
compatible = "arm,psci-0.2";
method = "smc";
};

pmufw: firmware {
compatible = "xlnx,zynqmp-pm";
method = "smc";
interrupt-parent = <>;
interrupts = <0 35 4>;
};

SW that does not have DT support, will either directly probe the SMC
interface or in some cases just assume it's there and use it.

ZynqMP-wise, Xen is in a little bit of an akward position by messing
guests up if they probe for non-existing SMC services but I don't think
it's that bad. Mainly because there's really very little ZynqMP guests
that need the firmware would be able todo without it.

For other platforms and services, I guess FW may very well be optional
and probing makes more sence. So getting support for gracefully returning
Unknown FID still important...


> The other issue is not all the firmware may be SMCCC capable. We may want in
> the future to support other convention to allow baremetal OS running on Xen.
> This means a guest should be able to detect the convention used.

Perhaps this could be done by injecting DT fragments like we do for passthrough?

Cheers,
Edgar

> 
> I don't have a clear answer here yet, but thought it would be good to start
> a conversation.
> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Tue, Aug 01, 2017 at 11:37:05AM +0100, Julien Grall wrote:
> Hi Edgar,
> 
> On 31/07/17 23:23, Edgar E. Iglesias wrote:
> >On Thu, Feb 09, 2017 at 12:32:09PM -0700, Tamas K Lengyel wrote:
> >>On Thu, Feb 9, 2017 at 11:43 AM, Stefano Stabellini
> >><sstabell...@kernel.org> wrote:
> >>>On Thu, 9 Feb 2017, Tamas K Lengyel wrote:
> >>>>On Thu, Feb 9, 2017 at 11:22 AM, Stefano Stabellini
> >>>><sstabell...@kernel.org> wrote:
> >>>>>On Thu, 9 Feb 2017, Edgar E. Iglesias wrote:
> >>>>>>On Thu, Feb 09, 2017 at 10:12:41AM +0100, Edgar E. Iglesias wrote:
> >>>>>>>On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:
> >>>>>>>>On Thu, 9 Feb 2017, Julien Grall wrote:
> >>>>>>>>>On 08/02/2017 23:28, Tamas K Lengyel wrote:
> >>>>>>>>>>On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall <julien.gr...@arm.com> 
> >>>>>>>>>>wrote:
> >>>>>>>>>>>Hi Tamas,
> >
> >..
> >
> >>>>In principle I have nothing against a command line option, but I don't
> >>>>really follow how that would help. The monitor system is disabled by
> >>>>default for all domains, so there is no problem with dom0 booting or
> >>>>any other domain needing to access the firmware. You specifically have
> >>>>to enable the monitoring for domains. Why is it a problem to have it
> >>>>be exclusive for just those domains where it is enabled?
> >>>
> >>>I am suggesting this solution because I expect many use-cases for memory
> >>>introspection that don't actually require any platform_hvc events to be
> >>>monitored at all. On the other end, I expect that on platforms where
> >>>platform_hvc is implemented, such as the ZynqMP, those calls are
> >>>important and should be handled in Xen in most cases.
> >>>
> >>>Looking at the code, does monitor.privileged_call_enabled only cover
> >>>SMC? Is monitor.privileged_call_enabled disabled by default?
> >>>If so, monitor.privileged_call_enabled could be the tunable I was
> >>>talking about. As long as enabling memory introspection doesn't
> >>>automatically forward platform_hvc events to the monitor, I am fine with
> >>>it.
> >>
> >>Yes, monitor.privileged_call_enabled only covers SMCs right now and it
> >>is disabled by default. It has to be enabled specifically for a
> >>domain.  Memory introspection is separate from this, that is handled
> >>by the mem_access system and it can be enabled separately from SMC
> >>monitoring.
> >>
> >>As for hypercalls that get handled by Xen, I don't really need to
> >>monitor those. If Xen would on the other hand go and call some
> >>firmware as a result of the hypercall, I would need to be able to deny
> >>that. So as long as XSM can be used to control HVC calls, that works
> >>for me just fine too.
> >
> >Hi again!
> >
> >This was quite a while ago but I think we kind of ended up with
> >monitor.privileged_call_enabled being a possible flag to conditionalize
> >the forwarding of firmware calls or not.
> >
> >There are at least 3 cases to consider at the moment:
> >1. Firmware calls over SMC (PSCI or other platform calls like EEMI)
> >2. Firmware calls over HVC Handled by Xen (PSCI and XEN Hypercalls)
> >3. Firmware calls over HVC Handled by platform specific code (e.g EEMI)
> >
> >
> >For #1 Firmware calls over SMC:
> >I've conditionalized all of it on monitor.privileged_call_enabled.
> >It's either the monitor or the firmware call handling, they
> >are mutually exclusive. Guests can still do PSCI over HVC.
> >
> >For #2, things work like today. This is PSCI and the Xen Hypercallsi over 
> >HVC.
> >
> >For #3, only platform code knows if the specific call will be handled
> >in Xen completely or if it will result in some kind of SMC to lower layers.
> >If monitor.privileged_call_enabled is on, I've made the ZynqMP
> >implementation gracefully NACK any call that would result in an SMC
> >issued by Xen.
> >
> >Are there any concerns around this?
> 
> You may want to have a look at the discussion about SMC and Xen:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2017-02/msg01226.html
> 
> IIRC, the consensus is to exclusively forward SMC to the monitor if enabled:
> 
> "10. Domains on which the mon

Re: [Xen-devel] [PATCH v1 0/1] xen/arm: zynqmp: Disable PCIe

[Edgar E. Iglesias]

On Mon, Jul 31, 2017 at 11:11:39PM +0100, Julien Grall wrote:
> 
> 
> On 31/07/2017 20:37, Edgar E. Iglesias wrote:
> >From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> >
> >Hi,
> 
> Hi Edgar,
> 
> 
> >We're seeing panics in dom0 with PCIe enabled due to what seems
> >to be wrongly created mappings by Xen. With older kernels we
> >didn't see the panics but PCIe wasn't functional in dom0.
> >
> >This disables the PCIe nodes on the ZynqMP until Xen/ARM gets
> >more PCIe support.
> 
> I feel a bit sad to ack a patch disabling PCIe in the ZynqMP.
> 
> Before doing that. Can you describe what is the exact problem with Xen? It
> might be possible that we don't parse correctly the device-tree.

Yes, it was related to the DTB since we had a workaround with a modified
DTB.

Anyway, I was to quick with sending out the patch.

We've been carrying this for a while and your questions prompted me
to have a look again and dig out the details. Turns out I can't reproduce it
with our latest kernel and the latest Xen. I don't know what fixed
it yet but PCIe works ATM both on QEMU models and on real HW.

I don't think it ever has before without modified DTBs.

So please ignore this patch for now :-)

Cheers,
Edgar


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Thu, Feb 09, 2017 at 12:32:09PM -0700, Tamas K Lengyel wrote:
> On Thu, Feb 9, 2017 at 11:43 AM, Stefano Stabellini
> <sstabell...@kernel.org> wrote:
> > On Thu, 9 Feb 2017, Tamas K Lengyel wrote:
> >> On Thu, Feb 9, 2017 at 11:22 AM, Stefano Stabellini
> >> <sstabell...@kernel.org> wrote:
> >> > On Thu, 9 Feb 2017, Edgar E. Iglesias wrote:
> >> >> On Thu, Feb 09, 2017 at 10:12:41AM +0100, Edgar E. Iglesias wrote:
> >> >> > On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:
> >> >> > > On Thu, 9 Feb 2017, Julien Grall wrote:
> >> >> > > > On 08/02/2017 23:28, Tamas K Lengyel wrote:
> >> >> > > > > On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall 
> >> >> > > > > <julien.gr...@arm.com> wrote:
> >> >> > > > > > Hi Tamas,

..

> >> In principle I have nothing against a command line option, but I don't
> >> really follow how that would help. The monitor system is disabled by
> >> default for all domains, so there is no problem with dom0 booting or
> >> any other domain needing to access the firmware. You specifically have
> >> to enable the monitoring for domains. Why is it a problem to have it
> >> be exclusive for just those domains where it is enabled?
> >
> > I am suggesting this solution because I expect many use-cases for memory
> > introspection that don't actually require any platform_hvc events to be
> > monitored at all. On the other end, I expect that on platforms where
> > platform_hvc is implemented, such as the ZynqMP, those calls are
> > important and should be handled in Xen in most cases.
> >
> > Looking at the code, does monitor.privileged_call_enabled only cover
> > SMC? Is monitor.privileged_call_enabled disabled by default?
> > If so, monitor.privileged_call_enabled could be the tunable I was
> > talking about. As long as enabling memory introspection doesn't
> > automatically forward platform_hvc events to the monitor, I am fine with
> > it.
> 
> Yes, monitor.privileged_call_enabled only covers SMCs right now and it
> is disabled by default. It has to be enabled specifically for a
> domain.  Memory introspection is separate from this, that is handled
> by the mem_access system and it can be enabled separately from SMC
> monitoring.
> 
> As for hypercalls that get handled by Xen, I don't really need to
> monitor those. If Xen would on the other hand go and call some
> firmware as a result of the hypercall, I would need to be able to deny
> that. So as long as XSM can be used to control HVC calls, that works
> for me just fine too.

Hi again!

This was quite a while ago but I think we kind of ended up with
monitor.privileged_call_enabled being a possible flag to conditionalize
the forwarding of firmware calls or not.

There are at least 3 cases to consider at the moment:
1. Firmware calls over SMC (PSCI or other platform calls like EEMI)
2. Firmware calls over HVC Handled by Xen (PSCI and XEN Hypercalls)
3. Firmware calls over HVC Handled by platform specific code (e.g EEMI)


For #1 Firmware calls over SMC:
I've conditionalized all of it on monitor.privileged_call_enabled.
It's either the monitor or the firmware call handling, they
are mutually exclusive. Guests can still do PSCI over HVC.

For #2, things work like today. This is PSCI and the Xen Hypercallsi over HVC.

For #3, only platform code knows if the specific call will be handled
in Xen completely or if it will result in some kind of SMC to lower layers.
If monitor.privileged_call_enabled is on, I've made the ZynqMP
implementation gracefully NACK any call that would result in an SMC
issued by Xen.

Are there any concerns around this?

I'll also send out code for review, it may be easier to follow :-)

Best regards,
Edgar




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v4]Proposal to allow setting up shared memory areas between VMs from xl config file

[Edgar E. Iglesias]

On Mon, Jul 31, 2017 at 02:30:47PM -0700, Stefano Stabellini wrote:
> On Mon, 31 Jul 2017, Edgar E. Iglesias wrote:
> > On Fri, Jul 28, 2017 at 09:03:15PM +0800, Zhongze Liu wrote:
> > > 
> > > 1. Motivation and Description
> > 
> > Hi,
> > 
> > I think this looks quite useful. I have a few comments inline.
> 
> Hi Edgar, thanks for giving it a look!
> 
> 
> > > 
> > > Virtual machines use grant table hypercalls to setup a share page for
> > > inter-VMs communications. These hypercalls are used by all PV
> > > protocols today. However, very simple guests, such as baremetal
> > > applications, might not have the infrastructure to handle the grant table.
> > > This project is about setting up several shared memory areas for inter-VMs
> > > communications directly from the VM config file.
> > > So that the guest kernel doesn't have to have grant table support (in the
> > > embedded space, this is not unusual) to be able to communicate with
> > > other guests.
> > > 
> > > 
> > > 2. Implementation Plan:
> > > 
> > > 
> > > ==
> > > 2.1 Introduce a new VM config option in xl:
> > > ==
> > > 
> > > 2.1.1 Design Goals
> > > ~~~
> > > 
> > > The shared areas should be shareable among several (>=2) VMs, so every 
> > > shared
> > > physical memory area is assigned to a set of VMs. Therefore, a “token” or
> > > “identifier” should be used here to uniquely identify a backing memory 
> > > area.
> > > A string no longer than 128 bytes is used here to serve the purpose.
> > > 
> > > The backing area would be taken from one domain, which we will regard
> > > as the "master domain", and this domain should be created prior to any
> > > other "slave domain"s. Again, we have to use some kind of tag to tell who
> > > is the "master domain".
> > > 
> > > And the ability to specify the permissions and cacheability (and 
> > > shareability
> > > for ARM guest's) of the pages to be shared should also be given to the 
> > > user.
> > > 
> > > 2.2.2 Syntax and Behavior
> > > ~
> > > The following example illustrates the syntax of the proposed config entry
> > > (suppose that we're on x86):
> > > 
> > > In xl config file of vm1:
> > >   static_shm = [ 'id=ID1, begin=0x10, end=0x20, role=master,
> > >   cache_policy=x86_normal, prot=r2',
> > > 
> > >  'id=ID2, begin=0x30, end=0x40, role=master' ]
> > > 
> > > In xl config file of vm2:
> > >   static_shm = [ 'id=ID1, offset = 0, begin=0x50, end=0x60,
> > > role=slave, prot=ro' ]
> > > 
> > > In xl config file of vm3:
> > >   static_shm = [ 'id=ID2, offset = 1, begin=0x69,
> > > end=0x80, role=slave, prot=ro' ]
> > > 
> > > where:
> > >   @id   The identifier of the backing memory area.
> > > Can be any string that matches the regexp "[^ 
> > > \t\n,]+"
> > > and no longer than 128 characters
> > > 
> > >   @offset   Can only appear when @role = slave. The sharing 
> > > will
> > > start from the beginning of backing memory area 
> > > plus
> > > this offset. If not set, it defaults to zero.
> > > Can be decimals or hexadecimals of the form 
> > > "0x2",
> > > and should be the multiple of the hypervisor page
> > > granularity (currently 4K on both ARM and x86).
> > > 
> > >   @begin/endThe boundaries of the shared memory area. The 
> > > format
> > > requirements are the same with @offset.
> > 
> > I'm assuming this is all specified in GFN and also not MFN contigous?
> > Would it be possible to allow the specification of MFN mappings
> > that are contigous?
> 
> That could be done with the iomem= parameter?

The missing part from the iomem paramater is the attributes, cacheability, 

Re: [Xen-devel] [RFC v2 2/6] xen/arm: Introduce platform_hvc

[Edgar E. Iglesias]

On Mon, Feb 13, 2017 at 02:08:43PM -0800, Stefano Stabellini wrote:
> On Tue, 7 Feb 2017, Edgar E. Iglesias wrote:
> > From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> > 
> > Introduce platform_hvc as a way to handle hypercalls that
> > Xen does not know about in a platform specific way. This
> > is particularly useful for implementing the SiP (SoC
> > implementation specific) service calls.
> > 
> > Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> > ---
> >  xen/arch/arm/platform.c| 8 
> >  xen/arch/arm/traps.c   | 3 +++
> >  xen/include/asm-arm/platform.h | 5 +
> >  3 files changed, 16 insertions(+)
> > 
> > diff --git a/xen/arch/arm/platform.c b/xen/arch/arm/platform.c
> > index 0af6d57..90ea6b8 100644
> > --- a/xen/arch/arm/platform.c
> > +++ b/xen/arch/arm/platform.c
> > @@ -127,6 +127,14 @@ void platform_poweroff(void)
> >  platform->poweroff();
> >  }
> >  
> > +bool platform_hvc(struct cpu_user_regs *regs)
> 
> This is fine, but we need a different name for it, as it can be used to
> handle both HVC and SMC calls. Maybe "firmware_call"?

Hi,

Sorry for the super long delay.. I'm looking at this again now.

Yes, you're right. I went with
platform_firmware_call() and platform->firmware_call().

I kept the platform_ prefix in the wrapper function to be
consistent with the other hooks.

Thanks,
Edgar


> 
> 
> > +{
> > +if ( platform && platform->hvc )
> > +return platform->hvc(regs);
> > +
> > +return false;
> > +}
> > +
> >  bool_t platform_has_quirk(uint32_t quirk)
> >  {
> >  uint32_t quirks = 0;
> > diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> > index c5a4d41..33950d9 100644
> > --- a/xen/arch/arm/traps.c
> > +++ b/xen/arch/arm/traps.c
> > @@ -44,6 +44,7 @@
> >  #include 
> >  #include 
> >  #include 
> > +#include 
> >  
> >  #include "decode.h"
> >  #include "vtimer.h"
> > @@ -1430,6 +1431,8 @@ static void do_trap_psci(struct cpu_user_regs *regs)
> >  }
> >  break;
> >  default:
> > +if ( platform_hvc(regs) )
> > +return;
> >  domain_crash_synchronous();
> >  return;
> >  }
> > diff --git a/xen/include/asm-arm/platform.h b/xen/include/asm-arm/platform.h
> > index 08010ba..4d51f0a 100644
> > --- a/xen/include/asm-arm/platform.h
> > +++ b/xen/include/asm-arm/platform.h
> > @@ -26,6 +26,10 @@ struct platform_desc {
> >  void (*reset)(void);
> >  /* Platform power-off */
> >  void (*poweroff)(void);
> > +/* Platform specific HVC handler.
> > + * Returns true if the call was handled and false if not.
> > + */
> > +bool (*hvc)(struct cpu_user_regs *regs);
> >  /*
> >   * Platform quirks
> >   * Defined has a function because a platform can support multiple
> > @@ -55,6 +59,7 @@ int platform_cpu_up(int cpu);
> >  #endif
> >  void platform_reset(void);
> >  void platform_poweroff(void);
> > +bool platform_hvc(struct cpu_user_regs *regs);
> >  bool_t platform_has_quirk(uint32_t quirk);
> >  bool_t platform_device_is_blacklisted(const struct dt_device_node *node);

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v4]Proposal to allow setting up shared memory areas between VMs from xl config file

[Edgar E. Iglesias]

On Fri, Jul 28, 2017 at 09:03:15PM +0800, Zhongze Liu wrote:
> 
> 1. Motivation and Description

Hi,

I think this looks quite useful. I have a few comments inline.


> 
> Virtual machines use grant table hypercalls to setup a share page for
> inter-VMs communications. These hypercalls are used by all PV
> protocols today. However, very simple guests, such as baremetal
> applications, might not have the infrastructure to handle the grant table.
> This project is about setting up several shared memory areas for inter-VMs
> communications directly from the VM config file.
> So that the guest kernel doesn't have to have grant table support (in the
> embedded space, this is not unusual) to be able to communicate with
> other guests.
> 
> 
> 2. Implementation Plan:
> 
> 
> ==
> 2.1 Introduce a new VM config option in xl:
> ==
> 
> 2.1.1 Design Goals
> ~~~
> 
> The shared areas should be shareable among several (>=2) VMs, so every shared
> physical memory area is assigned to a set of VMs. Therefore, a “token” or
> “identifier” should be used here to uniquely identify a backing memory area.
> A string no longer than 128 bytes is used here to serve the purpose.
> 
> The backing area would be taken from one domain, which we will regard
> as the "master domain", and this domain should be created prior to any
> other "slave domain"s. Again, we have to use some kind of tag to tell who
> is the "master domain".
> 
> And the ability to specify the permissions and cacheability (and shareability
> for ARM guest's) of the pages to be shared should also be given to the user.
> 
> 2.2.2 Syntax and Behavior
> ~
> The following example illustrates the syntax of the proposed config entry
> (suppose that we're on x86):
> 
> In xl config file of vm1:
>   static_shm = [ 'id=ID1, begin=0x10, end=0x20, role=master,
>   cache_policy=x86_normal, prot=r2',
> 
>  'id=ID2, begin=0x30, end=0x40, role=master' ]
> 
> In xl config file of vm2:
>   static_shm = [ 'id=ID1, offset = 0, begin=0x50, end=0x60,
> role=slave, prot=ro' ]
> 
> In xl config file of vm3:
>   static_shm = [ 'id=ID2, offset = 1, begin=0x69,
> end=0x80, role=slave, prot=ro' ]
> 
> where:
>   @id   The identifier of the backing memory area.
> Can be any string that matches the regexp "[^ \t\n,]+"
> and no longer than 128 characters
> 
>   @offset   Can only appear when @role = slave. The sharing will
> start from the beginning of backing memory area plus
> this offset. If not set, it defaults to zero.
> Can be decimals or hexadecimals of the form "0x2",
> and should be the multiple of the hypervisor page
> granularity (currently 4K on both ARM and x86).
> 
>   @begin/endThe boundaries of the shared memory area. The format
> requirements are the same with @offset.

I'm assuming this is all specified in GFN and also not MFN contigous?
Would it be possible to allow the specification of MFN mappings
that are contigous?

This would be useful to map specific kinds of memory (e.g On Chip RAMs).

Other use-cases are when there are not only guests sharing
the pages but also devices. In some cases these devs may be locked in
with low-latency access to specific memory regions.

Perhaps something like the following?
addr=gfn@
size=0x1000

with mfn being optional?


>   @role Can only be 'master' or 'slave', it defaults to 
> 'slave'.
> 
>   @prot When @role = master, this means the largest set of
> stage-2 permission flags that can be granted to the
> slave domains.
> When @role = slave, this means the stage-2 permission
> flags of the shared memory area.
> Currently only 'rw' is supported. If not set. it
> defaults to 'rw'.
> 
>   @cache_policy The stage-2 cacheability/shareability attributes of 
> the
> shared memory area. Currently, only two policies are
> supported:
>   * ARM_normal: Only applicable to ARM guests. This
> would mean Inner and Outer Write-Back
> Cacheable, and Inner Shareable.


Is there a reason not to set this to Outer Shareable?
Again, mainly useful when these pages get shared with devs as well.

The guest can always lower it to Inner Shareable via S1 tables if needed.

Best 

Re: [Xen-devel] [XenSummit 2017] Shared coprocessor framework followup

[Edgar E. Iglesias]

On Tue, Jul 18, 2017 at 08:10:15PM +0300, Andrii Anisov wrote:
> **Dear All,
> 
> During the developers summit a Shared Coprocessor Framework (SCF) concept
> was presented. Noticeable interest from community was discovered during
> discussions. So this is a call for all interested parties to collect a
> feedback and setup a collaboration.

Hi Andrii!


> 
> There are several topics I would like to collect responses from the
> community:
> - Who are interested in SCF design, discussions, development, usage,
> etc? Personalities or organizations.

Yes I'm interested in this. I'm not sure how much time I'll be able to
contribute but at least I can review proposals and hopefully look at
implementing a driver/backend that may be useful for our FPGA platforms.

Cheers,
Edgar


> - What devices (type of devices) are intended to be shared using SCF?
> - What are expected coprocessor sharing use-cases (i.e. DSP running
> different FW for different domains, etc).
> - If someone is willing to take a part in SCF design and development
> (core, API)?
> - If someone is willing to implement their coprocessor support (driver)
> for SCF?
> 
> I look forward to hearing from you.
> 
> -- 
> 
> *Andrii Anisov*
> 
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 0/1] xen/arm: zynqmp: Disable PCIe

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Hi,

We're seeing panics in dom0 with PCIe enabled due to what seems
to be wrongly created mappings by Xen. With older kernels we
didn't see the panics but PCIe wasn't functional in dom0.

This disables the PCIe nodes on the ZynqMP until Xen/ARM gets
more PCIe support.

Can we get this into stable-4.8 and 4.9?

Cheers,
Edgar

Edgar E. Iglesias (1):
  xen/arm: Disable PCIe on the ZynqMP

 xen/arch/arm/platforms/xilinx-zynqmp.c | 1 +
 1 file changed, 1 insertion(+)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 1/1] xen/arm: Disable PCIe on the ZynqMP

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Disable PCIe on the ZynqMP. Xen does not yet know how to map the
controller and dom0 fails to boot with the node enabled.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platforms/xilinx-zynqmp.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/arm/platforms/xilinx-zynqmp.c 
b/xen/arch/arm/platforms/xilinx-zynqmp.c
index 2adee91..822287b 100644
--- a/xen/arch/arm/platforms/xilinx-zynqmp.c
+++ b/xen/arch/arm/platforms/xilinx-zynqmp.c
@@ -29,6 +29,7 @@ static const struct dt_device_match zynqmp_blacklist_dev[] 
__initconst =
 {
 /* Power management is not yet supported.  */
 DT_MATCH_COMPATIBLE("xlnx,zynqmp-pm"),
+DT_MATCH_COMPATIBLE("xlnx,nwl-pcie-2.11"),
 { /* sentinel */ },
 };
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Next Xen ARM community call - Wednesday 2nd August 2017

[Edgar E. Iglesias]

Hi, this time works for me too. I'd like to have a slot to introduce and
discuss the work that Xilinx and Aggios are doing on Power management for
Xen/Arm.

Cheers,
Edgar

Sent from my phone

On Jul 26, 2017 6:59 PM, "Stefano Stabellini" 
wrote:

> On Wed, 26 Jul 2017, Julien Grall wrote:
> > Hi all,
> >
> > The next Xen ARM community call will be Wednesday 2nd August 2017 5pm
> BST.
> >
> > Do you have any specific topic you would like to discuss?
>
> These date and time work for me. I don't have any specific topics to
> discuss.
>
>
> > Call+44 1223 406065 (Local dial in)
> > and enter the access code below followed by # key.
> > Participant code: 4915191
> >
> > Mobile Auto Dial:
> > VoIP: voip://+441223406065;4915191#
> > iOS devices: +44 1223 406065,4915191 and press #
> > Other devices: +44 1223 406065x4915191#
> >
> > Additional Calling Information:
> >
> > UK +44 1142828002
> > US CA +1 4085761502
> > US TX +1 5123141073
> > JP +81 453455355
> > DE +49 8945604050
> > NO +47 73187518
> > SE +46 46313131
> > FR +33 497235101
> > TW +886 35657119
> > HU +36 13275600
> > IE +353 91337900
> >
> > Toll Free
> >
> > UK 0800 1412084
> > US +1 8668801148
> > CN +86 4006782367
> > IN 0008009868365
> > IN +918049282778
> > TW 08000 22065
> > HU 0680981587
> > IE 1800800022
> > KF +972732558877
>
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Proposal to allow setting up shared memory areas between VMs from xl config file

[Edgar E. Iglesias]

On Mon, May 22, 2017 at 02:14:41PM -0700, Stefano Stabellini wrote:
> On Mon, 22 May 2017, Ian Jackson wrote:
> > Stefano Stabellini writes ("Re: Proposal to allow setting up shared memory 
> > areas between VMs from xl config file"):
> > > In this scenario, she is going to write to the VM config files of the
> > > two apps that one page will be shared among the two, so that they can
> > > send each other messages. She will hard-code the address of the shared
> > > page in her "bare-metal" app.
> > 
> > Thanks.  This makes some sense.
> > 
> > How do these apps expect to interrupt each other, or do they poll in
> > the shared memory ?  What I'm getting at with this question is that
> > perhaps some event channels will need setting up.
> 
> As a matter of fact, I have been asking myself the same question. Nobody
> asked me explicitly for notifications support, so I didn't include it in
> the original project definition (it can always be added later) but I
> think it would be useful.
> 
> Edgar, Jarvis, do you have an opinion on this? Do (software) interrupts
> need to be setup together with the shared memory region to send
> notifications back and forth between the two guests, or are they
> unnecessary because the apps do polling anyway?

Hi Stefano,

Sorry, I haven't been following this thread in detail.

The requests I've heard of so far involve:

1. Static setup of memory/pages at a given guest physical address.
   Preferably by allowing the setup to control the real physical
   address aswell (e.g, to select on chip memories as the backing).
   Bonus for an option to let Xen dynamically allocate the physical
   memory.

   Preferably avoiding the need for hypercalls and such as the guest
   my be an unmodified program that runs natively (without Xen).

2. Interrupts would be done by means of IPIs like if running natively
   on HW. Either by some dedicated IP device or by using GIC PPIs/SGIs
   to raise interrupts on other cores. PPI's are a bit akward as
   it conflicts with the Xen model of multi-core intra-guest IPIs,
   as oppposed to inter-guest IPIs. SGI's accross guests could work.


> 
> Event channels are not as complex as grants, but they are not trivial
> either. Guests need to support the full event channel ABI even just to
> receive notifications from one event channel only (because they need to
> clear the pending bits), which is not simple and increases latency to
> de-multiplex events. See drivers/xen/events/events_2l.c and
> drivers/xen/events/events_base.c in Linux. I think we would have to
> introduce a simpler model, where each "notification channel" is not
> implemented by an event channel, but by a PPI or SGI instead. We expect
> only one or two to be used. PPIs and SGIs are interrupt classes on ARM,
> it is possible to allocate one or more for notification usage.

Yes, I wrote too fast, you're getting to the same point here...



> 
> I think it is probably best to leave notifications to the future.

Perhaps yes.

In the ZynqMP case, as a first step, we can use the dedicated IPI blocks.
It would simply involve mapping irq's and memory regions to the various guests
and they would be able to raise interrupts to each other by memory
writes to the IPI devices. Xen doesn't need to be involved more.
This should already work today.

Cheers,
Edgar


> 
> 
> > > There is no frontend and backend (as in the usual Xen meaning). In some
> > > configurations one app might be more critical than the other, but in
> > > some other scenarios they might have the same criticality.
> > 
> > Yes.
> > 
> > > If, as Jan pointed out, we need to call out explicitly which is the
> > > frontend and which is the backend for page ownership reasons, then I
> > > suggested we expose that configuration to the user, so that she can
> > > choose.
> > 
> > Indeed.
> > 
> > ISTM that this scenario doesn't depend on new hypervisor
> > functionality.  The toolstack could set up the appropriate page
> > sharing (presumably, this would be done with grants so that the result
> > is like something the guests could have done themselves.)
> 
> Right, I don't think we need new hypervisor functionalities. I don't
> have an opinion on whether it should be done with grants or with other
> hypercalls, although I have the feeling that it might be more difficult
> to achieve with grants. As long as it works... :-)
> 
> 
> > I see no objection to the libxl domain configuration file naming
> > guest-physical addresses for use in this way.
> >
> > One problem, though, is to do with startup order.  To do this in the
> > most natural way, one would want to start both guests at once so that
> > one would know their domids etc.  (Also that avoids questions like
> > `what if one of them crashes'...)
> > 
> > I'm not sure exactly how to fit this into the libxl model, which
> > mostly talks about one guest domain at a time; and each guest config
> > talks about persistent resources, rather than resources which are
> > somehow exposed by a 

Re: [Xen-devel] Xen ARM community call - meeting minutes and date for the next one

[Edgar E. Iglesias]

On Tue, Mar 28, 2017 at 04:23:03PM +0100, Julien Grall wrote:
> Hi all,
> 
> Apologies for the late sending, you will find at the end of the e-mail a
> summary of the discussion from the previous call. Feel free to reply if I
> missed some parts.
> 
> I suggest to have the next call on the 5th April at 5PM UTC. Any opinions?


That works for me, thanks!

Edgar


> 
> Also do you have any specific topic you would like to talk during this call?
> 
> Cheers,
> 
> == Attendees ==
> 
> Apologies if I misspelled any name.
> 
> Stefano, Aporeto
> Julien, ARM
> Oleksandr, EPAM
> Artem, EPAM
> Thanasis, OnApp
> Volodymir, EPAM
> 
> == Xen on ARM status ==
> 
> Over 100 patches in-flight for Xen on ARM:
> - PV protocols: Some are already accepted
> - NUMA support
> - GICv3 ITS support
> - Exposing and emulating a PL011 for guest
> - guest SMC forwarding for Xilinx platform
> - Interrupt latency improvement
> 
> == PV protocols ==
> 
> * PV protocols written by Stefano was merged after 10 months
> 
> Stefano: PV protocols review are moving faster
> Attendees agreed
> 
> * Audio protocol: close to be accepted
> * Display protocol: minor issue, a bit more design is required
> 
> Hopefully both will be ready for Xen 4.9
> 
> Oleksandr: What to do when the backend die?
> 
> (I cannot find any notes on it some I am not sure if we answered the question
> during the call. I suspect it has been asked to bring up the subject on the 
> ML).
> 
> == Interrupt latency ==
> 
> Stefano: Some improvement has been done but it is not possible to know whether
> it is good. Do you have any specific IRQ latency requirements?
> 
> Artem: There is no hard latency requirements in automotive, although many
> requirements depends on latency. For example:
> * Scheduling
> * GPU (implementation is sentive to interrupt latency)
> 
> Automotive is using a set of benchmark to find the virtualization overhead. 
> This
> should be low.
> 
> ACTION: Artem to send a list of benchmark
> 
> == SMC/HVC handling in Xen ==
> 
> Artem: Please review the proposal on the mailing list. See:
> 
> https://lists.xenproject.org/archives/html/xen-devel/2017-03/msg00430.html
> 
> == Deprivilege mode ==
> 
> EPAM are working on adding support for OP-TEE in Xen to allow multiple guest
> access the trusted firmware.
> 
> During the discussion on the design, it was suggested to move the SMC handling
> in a separate domain. This was tested using the VM event API and Mini-OS
> (upstream with Chen Baozi's series to support ARM64). The first results
> shows it is 10 times slower than handling SMC calls directly in the 
> hypervisor.
> 
> Volodymir is working on another approach to deprivilege the execution by
> implementing a Xen EL0.
> 
> == big.LITTLE support ==
> 
> Thanasis: Document discussed on the ML. Xen will split CPUs at boot time
> (big vs little). A series will be sent on the on the ML soon.
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: fix affected memory range by dcache clean functions

[Edgar E. Iglesias]

On Thu, Mar 02, 2017 at 05:15:26PM -0800, Stefano Stabellini wrote:
> clean_dcache_va_range and clean_and_invalidate_dcache_va_range don't
> calculate the range correctly when "end" is not cacheline aligned. As a
> result, the last cacheline is not skipped. Fix the issue by aligning the
> start address to the cacheline size.
> 
> In addition, make the code simpler and faster in
> invalidate_dcache_va_range, by removing the module operation and using
> bitmasks instead.
> 
> Signed-off-by: Stefano Stabellini <sstabell...@kernel.org>
> Reported-by: edgar.igles...@xilinx.com
> CC: edgar.igles...@xilinx.com

This looks good to me and it works on my side.
In the future, perhaps we could convert invalidate_dcache_va_range
to use a smaller implementation like clean_dcache_va_range().

Anyway:
Reviewed-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
Tested-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>

And super thanks for helping me resolve this issue!

Cheers,
Edgar


> ---
>  xen/include/asm-arm/page.h | 24 +++-
>  1 file changed, 11 insertions(+), 13 deletions(-)
> 
> diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
> index 86de0b6..4b46e88 100644
> --- a/xen/include/asm-arm/page.h
> +++ b/xen/include/asm-arm/page.h
> @@ -291,24 +291,20 @@ extern size_t cacheline_bytes;
>  
>  static inline int invalidate_dcache_va_range(const void *p, unsigned long 
> size)
>  {
> -size_t off;
>  const void *end = p + size;
> +size_t cacheline_mask = cacheline_bytes - 1;
>  
>  dsb(sy);   /* So the CPU issues all writes to the range */
>  
> -off = (unsigned long)p % cacheline_bytes;
> -if ( off )
> +if ( (uintptr_t)p & cacheline_mask )
>  {
> -p -= off;
> +p = (void *)((uintptr_t)p & ~cacheline_mask);
>  asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p));
>  p += cacheline_bytes;
> -size -= cacheline_bytes - off;
>  }
> -off = (unsigned long)end % cacheline_bytes;
> -if ( off )
> +if ( (uintptr_t)end & cacheline_mask )
>  {
> -end -= off;
> -size -= off;
> +end = (void *)((uintptr_t)end & ~cacheline_mask);
>  asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (end));
>  }
>  
> @@ -322,9 +318,10 @@ static inline int invalidate_dcache_va_range(const void 
> *p, unsigned long size)
>  
>  static inline int clean_dcache_va_range(const void *p, unsigned long size)
>  {
> -const void *end;
> +const void *end = p + size;
>  dsb(sy);   /* So the CPU issues all writes to the range */
> -for ( end = p + size; p < end; p += cacheline_bytes )
> +p = (void *)((uintptr_t)p & ~(cacheline_bytes - 1));
> +for ( ; p < end; p += cacheline_bytes )
>  asm volatile (__clean_dcache_one(0) : : "r" (p));
>  dsb(sy);   /* So we know the flushes happen before continuing */
>  /* ARM callers assume that dcache_* functions cannot fail. */
> @@ -334,9 +331,10 @@ static inline int clean_dcache_va_range(const void *p, 
> unsigned long size)
>  static inline int clean_and_invalidate_dcache_va_range
>  (const void *p, unsigned long size)
>  {
> -const void *end;
> +const void *end = p + size;
>  dsb(sy); /* So the CPU issues all writes to the range */
> -for ( end = p + size; p < end; p += cacheline_bytes )
> +p = (void *)((uintptr_t)p & ~(cacheline_bytes - 1));
> +for ( ; p < end; p += cacheline_bytes )
>  asm volatile (__clean_and_invalidate_dcache_one(0) : : "r" (p));
>  dsb(sy); /* So we know the flushes happen before continuing */
>  /* ARM callers assume that dcache_* functions cannot fail. */
> -- 
> 1.9.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] xen/arm and swiotlb-xen: possible data corruption

[Edgar E. Iglesias]

On Thu, Mar 02, 2017 at 02:39:55PM -0800, Stefano Stabellini wrote:
> On Thu, 2 Mar 2017, Julien Grall wrote:
> > Hi Stefano,
> > 
> > On 02/03/17 19:12, Stefano Stabellini wrote:
> > > On Thu, 2 Mar 2017, Julien Grall wrote:
> > > > On 02/03/17 08:53, Edgar E. Iglesias wrote:
> > > > > On Thu, Mar 02, 2017 at 09:38:37AM +0100, Edgar E. Iglesias wrote:
> > > > > > On Wed, Mar 01, 2017 at 05:05:21PM -0800, Stefano Stabellini wrote:
> > > Julien, from looking at the two diffs, this is simpler and nicer, but if
> > > you look at xen/include/asm-arm/page.h, my patch made
> > > clean_dcache_va_range consistent with invalidate_dcache_va_range. For
> > > consistency, I would prefer to deal with the two functions the same way.
> > > Although it is not a spec requirement, I also think that it is a good
> > > idea to issue cache flushes from cacheline aligned addresses, like
> > > invalidate_dcache_va_range does and Linux does, to make more obvious
> > > what is going on.
> > 
> > invalid_dcache_va_range is split because the cache instruction differs for 
> > the
> > start and end if unaligned. For them you want to use clean & invalidate 
> > rather
> > than invalidate.
> > 
> > If you look at the implementation of other cache helpers in Linux (see
> > dcache_by_line_op in arch/arm64/include/asm/assembler.h), they will only 
> > align
> > start & end.
> 
> I don't think so, unless I am reading dcache_by_line_op wrong.
> 
> 
> > Also, the invalid_dcache_va_range is using modulo which I would rather 
> > avoid.
> > The modulo in this case will not be optimized by the compiler because
> > cacheline_bytes is not a constant.
> 
> That is a good point. What if I replace the modulo op with
> 
>   p & (cacheline_bytes - 1)
> 
> in invalidate_dcache_va_range, then add the similar code to
> clean_dcache_va_range and clean_and_invalidate_dcache_va_range?


Yeah, if there was some kind of generic ALIGN or ROUND_DOWN macro we could do:

--- a/xen/include/asm-arm/page.h
+++ b/xen/include/asm-arm/page.h
@@ -325,7 +325,9 @@ static inline int clean_dcache_va_range(const void *p, 
unsigned long size)
 {
 const void *end;
 dsb(sy);   /* So the CPU issues all writes to the range */
-for ( end = p + size; p < end; p += cacheline_bytes )
+
+p = (void *)ALIGN((uintptr_t)p, cacheline_bytes);
+end = (void *)ROUNDUP((uintptr_t)p + size, cacheline_bytes);
+for ( ; p < end; p += cacheline_bytes )
 asm volatile (__clean_dcache_one(0) : : "r" (p));
 dsb(sy);   /* So we know the flushes happen before continuing */
 /* ARM callers assume that dcache_* functions cannot fail. */

I think that would achieve the same result as your patch Stefano?

Cheers,
Edgar


> 
> 
> > BTW, you would also need to fix clean_and_invalidate_dcache_va_range.
> 
> I'll do that, thanks for the reminder.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Thu, Feb 23, 2017 at 04:47:19PM +, Julien Grall wrote:
> 
> Hi Edgar,
> 
> On 22/02/17 04:03, Edgar E. Iglesias wrote:
> >On Mon, Feb 13, 2017 at 03:35:19PM +, Julien Grall wrote:
> >>On 02/02/17 15:33, Edgar E. Iglesias wrote:
> >>>On Wed, Feb 01, 2017 at 07:04:43PM +, Julien Grall wrote:
> >>>>On 31/01/2017 19:06, Edgar E. Iglesias wrote:
> >>>>>On Tue, Jan 31, 2017 at 05:09:53PM +, Julien Grall wrote:
> >>>I'll see if I can find working examples for PCIe on the ZCU102. Then I'll 
> >>>share
> >>>DTS, Kernel etc.
> >>
> >>I've found a device tree on the github from the ZCU102: zynqmp-zcu102.dts,
> >>it looks like there is no use of PHY for the pcie so far.
> >>
> >>Lets imagine in the future, pcie will use the PHY. If we decide to
> >>initialize the hostbridge in Xen, we would also have to pull the PHY code in
> >>the hypervisor. Leaving aside the problem to pull more code in Xen, this is
> >>not nice because the PHY is used by different components (e.g SATA, USB). So
> >>Xen and DOM0 would have to share the PHY.
> >>
> >>For Xen POV, the best solution would be the bootloader initializing the PHY
> >>because starting Xen. So we can keep all the hostbridge (initialization +
> >>access) in Xen.
> >>
> >>If it is not possible, then I would prefer to see the hostbridge
> >>initialization in DOM0.
> >
> >>>
> >>>I suspect that this setup has previously been done by the initial 
> >>>bootloader
> >>>auto-generated from design configuration tools.
> >>>
> >>>Now, this is moving into Linux.
> >>
> >>Do you know why they decide to move the code in Linux? What would be the
> >>problem to let the bootloader configuring the GT?
> >
> >
> >No, I'm not sure why this approach was not used. The only thing I can think 
> >of
> >is a runtime configuration approach.
> >
> >
> >>
> >>>There's a specific driver that does that but AFAICS, it has not been 
> >>>upstreamed yet.
> >>>You can see it here:
> >>>https://github.com/Xilinx/linux-xlnx/blob/master/drivers/phy/phy-zynqmp.c
> >>>
> >>>DTS nodes that need a PHY can then just refer to it, here's an example 
> >>>from SATA:
> >>> {
> >>>   phy-names = "sata-phy";
> >>>   phys = < PHY_TYPE_SATA 1 3 15000>;
> >>>};
> >>>
> >Yes, I agree that the GT setup in the bootloader is very attractive.
> >I don't think hte setup sequence is complicated, we can perhaps even do it
> >on the commandline in u-boot or xsdb. I'll have to check.
> 
> That might simplify things for Xen. I would be happy to consider any other
> solutions. It might probably be worth to kick a separate thread regarding
> how to support Xilinx hostcontroller in Xen.
> 
> For now, I will explain in the design document the different situation we
> can encounter with an hostbridge and will leave open the design for
> initialization bits.
> 
> 
> [...]
> 
> >>>>
> >>>>From a design point of view, it would make more sense to have the MSI
> >>>>controller driver in Xen as the hostbridge emulation for guest will also
> >>>>live there.
> >>>>
> >>>>So if we receive MSI in Xen, we need to figure out a way for DOM0 and 
> >>>>guest
> >>>>to receive MSI. The same way would be the best, and I guess non-PV if
> >>>>possible. I know you are looking to boot unmodified OS in a VM. This would
> >>>>mean we need to emulate the MSI controller and potentially xilinx PCI
> >>>>controller. How much are you willing to modify the OS?
> >>>
> >>>Today, we have not yet implemented PCIe drivers for our baremetal SDK. So
> >>>things are very open and we could design with pretty much anything in mind.
> >>>
> >>>Yes, we could perhaps include a very small model with most registers 
> >>>dummied.
> >>>Implementing the MSI read FIFO would allow us to:
> >>>
> >>>1. Inject the MSI doorbell SPI into guests. The guest will then see the 
> >>>same
> >>>  IRQ as on real HW.
> >>>
> >>>2. Guest reads host-controller registers (MSI FIFO) to get the signaled 
> >>>MSI.
> >>
> >>The Xilinx PCIe hostbridge is not the only hostbridge having MSI controller
> >>

Re: [Xen-devel] xen/arm and swiotlb-xen: possible data corruption

[Edgar E. Iglesias]

On Thu, Mar 02, 2017 at 09:38:37AM +0100, Edgar E. Iglesias wrote:
> On Wed, Mar 01, 2017 at 05:05:21PM -0800, Stefano Stabellini wrote:
> > Hi all,
> > 
> > Edgar reported a data corruption on network packets in dom0 when the
> > swiotlb-xen is in use. He also reported that the following patch "fixes"
> > the problem for him:
> > 
> >  static void __xen_dma_page_cpu_to_dev(struct device *hwdev, dma_addr_t 
> > handle,
> > size_t size, enum dma_data_direction dir)
> >  {
> > -   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, 
> > DMA_MAP);
> > +   printk("%s: addr=%lx size=%zd\n", __func__, handle, size);
> > +   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size + 64, 
> > dir, DMA_MAP);
> > 
> > I am thinking that the problem has something to do with cacheline
> > alignment on the Xen side
> > (xen/common/grant_table.c:__gnttab_cache_flush).
> > 
> > If op == GNTTAB_CACHE_INVAL, we call invalidate_dcache_va_range; if op
> > == GNTTAB_CACHE_CLEAN, we call clean_dcache_va_range instead. The
> > parameter, v, could be non-cacheline aligned.
> > 
> > invalidate_dcache_va_range is capable of handling a not aligned address,
> > while clean_dcache_va_range does not.
> > 
> > Edgar, does the appended patch fix the problem for you?
> 
> 
> Thanks Stefano,
> 
> This does indeed fix the issue for me.


Hi again,

Looking at the code, the problem here is that we may flush one cache line
less than expected.

This smaller patch fixes it for me too:
diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
index c492d6d..fa1b4dd 100644
--- a/xen/include/asm-arm/page.h
+++ b/xen/include/asm-arm/page.h
@@ -325,7 +325,9 @@ static inline int clean_dcache_va_range(const void *p, 
unsigned long size)
 {
 const void *end;
 dsb(sy);   /* So the CPU issues all writes to the range */
-for ( end = p + size; p < end; p += cacheline_bytes )
+
+end = (void *)ROUNDUP((uintptr_t)p + size, cacheline_bytes);
+for ( ; p < end; p += cacheline_bytes )
 asm volatile (__clean_dcache_one(0) : : "r" (p));
 dsb(sy);   /* So we know the flushes happen before continuing */
 /* ARM callers assume that dcache_* functions cannot fail. */


Anyway, I'm OK with either fix.

Cheers,
Edgar



> 
> Cheers,
> Edgar
> 
> 
> > 
> > ---
> > 
> > diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
> > index 86de0b6..9cdf2fb 100644
> > --- a/xen/include/asm-arm/page.h
> > +++ b/xen/include/asm-arm/page.h
> > @@ -322,10 +322,30 @@ static inline int invalidate_dcache_va_range(const 
> > void *p, unsigned long size)
> >  
> >  static inline int clean_dcache_va_range(const void *p, unsigned long size)
> >  {
> > -const void *end;
> > +size_t off;
> > +const void *end = p + size;
> > +
> >  dsb(sy);   /* So the CPU issues all writes to the range */
> > -for ( end = p + size; p < end; p += cacheline_bytes )
> > +
> > +off = (unsigned long)p % cacheline_bytes;
> > +if ( off )
> > +{
> > +p -= off;
> >  asm volatile (__clean_dcache_one(0) : : "r" (p));
> > +p += cacheline_bytes;
> > +size -= cacheline_bytes - off;
> > +}
> > +off = (unsigned long)end % cacheline_bytes;
> > +if ( off )
> > +{
> > +end -= off;
> > +size -= off;
> > +asm volatile (__clean_dcache_one(0) : : "r" (end));
> > +}
> > +
> > +for ( ; p < end; p += cacheline_bytes )
> > +asm volatile (__clean_dcache_one(0) : : "r" (p));
> > +
> >  dsb(sy);   /* So we know the flushes happen before continuing 
> > */
> >  /* ARM callers assume that dcache_* functions cannot fail. */
> >  return 0;
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] xen/arm and swiotlb-xen: possible data corruption

[Edgar E. Iglesias]

On Wed, Mar 01, 2017 at 05:05:21PM -0800, Stefano Stabellini wrote:
> Hi all,
> 
> Edgar reported a data corruption on network packets in dom0 when the
> swiotlb-xen is in use. He also reported that the following patch "fixes"
> the problem for him:
> 
>  static void __xen_dma_page_cpu_to_dev(struct device *hwdev, dma_addr_t 
> handle,
> size_t size, enum dma_data_direction dir)
>  {
> -   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size, dir, 
> DMA_MAP);
> +   printk("%s: addr=%lx size=%zd\n", __func__, handle, size);
> +   dma_cache_maint(handle & PAGE_MASK, handle & ~PAGE_MASK, size + 64, 
> dir, DMA_MAP);
> 
> I am thinking that the problem has something to do with cacheline
> alignment on the Xen side
> (xen/common/grant_table.c:__gnttab_cache_flush).
> 
> If op == GNTTAB_CACHE_INVAL, we call invalidate_dcache_va_range; if op
> == GNTTAB_CACHE_CLEAN, we call clean_dcache_va_range instead. The
> parameter, v, could be non-cacheline aligned.
> 
> invalidate_dcache_va_range is capable of handling a not aligned address,
> while clean_dcache_va_range does not.
> 
> Edgar, does the appended patch fix the problem for you?


Thanks Stefano,

This does indeed fix the issue for me.

Cheers,
Edgar


> 
> ---
> 
> diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
> index 86de0b6..9cdf2fb 100644
> --- a/xen/include/asm-arm/page.h
> +++ b/xen/include/asm-arm/page.h
> @@ -322,10 +322,30 @@ static inline int invalidate_dcache_va_range(const void 
> *p, unsigned long size)
>  
>  static inline int clean_dcache_va_range(const void *p, unsigned long size)
>  {
> -const void *end;
> +size_t off;
> +const void *end = p + size;
> +
>  dsb(sy);   /* So the CPU issues all writes to the range */
> -for ( end = p + size; p < end; p += cacheline_bytes )
> +
> +off = (unsigned long)p % cacheline_bytes;
> +if ( off )
> +{
> +p -= off;
>  asm volatile (__clean_dcache_one(0) : : "r" (p));
> +p += cacheline_bytes;
> +size -= cacheline_bytes - off;
> +}
> +off = (unsigned long)end % cacheline_bytes;
> +if ( off )
> +{
> +end -= off;
> +size -= off;
> +asm volatile (__clean_dcache_one(0) : : "r" (end));
> +}
> +
> +for ( ; p < end; p += cacheline_bytes )
> +asm volatile (__clean_dcache_one(0) : : "r" (p));
> +
>  dsb(sy);   /* So we know the flushes happen before continuing */
>  /* ARM callers assume that dcache_* functions cannot fail. */
>  return 0;

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2] xen/arm: introduce vwfi parameter

[Edgar E. Iglesias]

On Wed, Feb 22, 2017 at 09:29:47AM -0800, Stefano Stabellini wrote:
> Introduce new Xen command line parameter called "vwfi", which stands for
> virtual wfi. The default is "trap": Xen traps the guest wfi instruction
> and calls vcpu_block on the guest vcpu. The behavior can be changed
> setting vwfi to "native", in that case Xen doesn't trap the instruction
> at all, running it in guest context.
> 
> The result is strong reduction in irq latency (from 5000ns to 2000ns,
> measured using https://github.com/edgarigl/tbm, the physical timer, and
> 1 pcpu dedicated to 1 vcpu). The downside is that the scheduler thinks
> that the guest is busy when actually is sleeping, leading to suboptimal
> scheduling decisions.
> 
> Signed-off-by: Stefano Stabellini <sstabell...@kernel.org>
> CC: dario.faggi...@citrix.com
> CC: george.dun...@citrix.com
> CC: edgar.igles...@xilinx.com


Reviewed-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>


> ---
>  docs/misc/xen-command-line.markdown | 12 
>  xen/arch/arm/traps.c| 17 +++--
>  2 files changed, 27 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.markdown 
> b/docs/misc/xen-command-line.markdown
> index c0106fb..6dea172 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -1638,6 +1638,18 @@ Note that if **watchdog** option is also specified 
> vpmu will be turned off.
>  As the virtualisation is not 100% safe, don't use the vpmu flag on
>  production systems (see http://xenbits.xen.org/xsa/advisory-163.html)!
>  
> +### vwfi
> +> `= trap | native
> +
> +> Default: `trap`
> +
> +WFI is the ARM instruction to "wait for interrupt". This option, which
> +is ARM specific, changes the way guest WFI is implemented in Xen. By
> +default, Xen traps the instruction, then blocks the guest vcpu. When setting
> +vwfi to `native`, Xen doesn't trap the instruction, running it in guest
> +context. Setting vwfi to `native` reduces irq latency, but leads to
> +suboptimal scheduling decisions.
> +
>  ### watchdog
>  > `= force | `
>  
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 4f96a2b..3a40717 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -102,6 +102,19 @@ static int debug_stack_lines = 40;
>  
>  integer_param("debug_stack_lines", debug_stack_lines);
>  
> +static enum {
> + TRAP,
> + NATIVE,
> +} vwfi;
> +
> +static void __init parse_vwfi(const char *s)
> +{
> + if ( !strcmp(s, "native") )
> + vwfi = NATIVE;
> + else
> + vwfi = TRAP;
> +}
> +custom_param("vwfi", parse_vwfi);
>  
>  void init_traps(void)
>  {
> @@ -128,8 +141,8 @@ void init_traps(void)
>  
>  /* Setup hypervisor traps */
>  WRITE_SYSREG(HCR_PTW|HCR_BSU_INNER|HCR_AMO|HCR_IMO|HCR_FMO|HCR_VM|
> - HCR_TWE|HCR_TWI|HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB,
> - HCR_EL2);
> + (vwfi != NATIVE ? HCR_TWI : 0) |HCR_TWE|
> + HCR_TSC|HCR_TAC|HCR_SWIO|HCR_TIDCP|HCR_FB,HCR_EL2);
>  isb();
>  }
>  
> -- 
> 1.9.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Edgar E. Iglesias]

On Wed, Feb 22, 2017 at 09:22:25AM -0800, Stefano Stabellini wrote:
> On Wed, 22 Feb 2017, Edgar E. Iglesias wrote:
> > On Tue, Feb 21, 2017 at 07:20:29PM +, Julien Grall wrote:
> > > 
> > > 
> > > On 21/02/2017 18:30, Stefano Stabellini wrote:
> > > >On Tue, 21 Feb 2017, Julien Grall wrote:
> > > >>Hi Stefano,
> > > >>
> > > >>On 21/02/17 17:49, Stefano Stabellini wrote:
> > > >>>On Tue, 21 Feb 2017, Dario Faggioli wrote:
> > > >>>>On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote:
> > > >>>>Oh, actually, if --which I only now realize may be what you are
> > > >>>>referring to, since you're talking about "guest burning its credits"--
> > > >>>>you let the vCPU put the pCPU to sleep *but*, when it wakes up (or 
> > > >>>>when
> > > >>>>the scheduler runs again for whatever reason), you charge to it for 
> > > >>>>all
> > > >>>>the time the the pCPU was actually idle/sleeping, well, that may
> > > >>>>actually  not break scheduling, or cause disruption to the service of
> > > >>>>other vCPUs But indeed I'd consider it rather counter intuitive a
> > > >>>>behavior.
> > > >>>
> > > >>>How can this be safe? There could be no interrupts programmed to wake 
> > > >>>up
> > > >>>the pcpu at all. In fact, I don't think today there would be any, 
> > > >>>unless
> > > >>>we set one up in Xen for the specific purpose of interrupting the pcpu
> > > >>>sleep.
> > > >>>
> > > >>>I don't know the inner working of the scheduler, but does it always 
> > > >>>send
> > > >>>an interrupt to other pcpu to schedule something?
> > > >>
> > > >>You still seem to assume that WFI/WFE is the only way to get a vCPU
> > > >>unscheduled. If that was the case it would be utterly wrong because you 
> > > >>cannot
> > > >>expect a guest to use them.
> > > >>
> > > >>>
> > > >>>What if there are 2 vcpu pinned to the same pcpu? This cannot be fair.
> > > >>
> > > >>Why wouldn't it be fair? This is the same situation as a guest vCPU not 
> > > >>using
> > > >>WFI/WFE.
> > > >
> > > >I read your suggestion as trapping WFI in Xen, then, depending on
> > > >settings, executing WFI in the Xen trap handler to idle the pcpu. That
> > > >doesn't work. But I take you suggested not trapping wfi (remove
> > > >HCR_TWI), executing the instruction in guest context. That is what we
> > > >used to do in the early days (before a780f750). It should be safe and
> > > >possibly even quick. I'll rerun the numbers and let you know.
> > > 
> > > My first suggestion was to emulate WFI in Xen, which I agree is not safe 
> > > :).
> > > 
> > > I think not trapping WFI will have the best performance but may impact the
> > > credit of the vCPU as mentioned by Dario and George.
> > 
> > I agree, wfi in guest context or at least with everything prepared to 
> > return to
> > the current guest would be great.
> 
> And the new numbers look good:
> 
>  AVGMAX WARM_MAX
> credit1  1850 26501950
> credit2  1850 29501840 
> 
> We are hitting the same levels as the non-WFI case. Nice!

Yeah, very nice :-)

Thanks!
Edgar

> 
> 
> > An option to enable this would work fine for our use-cases. Or if we could
> > at runtime detect that it's the best approach given scheduling (i.e
> > exclusive vCPU/pCPU pinning) even better.
> 
> The option is easy and we can do it today. We might be able to do
> automatic enablement once we have a "nop" scheduler.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: introduce vwfi parameter

[Edgar E. Iglesias]

On Tue, Feb 21, 2017 at 07:20:29PM +, Julien Grall wrote:
> 
> 
> On 21/02/2017 18:30, Stefano Stabellini wrote:
> >On Tue, 21 Feb 2017, Julien Grall wrote:
> >>Hi Stefano,
> >>
> >>On 21/02/17 17:49, Stefano Stabellini wrote:
> >>>On Tue, 21 Feb 2017, Dario Faggioli wrote:
> On Tue, 2017-02-21 at 13:46 +, George Dunlap wrote:
> Oh, actually, if --which I only now realize may be what you are
> referring to, since you're talking about "guest burning its credits"--
> you let the vCPU put the pCPU to sleep *but*, when it wakes up (or when
> the scheduler runs again for whatever reason), you charge to it for all
> the time the the pCPU was actually idle/sleeping, well, that may
> actually  not break scheduling, or cause disruption to the service of
> other vCPUs But indeed I'd consider it rather counter intuitive a
> behavior.
> >>>
> >>>How can this be safe? There could be no interrupts programmed to wake up
> >>>the pcpu at all. In fact, I don't think today there would be any, unless
> >>>we set one up in Xen for the specific purpose of interrupting the pcpu
> >>>sleep.
> >>>
> >>>I don't know the inner working of the scheduler, but does it always send
> >>>an interrupt to other pcpu to schedule something?
> >>
> >>You still seem to assume that WFI/WFE is the only way to get a vCPU
> >>unscheduled. If that was the case it would be utterly wrong because you 
> >>cannot
> >>expect a guest to use them.
> >>
> >>>
> >>>What if there are 2 vcpu pinned to the same pcpu? This cannot be fair.
> >>
> >>Why wouldn't it be fair? This is the same situation as a guest vCPU not 
> >>using
> >>WFI/WFE.
> >
> >I read your suggestion as trapping WFI in Xen, then, depending on
> >settings, executing WFI in the Xen trap handler to idle the pcpu. That
> >doesn't work. But I take you suggested not trapping wfi (remove
> >HCR_TWI), executing the instruction in guest context. That is what we
> >used to do in the early days (before a780f750). It should be safe and
> >possibly even quick. I'll rerun the numbers and let you know.
> 
> My first suggestion was to emulate WFI in Xen, which I agree is not safe :).
> 
> I think not trapping WFI will have the best performance but may impact the
> credit of the vCPU as mentioned by Dario and George.

I agree, wfi in guest context or at least with everything prepared to return to
the current guest would be great.

An option to enable this would work fine for our use-cases. Or if we could
at runtime detect that it's the best approach given scheduling (i.e
exclusive vCPU/pCPU pinning) even better.

Cheers,
Edgar


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Mon, Feb 13, 2017 at 03:35:19PM +, Julien Grall wrote:
> On 02/02/17 15:33, Edgar E. Iglesias wrote:
> >On Wed, Feb 01, 2017 at 07:04:43PM +, Julien Grall wrote:
> >>On 31/01/2017 19:06, Edgar E. Iglesias wrote:
> >>>On Tue, Jan 31, 2017 at 05:09:53PM +, Julien Grall wrote:
> >>Thank you for the documentation. I am trying to understand if we could move
> >>initialization in Xen as suggested by Stefano. I looked at the driver in
> >>Linux and the code looks simple not many dependencies. However, I was not
> >>able to find where the Gigabit Transceivers are configured. Do you have any
> >>link to the code for that?
> >
> >Hi Julien,
> 
> Hi Edgar,

Hi Julien,

Sorry for the late reply..


> 
> >
> >I suspect that this setup has previously been done by the initial bootloader
> >auto-generated from design configuration tools.
> >
> >Now, this is moving into Linux.
> 
> Do you know why they decide to move the code in Linux? What would be the
> problem to let the bootloader configuring the GT?


No, I'm not sure why this approach was not used. The only thing I can think of
is a runtime configuration approach.


> 
> >There's a specific driver that does that but AFAICS, it has not been 
> >upstreamed yet.
> >You can see it here:
> >https://github.com/Xilinx/linux-xlnx/blob/master/drivers/phy/phy-zynqmp.c
> >
> >DTS nodes that need a PHY can then just refer to it, here's an example from 
> >SATA:
> > {
> >phy-names = "sata-phy";
> >phys = < PHY_TYPE_SATA 1 3 15000>;
> >};
> >
> >I'll see if I can find working examples for PCIe on the ZCU102. Then I'll 
> >share
> >DTS, Kernel etc.
> 
> I've found a device tree on the github from the ZCU102: zynqmp-zcu102.dts,
> it looks like there is no use of PHY for the pcie so far.
> 
> Lets imagine in the future, pcie will use the PHY. If we decide to
> initialize the hostbridge in Xen, we would also have to pull the PHY code in
> the hypervisor. Leaving aside the problem to pull more code in Xen, this is
> not nice because the PHY is used by different components (e.g SATA, USB). So
> Xen and DOM0 would have to share the PHY.
> 
> For Xen POV, the best solution would be the bootloader initializing the PHY
> because starting Xen. So we can keep all the hostbridge (initialization +
> access) in Xen.
> 
> If it is not possible, then I would prefer to see the hostbridge
> initialization in DOM0.

Yes, I agree that the GT setup in the bootloader is very attractive.
I don't think hte setup sequence is complicated, we can perhaps even do it
on the commandline in u-boot or xsdb. I'll have to check.


> 
> >
> >If you are looking for a platform to get started, an option could be if I 
> >get you a build of
> >our QEMU that includes models for the PCIe controller, MSI and SMMU 
> >connections.
> >These models are friendly wrt. PHY configs and initialization sequences, it 
> >will
> >accept pretty much any sequence and still work. This would allow you to 
> >focus on
> >architectural issues rather than exact details of init sequences (which we 
> >can
> >deal with later).
> 
> From my understanding the problem is where the hostbridge should be
> initialized. In an ideal world, I think this is the goal of the bootloader.
> If it is not possible then depending on the complexity, the initialization
> would have to be done either in Xen or DOM0.
> 
> I guess this could be decided on case by case basis. I will suggest
> different possibility in the design document.
> 
> [...]
> 
> >>
> >>From a design point of view, it would make more sense to have the MSI
> >>controller driver in Xen as the hostbridge emulation for guest will also
> >>live there.
> >>
> >>So if we receive MSI in Xen, we need to figure out a way for DOM0 and guest
> >>to receive MSI. The same way would be the best, and I guess non-PV if
> >>possible. I know you are looking to boot unmodified OS in a VM. This would
> >>mean we need to emulate the MSI controller and potentially xilinx PCI
> >>controller. How much are you willing to modify the OS?
> >
> >Today, we have not yet implemented PCIe drivers for our baremetal SDK. So
> >things are very open and we could design with pretty much anything in mind.
> >
> >Yes, we could perhaps include a very small model with most registers dummied.
> >Implementing the MSI read FIFO would allow us to:
> >
> >1. Inject the MSI doorbell SPI into guests. The guest will then see the same
> >   IRQ as on real HW.
> >

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Thu, Feb 09, 2017 at 12:42:09PM -0700, Tamas K Lengyel wrote:
> On Thu, Feb 9, 2017 at 11:39 AM, Julien Grall  wrote:
> > Hi Tamas,
> >
> > On 02/09/2017 06:11 PM, Tamas K Lengyel wrote:
> >>
> >> On Wed, Feb 8, 2017 at 5:08 PM, Julien Grall  wrote:
> >>>
> >>> On 08/02/2017 23:28, Tamas K Lengyel wrote:
> 
>  On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall 
>  wrote:
> >>>
> >>> You haven't understood my point. Xen is currently emulating PSCI call for
> >>> the guest to allow powering up and down the CPUs and other stuff. If you
> >>> decide to trap all the SMCs, you would have to handle them.
> >>
> >>
> >> Sure, it's more work on the monitor side, but other then that, what's
> >> the problem?
> >
> >
> > Because you will have to introduce hypercalls to get all the necessary
> > information from Xen that will not be available from outside.
>
> > Given that SMC has been designed to target different services (PSCI, Trusted
> > OS...) it would be normal to have monitor app only monitoring a certain set
> > of SMC call. You cannot deny a such use case as it would avoid an monitor
> > app to handle every single call that would be consumed by XEN but not
> > forwarded to the secure firmware.
> >
> 
> I have nothing against introducing a fine-tune option to the SMC
> monitoring system so the monitor app can determine if it wants all
> SMCs or only a subset. At the moment I don't know of any usecase that
> would require this option. I certainly don't need it. If this option
> gets implemented by someone, I would be happy to take it.

Well, the reason it would be useful is the other way around.
On for example ZynqMP, enabling the monitor is useless since it will
turn off the ability to use the vital FW APIs needed for device
passthrough.

So the monitor only works for PV guests that call SMC APIs to some
imaginary Firmware.

While a monitor that didn't insist in consuming all SMC calls,
could very well be useful for monitoring/tracing purposes or
other stuff even with guests that actually use a "real" FW API.

I don't think we need to implement support for the latter right away,
we can incrementally add support for these things (I hope).

Cheers,
Edgar





___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Wed, Feb 08, 2017 at 10:04:53PM +, Julien Grall wrote:
> Hi Tamas,
> 
> Can you please try to configure your e-mail client to use '>' rather than '
> '? It makes quite hard to read the e-mail.
> 
> On 08/02/2017 20:15, Tamas K Lengyel wrote:
> >
> >
> >On Wed, Feb 8, 2017 at 12:40 PM, Edgar E. Iglesias
> ><edgar.igles...@gmail.com <mailto:edgar.igles...@gmail.com>> wrote:
> >On Wed, Feb 08, 2017 at 06:29:13PM +0100, Edgar E. Iglesias wrote:
> 
> >If platform_hvc() consumes an SMC, it's considered part of the Xen API.
> >Visible but not filterable by a monitor.
> >
> >
> >Platforms can then dictate which SMC calls are better handled within
> >Xen and
> >which ones can be exposed to dom0 user-space.
> >
> >In addition, there could be a hypercall to disable platform specific
> >handling
> >in Xen alltogether for a given guest. Then everything goes to dom0
> >user-space.
> >
> >It's a little messy...
> >
> >
> >
> >That is messy and I would not want any SMCs reaching the firmware when
> >the monitor application is in use. The monitor interface is disabled by
> >default and there aren't any known usecases where the SMC has to reach
> >both the firmware and the monitor application as well. So I think it is
> >safe to just make the two things mutually exclusive.
> 
> If you look at the SMC Calling Convention [1] both HVC and SMC are
> considered a conduit for service call to the secure firmware or hypervisor.
> It would be up to the hypervisor deciding what to do.
> 
> Lets imagine the guest is deciding to use HVC to access the secure firmware
> (AFAIU this patch series is adding that), are you going to monitor all the

Hi Julien,

My patch enables HVC and SMC to reach the platform specific calls. I didn't
connect PSCI over SMC though, I forgot about that but I definitely think
it makes sense to do so. Unmodified guest code would use SMC for everything
when running without a Hypervisor, so preferably the same would work on top
of Xen.

I'll fix that for v3.

Thanks,
Edgar




> HVCs (including hypercall)?
> 
> Similarly, non-modified baremetal app could use SMC to power on/off the vCPU
> (see PSCI spec). Will you emulate that in the monitor app?
> 
> So I think we need to be consistent across HVC and SMC call. And mutually
> exclusive does not sound right to me for HVC.
> 
> Cheers,
> 
> [1] 
> http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.den0028b/index.html
> 
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Thu, Feb 09, 2017 at 10:12:41AM +0100, Edgar E. Iglesias wrote:
> On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:
> > On Thu, 9 Feb 2017, Julien Grall wrote:
> > > On 08/02/2017 23:28, Tamas K Lengyel wrote:
> > > > On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall <julien.gr...@arm.com> 
> > > > wrote:
> > > > > Hi Tamas,
> > > > > 
> > > > > Can you please try to configure your e-mail client to use '>' rather 
> > > > > than
> > > > > '
> > > > > '? It makes quite hard to read the e-mail.
> > > > 
> > > > Hm, not sure why it switched but should be fine now.
> > > > 
> > > > > On 08/02/2017 20:15, Tamas K Lengyel wrote:
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > On Wed, Feb 8, 2017 at 12:40 PM, Edgar E. Iglesias
> > > > > > <edgar.igles...@gmail.com <mailto:edgar.igles...@gmail.com>> wrote:
> > > > > > On Wed, Feb 08, 2017 at 06:29:13PM +0100, Edgar E. Iglesias 
> > > > > > wrote:
> > > > > 
> > > > > 
> > > > > > If platform_hvc() consumes an SMC, it's considered part of the 
> > > > > > Xen
> > > > > > API.
> > > > > > Visible but not filterable by a monitor.
> > > > > > 
> > > > > > 
> > > > > > Platforms can then dictate which SMC calls are better handled 
> > > > > > within
> > > > > > Xen and
> > > > > > which ones can be exposed to dom0 user-space.
> > > > > > 
> > > > > > In addition, there could be a hypercall to disable platform 
> > > > > > specific
> > > > > > handling
> > > > > > in Xen alltogether for a given guest. Then everything goes to 
> > > > > > dom0
> > > > > > user-space.
> > > > > > 
> > > > > > It's a little messy...
> > > > > > 
> > > > > > 
> > > > > > 
> > > > > > That is messy and I would not want any SMCs reaching the firmware 
> > > > > > when
> > > > > > the monitor application is in use. The monitor interface is 
> > > > > > disabled by
> > > > > > default and there aren't any known usecases where the SMC has to 
> > > > > > reach
> > > > > > both the firmware and the monitor application as well. So I think 
> > > > > > it is
> > > > > > safe to just make the two things mutually exclusive.
> > > > > 
> > > > > 
> > > > > If you look at the SMC Calling Convention [1] both HVC and SMC are
> > > > > considered a conduit for service call to the secure firmware or
> > > > > hypervisor.
> > > > > It would be up to the hypervisor deciding what to do.
> > > > > 
> > > > > Lets imagine the guest is deciding to use HVC to access the secure
> > > > > firmware
> > > > > (AFAIU this patch series is adding that), are you going to monitor 
> > > > > all the
> > > > > HVCs (including hypercall)?
> > > > 
> > > > There are some fundamental differences between HVC and SMC calls
> > > > though. An HVC can only land in the hypervisor, so as a hypercall, I
> > > > would expect it to be something I can deny via XSM. That is a
> > > > sufficient option for now to block the path to the firmware. If we end
> > > > up needing to support an application that uses that hypercall for
> > > > something critical, then yes, it would also need to be hooked into the
> > > > monitor system. At the moment this is not necessary.
> > > 
> > > My point is not about what is necessary at the moment. But what is right
> > > things to do. If you look at the spec, HVC are not only for hypercall, 
> > > but any
> > > other kind of services. Why would you deny something that is valid from 
> > > the
> > > specification (see 5.2.1)?
> > > 
> > > "The SMC calling convention, however, does not specify which instruction
> > > (either SMC or HVC) to use to invoke a
> > > particular service."
> > 
> > To have a generic solution, we need a way to specify a set of H

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Wed, Feb 08, 2017 at 05:20:44PM -0800, Stefano Stabellini wrote:
> On Thu, 9 Feb 2017, Julien Grall wrote:
> > On 08/02/2017 23:28, Tamas K Lengyel wrote:
> > > On Wed, Feb 8, 2017 at 3:04 PM, Julien Grall <julien.gr...@arm.com> wrote:
> > > > Hi Tamas,
> > > > 
> > > > Can you please try to configure your e-mail client to use '>' rather 
> > > > than
> > > > '
> > > > '? It makes quite hard to read the e-mail.
> > > 
> > > Hm, not sure why it switched but should be fine now.
> > > 
> > > > On 08/02/2017 20:15, Tamas K Lengyel wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > On Wed, Feb 8, 2017 at 12:40 PM, Edgar E. Iglesias
> > > > > <edgar.igles...@gmail.com <mailto:edgar.igles...@gmail.com>> wrote:
> > > > > On Wed, Feb 08, 2017 at 06:29:13PM +0100, Edgar E. Iglesias wrote:
> > > > 
> > > > 
> > > > > If platform_hvc() consumes an SMC, it's considered part of the Xen
> > > > > API.
> > > > > Visible but not filterable by a monitor.
> > > > > 
> > > > > 
> > > > > Platforms can then dictate which SMC calls are better handled 
> > > > > within
> > > > > Xen and
> > > > > which ones can be exposed to dom0 user-space.
> > > > > 
> > > > > In addition, there could be a hypercall to disable platform 
> > > > > specific
> > > > > handling
> > > > > in Xen alltogether for a given guest. Then everything goes to dom0
> > > > > user-space.
> > > > > 
> > > > > It's a little messy...
> > > > > 
> > > > > 
> > > > > 
> > > > > That is messy and I would not want any SMCs reaching the firmware when
> > > > > the monitor application is in use. The monitor interface is disabled 
> > > > > by
> > > > > default and there aren't any known usecases where the SMC has to reach
> > > > > both the firmware and the monitor application as well. So I think it 
> > > > > is
> > > > > safe to just make the two things mutually exclusive.
> > > > 
> > > > 
> > > > If you look at the SMC Calling Convention [1] both HVC and SMC are
> > > > considered a conduit for service call to the secure firmware or
> > > > hypervisor.
> > > > It would be up to the hypervisor deciding what to do.
> > > > 
> > > > Lets imagine the guest is deciding to use HVC to access the secure
> > > > firmware
> > > > (AFAIU this patch series is adding that), are you going to monitor all 
> > > > the
> > > > HVCs (including hypercall)?
> > > 
> > > There are some fundamental differences between HVC and SMC calls
> > > though. An HVC can only land in the hypervisor, so as a hypercall, I
> > > would expect it to be something I can deny via XSM. That is a
> > > sufficient option for now to block the path to the firmware. If we end
> > > up needing to support an application that uses that hypercall for
> > > something critical, then yes, it would also need to be hooked into the
> > > monitor system. At the moment this is not necessary.
> > 
> > My point is not about what is necessary at the moment. But what is right
> > things to do. If you look at the spec, HVC are not only for hypercall, but 
> > any
> > other kind of services. Why would you deny something that is valid from the
> > specification (see 5.2.1)?
> > 
> > "The SMC calling convention, however, does not specify which instruction
> > (either SMC or HVC) to use to invoke a
> > particular service."
> 
> To have a generic solution, we need a way to specify a set of HVC/SMC
> calls that get monitored and a set that get handled in Xen (platform
> specific or otherwise). I think it is OK not to do both, at least at the
> beginning, but we might want to add that feature in the future.
> 
> As much as I would like to see that, in respect to this series, I don't
> think we should ask Edgar to introduce such a mechanism. However, we do
> need to decide what Xen should do when platform_hvc is implemented and
> monitor is also enabled.
> 
> I think the default should be to only call platform_hvc, because there
> are many valid monitoring use-cases which don't require those few
> platf

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Wed, Feb 08, 2017 at 06:29:13PM +0100, Edgar E. Iglesias wrote:
> On Wed, Feb 08, 2017 at 04:58:44PM +, Julien Grall wrote:
> > Hi Tamas,
> > 
> > On 08/02/17 16:40, Tamas K Lengyel wrote:
> > >On Wed, Feb 8, 2017 at 1:31 AM, Edgar E. Iglesias
> > ><edgar.igles...@xilinx.com <mailto:edgar.igles...@xilinx.com>> wrote:
> > >On Tue, Feb 07, 2017 at 05:24:03PM -0700, Tamas K Lengyel wrote:
> > >> On Tue, Feb 7, 2017 at 1:51 PM, Julien Grall <julien.gr...@arm.com
> > ><mailto:julien.gr...@arm.com>> wrote:
> > >I considered this approach a bit but it has several problems IMO.
> > >These may not be unsolvable or even problems for monitoring but
> > >they do introduce complexity into the solution.
> > >
> > >1. Some SMC calls may depend on the core they are issued from.
> > >   If taking a detour to dom0, this becomes messy to guarantee.
> > >
> > >2. Overall complexity increases very significantly and it becomes
> > >   quite hard to follow/review how these calls get handled.
> > >   In particular once you consider solving #1.
> > >
> > >3. There are certain calls that perhaps not even dom0 should have
> > >   direct access to. This means that Xen may need to filter some of
> > >   them anyway.
> > >
> > >Some examples may be:
> > >
> > >SMC calls:
> > >* To directly turn off or suspend cores
> > >* To turn off DDR or RAMs that Xen is using
> > >* To a solution specific Trusted OS pinned to a specific core
> > >* For PSCI
> > >* Etc
> > >
> > >I would prefer if we could find a way for monitoring to play nicely
> > >with Xen implementing the SMC mediators.
> > >Perhaps we could allow calls that Xen consumes to be 
> > > monitored/inspected
> > >but not modified. Or there might be other ways.
> > >
> > >Best regards,
> > >Edgar
> > >
> > >
> > >Hi Edgar,
> > >certainly there are many cases where the system would become very
> > >complex when there is functionality like what you describe in the TZ
> > >that needs to be made accessible via SMC. The setup I described is
> > >experimental only, and the underlying assumption is that the TZ is
> > >working jointly with the monitor application (ie. both are aware of each
> > >other). So it is really not intended to work with just any firmware.
> > 
> > How do you expect TrustZone to work with the monitor application? If you
> > think about modifying Trustzone, it seems a requirement difficult to achieve
> > as some Trusted OS are proprietary or difficult to replace on a phone.
> > 
> > >
> > >So I think for the sake of reducing complexity, having the monitor
> > >system be exclusive when enabled should make everyone's life simpler.
> > >Having a passive monitoring mode as you suggest is certainly an option,
> > >although it should not be the only option, exclusive routing of SMCs
> > >through monitor applications should still be available to be configured
> > >by the user. Since I really don't know of any usecases where passive
> > >monitoring of SMCs is required, I don't think we should go that route.
> > 
> > I see the SMC trap similar to a register trap. The monitor app will look at
> > the register value and potentially modify it. What would be the issue to do
> > the same for SMC?
> 
> Yes, I think this would work if we can keep the SMC processing on the same
> core (after it's been accepted by monitor filters). If not accepted by
> filters, we'd just ignore the SMC processing and return registers provided
> by the monitor (or something along those lines).
> 
> 
> > I think a such model would fit the requirement for everyone. The monitor app
> > can filter if necessary, and Xen would handle the mediation between multiple

Any ideas on how we can support monitor filtering while at the same time
having Xen processing calls? How do we wait for a response from the monitor
and then (if allowed) issue the SMC from the right core?

Or we could make these approaches mutually exclusive.
If platform_hvc() consumes an SMC, it's considered part of the Xen API.
Visible but not filterable by a monitor.

Platforms can then dictate which SMC calls are better handled within Xen and
which ones can be exposed to dom0 user-space.

In addition, there could be a hypercall to disable platform specific handling
in Xen alltogether for a given guest. Then everyth

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Wed, Feb 08, 2017 at 04:58:44PM +, Julien Grall wrote:
> Hi Tamas,
> 
> On 08/02/17 16:40, Tamas K Lengyel wrote:
> >On Wed, Feb 8, 2017 at 1:31 AM, Edgar E. Iglesias
> ><edgar.igles...@xilinx.com <mailto:edgar.igles...@xilinx.com>> wrote:
> >On Tue, Feb 07, 2017 at 05:24:03PM -0700, Tamas K Lengyel wrote:
> >> On Tue, Feb 7, 2017 at 1:51 PM, Julien Grall <julien.gr...@arm.com
> ><mailto:julien.gr...@arm.com>> wrote:
> >I considered this approach a bit but it has several problems IMO.
> >These may not be unsolvable or even problems for monitoring but
> >they do introduce complexity into the solution.
> >
> >1. Some SMC calls may depend on the core they are issued from.
> >   If taking a detour to dom0, this becomes messy to guarantee.
> >
> >2. Overall complexity increases very significantly and it becomes
> >   quite hard to follow/review how these calls get handled.
> >   In particular once you consider solving #1.
> >
> >3. There are certain calls that perhaps not even dom0 should have
> >   direct access to. This means that Xen may need to filter some of
> >   them anyway.
> >
> >Some examples may be:
> >
> >SMC calls:
> >* To directly turn off or suspend cores
> >* To turn off DDR or RAMs that Xen is using
> >* To a solution specific Trusted OS pinned to a specific core
> >* For PSCI
> >* Etc
> >
> >I would prefer if we could find a way for monitoring to play nicely
> >with Xen implementing the SMC mediators.
> >Perhaps we could allow calls that Xen consumes to be monitored/inspected
> >but not modified. Or there might be other ways.
> >
> >Best regards,
> >Edgar
> >
> >
> >Hi Edgar,
> >certainly there are many cases where the system would become very
> >complex when there is functionality like what you describe in the TZ
> >that needs to be made accessible via SMC. The setup I described is
> >experimental only, and the underlying assumption is that the TZ is
> >working jointly with the monitor application (ie. both are aware of each
> >other). So it is really not intended to work with just any firmware.
> 
> How do you expect TrustZone to work with the monitor application? If you
> think about modifying Trustzone, it seems a requirement difficult to achieve
> as some Trusted OS are proprietary or difficult to replace on a phone.
> 
> >
> >So I think for the sake of reducing complexity, having the monitor
> >system be exclusive when enabled should make everyone's life simpler.
> >Having a passive monitoring mode as you suggest is certainly an option,
> >although it should not be the only option, exclusive routing of SMCs
> >through monitor applications should still be available to be configured
> >by the user. Since I really don't know of any usecases where passive
> >monitoring of SMCs is required, I don't think we should go that route.
> 
> I see the SMC trap similar to a register trap. The monitor app will look at
> the register value and potentially modify it. What would be the issue to do
> the same for SMC?

Yes, I think this would work if we can keep the SMC processing on the same
core (after it's been accepted by monitor filters). If not accepted by
filters, we'd just ignore the SMC processing and return registers provided
by the monitor (or something along those lines).


> I think a such model would fit the requirement for everyone. The monitor app
> can filter if necessary, and Xen would handle the mediation between multiple
> guests. I would also recommend to read the thread about OP-TEE support in
> Xen (see [1]).
>

We have a similar issue with interrupts from Firmware. I've not implemented
this part in this series but at some point I'm going to have to.

Essentially I'm looking at handling an IPI in Xen and forwarding an SGI or
a virtual inject of the IPI to the guest that was targeted. Future work...

Cheers,
Edgar

> Cheers,
> 
> [1]
> https://lists.xenproject.org/archives/html/xen-devel/2016-11/msg02220.html
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

On Tue, Feb 07, 2017 at 05:24:03PM -0700, Tamas K Lengyel wrote:
> On Tue, Feb 7, 2017 at 1:51 PM, Julien Grall <julien.gr...@arm.com> wrote:
> 
> > Hi Tamas,
> >
> > On 07/02/2017 20:38, Tamas K Lengyel wrote:
> >
> >>
> >>
> >> On Tue, Feb 7, 2017 at 12:42 PM, Edgar E. Iglesias
> >> <edgar.igles...@gmail.com <mailto:edgar.igles...@gmail.com>> wrote:
> >>
> >> From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com
> >> <mailto:edgar.igles...@xilinx.com>>
> >>
> >> Allow platform_hvc to handle guest SMC calls (as well as
> >> HVC calls) in a platform specific way.
> >>
> >> Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com
> >> <mailto:edgar.igles...@xilinx.com>>
> >> ---
> >>  xen/arch/arm/traps.c | 5 +
> >>  1 file changed, 5 insertions(+)
> >>
> >> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> >> index 33950d9..1bedc6e 100644
> >> --- a/xen/arch/arm/traps.c
> >> +++ b/xen/arch/arm/traps.c
> >> @@ -2623,6 +2623,11 @@ static void do_trap_smc(struct cpu_user_regs
> >> *regs, const union hsr hsr)
> >>  if ( current->domain->arch.monitor.privileged_call_enabled )
> >>  rc = monitor_smc();
> >>
> >>
> >> I think you should check that rc is still 0 at this point as the vCPU
> >> might already be paused and the event forwarded to a monitor subscriber.
> >>
> >
> > SMC are used to access the secure firmware (e.g power management) and will
> > be used by the guest to access the secure firmware. Today, the code is
> > expecting all event to be trapped by the monitor app and software emulated.
> > However, some SMCs may be needed to be forwarded to the secure firmware,
> > how do you expect it to work together?
> >
> > It is something I already brought up when SMC trap was added and it is
> > probably time to figure out what to do because this will not be the first
> > series bringing the problem. For instance if you want to do video decoding
> > or even payment on Android you may need to access the secure firmware for
> > cryptography. At the same time, you also want to be able to monitor your
> > guest.
> >
> 
> 
> Hi Julien,
> monitoring SMCs using the monitor system should be incompatible with Xen
> routing the SMCs elsewhere. Since the monitor system is disabled by default
> I think this should be fine for everyone and not get in the way of people
> accessing the firmware in other usecases or routing SMCs elsewhere as
> needed.
> 
> As for applications that want to use SMC monitoring but also access the
> firmware, it can be accomplished by the monitor application on behalf of
> the VM.  While this adds a detour, this detour is by design as it adds a
> layer between untrusted VMs and the TZ so that any potential exploit
> targeting the TZ would first have to go through the monitor application
> (see https://www.sec.in.tum.de/publications/publication/322 for more info
> on the idea).

I considered this approach a bit but it has several problems IMO.
These may not be unsolvable or even problems for monitoring but
they do introduce complexity into the solution.

1. Some SMC calls may depend on the core they are issued from.
   If taking a detour to dom0, this becomes messy to guarantee.

2. Overall complexity increases very significantly and it becomes
   quite hard to follow/review how these calls get handled.
   In particular once you consider solving #1.

3. There are certain calls that perhaps not even dom0 should have
   direct access to. This means that Xen may need to filter some of
   them anyway.

Some examples may be:

SMC calls:
* To directly turn off or suspend cores
* To turn off DDR or RAMs that Xen is using
* To a solution specific Trusted OS pinned to a specific core
* For PSCI
* Etc

I would prefer if we could find a way for monitoring to play nicely
with Xen implementing the SMC mediators.
Perhaps we could allow calls that Xen consumes to be monitored/inspected
but not modified. Or there might be other ways.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 5/6] xen/arm: zynqmp: Forward plaform specific firmware calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Implement an EEMI mediator and forward platform specific
firmware calls from guests to firmware.

The EEMI mediator is responsible for implementing access
controls modifying or blocking calls that try to operate
on setup for devices that are not under the calling guest's
control.

EEMI:
https://www.xilinx.com/support/documentation/user_guides/ug1200-eemi-api.pdf

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platforms/Makefile|   1 +
 xen/arch/arm/platforms/xilinx-zynqmp-eemi.c| 794 +
 xen/arch/arm/platforms/xilinx-zynqmp.c |  18 +
 xen/include/asm-arm/platforms/xilinx-zynqmp-eemi.h | 337 +
 xen/include/asm-arm/platforms/xilinx-zynqmp-mm.h   | 284 
 5 files changed, 1434 insertions(+)
 create mode 100644 xen/arch/arm/platforms/xilinx-zynqmp-eemi.c
 create mode 100644 xen/include/asm-arm/platforms/xilinx-zynqmp-eemi.h
 create mode 100644 xen/include/asm-arm/platforms/xilinx-zynqmp-mm.h

diff --git a/xen/arch/arm/platforms/Makefile b/xen/arch/arm/platforms/Makefile
index 49fa683..b58b71f 100644
--- a/xen/arch/arm/platforms/Makefile
+++ b/xen/arch/arm/platforms/Makefile
@@ -8,3 +8,4 @@ obj-$(CONFIG_ARM_64) += seattle.o
 obj-$(CONFIG_ARM_32) += sunxi.o
 obj-$(CONFIG_ARM_64) += xgene-storm.o
 obj-$(CONFIG_ARM_64) += xilinx-zynqmp.o
+obj-$(CONFIG_ARM_64) += xilinx-zynqmp-eemi.o
diff --git a/xen/arch/arm/platforms/xilinx-zynqmp-eemi.c 
b/xen/arch/arm/platforms/xilinx-zynqmp-eemi.c
new file mode 100644
index 000..c2c184d
--- /dev/null
+++ b/xen/arch/arm/platforms/xilinx-zynqmp-eemi.c
@@ -0,0 +1,794 @@
+/*
+ * xen/arch/arm/platforms/xilinx-zynqmp-eemi.c
+ *
+ * Xilinx ZynqMP EEMI API mediator.
+ *
+ * Copyright (c) 2017 Xilinx Inc.
+ * Written by Edgar E. Iglesias <edgar.igles...@xilinx.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+/*
+ * Mediator for the EEMI Power Mangement API.
+ *
+ * Refs:
+ * https://www.xilinx.com/support/documentation/user_guides/ug1200-eemi-api.pdf
+ *
+ * Background:
+ * The ZynqMP has a subsystem named the PMU with a CPU and special devices
+ * dedicated to running Power Management Firmware. Other masters in the
+ * system need to send requests to the PMU in order to for example:
+ * * Manage power state
+ * * Configure clocks
+ * * Program bitstreams for the programmable logic
+ * * etc
+ *
+ * Allthough the details of the setup are configurable, in the common case
+ * the PMU lives in the Secure world. NS World cannot directly communicate
+ * with it and must use proxy services from ARM Trusted Firmware to reach
+ * the PMU.
+ *
+ * Power Management on the ZynqMP is implemented in a layered manner.
+ * The PMU knows about various masters and will enforce access controls
+ * based on a pre-configured partitioning. This configuration dictates
+ * which devices are owned by the various masters and the PMU FW makes sure
+ * that a given master cannot turn off a device that it does not own or that
+ * is in use by other masters.
+ *
+ * The PMU is not aware of multiple execution states in masters.
+ * For example, it treats the ARMv8 cores as single units and does not
+ * distinguish between Secure vs NS OS:s nor does it know about Hypervisors
+ * and multiple guests. It is up to software on the ARMv8 cores to present
+ * a unified view of its power requirements.
+ *
+ * To implement this unified view, ARM Trusted Firmware at EL3 provides
+ * access to the PM API via SMC calls. ARM Trusted Firmware is responsible
+ * for mediating between the Secure and the NS world, rejecting SMC calls
+ * that request changes that are not allowed.
+ *
+ * Xen running above ATF owns the NS world and is responsible for presenting
+ * unified PM requests taking all guests and the hypervisor into account.
+ *
+ * Implementation:
+ * The PM API contains different classes of calls.
+ * Certain calls are harmless to expose to any guest.
+ * These include calls to get the PM API Version, or to read out the version
+ * of the chip we're running on.
+ *
+ * Other calls are more problematic. Here're some:
+ * * Power requests for nodes (i.e turn on or off a given device)
+ * * Reset line control (i.e reset a given device)
+ * * MMIO access (i.e directly access clock and reset registers)
+ *
+ * Power requests for nodes:
+ * In order to correctly mediate these calls, we need to know if
+ * guests issuing these calls have ownership of the given device.

[Xen-devel] [RFC v2 6/6] xen/arm: zynqmp: Remove blacklist of ZynqMP's PM node

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Stop blacklisting ZynqMP's power management node.
This is now possible since we allow the hardware domain to
issue HVC/SMC calls to firmware.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platforms/xilinx-zynqmp.c | 8 
 1 file changed, 8 deletions(-)

diff --git a/xen/arch/arm/platforms/xilinx-zynqmp.c 
b/xen/arch/arm/platforms/xilinx-zynqmp.c
index d826282..a2128e2 100644
--- a/xen/arch/arm/platforms/xilinx-zynqmp.c
+++ b/xen/arch/arm/platforms/xilinx-zynqmp.c
@@ -26,13 +26,6 @@ static const char * const zynqmp_dt_compat[] __initconst =
 NULL
 };
 
-static const struct dt_device_match zynqmp_blacklist_dev[] __initconst =
-{
-/* Power management is not yet supported.  */
-DT_MATCH_COMPATIBLE("xlnx,zynqmp-pm"),
-{ /* sentinel */ },
-};
-
 bool zynqmp_hvc(struct cpu_user_regs *regs)
 {
 register_t ret[4] = { 0 };
@@ -52,7 +45,6 @@ bool zynqmp_hvc(struct cpu_user_regs *regs)
 PLATFORM_START(xgene_storm, "Xilinx ZynqMP")
 .compatible = zynqmp_dt_compat,
 .hvc = zynqmp_hvc,
-.blacklist_dev = zynqmp_blacklist_dev,
 PLATFORM_END
 
 /*
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 2/6] xen/arm: Introduce platform_hvc

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Introduce platform_hvc as a way to handle hypercalls that
Xen does not know about in a platform specific way. This
is particularly useful for implementing the SiP (SoC
implementation specific) service calls.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platform.c| 8 
 xen/arch/arm/traps.c   | 3 +++
 xen/include/asm-arm/platform.h | 5 +
 3 files changed, 16 insertions(+)

diff --git a/xen/arch/arm/platform.c b/xen/arch/arm/platform.c
index 0af6d57..90ea6b8 100644
--- a/xen/arch/arm/platform.c
+++ b/xen/arch/arm/platform.c
@@ -127,6 +127,14 @@ void platform_poweroff(void)
 platform->poweroff();
 }
 
+bool platform_hvc(struct cpu_user_regs *regs)
+{
+if ( platform && platform->hvc )
+return platform->hvc(regs);
+
+return false;
+}
+
 bool_t platform_has_quirk(uint32_t quirk)
 {
 uint32_t quirks = 0;
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index c5a4d41..33950d9 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -44,6 +44,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "decode.h"
 #include "vtimer.h"
@@ -1430,6 +1431,8 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 }
 break;
 default:
+if ( platform_hvc(regs) )
+return;
 domain_crash_synchronous();
 return;
 }
diff --git a/xen/include/asm-arm/platform.h b/xen/include/asm-arm/platform.h
index 08010ba..4d51f0a 100644
--- a/xen/include/asm-arm/platform.h
+++ b/xen/include/asm-arm/platform.h
@@ -26,6 +26,10 @@ struct platform_desc {
 void (*reset)(void);
 /* Platform power-off */
 void (*poweroff)(void);
+/* Platform specific HVC handler.
+ * Returns true if the call was handled and false if not.
+ */
+bool (*hvc)(struct cpu_user_regs *regs);
 /*
  * Platform quirks
  * Defined has a function because a platform can support multiple
@@ -55,6 +59,7 @@ int platform_cpu_up(int cpu);
 #endif
 void platform_reset(void);
 void platform_poweroff(void);
+bool platform_hvc(struct cpu_user_regs *regs);
 bool_t platform_has_quirk(uint32_t quirk);
 bool_t platform_device_is_blacklisted(const struct dt_device_node *node);
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 0/6] zynqmp: Add forwarding of platform specific firmware calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

The ZynqMP software stack has an extensive Firmware API based on EEMI:
https://www.xilinx.com/support/documentation/user_guides/ug1200-eemi-api.pdf
https://www.xilinx.com/support/documentation/user_guides/ug1199-zynq-power-management.pdf

Software at EL1 is increasingly dependant on this API. In fact,
a large set of use-cases for Linux already require access to firmware.

Services provided by the API, among other things include:
* Clock Management
* PHY Configuration
* Power Management
* FPGA configuration
* etc

Adding support for accessing firmware can be done in two steps.

1. Allow the hw domain (dom0) to call into firmware. This enables
   all devices to work with PV setups.

2. Allow other guests to use the Firmware API. This will be needed
   to fully support device pass-through.
   We cannot give un-supervised access to unprivileged guests, so
   the idea is to mediate these calls in Xen or in dom0.
   Mediation mainly consists of verfifying access to calls,
   modifying calls and sometimes blocking them.

This patch series implements step #2.

There are quite a few options on how to implement step #2, e.g:
1. Trap HVC/SMC calls from unpriv guests and handle them in dom0.
   This could be done in a user-space agent.
2. Add a PV protocol carrying these calls.
3. Add several generic PV protocols carrying a set of platform independent
   generic calls that get translated or mapped to firmware calls
   in dom0. E.g PV-Clock, PV-Power, PV-FPGA.
4. Same as #1 but doing it all in Xen.
5. Others??

A benefit with #1 and #4 is that we could support unmodified bare-metal
guests. Such guests would continue to issue SMC calls from EL1 just as
if they were running without a hypervisor.

This patch series implements step #2 according to option #4.

EEMI has 3 kinds of resources it operates on.
1. Nodes (can be anything, a device, a CPU, a bus, a PLL)
2. Reset lines
3. MMIO accesses (i.e direct register access)

This implementation relies on the assumption that these
can be associated to a real device with some kind of
register area controlling it. If a guest has access to these
registers (e.g by being granted access via iomem), the
guest is allowed to control the node.

So, we map Nodes and rest-lines to memory addresses and check if
the guest issuing the call has access the device's control regs.

For direct MMIO calls, we map addresses and bitfields within registers
into nodes that get checked for guest ownership.

Guests are only allowed to power up/down and reset devices that are
in their control.

Certain calls, nodes and registers cannot be mapped (yet) to guests.
These fall mostly under dom0's control. A few calls are never allowed
(e.g turning off the current CPU or powering off DDR).

Any feedback would be much appreciated.

Thanks and Best regards!
Edgar

ChangeLog:

v1 -> v2:
* Advance PC after successfully processing an SMC
* Add implementation of a ZynqMP EEMI Mediator

Edgar E. Iglesias (6):
  xen/arm: traps: Reorder early overwrite of FID
  xen/arm: Introduce platform_hvc
  xen/arm: Allow platform_hvc to handle guest SMC calls
  xen/arm: Introduce call_smcc64
  xen/arm: zynqmp: Forward plaform specific firmware calls
  xen/arm: zynqmp: Remove blacklist of ZynqMP's PM node

 xen/arch/arm/platform.c|   8 +
 xen/arch/arm/platforms/Makefile|   1 +
 xen/arch/arm/platforms/xilinx-zynqmp-eemi.c| 794 +
 xen/arch/arm/platforms/xilinx-zynqmp.c |  22 +-
 xen/arch/arm/traps.c   |  15 +-
 xen/include/asm-arm/platform.h |   5 +
 xen/include/asm-arm/platforms/xilinx-zynqmp-eemi.h | 337 +
 xen/include/asm-arm/platforms/xilinx-zynqmp-mm.h   | 284 
 xen/include/asm-arm/processor.h|  40 ++
 9 files changed, 1498 insertions(+), 8 deletions(-)
 create mode 100644 xen/arch/arm/platforms/xilinx-zynqmp-eemi.c
 create mode 100644 xen/include/asm-arm/platforms/xilinx-zynqmp-eemi.h
 create mode 100644 xen/include/asm-arm/platforms/xilinx-zynqmp-mm.h

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 4/6] xen/arm: Introduce call_smcc64

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Introduce call_smcc64, a helper function to issue 64-bit SMC
calls that follow the SMC Calling Convention. This includes
returning up to 4 64-bit values.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/include/asm-arm/processor.h | 40 
 1 file changed, 40 insertions(+)

diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index afc0e9a..a604f8c 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -706,6 +706,46 @@ void vcpu_regs_user_to_hyp(struct vcpu *vcpu,
 int call_smc(register_t function_id, register_t arg0, register_t arg1,
  register_t arg2);
 
+/*
+ * Helper to issue a 64-bit SMC according to the SMC Calling Convention.
+ *
+ * @fid:  Function Identifier
+ * @a0 - a5:  6 arguments
+ * @ret:  Pointer to 4 register_t carrying return values
+ */
+static inline register_t call_smcc64(register_t fid,
+ register_t a0,
+ register_t a1,
+ register_t a2,
+ register_t a3,
+ register_t a4,
+ register_t a5,
+ register_t *ret)
+{
+register register_t x0 asm("x0") = fid;
+register register_t x1 asm("x1") = a0;
+register register_t x2 asm("x2") = a1;
+register register_t x3 asm("x3") = a2;
+register register_t x4 asm("x4") = a3;
+register register_t x5 asm("x5") = a4;
+register register_t x6 asm("x6") = a5;
+
+asm volatile ("smc #0\n"
+  : "+r" (x0), "+r" (x1), "+r" (x2), "+r" (x3),
+"+r" (x4), "+r" (x5), "+r" (x6)
+  :
+  : "x7", "x8", "x9", "x10", "x11", "x12",
+"x13", "x14", "x15", "x16", "x17" );
+
+if (ret) {
+ret[0] = x0;
+ret[1] = x1;
+ret[2] = x2;
+ret[3] = x3;
+}
+return x0;
+}
+
 void do_trap_guest_error(struct cpu_user_regs *regs);
 
 #endif /* __ASSEMBLY__ */
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Allow platform_hvc to handle guest SMC calls (as well as
HVC calls) in a platform specific way.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/traps.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 33950d9..1bedc6e 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2623,6 +2623,11 @@ static void do_trap_smc(struct cpu_user_regs *regs, 
const union hsr hsr)
 if ( current->domain->arch.monitor.privileged_call_enabled )
 rc = monitor_smc();
 
+if ( platform_hvc(regs) ) {
+advance_pc(regs, hsr);
+rc = 1;
+}
+
 if ( rc != 1 )
 inject_undef_exception(regs, hsr);
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v2 1/6] xen/arm: traps: Reorder early overwrite of FID

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Move the early setting of PSCI_RESULT_REG to a later stage
avoiding the early override of the FID that's stored in
the same register.

No functional change.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/traps.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 92b1d80..c5a4d41 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1335,8 +1335,6 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 {
 register_t fid = PSCI_ARG(regs,0);
 
-/* preloading in case psci_mode_check fails */
-PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 switch( fid )
 {
 case PSCI_cpu_off:
@@ -1369,6 +1367,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU:
 case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU:
 perfc_incr(vpsci_migrate_info_up_cpu);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 PSCI_RESULT_REG(regs) = do_psci_0_2_migrate_info_up_cpu();
 break;
@@ -1385,6 +1384,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_CPU_ON:
 case PSCI_0_2_FN64_CPU_ON:
 perfc_incr(vpsci_cpu_on);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 register_t vcpuid = PSCI_ARG(regs,1);
@@ -1397,6 +1397,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_CPU_SUSPEND:
 case PSCI_0_2_FN64_CPU_SUSPEND:
 perfc_incr(vpsci_cpu_suspend);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 uint32_t pstate = PSCI_ARG32(regs,1);
@@ -1409,6 +1410,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_AFFINITY_INFO:
 case PSCI_0_2_FN64_AFFINITY_INFO:
 perfc_incr(vpsci_cpu_affinity_info);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 register_t taff = PSCI_ARG(regs,1);
@@ -1420,6 +1422,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_MIGRATE:
 case PSCI_0_2_FN64_MIGRATE:
 perfc_incr(vpsci_cpu_migrate);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 uint32_t tcpu = PSCI_ARG32(regs,1);
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Thu, Feb 02, 2017 at 03:12:52PM -0800, Stefano Stabellini wrote:
> On Thu, 2 Feb 2017, Edgar E. Iglesias wrote:
> > On Wed, Feb 01, 2017 at 07:04:43PM +, Julien Grall wrote:
> > > Hi Edgar,
> > > 
> > > On 31/01/2017 19:06, Edgar E. Iglesias wrote:
> > > >On Tue, Jan 31, 2017 at 05:09:53PM +, Julien Grall wrote:
> > > >>On 31/01/17 16:53, Edgar E. Iglesias wrote:
> > > >>>On Wed, Jan 25, 2017 at 06:53:20PM +, Julien Grall wrote:
> > > >>>>On 24/01/17 20:07, Stefano Stabellini wrote:
> > > >>>>>On Tue, 24 Jan 2017, Julien Grall wrote:
> > > >>>>For generic host bridge, the initialization is inexistent. However 
> > > >>>>some host
> > > >>>>bridge (e.g xgene, xilinx) may require some specific setup and also
> > > >>>>configuring clocks. Given that Xen only requires to access the 
> > > >>>>configuration
> > > >>>>space, I was thinking to let DOM0 initialization the host bridge. 
> > > >>>>This would
> > > >>>>avoid to import a lot of code in Xen, however this means that we need 
> > > >>>>to
> > > >>>>know when the host bridge has been initialized before accessing the
> > > >>>>configuration space.
> > > >>>
> > > >>>
> > > >>>Yes, that's correct.
> > > >>>There's a sequence on the ZynqMP that involves assiging Gigabit 
> > > >>>Transceivers
> > > >>>to PCI (GTs are shared among PCIe, USB, SATA and the Display Port),
> > > >>>enabling clocks and configuring a few registers to enable ECAM and MSI.
> > > >>>
> > > >>>I'm not sure if this could be done prior to starting Xen. Perhaps.
> > > >>>If so, bootloaders would have to know a head of time what devices
> > > >>>the GTs are supposed to be configured for.
> > > >>
> > > >>I've got further questions regarding the Gigabit Transceivers. You 
> > > >>mention
> > > >>they are shared, do you mean that multiple devices can use a GT at the 
> > > >>same
> > > >>time? Or the software is deciding at startup which device will use a 
> > > >>given
> > > >>GT? If so, how does the software make this decision?
> > > >
> > > >Software will decide at startup. AFAIK, the allocation is normally done
> > > >once but I guess that in theory you could design boards that could switch
> > > >at runtime. I'm not sure we need to worry about that use-case though.
> > > >
> > > >The details can be found here:
> > > >https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf
> > > >
> > > >I suggest looking at pages 672 and 733.
> > > 
> > > Thank you for the documentation. I am trying to understand if we could 
> > > move
> > > initialization in Xen as suggested by Stefano. I looked at the driver in
> > > Linux and the code looks simple not many dependencies. However, I was not
> > > able to find where the Gigabit Transceivers are configured. Do you have 
> > > any
> > > link to the code for that?
> > 
> > Hi Julien,
> > 
> > I suspect that this setup has previously been done by the initial bootloader
> > auto-generated from design configuration tools.
> > 
> > Now, this is moving into Linux.
> > There's a specific driver that does that but AFAICS, it has not been 
> > upstreamed yet.
> > You can see it here:
> > https://github.com/Xilinx/linux-xlnx/blob/master/drivers/phy/phy-zynqmp.c
> > 
> > DTS nodes that need a PHY can then just refer to it, here's an example from 
> > SATA:
> >  {
> > phy-names = "sata-phy";
> > phys = < PHY_TYPE_SATA 1 3 15000>;
> > };
> > 
> > I'll see if I can find working examples for PCIe on the ZCU102. Then I'll 
> > share
> > DTS, Kernel etc.
> > 
> > If you are looking for a platform to get started, an option could be if I 
> > get you a build of
> > our QEMU that includes models for the PCIe controller, MSI and SMMU 
> > connections.
> > These models are friendly wrt. PHY configs and initialization sequences, it 
> > will
> > accept pretty much any sequence and still work. This would allow you to 
> > focus on
> > architectural issues rather than e

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Wed, Feb 01, 2017 at 07:04:43PM +, Julien Grall wrote:
> Hi Edgar,
> 
> On 31/01/2017 19:06, Edgar E. Iglesias wrote:
> >On Tue, Jan 31, 2017 at 05:09:53PM +, Julien Grall wrote:
> >>On 31/01/17 16:53, Edgar E. Iglesias wrote:
> >>>On Wed, Jan 25, 2017 at 06:53:20PM +, Julien Grall wrote:
> >>>>On 24/01/17 20:07, Stefano Stabellini wrote:
> >>>>>On Tue, 24 Jan 2017, Julien Grall wrote:
> >>>>For generic host bridge, the initialization is inexistent. However some 
> >>>>host
> >>>>bridge (e.g xgene, xilinx) may require some specific setup and also
> >>>>configuring clocks. Given that Xen only requires to access the 
> >>>>configuration
> >>>>space, I was thinking to let DOM0 initialization the host bridge. This 
> >>>>would
> >>>>avoid to import a lot of code in Xen, however this means that we need to
> >>>>know when the host bridge has been initialized before accessing the
> >>>>configuration space.
> >>>
> >>>
> >>>Yes, that's correct.
> >>>There's a sequence on the ZynqMP that involves assiging Gigabit 
> >>>Transceivers
> >>>to PCI (GTs are shared among PCIe, USB, SATA and the Display Port),
> >>>enabling clocks and configuring a few registers to enable ECAM and MSI.
> >>>
> >>>I'm not sure if this could be done prior to starting Xen. Perhaps.
> >>>If so, bootloaders would have to know a head of time what devices
> >>>the GTs are supposed to be configured for.
> >>
> >>I've got further questions regarding the Gigabit Transceivers. You mention
> >>they are shared, do you mean that multiple devices can use a GT at the same
> >>time? Or the software is deciding at startup which device will use a given
> >>GT? If so, how does the software make this decision?
> >
> >Software will decide at startup. AFAIK, the allocation is normally done
> >once but I guess that in theory you could design boards that could switch
> >at runtime. I'm not sure we need to worry about that use-case though.
> >
> >The details can be found here:
> >https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf
> >
> >I suggest looking at pages 672 and 733.
> 
> Thank you for the documentation. I am trying to understand if we could move
> initialization in Xen as suggested by Stefano. I looked at the driver in
> Linux and the code looks simple not many dependencies. However, I was not
> able to find where the Gigabit Transceivers are configured. Do you have any
> link to the code for that?

Hi Julien,

I suspect that this setup has previously been done by the initial bootloader
auto-generated from design configuration tools.

Now, this is moving into Linux.
There's a specific driver that does that but AFAICS, it has not been upstreamed 
yet.
You can see it here:
https://github.com/Xilinx/linux-xlnx/blob/master/drivers/phy/phy-zynqmp.c

DTS nodes that need a PHY can then just refer to it, here's an example from 
SATA:
 {
phy-names = "sata-phy";
phys = < PHY_TYPE_SATA 1 3 15000>;
};

I'll see if I can find working examples for PCIe on the ZCU102. Then I'll share
DTS, Kernel etc.

If you are looking for a platform to get started, an option could be if I get 
you a build of
our QEMU that includes models for the PCIe controller, MSI and SMMU connections.
These models are friendly wrt. PHY configs and initialization sequences, it will
accept pretty much any sequence and still work. This would allow you to focus on
architectural issues rather than exact details of init sequences (which we can
deal with later).



> 
> This would also mean that the MSI interrupt controller will be moved in Xen.
> Which I think is a more sensible design (see more below).
> 
> >>
> >>>>  - For all other host bridges => I don't know if there are host bridges
> >>>>falling under this category. I also don't have any idea how to handle 
> >>>>this.
> >>>>
> >>>>>
> >>>>>Otherwise, if Dom0 is the only one to drive the physical host bridge,
> >>>>>and Xen is the one to provide the emulated host bridge, how are DomU PCI
> >>>>>config reads and writes supposed to work in details?
> >>>>
> >>>>I think I have answered to this question with my explanation above. Let me
> >>>>know if it is not the case.
> >>>>
> >>>>>How is MSI configuration supposed to work?
> >>>>
> >>>>

Re: [Xen-devel] [PATCH v2 2/3] Revert "xen/arm: Map mmio-sram nodes as un-cached memory"

[Edgar E. Iglesias]

On Tue, Jan 31, 2017 at 02:18:40PM -0800, Stefano Stabellini wrote:
> On Tue, 31 Jan 2017, Julien Grall wrote:
> > Hi Edgar,
> > 
> > On 26/01/17 13:16, Edgar E. Iglesias wrote:
> > > From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> > > 
> > > This reverts commit 1e75ed8b64bc1a9b47e540e6f100f17ec6d97f1b.
> > 
> > I would add a word explain why it is reverted. Something like: "The default
> > attribute mapping for MMIO as been relaxed and now rely on the hardware 
> > domain
> > to set the correct memory attribute".
> > 
> > Also, your signed-off is missing.
> 
> Assuming Edgar is OK with this, I fixed it up and committed. Edgar,
> please let me know if it's a problem.

Ah, thanks!

Our emails crossed paths as I submitted v3 :-)

Feel free to keep the commited version and ignore v3.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 2/3] Revert "xen/arm: Map mmio-sram nodes as un-cached memory"

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This reverts commit 1e75ed8b64bc1a9b47e540e6f100f17ec6d97f1b.

The default attribute mappings for MMIO have been relaxed, we now rely
on the hardware domain to set correct memory attributes.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 30 --
 1 file changed, 30 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4fc467c..c97a1f5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -48,20 +48,6 @@ struct map_range_data
 p2m_type_t p2mt;
 };
 
-static const struct dt_device_match dev_map_attrs[] __initconst =
-{
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-__DT_MATCH_PROP("no-memory-wc"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_dev,
-},
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
-},
-{ /* sentinel */ },
-};
-
 //#define DEBUG_11_ALLOCATION
 #ifdef DEBUG_11_ALLOCATION
 # define D11PRINT(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
@@ -1170,21 +1156,6 @@ static int handle_device(struct domain *d, struct 
dt_device_node *dev,
 return 0;
 }
 
-static p2m_type_t lookup_map_attr(struct dt_device_node *node,
-  p2m_type_t parent_p2mt)
-{
-const struct dt_device_match *r;
-
-/* Search and if nothing matches, use the parent's attributes.  */
-r = dt_match_node(dev_map_attrs, node);
-
-/*
- * If this node does not dictate specific mapping attributes,
- * it inherits its parent's attributes.
- */
-return r ? (uintptr_t) r->data : parent_p2mt;
-}
-
 static int handle_node(struct domain *d, struct kernel_info *kinfo,
struct dt_device_node *node,
p2m_type_t p2mt)
@@ -1275,7 +1246,6 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
"WARNING: Path %s is reserved, skip the node as we may re-use 
the path.\n",
path);
 
-p2mt = lookup_map_attr(node, p2mt);
 res = handle_device(d, node, p2mt);
 if ( res)
 return res;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 0/3] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This relaxes the hw domains S2 mapping attributes to p2m_mmio_direct_c.

Eventually, I think we should allow guests to execute code from on chip
memories. We could follow-up with a patch that allows execution for
p2m_mmio_direct_c and p2m_mmio_direct_nc. Does that sound OK?

Cheers,
Edgar

ChangeLog:

v2 -> v3:
* Add rationale and SoB line to commit message of patch #2

v1 -> v2:
* Fixed typo in commit message
* Add rationale in commit message
* Keep p2m attributes plumbing
* Add relaxation of attributes for ACPI hw domains

Edgar E. Iglesias (3):
  xen/arm: dt: Relax hw domain mapping attributes to p2m_mmio_direct_c
  Revert "xen/arm: Map mmio-sram nodes as un-cached memory"
  xen/arm: acpi: Relax hw domain mapping attributes to p2m_mmio_direct_c

 xen/arch/arm/domain_build.c | 32 +---
 xen/arch/arm/p2m.c  |  2 +-
 xen/include/public/memory.h |  6 +++---
 3 files changed, 5 insertions(+), 35 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 3/3] xen/arm: acpi: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Since the hardware domain is a trusted domain, we extend the
trust to include making final decisions on what attributes to
use when mapping memory regions.

For ACPI configured hardware domains, this patch relaxes the hardware
domains mapping attributes to p2m_mmio_direct_c. This will allow the
hardware domain to control the attributes via its S1 mappings.

Acked-by: Julien Grall <julien.gr...@arm.com>
Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/p2m.c  | 2 +-
 xen/include/public/memory.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 09ceb37..cb5a45d 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1185,7 +1185,7 @@ int map_dev_mmio_region(struct domain *d,
 if ( !(nr && iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn) + nr - 1)) )
 return 0;
 
-res = map_mmio_regions(d, gfn, nr, mfn);
+res = p2m_insert_mapping(d, gfn, nr, mfn, p2m_mmio_direct_c);
 if ( res < 0 )
 {
 printk(XENLOG_G_ERR "Unable to map MFNs [%#"PRI_mfn" - %#"PRI_mfn" in 
Dom%d\n",
diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
index e633047..6eee0c8 100644
--- a/xen/include/public/memory.h
+++ b/xen/include/public/memory.h
@@ -222,9 +222,9 @@ DEFINE_XEN_GUEST_HANDLE(xen_machphys_mapping_t);
 * XENMEM_add_to_physmap_batch only. */
 #define XENMAPSPACE_dev_mmio 5 /* device mmio region
   ARM only; the region is mapped in
-  Stage-2 using the memory attribute
-  "Device-nGnRE" (previously named
-  "Device" on ARMv7) */
+  Stage-2 using the Normal Memory
+  Inner/Outer Write-Back Cacheable
+  memory attribute. */
 /* ` } */
 
 /*
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v3 1/3] xen/arm: dt: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Since the hardware domain is a trusted domain, we extend the
trust to include making final decisions on what attributes to
use when mapping memory regions.

For device-tree configured hardware domains, this patch relaxes
the hardware domains mapping attributes to p2m_mmio_direct_c.
This will allow the hardware domain to control the attributes
via its S1 mappings.

Reviewed-by: Julien Grall <julien.gr...@arm.com>
Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 63301e6..4fc467c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1330,7 +1330,7 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 
 static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
 {
-const p2m_type_t default_p2mt = p2m_mmio_direct_dev;
+const p2m_type_t default_p2mt = p2m_mmio_direct_c;
 const void *fdt;
 int new_size;
 int ret;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/3] Revert "xen/arm: Map mmio-sram nodes as un-cached memory"

[Edgar E. Iglesias]

On Tue, Jan 31, 2017 at 04:35:09PM +, Julien Grall wrote:
> Hi Edgar,
> 
> On 26/01/17 13:16, Edgar E. Iglesias wrote:
> >From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> >
> >This reverts commit 1e75ed8b64bc1a9b47e540e6f100f17ec6d97f1b.
> 
> I would add a word explain why it is reverted. Something like: "The default
> attribute mapping for MMIO as been relaxed and now rely on the hardware
> domain to set the correct memory attribute".
> 
> Also, your signed-off is missing.

Thanks Julien,

I've fixed this and will send out a v3 shortly.

Best regards,
Edgar


> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Tue, Jan 31, 2017 at 05:09:53PM +, Julien Grall wrote:
> Hi Edgar,
> 
> Thank you for the feedbacks.

Hi Julien,

> 
> On 31/01/17 16:53, Edgar E. Iglesias wrote:
> >On Wed, Jan 25, 2017 at 06:53:20PM +, Julien Grall wrote:
> >>On 24/01/17 20:07, Stefano Stabellini wrote:
> >>>On Tue, 24 Jan 2017, Julien Grall wrote:
> >>For generic host bridge, the initialization is inexistent. However some host
> >>bridge (e.g xgene, xilinx) may require some specific setup and also
> >>configuring clocks. Given that Xen only requires to access the configuration
> >>space, I was thinking to let DOM0 initialization the host bridge. This would
> >>avoid to import a lot of code in Xen, however this means that we need to
> >>know when the host bridge has been initialized before accessing the
> >>configuration space.
> >
> >
> >Yes, that's correct.
> >There's a sequence on the ZynqMP that involves assiging Gigabit Transceivers
> >to PCI (GTs are shared among PCIe, USB, SATA and the Display Port),
> >enabling clocks and configuring a few registers to enable ECAM and MSI.
> >
> >I'm not sure if this could be done prior to starting Xen. Perhaps.
> >If so, bootloaders would have to know a head of time what devices
> >the GTs are supposed to be configured for.
> 
> I've got further questions regarding the Gigabit Transceivers. You mention
> they are shared, do you mean that multiple devices can use a GT at the same
> time? Or the software is deciding at startup which device will use a given
> GT? If so, how does the software make this decision?

Software will decide at startup. AFAIK, the allocation is normally done
once but I guess that in theory you could design boards that could switch
at runtime. I'm not sure we need to worry about that use-case though.

The details can be found here:
https://www.xilinx.com/support/documentation/user_guides/ug1085-zynq-ultrascale-trm.pdf

I suggest looking at pages 672 and 733.



> 
> >>- For all other host bridges => I don't know if there are host bridges
> >>falling under this category. I also don't have any idea how to handle this.
> >>
> >>>
> >>>Otherwise, if Dom0 is the only one to drive the physical host bridge,
> >>>and Xen is the one to provide the emulated host bridge, how are DomU PCI
> >>>config reads and writes supposed to work in details?
> >>
> >>I think I have answered to this question with my explanation above. Let me
> >>know if it is not the case.
> >>
> >>> How is MSI configuration supposed to work?
> >>
> >>For GICv3 ITS, the MSI will be configured with the eventID (it is uniq
> >>per-device) and the address of the doorbell. The linkage between the LPI and
> >>"MSI" will be done through the ITS.
> >>
> >>For GICv2m, the MSI will be configured with an SPIs (or offset on some
> >>GICv2m) and the address of the doorbell. Note that for DOM0 SPIs are mapped
> >>1:1.
> >>
> >>So in both case, I don't think it is necessary to trap MSI configuration for
> >>DOM0. This may not be true if we want to handle other MSI controller.
> >>
> >>I have in mind the xilinx MSI controller (embedded in the host bridge? [4])
> >>and xgene MSI controller ([5]). But I have no idea how they work and if we
> >>need to support them. Maybe Edgar could share details on the Xilinx one?
> >
> >
> >The Xilinx controller has 2 dedicated SPIs and pages for MSIs. AFAIK, 
> >there's no
> >way to protect the MSI doorbells from mal-configured end-points raising 
> >malicious EventIDs.
> >So perhaps trapped config accesses from domUs can help by adding this 
> >protection
> >as drivers configure the device.
> >
> >On Linux, Once MSI's hit, the kernel takes the SPI interrupts, reads
> >out the EventID from a FIFO in the controller and injects a new IRQ into
> >the kernel.
> 
> It might be early to ask, but how do you expect  MSI to work with DOMU on
> your hardware? Does your MSI controller supports virtualization? Or are you
> looking for a different way to inject MSI?

MSI support in HW is quite limited to support domU and will require SW hacks :-(

Anyway, something along the lines of this might work:

* Trap domU CPU writes to MSI descriptors in config space.
  Force real MSI descriptors to the address of the door bell area.
  Force real MSI descriptors to use a specific device unique Event ID allocated 
by Xen.
  Remember what EventID domU requested per device and descriptor.

* Xen or Dom0 take the real SPI generated when device writes into the doorbell 
area.
  At this point, we can read out the EventID from the MSI FIFO and map it to 
the one requested from domU.
  Xen or Dom0 inject the expected EventID into domU

Do you have any good ideas? :-)

Cheers,
Edgar


> 
> >
> >I hope that helps!
> 
> It helped thank you!
> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Wed, Jan 25, 2017 at 06:53:20PM +, Julien Grall wrote:
> Hi Stefano,
> 
> On 24/01/17 20:07, Stefano Stabellini wrote:
> >On Tue, 24 Jan 2017, Julien Grall wrote:
> ## Discovering and register hostbridge
> 
> Both ACPI and Device Tree do not provide enough information to fully
> instantiate an host bridge driver. In the case of ACPI, some data may come
> from ASL,
> >>>
> >>>The data available from ASL is just to initialize quirks and non-ECAM
> >>>controllers, right? Given that SBSA mandates ECAM, and we assume that
> >>>ACPI is mostly (if not only) for servers, then I think it is safe to say
> >>>that in the case of ACPI we should have all the info to fully
> >>>instantiate an host bridge driver.
> >>
> >>From the spec, the MCFG will only describe host bridge available at boot 
> >>(see
> >>4.2 in "PCI firmware specification, rev 3.2"). All the other host bridges 
> >>will
> >>be described in ASL.
> >>
> >>So we need DOM0 to feed Xen about the latter host bridges.
> >
> >Unfortunately PCI specs are only accessible by PCI SIG members
> >organizations. In other words, I cannot read the doc.
> >
> >Could you please explain what kind of host bridges are not expected to
> >be available at boot? Do you know of any examples?
> 
> Roger answered to this answer in on a reply to this e-mail. So I will skip
> it. Let me know if you need for details.
> 
> >
> >
> whilst for Device Tree the segment number is not available.
> 
> So Xen needs to rely on DOM0 to discover the host bridges and notify Xen
> with all the relevant informations. This will be done via a new hypercall
> PHYSDEVOP_pci_host_bridge_add. The layout of the structure will be:
> >>>
> >>>I understand that the main purpose of this hypercall is to get Xen and Dom0
> >>>to
> >>>agree on the segment numbers, but why is it necessary? If Dom0 has an
> >>>emulated contoller like any other guest, do we care what segment numbers
> >>>Dom0 will use?
> >>
> >>I was not planning to have a emulated controller for DOM0. The physical one 
> >>is
> >>not necessarily ECAM compliant so we would have to either emulate the 
> >>physical
> >>one (meaning multiple different emulation) or an ECAM compliant.
> >>
> >>The latter is not possible because you don't know if there is enough free 
> >>MMIO
> >>space for the emulation.
> >>
> >>In the case on ARM, I don't see much the point to emulate the host bridge 
> >>for
> >>DOM0. The only thing we need in Xen is to access the configuration space, we
> >>don't have about driving the host bridge. So I would let DOM0 dealing with
> >>that.
> >>
> >>Also, I don't see any reason for ARM to trap DOM0 configuration space 
> >>access.
> >>The MSI will be configured using the interrupt controller and it is a 
> >>trusted
> >>Domain.
> >
> >These last you sentences raise a lot of questions. Maybe I am missing
> >something. You might want to clarify the strategy for Dom0 and DomUs,
> >and how they differ, in the next version of the doc.
> >
> >At some point you wrote "Instantiation of a specific driver for the host
> >controller can be easily done if Xen has the information to detect it.
> >However, those drivers may require resources described in ASL." Does it
> >mean you plan to drive the physical host bridge from Xen and Dom0
> >simultaneously?
> 
> I may miss some bits, so feel free to correct me if I am wrong.
> 
> My understanding is host bridge can be divided in 2 parts:
>   - Initialization of the host bridge
>   - Access the configuration space
> 
> For generic host bridge, the initialization is inexistent. However some host
> bridge (e.g xgene, xilinx) may require some specific setup and also
> configuring clocks. Given that Xen only requires to access the configuration
> space, I was thinking to let DOM0 initialization the host bridge. This would
> avoid to import a lot of code in Xen, however this means that we need to
> know when the host bridge has been initialized before accessing the
> configuration space.


Yes, that's correct.
There's a sequence on the ZynqMP that involves assiging Gigabit Transceivers
to PCI (GTs are shared among PCIe, USB, SATA and the Display Port),
enabling clocks and configuring a few registers to enable ECAM and MSI.

I'm not sure if this could be done prior to starting Xen. Perhaps.
If so, bootloaders would have to know a head of time what devices
the GTs are supposed to be configured for.



> 
> Now regarding the configuration space, I think we can divide in 2 category:
>   - indirect access, the configuration space are multiplexed. An example
> would be the legacy method on x86 (e.g 0xcf8 and 0xcfc). A similar method is
> used for x-gene PCI driver ([1]).
>   - ECAM like access, where each PCI configuration space will have it is 
> own
> address space. I said "ECAM like" because some host bridge will require some
> bits fiddling when accessing register (see thunder-ecam [2])
> 
> There are also host bridges that mix both indirect 

Re: [Xen-devel] [RFC v1 5/6] xen/arm: zynqmp: Forward plaform specific firmware calls

[Edgar E. Iglesias]

On Mon, Jan 30, 2017 at 06:21:54PM +0100, Edgar E. Iglesias wrote:
> From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> 
> Forward platform specific firmware calls from the hardware
> domain to firmware.
> 
> Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> ---
>  xen/arch/arm/platforms/xilinx-zynqmp.c | 63 
> ++
>  1 file changed, 63 insertions(+)
> 
> diff --git a/xen/arch/arm/platforms/xilinx-zynqmp.c 
> b/xen/arch/arm/platforms/xilinx-zynqmp.c
> index 2adee91..bde7f52 100644
> --- a/xen/arch/arm/platforms/xilinx-zynqmp.c
> +++ b/xen/arch/arm/platforms/xilinx-zynqmp.c
> @@ -19,6 +19,46 @@
>  
>  #include 
>  
> +/* Service calls.  */
> +#define SVC_MASK 0x3F00
> +#define SVC_SIP  0x0200/* SoC Implementation Specific.  
> */
> +
> +/* SMC function IDs for SiP Service queries */
> +#define ZYNQMP_SIP_SVC_CALL_COUNT   0xff00
> +#define ZYNQMP_SIP_SVC_UID  0xff01
> +#define ZYNQMP_SIP_SVC_VERSION  0xff03
> +
> +enum pm_api_id {
> +/* Miscellaneous API functions: */
> +GET_API_VERSION = 1,
> +SET_CONFIGURATION,

These pm_api_id enums are a left over and not needed, I'll remove them from
future versions of this patch.

Cheers,
Edgar


> +GET_NODE_STATUS,
> +GET_OPERATING_CHARACTERISTIC,
> +REGISTER_NOTIFIER,
> +/* API for suspending of PUs: */
> +REQUEST_SUSPEND,
> +SELF_SUSPEND,
> +FORCE_POWERDOWN,
> +ABORT_SUSPEND,
> +REQUEST_WAKEUP,
> +SET_WAKEUP_SOURCE,
> +SYSTEM_SHUTDOWN,
> +/* API for managing PM slaves: */
> +REQUEST_NODE,
> +RELEASE_NODE,
> +SET_REQUIREMENT,
> +SET_MAX_LATENCY,
> +/* Direct control API functions: */
> +RESET_ASSERT,
> +RESET_GET_STATUS,
> +MMIO_WRITE,
> +MMIO_READ,
> +PM_INIT,
> +FPGA_LOAD,
> +FPGA_GET_STATUS,
> +GET_CHIPID,
> +};
> +
>  static const char * const zynqmp_dt_compat[] __initconst =
>  {
>  "xlnx,zynqmp",
> @@ -32,8 +72,31 @@ static const struct dt_device_match zynqmp_blacklist_dev[] 
> __initconst =
>  { /* sentinel */ },
>  };
>  
> +bool zynqmp_hvc(struct cpu_user_regs *regs)
> +{
> +uint32_t fid = regs->x0;
> +uint32_t svc = fid & SVC_MASK;
> +register_t ret[4];
> +
> +/* We only forward SiP service calls from the hw domain to firmware.  */
> +if ( svc != SVC_SIP || !is_hardware_domain(current->domain) )
> +return false;
> +
> +/* Forward the call.  */
> +call_smcc64(regs->x0, regs->x1, regs->x2, regs->x3,
> +regs->x4, regs->x5, regs->x6, ret);
> +
> +/* Transfer return values into guest registers.  */
> +regs->x0 = ret[0];
> +regs->x1 = ret[1];
> +regs->x2 = ret[2];
> +regs->x3 = ret[3];
> +return true;
> +}
> +
>  PLATFORM_START(xgene_storm, "Xilinx ZynqMP")
>  .compatible = zynqmp_dt_compat,
> +.hvc = zynqmp_hvc,
>  .blacklist_dev = zynqmp_blacklist_dev,
>  PLATFORM_END
>  
> -- 
> 2.7.4
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 1/6] xen/arm: traps: Reorder early overwrite of FID

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Move the early setting of PSCI_RESULT_REG to a later stage
avoiding the early override of the FID that's stored in
the same register.

No functional change.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/traps.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 35d8e8b..a5fbf1e 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -1334,8 +1334,6 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 {
 register_t fid = PSCI_ARG(regs,0);
 
-/* preloading in case psci_mode_check fails */
-PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 switch( fid )
 {
 case PSCI_cpu_off:
@@ -1368,6 +1366,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_MIGRATE_INFO_UP_CPU:
 case PSCI_0_2_FN64_MIGRATE_INFO_UP_CPU:
 perfc_incr(vpsci_migrate_info_up_cpu);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 PSCI_RESULT_REG(regs) = do_psci_0_2_migrate_info_up_cpu();
 break;
@@ -1384,6 +1383,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_CPU_ON:
 case PSCI_0_2_FN64_CPU_ON:
 perfc_incr(vpsci_cpu_on);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 register_t vcpuid = PSCI_ARG(regs,1);
@@ -1396,6 +1396,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_CPU_SUSPEND:
 case PSCI_0_2_FN64_CPU_SUSPEND:
 perfc_incr(vpsci_cpu_suspend);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 uint32_t pstate = PSCI_ARG32(regs,1);
@@ -1408,6 +1409,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_AFFINITY_INFO:
 case PSCI_0_2_FN64_AFFINITY_INFO:
 perfc_incr(vpsci_cpu_affinity_info);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 register_t taff = PSCI_ARG(regs,1);
@@ -1419,6 +1421,7 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 case PSCI_0_2_FN_MIGRATE:
 case PSCI_0_2_FN64_MIGRATE:
 perfc_incr(vpsci_cpu_migrate);
+PSCI_RESULT_REG(regs) = PSCI_INVALID_PARAMETERS;
 if ( psci_mode_check(current->domain, fid) )
 {
 uint32_t tcpu = PSCI_ARG32(regs,1);
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 3/6] xen/arm: Allow platform_hvc to handle guest SMC calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Allow platform_hvc to handle guest SMC calls (as well as
HVC calls) in a platform specific way.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/traps.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index e8cd111..1925122 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -2622,6 +2622,9 @@ static void do_trap_smc(struct cpu_user_regs *regs, const 
union hsr hsr)
 if ( current->domain->arch.monitor.privileged_call_enabled )
 rc = monitor_smc();
 
+if ( platform_hvc(regs) )
+rc = 1;
+
 if ( rc != 1 )
 inject_undef_exception(regs, hsr);
 }
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 5/6] xen/arm: zynqmp: Forward plaform specific firmware calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Forward platform specific firmware calls from the hardware
domain to firmware.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platforms/xilinx-zynqmp.c | 63 ++
 1 file changed, 63 insertions(+)

diff --git a/xen/arch/arm/platforms/xilinx-zynqmp.c 
b/xen/arch/arm/platforms/xilinx-zynqmp.c
index 2adee91..bde7f52 100644
--- a/xen/arch/arm/platforms/xilinx-zynqmp.c
+++ b/xen/arch/arm/platforms/xilinx-zynqmp.c
@@ -19,6 +19,46 @@
 
 #include 
 
+/* Service calls.  */
+#define SVC_MASK 0x3F00
+#define SVC_SIP  0x0200/* SoC Implementation Specific.  */
+
+/* SMC function IDs for SiP Service queries */
+#define ZYNQMP_SIP_SVC_CALL_COUNT   0xff00
+#define ZYNQMP_SIP_SVC_UID  0xff01
+#define ZYNQMP_SIP_SVC_VERSION  0xff03
+
+enum pm_api_id {
+/* Miscellaneous API functions: */
+GET_API_VERSION = 1,
+SET_CONFIGURATION,
+GET_NODE_STATUS,
+GET_OPERATING_CHARACTERISTIC,
+REGISTER_NOTIFIER,
+/* API for suspending of PUs: */
+REQUEST_SUSPEND,
+SELF_SUSPEND,
+FORCE_POWERDOWN,
+ABORT_SUSPEND,
+REQUEST_WAKEUP,
+SET_WAKEUP_SOURCE,
+SYSTEM_SHUTDOWN,
+/* API for managing PM slaves: */
+REQUEST_NODE,
+RELEASE_NODE,
+SET_REQUIREMENT,
+SET_MAX_LATENCY,
+/* Direct control API functions: */
+RESET_ASSERT,
+RESET_GET_STATUS,
+MMIO_WRITE,
+MMIO_READ,
+PM_INIT,
+FPGA_LOAD,
+FPGA_GET_STATUS,
+GET_CHIPID,
+};
+
 static const char * const zynqmp_dt_compat[] __initconst =
 {
 "xlnx,zynqmp",
@@ -32,8 +72,31 @@ static const struct dt_device_match zynqmp_blacklist_dev[] 
__initconst =
 { /* sentinel */ },
 };
 
+bool zynqmp_hvc(struct cpu_user_regs *regs)
+{
+uint32_t fid = regs->x0;
+uint32_t svc = fid & SVC_MASK;
+register_t ret[4];
+
+/* We only forward SiP service calls from the hw domain to firmware.  */
+if ( svc != SVC_SIP || !is_hardware_domain(current->domain) )
+return false;
+
+/* Forward the call.  */
+call_smcc64(regs->x0, regs->x1, regs->x2, regs->x3,
+regs->x4, regs->x5, regs->x6, ret);
+
+/* Transfer return values into guest registers.  */
+regs->x0 = ret[0];
+regs->x1 = ret[1];
+regs->x2 = ret[2];
+regs->x3 = ret[3];
+return true;
+}
+
 PLATFORM_START(xgene_storm, "Xilinx ZynqMP")
 .compatible = zynqmp_dt_compat,
+.hvc = zynqmp_hvc,
 .blacklist_dev = zynqmp_blacklist_dev,
 PLATFORM_END
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 6/6] xen/arm: zynqmp: Remove blacklist of ZynqMP's PM node

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Stop blacklisting ZynqMP's power management node.
This is now possible since we allow the hardware domain to
issue HVC/SMC calls to firmware.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platforms/xilinx-zynqmp.c | 8 
 1 file changed, 8 deletions(-)

diff --git a/xen/arch/arm/platforms/xilinx-zynqmp.c 
b/xen/arch/arm/platforms/xilinx-zynqmp.c
index bde7f52..d21bd87 100644
--- a/xen/arch/arm/platforms/xilinx-zynqmp.c
+++ b/xen/arch/arm/platforms/xilinx-zynqmp.c
@@ -65,13 +65,6 @@ static const char * const zynqmp_dt_compat[] __initconst =
 NULL
 };
 
-static const struct dt_device_match zynqmp_blacklist_dev[] __initconst =
-{
-/* Power management is not yet supported.  */
-DT_MATCH_COMPATIBLE("xlnx,zynqmp-pm"),
-{ /* sentinel */ },
-};
-
 bool zynqmp_hvc(struct cpu_user_regs *regs)
 {
 uint32_t fid = regs->x0;
@@ -97,7 +90,6 @@ bool zynqmp_hvc(struct cpu_user_regs *regs)
 PLATFORM_START(xgene_storm, "Xilinx ZynqMP")
 .compatible = zynqmp_dt_compat,
 .hvc = zynqmp_hvc,
-.blacklist_dev = zynqmp_blacklist_dev,
 PLATFORM_END
 
 /*
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 4/6] xen/arm: Introduce call_smcc64

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Introduce call_smcc64, a helper function to issue 64-bit SMC
calls that follow the SMC Calling Convention. This includes
returning up to 4 64-bit values.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/include/asm-arm/processor.h | 40 
 1 file changed, 40 insertions(+)

diff --git a/xen/include/asm-arm/processor.h b/xen/include/asm-arm/processor.h
index afc0e9a..a604f8c 100644
--- a/xen/include/asm-arm/processor.h
+++ b/xen/include/asm-arm/processor.h
@@ -706,6 +706,46 @@ void vcpu_regs_user_to_hyp(struct vcpu *vcpu,
 int call_smc(register_t function_id, register_t arg0, register_t arg1,
  register_t arg2);
 
+/*
+ * Helper to issue a 64-bit SMC according to the SMC Calling Convention.
+ *
+ * @fid:  Function Identifier
+ * @a0 - a5:  6 arguments
+ * @ret:  Pointer to 4 register_t carrying return values
+ */
+static inline register_t call_smcc64(register_t fid,
+ register_t a0,
+ register_t a1,
+ register_t a2,
+ register_t a3,
+ register_t a4,
+ register_t a5,
+ register_t *ret)
+{
+register register_t x0 asm("x0") = fid;
+register register_t x1 asm("x1") = a0;
+register register_t x2 asm("x2") = a1;
+register register_t x3 asm("x3") = a2;
+register register_t x4 asm("x4") = a3;
+register register_t x5 asm("x5") = a4;
+register register_t x6 asm("x6") = a5;
+
+asm volatile ("smc #0\n"
+  : "+r" (x0), "+r" (x1), "+r" (x2), "+r" (x3),
+"+r" (x4), "+r" (x5), "+r" (x6)
+  :
+  : "x7", "x8", "x9", "x10", "x11", "x12",
+"x13", "x14", "x15", "x16", "x17" );
+
+if (ret) {
+ret[0] = x0;
+ret[1] = x1;
+ret[2] = x2;
+ret[3] = x3;
+}
+return x0;
+}
+
 void do_trap_guest_error(struct cpu_user_regs *regs);
 
 #endif /* __ASSEMBLY__ */
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 0/6] zynqmp: Add forwarding of platform specific firmware calls

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

The ZynqMP software stack has an extensive Firmware API.
Software at EL1 is increasingly dependant on this API. In fact,
a large set of use-cases for Linux already require access
to firmware. In a couple of months, we expect that most setups
will need it, so this is becoming somewhat urgent.

Services provided by the API, among other things include:
* Clock Management
* PHY Configuration
* Power Management
* FPGA configuration
* etc

Adding support for accessing firmware comes in two steps.

1. Allow the hw domain (dom0) to call into firmware. This enables
   all devices to work with PV setups.

2. Allow other guests to use the Firmware API. This will be needed
   to fully support Device pass-through.
   We cannot give un-supervised access to unprivileged guests, so
   the idea is to mediate these calls in Xen or preferably in
   dom0. Mediation mainly consists of ref-counting and filtering.

This RFC patch series implements step #1.

There are quite a few options on how to implement step #2, e.g:
1. Trap HVC/SMC calls from unpriv guests and handle them in dom0.
   This could be done in a user-space agent.
2. Add a PV protocol carrying these calls. This is similar to #1.
3. Add several generic PV protocols carrying a set of platform independent
   generic calls that get translated or mapped to firmware calls
   in dom0. E.g PV-Clock, PV-Power, PV-FPGA.
4. Others??

A benefit with #1 is that we could support unmodified bare-metal
guests. Such guests would continue to issue SMC calls from EL1 just as
if they were running without a hypervisor.

Best regards,
Edgar

Edgar E. Iglesias (6):
  xen/arm: traps: Reorder early overwrite of FID
  xen/arm: Introduce platform_hvc
  xen/arm: Allow platform_hvc to handle guest SMC calls
  xen/arm: Introduce call_smcc64
  xen/arm: zynqmp: Forward plaform specific firmware calls
  xen/arm: zynqmp: Remove blacklist of ZynqMP's PM node

 xen/arch/arm/platform.c|  8 
 xen/arch/arm/platforms/xilinx-zynqmp.c | 67 +++---
 xen/arch/arm/traps.c   | 13 ++-
 xen/include/asm-arm/platform.h |  5 +++
 xen/include/asm-arm/processor.h| 40 
 5 files changed, 125 insertions(+), 8 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC v1 2/6] xen/arm: Introduce platform_hvc

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Introduce platform_hvc as a way to handle hypercalls that
Xen does not know about in a platform specific way. This
is particularly useful for implementing the SiP (SoC
implementation specific) service calls.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/platform.c| 8 
 xen/arch/arm/traps.c   | 3 +++
 xen/include/asm-arm/platform.h | 5 +
 3 files changed, 16 insertions(+)

diff --git a/xen/arch/arm/platform.c b/xen/arch/arm/platform.c
index 0af6d57..90ea6b8 100644
--- a/xen/arch/arm/platform.c
+++ b/xen/arch/arm/platform.c
@@ -127,6 +127,14 @@ void platform_poweroff(void)
 platform->poweroff();
 }
 
+bool platform_hvc(struct cpu_user_regs *regs)
+{
+if ( platform && platform->hvc )
+return platform->hvc(regs);
+
+return false;
+}
+
 bool_t platform_has_quirk(uint32_t quirk)
 {
 uint32_t quirks = 0;
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index a5fbf1e..e8cd111 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -43,6 +43,7 @@
 #include 
 #include 
 #include 
+#include 
 
 #include "decode.h"
 #include "vtimer.h"
@@ -1429,6 +1430,8 @@ static void do_trap_psci(struct cpu_user_regs *regs)
 }
 break;
 default:
+if ( platform_hvc(regs) )
+return;
 domain_crash_synchronous();
 return;
 }
diff --git a/xen/include/asm-arm/platform.h b/xen/include/asm-arm/platform.h
index 08010ba..4d51f0a 100644
--- a/xen/include/asm-arm/platform.h
+++ b/xen/include/asm-arm/platform.h
@@ -26,6 +26,10 @@ struct platform_desc {
 void (*reset)(void);
 /* Platform power-off */
 void (*poweroff)(void);
+/* Platform specific HVC handler.
+ * Returns true if the call was handled and false if not.
+ */
+bool (*hvc)(struct cpu_user_regs *regs);
 /*
  * Platform quirks
  * Defined has a function because a platform can support multiple
@@ -55,6 +59,7 @@ int platform_cpu_up(int cpu);
 #endif
 void platform_reset(void);
 void platform_poweroff(void);
+bool platform_hvc(struct cpu_user_regs *regs);
 bool_t platform_has_quirk(uint32_t quirk);
 bool_t platform_device_is_blacklisted(const struct dt_device_node *node);
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

On Thu, Jan 26, 2017 at 12:58:52PM +, Julien Grall wrote:
> Hi Edgar,

Hi Julien,

> 
> On 26/01/2017 12:52, Edgar E. Iglesias wrote:
> >On Thu, Jan 19, 2017 at 12:40:45PM +, Julien Grall wrote:
> >>
> >>On 10/01/2017 11:37, Edgar E. Iglesias wrote:
> >>>From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> >>>
> >>>Relax the hardware domains mapping attributes to p2m_mmio_direct_c.
> >>>This will allow the hardware domain to fully control the
> >>>attribtues via its S1 mappings.
> >>
> >>s/attribtues/attributes/
> >
> >Fixed for v2.
> >
> >
> >>
> >>I would like some rationale in the commit message to explain why it is fine
> >>to do this relaxation (e.g the hardware domain is a trusted domain).
> >
> >I've added the following for v2:
> >Since the hardware domain is a trusted domain, we extend the
> >trust to include making final decisions on what attributes to
> >use when mapping memory regions.
> >
> >For device-tree configured hardware domains, this patch relaxes
> 
> I would drop the "For device-tree configured hardware domains" as you will
> also fix ACPI. The rest looks good to me.

Sorry, saw this a bit late but the series has one patch fixing DT and
another fixing ACPI. If you prefer I can squash them in a v3 or perhaps
Stefano can squash it as he commits them.


> 
> >the hardware domains mapping attributes to p2m_mmio_direct_c.
> >This will allow the hardware domain to control the attributes
> >via its S1 mappings.
> >
> >>
> >>A such relaxation would probably be necessary for the ACPI case too (see
> >>map_dev_mmio_region).
> >
> >I don't have testcases for ACPI but I'll try to fix it.
> >
> >Please correct me if I'm wrong. IIUC, when using ACPI, we map in a few
> >selected devices (UART, GIC, SMMU, RAM) to dom0 but leave the rest unmapped.
> >Dom0 then parses ACPI tables and issues hypervisor calls to map individual
> >devices (XENMEM_add_to_physmap with XENMAPSPACE_dev_mmio).
> 
> That is correct.
> 
> >
> >Since XENMEM_add_to_physmap with XENMAPSPACE_dev_mmio is only used
> >for dom0 mappings, I think this relaxation would be safe:
> >+++ b/xen/arch/arm/p2m.c
> >@@ -1185,7 +1185,7 @@ int map_dev_mmio_region(struct domain *d,
> > if ( !(nr && iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn) + nr - 
> > 1)) )
> > return 0;
> >
> >-res = map_mmio_regions(d, gfn, nr, mfn);
> >+res = p2m_insert_mapping(d, gfn, nr, mfn, p2m_mmio_direct_c);
> > if ( res < 0 )
> > {
> 
> This change looks good to me. I will give a try when the new version will be
> sent.

Thanks!

> >
> >Anyway, I'll send the v2 series out and we can discuss from there.
> 
> Can you also please modify the comment on "XENMEMSPACE_dev_mmio" in
> xen/include/public/memory.h regarding the memory attribute used to map?

Yes, I've updated the comment.

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 1/3] xen/arm: dt: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Since the hardware domain is a trusted domain, we extend the
trust to include making final decisions on what attributes to
use when mapping memory regions.

For device-tree configured hardware domains, this patch relaxes
the hardware domains mapping attributes to p2m_mmio_direct_c.
This will allow the hardware domain to control the attributes
via its S1 mappings.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 63301e6..4fc467c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1330,7 +1330,7 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 
 static int prepare_dtb(struct domain *d, struct kernel_info *kinfo)
 {
-const p2m_type_t default_p2mt = p2m_mmio_direct_dev;
+const p2m_type_t default_p2mt = p2m_mmio_direct_c;
 const void *fdt;
 int new_size;
 int ret;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 0/3] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This relaxes the hw domains S2 mapping attributes to p2m_mmio_direct_c.

Eventually, I think we should allow guests to execute code from on chip
memories. We could follow-up with a patch that allows execution for
p2m_mmio_direct_c and p2m_mmio_direct_nc. Does that sound OK?

Cheers,
Edgar

ChangeLog:

v1 -> v2:
* Fixed typo in commit message
* Add rationale in commit message
* Keep p2m attributes plumbing
* Add relaxation of attributes for ACPI hw domains

Edgar E. Iglesias (3):
  xen/arm: dt: Relax hw domain mapping attributes to p2m_mmio_direct_c
  Revert "xen/arm: Map mmio-sram nodes as un-cached memory"
  xen/arm: acpi: Relax hw domain mapping attributes to p2m_mmio_direct_c

 xen/arch/arm/domain_build.c | 32 +---
 xen/arch/arm/p2m.c  |  2 +-
 xen/include/public/memory.h |  6 +++---
 3 files changed, 5 insertions(+), 35 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 3/3] xen/arm: acpi: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Since the hardware domain is a trusted domain, we extend the
trust to include making final decisions on what attributes to
use when mapping memory regions.

For ACPI configured hardware domains, this patch relaxes the hardware
domains mapping attributes to p2m_mmio_direct_c. This will allow the
hardware domain to control the attributes via its S1 mappings.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/p2m.c  | 2 +-
 xen/include/public/memory.h | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 09ceb37..cb5a45d 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -1185,7 +1185,7 @@ int map_dev_mmio_region(struct domain *d,
 if ( !(nr && iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn) + nr - 1)) )
 return 0;
 
-res = map_mmio_regions(d, gfn, nr, mfn);
+res = p2m_insert_mapping(d, gfn, nr, mfn, p2m_mmio_direct_c);
 if ( res < 0 )
 {
 printk(XENLOG_G_ERR "Unable to map MFNs [%#"PRI_mfn" - %#"PRI_mfn" in 
Dom%d\n",
diff --git a/xen/include/public/memory.h b/xen/include/public/memory.h
index e633047..6eee0c8 100644
--- a/xen/include/public/memory.h
+++ b/xen/include/public/memory.h
@@ -222,9 +222,9 @@ DEFINE_XEN_GUEST_HANDLE(xen_machphys_mapping_t);
 * XENMEM_add_to_physmap_batch only. */
 #define XENMAPSPACE_dev_mmio 5 /* device mmio region
   ARM only; the region is mapped in
-  Stage-2 using the memory attribute
-  "Device-nGnRE" (previously named
-  "Device" on ARMv7) */
+  Stage-2 using the Normal Memory
+  Inner/Outer Write-Back Cacheable
+  memory attribute. */
 /* ` } */
 
 /*
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 2/3] Revert "xen/arm: Map mmio-sram nodes as un-cached memory"

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This reverts commit 1e75ed8b64bc1a9b47e540e6f100f17ec6d97f1b.
---
 xen/arch/arm/domain_build.c | 30 --
 1 file changed, 30 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 4fc467c..c97a1f5 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -48,20 +48,6 @@ struct map_range_data
 p2m_type_t p2mt;
 };
 
-static const struct dt_device_match dev_map_attrs[] __initconst =
-{
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-__DT_MATCH_PROP("no-memory-wc"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_dev,
-},
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
-},
-{ /* sentinel */ },
-};
-
 //#define DEBUG_11_ALLOCATION
 #ifdef DEBUG_11_ALLOCATION
 # define D11PRINT(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
@@ -1170,21 +1156,6 @@ static int handle_device(struct domain *d, struct 
dt_device_node *dev,
 return 0;
 }
 
-static p2m_type_t lookup_map_attr(struct dt_device_node *node,
-  p2m_type_t parent_p2mt)
-{
-const struct dt_device_match *r;
-
-/* Search and if nothing matches, use the parent's attributes.  */
-r = dt_match_node(dev_map_attrs, node);
-
-/*
- * If this node does not dictate specific mapping attributes,
- * it inherits its parent's attributes.
- */
-return r ? (uintptr_t) r->data : parent_p2mt;
-}
-
 static int handle_node(struct domain *d, struct kernel_info *kinfo,
struct dt_device_node *node,
p2m_type_t p2mt)
@@ -1275,7 +1246,6 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
"WARNING: Path %s is reserved, skip the node as we may re-use 
the path.\n",
path);
 
-p2mt = lookup_map_attr(node, p2mt);
 res = handle_device(d, node, p2mt);
 if ( res)
 return res;
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

On Thu, Jan 19, 2017 at 10:46:03AM -0800, Stefano Stabellini wrote:
> On Thu, 19 Jan 2017, Julien Grall wrote:
> > Hi Edgar,
> > 
> > On 10/01/2017 11:37, Edgar E. Iglesias wrote:
> > > From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> > > 
> > > Relax the hardware domains mapping attributes to p2m_mmio_direct_c.
> > > This will allow the hardware domain to fully control the
> > > attribtues via its S1 mappings.
> > 
> > s/attribtues/attributes/
> > 
> > I would like some rationale in the commit message to explain why it is fine 
> > to
> > do this relaxation (e.g the hardware domain is a trusted domain).
> > 
> > A such relaxation would probably be necessary for the ACPI case too (see
> > map_dev_mmio_region).
> > 
> > Also, this is a revert of patch 1e75ed8 and 9eed361 + relaxation, I would
> > either mention it in the commit message. Or send separate patch to revert 
> > both
> > of them. Either way will be fine by me.
> > 
> > > 
> > > Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> > > ---
> > >  xen/arch/arm/domain_build.c | 63
> > > ++---
> > >  1 file changed, 13 insertions(+), 50 deletions(-)
> > > 
> > > diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> > > index 07b868d..a3521c7 100644
> > > --- a/xen/arch/arm/domain_build.c
> > > +++ b/xen/arch/arm/domain_build.c
> > > @@ -48,20 +48,6 @@ struct map_range_data
> > >  p2m_type_t p2mt;
> > >  };
> > > 
> > > -static const struct dt_device_match dev_map_attrs[] __initconst =
> > > -{
> > > -{
> > > -__DT_MATCH_COMPATIBLE("mmio-sram"),
> > > -__DT_MATCH_PROP("no-memory-wc"),
> > > -.data = (void *) (uintptr_t) p2m_mmio_direct_dev,
> > > -},
> > > -{
> > > -__DT_MATCH_COMPATIBLE("mmio-sram"),
> > > -.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
> > > -},
> > > -{ /* sentinel */ },
> > > -};
> > > -
> > >  //#define DEBUG_11_ALLOCATION
> > >  #ifdef DEBUG_11_ALLOCATION
> > >  # define D11PRINT(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
> > > @@ -1005,8 +991,7 @@ static int map_range_to_domain(const struct
> > > dt_device_node *dev,
> > > u64 addr, u64 len,
> > > void *data)
> > >  {
> > > -struct map_range_data *mr_data = data;
> > 
> > I would actually prefer to keep the plumbing and only remove dev_map_attrs.
> > Stefano do you have any opinions?
> 
> I would keep the plumbings too (9eed361a), but I am fine either way.


Thanks, I've changed the patch to keep the plumbing from 9eed361a.

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

On Thu, Jan 19, 2017 at 12:40:45PM +, Julien Grall wrote:
> Hi Edgar,

Hi Julien,

> 
> On 10/01/2017 11:37, Edgar E. Iglesias wrote:
> >From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> >
> >Relax the hardware domains mapping attributes to p2m_mmio_direct_c.
> >This will allow the hardware domain to fully control the
> >attribtues via its S1 mappings.
> 
> s/attribtues/attributes/

Fixed for v2.


> 
> I would like some rationale in the commit message to explain why it is fine
> to do this relaxation (e.g the hardware domain is a trusted domain).

I've added the following for v2:
Since the hardware domain is a trusted domain, we extend the
trust to include making final decisions on what attributes to
use when mapping memory regions.

For device-tree configured hardware domains, this patch relaxes
the hardware domains mapping attributes to p2m_mmio_direct_c.
This will allow the hardware domain to control the attributes
via its S1 mappings.

> 
> A such relaxation would probably be necessary for the ACPI case too (see
> map_dev_mmio_region).

I don't have testcases for ACPI but I'll try to fix it.

Please correct me if I'm wrong. IIUC, when using ACPI, we map in a few
selected devices (UART, GIC, SMMU, RAM) to dom0 but leave the rest unmapped.
Dom0 then parses ACPI tables and issues hypervisor calls to map individual
devices (XENMEM_add_to_physmap with XENMAPSPACE_dev_mmio).

Since XENMEM_add_to_physmap with XENMAPSPACE_dev_mmio is only used
for dom0 mappings, I think this relaxation would be safe:
+++ b/xen/arch/arm/p2m.c
@@ -1185,7 +1185,7 @@ int map_dev_mmio_region(struct domain *d,
 if ( !(nr && iomem_access_permitted(d, mfn_x(mfn), mfn_x(mfn) + nr - 1)) )
 return 0;
 
-res = map_mmio_regions(d, gfn, nr, mfn);
+res = p2m_insert_mapping(d, gfn, nr, mfn, p2m_mmio_direct_c);
 if ( res < 0 )
 {

Anyway, I'll send the v2 series out and we can discuss from there.


> Also, this is a revert of patch 1e75ed8 and 9eed361 + relaxation, I would
> either mention it in the commit message. Or send separate patch to revert
> both of them. Either way will be fine by me.


I've changed v2 to keep the plumbing but revert 9eed361.

Thanks,
Edgar


> 
> >
> >Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> >---
> > xen/arch/arm/domain_build.c | 63 
> > ++---
> > 1 file changed, 13 insertions(+), 50 deletions(-)
> >
> >diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> >index 07b868d..a3521c7 100644
> >--- a/xen/arch/arm/domain_build.c
> >+++ b/xen/arch/arm/domain_build.c
> >@@ -48,20 +48,6 @@ struct map_range_data
> > p2m_type_t p2mt;
> > };
> >
> >-static const struct dt_device_match dev_map_attrs[] __initconst =
> >-{
> >-{
> >-__DT_MATCH_COMPATIBLE("mmio-sram"),
> >-__DT_MATCH_PROP("no-memory-wc"),
> >-.data = (void *) (uintptr_t) p2m_mmio_direct_dev,
> >-},
> >-{
> >-__DT_MATCH_COMPATIBLE("mmio-sram"),
> >-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
> >-},
> >-{ /* sentinel */ },
> >-};
> >-
> > //#define DEBUG_11_ALLOCATION
> > #ifdef DEBUG_11_ALLOCATION
> > # define D11PRINT(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
> >@@ -1005,8 +991,7 @@ static int map_range_to_domain(const struct 
> >dt_device_node *dev,
> >u64 addr, u64 len,
> >void *data)
> > {
> >-struct map_range_data *mr_data = data;
> 
> I would actually prefer to keep the plumbing and only remove dev_map_attrs.
> Stefano do you have any opinions?
> 
> If we are going to remove the plumbing, you would need to remove
> map_range_data which has been introduced by the plumbing patch.
> 
> Cheers,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 1/1] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Relax the hardware domains mapping attributes to p2m_mmio_direct_c.
This will allow the hardware domain to fully control the
attribtues via its S1 mappings.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 63 ++---
 1 file changed, 13 insertions(+), 50 deletions(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 07b868d..a3521c7 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -48,20 +48,6 @@ struct map_range_data
 p2m_type_t p2mt;
 };
 
-static const struct dt_device_match dev_map_attrs[] __initconst =
-{
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-__DT_MATCH_PROP("no-memory-wc"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_dev,
-},
-{
-__DT_MATCH_COMPATIBLE("mmio-sram"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
-},
-{ /* sentinel */ },
-};
-
 //#define DEBUG_11_ALLOCATION
 #ifdef DEBUG_11_ALLOCATION
 # define D11PRINT(fmt, args...) printk(XENLOG_DEBUG fmt, ##args)
@@ -1005,8 +991,7 @@ static int map_range_to_domain(const struct dt_device_node 
*dev,
u64 addr, u64 len,
void *data)
 {
-struct map_range_data *mr_data = data;
-struct domain *d = mr_data->d;
+struct domain *d = data;
 bool_t need_mapping = !dt_device_for_passthrough(dev);
 int res;
 
@@ -1027,7 +1012,7 @@ static int map_range_to_domain(const struct 
dt_device_node *dev,
_gfn(paddr_to_pfn(addr)),
DIV_ROUND_UP(len, PAGE_SIZE),
_mfn(paddr_to_pfn(addr)),
-   mr_data->p2mt);
+   p2m_mmio_direct_c);
 
 if ( res < 0 )
 {
@@ -1039,8 +1024,8 @@ static int map_range_to_domain(const struct 
dt_device_node *dev,
 }
 }
 
-dt_dprintk("  - MMIO: %010"PRIx64" - %010"PRIx64" P2MType=%x\n",
-   addr, addr + len, mr_data->p2mt);
+dt_dprintk("  - MMIO: %010"PRIx64" - %010"PRIx64"\n",
+   addr, addr + len);
 
 return 0;
 }
@@ -1051,10 +1036,8 @@ static int map_range_to_domain(const struct 
dt_device_node *dev,
  * the child resources available to domain 0.
  */
 static int map_device_children(struct domain *d,
-   const struct dt_device_node *dev,
-   p2m_type_t p2mt)
+   const struct dt_device_node *dev)
 {
-struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
 int ret;
 
 if ( dt_device_type_is_equal(dev, "pci") )
@@ -1066,7 +1049,7 @@ static int map_device_children(struct domain *d,
 if ( ret < 0 )
 return ret;
 
-ret = dt_for_each_range(dev, _range_to_domain, _data);
+ret = dt_for_each_range(dev, _range_to_domain, d);
 if ( ret < 0 )
 return ret;
 }
@@ -1082,8 +1065,7 @@ static int map_device_children(struct domain *d,
  *  - Assign the device to the guest if it's protected by an IOMMU
  *  - Map the IRQs and iomem regions to DOM0
  */
-static int handle_device(struct domain *d, struct dt_device_node *dev,
- p2m_type_t p2mt)
+static int handle_device(struct domain *d, struct dt_device_node *dev)
 {
 unsigned int nirq;
 unsigned int naddr;
@@ -1149,7 +1131,6 @@ static int handle_device(struct domain *d, struct 
dt_device_node *dev,
 /* Give permission and map MMIOs */
 for ( i = 0; i < naddr; i++ )
 {
-struct map_range_data mr_data = { .d = d, .p2mt = p2mt };
 res = dt_device_get_address(dev, i, , );
 if ( res )
 {
@@ -1158,36 +1139,20 @@ static int handle_device(struct domain *d, struct 
dt_device_node *dev,
 return res;
 }
 
-res = map_range_to_domain(dev, addr, size, _data);
+res = map_range_to_domain(dev, addr, size, d);
 if ( res )
 return res;
 }
 
-res = map_device_children(d, dev, p2mt);
+res = map_device_children(d, dev);
 if ( res )
 return res;
 
 return 0;
 }
 
-static p2m_type_t lookup_map_attr(struct dt_device_node *node,
-  p2m_type_t parent_p2mt)
-{
-const struct dt_device_match *r;
-
-/* Search and if nothing matches, use the parent's attributes.  */
-r = dt_match_node(dev_map_attrs, node);
-
-/*
- * If this node does not dictate specific mapping attributes,
- * it inherits its parent's attributes.
- */
-return r ? (uintptr_t) r->data : parent_p2mt;
-}
-
 static int handle_node(struct domain *d, struct kernel_info *kinfo,
-   stru

[Xen-devel] [PATCH v1 0/1] xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This relaxes the hw domains S2 mapping attributes to p2m_mmio_direct_c.

Eventually, I think we should allow guests to execute code from on chip
memories. We could follow-up with a patch that allows execution for
p2m_mmio_direct_c and p2m_mmio_direct_nc. Does that sound OK?

Cheers,
Edgar

Edgar E. Iglesias (1):
  xen/arm: Relax hw domain mapping attributes to p2m_mmio_direct_c

 xen/arch/arm/domain_build.c | 63 ++---
 1 file changed, 13 insertions(+), 50 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Fri, Jan 06, 2017 at 01:12:44PM -0800, Stefano Stabellini wrote:
> On Fri, 6 Jan 2017, Edgar E. Iglesias wrote:
> > On Thu, Dec 29, 2016 at 02:04:15PM +, Julien Grall wrote:
> > > Hi all,
> > > 
> > > The document below is an early version of a design
> > > proposal for PCI Passthrough in Xen. It aims to
> > > describe from an high level perspective the interaction
> > > with the different subsystems and how guest will be able
> > > to discover and access PCI.
> > > 
> > > I am aware that a similar design has been posted recently
> > > by Cavium (see [1]), however the approach to expose PCI
> > > to guest is different. We have request to run unmodified
> > > baremetal OS on Xen, a such guest would directly
> > > access the devices and no PV drivers will be used.
> > > 
> > > That's why this design is based on emulating a root controller.
> > > This also has the advantage to have the VM interface as close
> > > as baremetal allowing the guest to use firmware tables to discover
> > > the devices.
> > > 
> > > Currently on ARM, Xen does not have any knowledge about PCI devices.
> > > This means that IOMMU and interrupt controller (such as ITS)
> > > requiring specific configuration will not work with PCI even with
> > > DOM0.
> > > 
> > > The PCI Passthrough work could be divided in 2 phases:
> > >   * Phase 1: Register all PCI devices in Xen => will allow
> > >  to use ITS and SMMU with PCI in Xen
> > > * Phase 2: Assign devices to guests
> > > 
> > > This document aims to describe the 2 phases, but for now only phase
> > > 1 is fully described.
> > 
> > Thanks Julien,
> > 
> > A question.
> > IIUC, Dom0 will own the real host bridge and DomUs will access a
> > virtual emulated one.
> > In the case of an ECAM compatible host bridge that only needs to
> > be initialized via a host bridge specific register sequence,
> > do I understand correctly that the amount of emulation would be
> > very small (just enough to fool the guest that the init sequence
> > passed). Beyond that we could have a generic ECAM emulator/mappings?
> 
> I think so.
> 
> 
> > How will we handle BAR setups?
> > Will we filter and make sure guests don't try to do funny stuff?
> > Perhaps Xen already has code for this (I'm guessing it does).
> 
> Yes, we'll have to filter guest accesses. There is already some code in
> Xen to do that, especially in regard to MSI and MSI-X setup.
> 

Thanks for clarifying!

I think the proposal looks good so far.

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [early RFC] ARM PCI Passthrough design document

[Edgar E. Iglesias]

On Thu, Dec 29, 2016 at 02:04:15PM +, Julien Grall wrote:
> Hi all,
> 
> The document below is an early version of a design
> proposal for PCI Passthrough in Xen. It aims to
> describe from an high level perspective the interaction
> with the different subsystems and how guest will be able
> to discover and access PCI.
> 
> I am aware that a similar design has been posted recently
> by Cavium (see [1]), however the approach to expose PCI
> to guest is different. We have request to run unmodified
> baremetal OS on Xen, a such guest would directly
> access the devices and no PV drivers will be used.
> 
> That's why this design is based on emulating a root controller.
> This also has the advantage to have the VM interface as close
> as baremetal allowing the guest to use firmware tables to discover
> the devices.
> 
> Currently on ARM, Xen does not have any knowledge about PCI devices.
> This means that IOMMU and interrupt controller (such as ITS)
> requiring specific configuration will not work with PCI even with
> DOM0.
> 
> The PCI Passthrough work could be divided in 2 phases:
>   * Phase 1: Register all PCI devices in Xen => will allow
>  to use ITS and SMMU with PCI in Xen
> * Phase 2: Assign devices to guests
> 
> This document aims to describe the 2 phases, but for now only phase
> 1 is fully described.

Thanks Julien,

A question.
IIUC, Dom0 will own the real host bridge and DomUs will access a
virtual emulated one.
In the case of an ECAM compatible host bridge that only needs to
be initialized via a host bridge specific register sequence,
do I understand correctly that the amount of emulation would be
very small (just enough to fool the guest that the init sequence
passed). Beyond that we could have a generic ECAM emulator/mappings?

How will we handle BAR setups?
Will we filter and make sure guests don't try to do funny stuff?
Perhaps Xen already has code for this (I'm guessing it does).

Thanks,
Edgar



> 
> I have sent the design document to start to gather feedback on
> phase 1.
> 
> Cheers,
> 
> [1] https://lists.xen.org/archives/html/xen-devel/2016-12/msg00224.html 
> 
> 
> % PCI pass-through support on ARM
> % Julien Grall 
> % Draft A
> 
> # Preface
> 
> This document aims to describe the components required to enable PCI
> passthrough on ARM.
> 
> This is an early draft and some questions are still unanswered, when this is
> the case the text will contain XXX.
> 
> # Introduction
> 
> PCI passthrough allows to give control of physical PCI devices to guest. This
> means that the guest will have full and direct access to the PCI device.
> 
> ARM is supporting one kind of guest that is exploiting as much as possible
> virtualization support in hardware. The guest will rely on PV driver only
> for IO (e.g block, network), interrupts will come through the virtualized
> interrupt controller. This means that there are no big changes required
> within the kernel.
> 
> By consequence, it would be possible to replace PV drivers by assigning real
> devices to the guest for I/O access. Xen on ARM would therefore be able to
> run unmodified operating system.
> 
> To achieve this goal, it looks more sensible to go towards emulating the
> host bridge (we will go into more details later). A guest would be able
> to take advantage of the firmware tables and obviating the need for a specific
> driver for Xen.
> 
> Thus in this document we follow the emulated host bridge approach.
> 
> # PCI terminologies
> 
> Each PCI device under a host bridge is uniquely identified by its Requester ID
> (AKA RID). A Requester ID is a triplet of Bus number, Device number, and
> Function.
> 
> When the platform has multiple host bridges, the software can add fourth
> number called Segment to differentiate host bridges. A PCI device will
> then uniquely by segment:bus:device:function (AKA SBDF).
> 
> So given a specific SBDF, it would be possible to find the host bridge and the
> RID associated to a PCI device.
> 
> # Interaction of the PCI subsystem with other subsystems
> 
> In order to have a PCI device fully working, Xen will need to configure
> other subsystems subsytems such as the SMMU and the Interrupt Controller.
> 
> The interaction expected between the PCI subsystem and the other is:
> * Add a device
> * Remove a device
> * Assign a device to a guest
> * Deassign a device from a guest
> 
> XXX: Detail the interaction when assigning/deassigning device
> 
> The following subsections will briefly describe the interaction from an
> higher level perspective. Implementation details (callback, structure...)
> is out of scope.
> 
> ## SMMU
> 
> The SMMU will be used to isolate the PCI device when accessing the memory
> (for instance DMA and MSI Doorbells). Often the SMMU will be configured using
> a StreamID (SID) that can be deduced from the RID with the help of the 
> firmware
> tables (see below).
> 
> Whilst in theory all 

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

On Mon, Dec 19, 2016 at 04:01:00PM -0800, Stefano Stabellini wrote:
> On Mon, 19 Dec 2016, Julien Grall wrote:
> > Hi Edgar,
> > 
> > On 16/12/2016 18:04, Edgar E. Iglesias wrote:
> > > On Fri, Dec 16, 2016 at 04:12:00PM +, Julien Grall wrote:
> > > > On 15/12/16 11:26, Edgar E. Iglesias wrote:
> > > > > From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> > > > > 
> > > > > Relax the mapping of mmio-sram nodes that do not set the
> > > > > no-memory-wc property to cached normal memory.
> > > > > 
> > > > > Rationale:
> > > > > Allthough on chip memories are relatively fast compared to
> > > > 
> > > > s/Allthough/Although/
> > > 
> > > Fixed for v2.
> > > 
> > > > 
> > > > > off-chip memories, large OCMs are still significantly slower
> > > > 
> > > > May I ask what does OCMs mean?
> > > 
> > > It means On Chip Memories. I can spell it out in v2.
> > 
> > Yes please. I was not able to find the acronym with a quick search Google.
> > 
> > [...]
> > 
> > > > I consider it as the most trusted domain and has other way to mess up 
> > > > the
> > > > platform. So I am fine with this relaxation for the hardware domain (AKA
> > > > DOM0).
> > > > 
> > > > However, I have the feeling that we need this kind of relaxation for 
> > > > many
> > > > devices. For instance prefetchable memory BARs for PCI would have to be
> > > > cacheable, same for integrated graphic cards.
> > > 
> > > Yes, I agree.
> > > 
> > > > 
> > > > I am wondering whether for DOM0 only (*not the other guests), we should
> > > > relax the default stage-2 attribute mapping to p2m_mmio_direct_c.
> > > > 
> > > > So we would let DOM0 in charge of the final attribute. This may boost 
> > > > the
> > > > performance of memory access in DOM0.
> > > > 
> > > > Any opinions?
> > > 
> > > I think it makes sense. We had a discussiong about it a while back ago:
> > > https://lists.xenproject.org/archives/html/xen-devel/2015-05/msg02349.html
> > > 
> > > The concerns that were raised were wether there could be devices that
> > > could behave badly and possibly giving dom0 access to trig undefined or
> > > unpredictable behavior that could be exploited.
> > 
> > I think it would be fine for DOM0. It is already a trusted domain and have
> > other way to take down the platform.
> > 
> > > 
> > > If dom0 accesses devices through a cache, access patterns will change.
> > > In theory it may trig unexpected behaviour in some device. It's hard
> > > to say unless talking about specific chips and implementations.
> > > 
> > > For example, given dom0 access to a DMA capable device may also achieve
> > > the same. I.e access patterns from DMA units differ from the ones
> > > originating from a CPU using uncached device memory. It could trig
> > > similar kind of errors.
> > 
> > Another example is having the device mapped with different in 2 places with
> > different cacheability attributes. The data accessed would not be coherent.
> > But I believe that should not lead to a security issue.
> > 
> > > 
> > > It would be interesting to see a concrete example of a problematic
> > > system.
> > 
> > I believe it would depend a lot on how the platform have been designed.
> > 
> > I think we have two choices to go forward:
> > 1) Relax the memory attribute on case by case. DOM0 would not be able
> > to exploit potential undefined behavior. However, this means that code is 
> > been
> > added for any new device (e.g compatible string).
> > 2) Relax the memory attribute on every case. DOM0 would be able to
> > exploit potential undefined behavior. On the benefit side, every devices can
> > be used at its full performance without any change required in Xen.
> > 
> > In the case of ACPI, we rely on DOM0 to provide the correct mapping 
> > attribute
> > when asking to do the stage-2 mapping (see XENMAPSPACE_dev_mmio). Note that
> > currently, the MMIO are always mapped with the attribute Device-nGnRE. There
> > is plan to change that in the future. So ACPI is implementing 2).
> > 
> > Device Tree is currently implementing 1). But I would like to see the 
> > behavior
> > of Xen the same no matter the firmware tables used. So I would lean towards
> > 2).
> 
> That's fine by me.


OK, I've sent a v2 of the mmio-sram patch to avoid delays with that part.

But now that we seem to be in agreement with going for relaxing the default
S2 mappings, perhaps we should ditch the whitelist/blacklist mechanism?
Or does someone see a need for a blacklist?

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 0/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This patch changes the mapping from non-cached to cached
for mmio-sram nodes that do not have the no-memory-wc property.
This is a hang-over from 4.8 since the mmio-sram patches went
in late in the cycle.

I've explained the rationale in the commit message:
Although on chip memories are relatively fast compared to
off-chip memories, large on chip memories are still
significantly slower than L1 caches. Depending on the
memory, either cached or uncached may make most sense.

Also, hardware domains may like to use the memory in a
cache-coherent way to avoid SW managed coherency.

By mapping it cached at S2, we let hardware domains select
cacheability via S1 mappings.

This was also discussed here:
https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg02412.html

Were we reached the conclusion that we should be using the most relaxed
attributes as long as we don't compromise security.

Since mmio-sram DTS noders have a property (no-memory-wc) to describe
srams with access restrictions, we can select the most relaxed mode
for unrestricted ones.

Cheers,
Edgar

ChangeLog:

v1 -> v2:
* Spell out On Chip Memories (OCMs)
* Avoid dom0 in preference of hardware domains
* Fix spelling of although

Edgar E. Iglesias (1):
  xen/arm: Map mmio-sram nodes as cached memory

 xen/arch/arm/domain_build.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v2 1/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Relax the mapping of mmio-sram nodes that do not set the
no-memory-wc property to cached normal memory.

Rationale:
Although on chip memories are relatively fast compared to
off-chip memories, large on chip memories are still
significantly slower than L1 caches. Depending on the
memory, either cached or uncached may make most sense.

Also, hardware domains may like to use the memory in a
cache-coherent way to avoid SW managed coherency.

By mapping it cached at S2, we let hardware domains select
cacheability via S1 mappings.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 07b868d..08646b8 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -56,8 +56,20 @@ static const struct dt_device_match dev_map_attrs[] 
__initconst =
 .data = (void *) (uintptr_t) p2m_mmio_direct_dev,
 },
 {
+/*
+ * Although on chip memories are relatively fast compared to
+ * off-chip memories, large on chip memories are still
+ * significantly slower than L1 caches. Depending on the
+ * memory, either cached or uncached may make most sense.
+ *
+ * Also, hardware domains may like to use the memory in a
+ * cache-coherent way to avoid SW managed coherency.
+ *
+ * By mapping it cached at S2, we let hardware domains select
+ * cacheability via S1 mappings.
+ */
 __DT_MATCH_COMPATIBLE("mmio-sram"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
+.data = (void *) (uintptr_t) p2m_mmio_direct_c,
 },
 { /* sentinel */ },
 };
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

On Fri, Dec 16, 2016 at 04:12:00PM +, Julien Grall wrote:
> Hi Edgar,
> 
> On 15/12/16 11:26, Edgar E. Iglesias wrote:
> >From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> >
> >Relax the mapping of mmio-sram nodes that do not set the
> >no-memory-wc property to cached normal memory.
> >
> >Rationale:
> >Allthough on chip memories are relatively fast compared to
> 
> s/Allthough/Although/

Fixed for v2.

> 
> >off-chip memories, large OCMs are still significantly slower
> 
> May I ask what does OCMs mean?

It means On Chip Memories. I can spell it out in v2.

> 
> >than L1 caches. Depending on the memory, either cached or
> >uncached may make most sense.
> >
> >Also, dom0 may like to use the memory in a cache-coherent way
> >to avoid SW managed coherency.
> >
> >By mapping it cached at S2, we let dom0 select cacheability
> >via S1 mappings.
> >
> >Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> >---
> > xen/arch/arm/domain_build.c | 14 +-
> > 1 file changed, 13 insertions(+), 1 deletion(-)
> >
> >diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
> >index 07b868d..3a7c1ee 100644
> >--- a/xen/arch/arm/domain_build.c
> >+++ b/xen/arch/arm/domain_build.c
> >@@ -56,8 +56,20 @@ static const struct dt_device_match dev_map_attrs[] 
> >__initconst =
> > .data = (void *) (uintptr_t) p2m_mmio_direct_dev,
> > },
> > {
> >+/*
> >+ * Allthough on chip memories are relatively fast compared to
> 
> s/Allthough/Although/

Fixed for v2.

> 
> >+ * off-chip memories, large OCMs are still significantly slower
> >+ * than L1 caches. Depending on the memory, either cached or
> >+ * uncached may make most sense.
> >+ *
> >+ * Also, dom0 may like to use the memory in a cache-coherent
> 
> We prefer to use the term "hardware domain" rather than DOM0 when possible
> in the code.

Sounds good, I've changed it in code and commit msg for v2.

> 
> >+ * way to avoid SW managed coherency.
> >+ *
> >+ * By mapping it cached at S2, we let dom0 select cacheability
> >+ * via S1 mappings.
> >+ */
> > __DT_MATCH_COMPATIBLE("mmio-sram"),
> >-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
> >+.data = (void *) (uintptr_t) p2m_mmio_direct_c,
> 
> I consider it as the most trusted domain and has other way to mess up the
> platform. So I am fine with this relaxation for the hardware domain (AKA
> DOM0).
> 
> However, I have the feeling that we need this kind of relaxation for many
> devices. For instance prefetchable memory BARs for PCI would have to be
> cacheable, same for integrated graphic cards.

Yes, I agree.

> 
> I am wondering whether for DOM0 only (*not the other guests), we should
> relax the default stage-2 attribute mapping to p2m_mmio_direct_c.
> 
> So we would let DOM0 in charge of the final attribute. This may boost the
> performance of memory access in DOM0.
> 
> Any opinions?

I think it makes sense. We had a discussiong about it a while back ago:
https://lists.xenproject.org/archives/html/xen-devel/2015-05/msg02349.html

The concerns that were raised were wether there could be devices that
could behave badly and possibly giving dom0 access to trig undefined or
unpredictable behavior that could be exploited.

If dom0 accesses devices through a cache, access patterns will change.
In theory it may trig unexpected behaviour in some device. It's hard
to say unless talking about specific chips and implementations.

For example, given dom0 access to a DMA capable device may also achieve
the same. I.e access patterns from DMA units differ from the ones
originating from a CPU using uncached device memory. It could trig
similar kind of errors.

It would be interesting to see a concrete example of a problematic
system.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 2/2] tools/xenstat: Don't disable xentop when cross-compiling

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This partially reverts 16504669c5cbb8b195d20412aadc838da5c428f7
since xentop cross-compiles fine.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 tools/xenstat/Makefile | 4 
 1 file changed, 4 deletions(-)

diff --git a/tools/xenstat/Makefile b/tools/xenstat/Makefile
index 09ec1b7..90d033a 100644
--- a/tools/xenstat/Makefile
+++ b/tools/xenstat/Makefile
@@ -3,11 +3,7 @@ include $(XEN_ROOT)/tools/Rules.mk
 
 SUBDIRS :=
 SUBDIRS += libxenstat
-
-# This doesn't cross-compile (cross-compile environments rarely have curses)
-ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
 SUBDIRS += xentop
-endif
 
 .PHONY: all install clean distclean
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 0/2] tools: Allow cross-compilation of xentop

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

We usually cross-compile the Xen tools for ARM in Yocto, including
support for xentop.

These patches remove two blockers in the Makefiles for cross-compiled
xentop.

Cheers,
Edgar

Edgar E. Iglesias (2):
  tools/xenstat: Remove redundant check for curses.h
  tools/xenstat: Don't disable xentop when cross-compiling

 tools/xenstat/Makefile | 6 --
 1 file changed, 6 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 1/2] tools/xenstat: Remove redundant check for curses.h

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This check for curses.h does not consider cross-compilation.
It only checks host paths.

Luckily, commit 65da4913214120ddc95bd846cb3649a29f87146a
introduced proper configure checks for ncurses so we can
remove the redundant check in the Makefile.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 tools/xenstat/Makefile | 2 --
 1 file changed, 2 deletions(-)

diff --git a/tools/xenstat/Makefile b/tools/xenstat/Makefile
index 901be4a..09ec1b7 100644
--- a/tools/xenstat/Makefile
+++ b/tools/xenstat/Makefile
@@ -6,10 +6,8 @@ SUBDIRS += libxenstat
 
 # This doesn't cross-compile (cross-compile environments rarely have curses)
 ifeq ($(XEN_COMPILE_ARCH),$(XEN_TARGET_ARCH))
-ifeq ($(wildcard /usr/include/curses.h),/usr/include/curses.h)
 SUBDIRS += xentop
 endif
-endif
 
 .PHONY: all install clean distclean
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 1/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Relax the mapping of mmio-sram nodes that do not set the
no-memory-wc property to cached normal memory.

Rationale:
Allthough on chip memories are relatively fast compared to
off-chip memories, large OCMs are still significantly slower
than L1 caches. Depending on the memory, either cached or
uncached may make most sense.

Also, dom0 may like to use the memory in a cache-coherent way
to avoid SW managed coherency.

By mapping it cached at S2, we let dom0 select cacheability
via S1 mappings.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index 07b868d..3a7c1ee 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -56,8 +56,20 @@ static const struct dt_device_match dev_map_attrs[] 
__initconst =
 .data = (void *) (uintptr_t) p2m_mmio_direct_dev,
 },
 {
+/*
+ * Allthough on chip memories are relatively fast compared to
+ * off-chip memories, large OCMs are still significantly slower
+ * than L1 caches. Depending on the memory, either cached or
+ * uncached may make most sense.
+ *
+ * Also, dom0 may like to use the memory in a cache-coherent
+ * way to avoid SW managed coherency.
+ *
+ * By mapping it cached at S2, we let dom0 select cacheability
+ * via S1 mappings.
+ */
 __DT_MATCH_COMPATIBLE("mmio-sram"),
-.data = (void *) (uintptr_t) p2m_mmio_direct_nc,
+.data = (void *) (uintptr_t) p2m_mmio_direct_c,
 },
 { /* sentinel */ },
 };
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 0/1] xen/arm: Map mmio-sram nodes as cached memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This patch changes the mapping from non-cached to cached
for mmio-sram nodes that do not have the no-memory-wc property.
This is a hang-over from 4.8 since the mmio-sram patches went
in late in the cycle.

I've explained the rationale in the commit message:
Rationale:
Allthough on chip memories are relatively fast compared to
off-chip memories, large OCMs are still significantly slower
than L1 caches. Depending on the memory, either cached or
uncached may make most sense.

Also, dom0 may like to use the memory in a cache-coherent way
to avoid SW managed coherency.

By mapping it cached at S2, we let dom0 select cacheability
via S1 mappings.

This was also discussed here:
https://lists.xenproject.org/archives/html/xen-devel/2016-05/msg02412.html

Cheers,
Edgar

Edgar E. Iglesias (1):
  xen/arm: Map mmio-sram nodes as cached memory

 xen/arch/arm/domain_build.c | 14 +-
 1 file changed, 13 insertions(+), 1 deletion(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM community call - meeting minutes and date for the next one

[Edgar E. Iglesias]

On Wed, Dec 14, 2016 at 03:59:00PM +0100, Dario Faggioli wrote:
> On Tue, 2016-12-13 at 19:00 +, Julien Grall wrote:
> > Hi all,
> > 
> > Stefano suggested to move the call at 5pm and I haven't seen any 
> > disagreement.
> > 
> > So the call will be tomorrow (Wednesday 14th December at 5pm).


Hi all,

Thanks for the call today. I'm sending a link to EEMI the
power management framework that Xilinx co-developed with Aggios.
I don't have a link to the ARM specs for SCMI. I found some slides
though. Perhaps Julien has specs?

SCMI:
http://www.slideshare.net/linaroorg/las16200-scmi-system-management-and-control-interface

EEMI:
https://www.xilinx.com/support/documentation/user_guides/ug1200-eemi-api.pdf

Roughly, EEMI works top down by having the various SW layers request
power down/up and other settings to the layer beneath. The various
layers are responsible for recounting so that we don't shut down
something that is in use and also to isolate so that we don't
turn on or off anything that we don't own.

At the moment we tunnel API calls via SMC calls to EL3. For calls
that ATF at EL3 can implement it answers directly. If not, these
calls further propagate over a shared memory / IPI interface to
a dedicated tiny core (PMU) with power management firmware.

The PMU is responsible for mediating power/clock request changes
and making sure that the requesters don't mess with devices or
settings that don't belong to them.

So top down the idea is:

EL0  User-space uses devices (open/close or whatever).
EL1  OS/Kernel refcounts per process and requests power down/up of devs when 
not used.
EL2  Hypervisor (or dom0 or something not yet implemented) refcounts per guest 
and forwards the request down the chain if appropriate.
EL3  Trusted Firmware mediates between Normal and Secure world.

PMU  Mediates between different CPU clusters and Programmable logic.

Best regards,
Edgar





___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM community call - meeting minutes and date for the next one

[Edgar E. Iglesias]

On Tue, Dec 06, 2016 at 11:02:20AM -0800, Stefano Stabellini wrote:
> On Tue, 6 Dec 2016, Julien Grall wrote:
> > Hi all,
> > 
> > Apologies for the late sending. You will find at the end of the mail a 
> > summary
> > of the discussion. Feel free to reply if I missed some parts.
> > 
> > At the end of the call we agreed to have another meeting before the end of 
> > the
> > year. Given that the Christmas is approaching and some people may take 
> > holidays
> > around, I would suggest to do the meeting on Wednesday 14th December at 4pm
> > UTC. Any opinions?
> 
> Would it be possible to move this call to 5pm UTC (this one, not
> necessarily all the future calls)?

Hi All,

I'm up for a call the 14th. 4 or 5pm UTC both work for me.

Best regards,
Edgar


> 
> 
> 
> > Also, do you have any specific topic you would like to talk during the next 
> > call?
> > 
> > Cheers,
> > 
> > == Attendees ==
> > 
> > Apologies if I misspelled any name.
> > 
> > * Stefano Stabellini:
> > Aporeto, Xen ARM maintainer
> > * Julien Grall
> > ARM, Xen ARM maintainer
> > * Artem Mygaiev, Andrii Anisov, Alexandre Agizim:
> > EPAM, Embedded and automotive application for Xen
> > * Steward Hildebrand:
> > Dornerworks, help customer to use Xen in their product
> > * Edgar Iglesias:
> > Xilinx, Embedded and datacenter application
> > * Meng Xu:
> > University of Pennsylvania, RT Xen. Looking for improving performance 
> > for
> > real-time application in virtualised environment
> > * Bosch (sorry I forgot the name of the attendees): Car multimedia
> > * Anastassios Nanos:
> > OnApp, using Xen on lower power server
> > * Dario Faggioli:
> > Citrix, scheduler maintainer. Interested in following big.LITTLE 
> > discussion
> > 
> > == Features ==
> > 
> > === Co-processor architecture ===
> > 
> > Artem:  EPAM is working on a co-processor framework to share resources 
> > between
> > guests (see a RFC of the design document [1]). A prototype has been
> > created by sharing a GPU between guests, the overhead is ~5% compare
> > to native.
> > Julien: concerned about context switch time
> > Stefano: concerned about the size of the emulator and security impact
> > Edgar:  it might not be possible to know the FPGA accelerator when building
> > Xen.
> > Stefano: having the emulator in Xen EL1 would be better for protection
> > Andrii: it is not necessary to implement a full co-processor emulator. It is
> > sufficient to emulation the behavior on register write.
> > 
> > ACTION: Continue the discussion on the mailing list.
> > 
> > === big.LITTLE ===
> > 
> > Anastassios:interested in knowing the status of big.LITTLE support in 
> > Xen
> > Dario:  went through the thread on the ML. The consensus seems to be
> > based on vcpu pin/affinity.
> > Anastassios:There are issue the way xen handles boot code. Wrong errata
> > for cpus.
> > Julien: Core could have different errata and features. Errata 
> > already
> > works today.
> > We need a summary of the discussion.
> > 
> > Dario stepped in to write a summary.
> > 
> > Anastassios:What is the best board to work big.LITTLE with Xen?
> > 
> > ?:  Renesas
> > 
> > ACTION: Dario to write a summary of the big.LITTLE discussions.
> > 
> > === Secure Call Monitor (SMC) from guests ===
> > 
> > Andrii/Artem:   EPAM is working on allowing a guest to make call to TEE (e.g
> > OPTEE). They are working in collaboration with Linaro to
> > make OPTEE  virtualization aware. A design document has been
> > posted on the ML (see [3]).
> > Edgar:  interested on this. Trusted world need to be accessed to
> > manage FPGA and for power management
> > 
> > === Running unmodified baremetal software in a guest ===
> > 
> > Edgar:  Xilinx is working on running unmodified baremetal software
> > in a guest. Piece of work required:
> > - Mapping memory area with different memory attribut
> >   to domU
> > - Replicate host memory map
> > - Exposing a vUART to guest (see [4] for the discussion)
> > Steward:vUART is very important, especially for baremetal guests
> > Stefano:it would need to be loggable
> > Julien: we would have the same issue in the future to be compliant
> > with the VM Spec (see [5]).
> > 
> > === Areas of concern ===
> > 
> > Bosch: problem with xen-swiotlb. It does not work properly on renesas board.
> > Stefano: please report the error on the ML
> > 
> > ACTION: Bosch to send a bug report regarding xen-swiotlb
> > 
> > Edgar:  IOMMU could not be used by the guest (Stage-1). This would be useful
> > to implement driver in userspace.
> > Julien: When will it be required?
> > Edgar:  It is a trend
> > 
> > 

Re: [Xen-devel] Please Welcome Our New Release Manager

[Edgar E. Iglesias]

On Mon, Dec 05, 2016 at 12:36:46PM -0800, Stefano Stabellini wrote:
> On Mon, 5 Dec 2016, Wei Liu wrote:
> > Dear community members,
> > 
> > I'm pleased to announce that Julien Grall  will be
> > the Release Manager for the next Xen release.
> > 
> > The appointment was voted by the Committers and the vote passed.
> > 
> > Julien has done excellent jobs in many aspects. He has been an active
> > developer for the past few years and contributed a lot of code for Xen
> > on ARM. He has been doing a good job in co-maintaining Xen on ARM with
> > Stefano Stabellini. Particularly in 4.8 release, he showed his ability
> > to make balanced decisions and influence other contributors to move
> > various projects forward. He also expressed desire to work with greater
> > Xen community and make bigger impact.
> > 
> > All in all, we believe Julien will do a good job in managing the next
> > release. Thanks Julien for stepping up.
> 
> Congratulations!

Congrats Julien! :-)

Cheers,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen virtual IOMMU high level design doc

[Edgar E. Iglesias]

On Thu, Nov 24, 2016 at 02:49:41PM +0800, Lan Tianyu wrote:
> On 2016年11月24日 12:09, Edgar E. Iglesias wrote:
> >>>> Hi,
> >>>> > > >
> >>>> > > > I have a few questions.
> >>>> > > >
> >>>> > > > If I understand correctly, you'll be emulating an Intel IOMMU in 
> >>>> > > > Xen.
> >>>> > > > So guests will essentially create intel iommu style page-tables.
> >>>> > > >
> >>>> > > > If we were to use this on Xen/ARM, we would likely be modelling an 
> >>>> > > > ARM
> >>>> > > > SMMU as a vIOMMU. Since Xen on ARM does not use QEMU for 
> >>>> > > > emulation, the
> >>>> > > > hypervisor OPs for QEMUs xen dummy IOMMU queries would not really 
> >>>> > > > be used.
> >>>> > > > Do I understand this correctly?
> >>> > > 
> >>> > > I think they could be called from the toolstack. This is why I was
> >>> > > saying in the other thread that the hypercalls should be general 
> >>> > > enough
> >>> > > that QEMU is not the only caller.
> >>> > > 
> >>> > > For PVH and ARM guests, the toolstack should be able to setup the 
> >>> > > vIOMMU
> >>> > > on behalf of the guest without QEMU intervention.
> > OK, I see. Or, I think I understand, not sure :-)
> > 
> > In QEMU when someone changes mappings in an IOMMU there will be a notifier
> > to tell caches upstream that mappings have changed. I think we will need to
> > prepare for that. I.e when TCG CPUs sit behind an IOMMU.
> 
> For Xen side, we may notify pIOMMU driver about mapping change via
> calling pIOMMU driver's API in vIOMMU.

I was refering to the other way around. When a guest modifies the mappings
for a vIOMMU, the driver domain with QEMU and vDevices needs to be notified.

I couldn't find any mention of this in the document...


> 
> > 
> > Another area that may need change is that on ARM we need the map-query to 
> > return
> > the memory attributes for the given mapping. Today QEMU or any emulator 
> > doesn't use it much but in the future things may change.

What about the mem attributes?
It's very likely we'll add support for memory attributes for IOMMU's in QEMU
at some point.
Emulated IOMMU's will thus have the ability to modify attributes (i.e 
SourceID's,
cacheability, etc). Perhaps we could allocate or reserve an uint64_t
for attributes TBD later in the query struct.



> > 
> > For SVM, whe will also need to deal with page-table faults by the IOMMU.
> > So I think there will need to be a channel from Xen to Guesrt to report 
> > these.
> 
> Yes, vIOMMU should forward the page-fault event to guest. For VTD side,
> we will trigger VTD's interrupt to notify guest about the event.

OK, Cool.

Perhaps you should document how this (and the map/unmap notifiers) will work?

I also think it would be a good idea to add a little more introduction so that
some of the questions we've been asking regarding the general design are easier
to grasp.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Wed, Nov 23, 2016 at 06:32:41PM +, Julien Grall wrote:
> Hi Edgar,
> 
> On 23/11/16 16:38, Edgar E. Iglesias wrote:
> >On Wed, Nov 23, 2016 at 06:26:06PM +0200, Artem Mygaiev wrote:
> >>On 23.11.16 17:19, Julien Grall wrote:
> >>>There is a couple of usecase where we cannot use PV console:
> >>>- Running unmodified baremetal application as guest. Those
> >>>applications will not have PV driver.
> >>Well, I somehow missed that requirement is to run *unmodified*
> >>applications... I think it will be pretty hard to provide all the HW
> >>infrastructure expected and UARTs as I know them have all possible
> >>issues & workarounds.
> >
> >Hi Artem,
> >
> >The way I see this is not for Xen to provide support for any HW for any
> >platform on all platforms.
> >
> >The way I see it is that on a given platform, for example the ZynqMP.
> >You can run Xen as a partitioning Hypervisor essentially using device
> >passthrough to assign various real HW devices to each VM.
> >
> >The various guests can run unmodified because they each see a subset
> >of the platform.
> >
> >It is useful to have vuarts that go are shared into a single one.
> 
> To expand on the vuart, this is a small emulator to handle write from a
> guest. This was originally designed to handle DOM0 early console that is
> hardcoded on ARM32.
> 
> As mentioned by Artem, read would be more complex as it will mean per-UART
> emulator. IIRC, this would not be a problem for you, right?

Hi Julien,

Right, we wouldn't want to use vuarts for reading.

In our case, I was thinking that guests that need input from serial ports
or that require specific BAUD rates will have to be assigned one of the
UARTs.

For guests that just want to output informative log messages onto the
UART, we can use vuarts.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen virtual IOMMU high level design doc

[Edgar E. Iglesias]

On Thu, Nov 24, 2016 at 02:00:21AM +, Tian, Kevin wrote:
> > From: Stefano Stabellini [mailto:sstabell...@kernel.org]
> > Sent: Thursday, November 24, 2016 3:09 AM
> > 
> > On Wed, 23 Nov 2016, Edgar E. Iglesias wrote:
> > > On Wed, Aug 17, 2016 at 08:05:51PM +0800, Lan, Tianyu wrote:
> > > > Hi All:
> > > >  The following is our Xen vIOMMU high level design for detail
> > > > discussion. Please have a look. Very appreciate for your comments.
> > > > This design doesn't cover changes when root port is moved to hypervisor.
> > > > We may design it later.
> > >
> > > Hi,
> > >
> > > I have a few questions.
> > >
> > > If I understand correctly, you'll be emulating an Intel IOMMU in Xen.
> > > So guests will essentially create intel iommu style page-tables.
> > >
> > > If we were to use this on Xen/ARM, we would likely be modelling an ARM
> > > SMMU as a vIOMMU. Since Xen on ARM does not use QEMU for emulation, the
> > > hypervisor OPs for QEMUs xen dummy IOMMU queries would not really be used.
> > > Do I understand this correctly?
> > 
> > I think they could be called from the toolstack. This is why I was
> > saying in the other thread that the hypercalls should be general enough
> > that QEMU is not the only caller.
> > 
> > For PVH and ARM guests, the toolstack should be able to setup the vIOMMU
> > on behalf of the guest without QEMU intervention.

OK, I see. Or, I think I understand, not sure :-)

In QEMU when someone changes mappings in an IOMMU there will be a notifier
to tell caches upstream that mappings have changed. I think we will need to
prepare for that. I.e when TCG CPUs sit behind an IOMMU.

Another area that may need change is that on ARM we need the map-query to return
the memory attributes for the given mapping. Today QEMU or any emulator 
doesn't use it much but in the future things may change.

For SVM, whe will also need to deal with page-table faults by the IOMMU.
So I think there will need to be a channel from Xen to Guesrt to report these.

For example, what happens when a guest assigned DMA unit page-faults?
Xen needs to know how to forward this fault back to guest for fixup and the
guest needs to be able to fix it and tell the device that it's OK to contine.
E.g PCI PRI or similar.


> > > Has a platform agnostic PV-IOMMU been considered to support 2-stage
> > > translation (i.e VFIO in the guest)? Perhaps that would hurt map/unmap
> > > performance too much?
> > 
> > That's an interesting idea. I don't know if that's feasible, but if it
> > is not, then we need to be able to specify the PV-IOMMU type in the
> > hypercalls, so that you would get Intel IOMMU on x86 and SMMU on ARM.
> > 
> > 
> 
> Not considered yet. PV is always possible as we've done for other I/O
> devices. Ideally it could be designed being more efficient than full
> emulation of vendor specific IOMMU, but also means requirement of
> maintaining a new guest IOMMU driver and limitation of supporting
> only newer version guest OSes. It's a tradeoff... at least not compelling 
> now (may consider when we see a real need in the future).

Agreed. Thanks.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen virtual IOMMU high level design doc

[Edgar E. Iglesias]

On Wed, Aug 17, 2016 at 08:05:51PM +0800, Lan, Tianyu wrote:
> Hi All:
>  The following is our Xen vIOMMU high level design for detail
> discussion. Please have a look. Very appreciate for your comments.
> This design doesn't cover changes when root port is moved to hypervisor.
> We may design it later.

Hi,

I have a few questions.

If I understand correctly, you'll be emulating an Intel IOMMU in Xen.
So guests will essentially create intel iommu style page-tables.

If we were to use this on Xen/ARM, we would likely be modelling an ARM
SMMU as a vIOMMU. Since Xen on ARM does not use QEMU for emulation, the
hypervisor OPs for QEMUs xen dummy IOMMU queries would not really be used.
Do I understand this correctly?

Has a platform agnostic PV-IOMMU been considered to support 2-stage
translation (i.e VFIO in the guest)? Perhaps that would hurt map/unmap
performance too much?

Best regards,
Edgar




> 
> 
> Content:
> ===
> 1. Motivation of vIOMMU
>   1.1 Enable more than 255 vcpus
>   1.2 Support VFIO-based user space driver
>   1.3 Support guest Shared Virtual Memory (SVM)
> 2. Xen vIOMMU Architecture
>   2.1 2th level translation overview
>   2.2 Interrupt remapping overview
> 3. Xen hypervisor
>   3.1 New vIOMMU hypercall interface
>   3.2 2nd level translation
>   3.3 Interrupt remapping
>   3.4 1st level translation
>   3.5 Implementation consideration
> 4. Qemu
>   4.1 Qemu vIOMMU framework
>   4.2 Dummy xen-vIOMMU driver
>   4.3 Q35 vs. i440x
>   4.4 Report vIOMMU to hvmloader
> 
> 
> 1 Motivation for Xen vIOMMU
> ===
> 1.1 Enable more than 255 vcpu support
> HPC virtualization requires more than 255 vcpus support in a single VM
> to meet parallel computing requirement. More than 255 vcpus support
> requires interrupt remapping capability present on vIOMMU to deliver
> interrupt to #vcpu >255 Otherwise Linux guest fails to boot up with >255
> vcpus if interrupt remapping is absent.
> 
> 
> 1.2 Support VFIO-based user space driver (e.g. DPDK) in the guest
> It relies on the 2nd level translation capability (IOVA->GPA) on
> vIOMMU. pIOMMU 2nd level becomes a shadowing structure of
> vIOMMU to isolate DMA requests initiated by user space driver.
> 
> 
> 1.3 Support guest SVM (Shared Virtual Memory)
> It relies on the 1st level translation table capability (GVA->GPA) on
> vIOMMU. pIOMMU needs to enable both 1st level and 2nd level translation
> in nested mode (GVA->GPA->HPA) for passthrough device. IGD passthrough
> is the main usage today (to support OpenCL 2.0 SVM feature). In the
> future SVM might be used by other I/O devices too.
> 
> 2. Xen vIOMMU Architecture
> 
> 
> * vIOMMU will be inside Xen hypervisor for following factors
>   1) Avoid round trips between Qemu and Xen hypervisor
>   2) Ease of integration with the rest of the hypervisor
>   3) HVMlite/PVH doesn't use Qemu
> * Dummy xen-vIOMMU in Qemu as a wrapper of new hypercall to create
> /destory vIOMMU in hypervisor and deal with virtual PCI device's 2th
> level translation.
> 
> 2.1 2th level translation overview
> For Virtual PCI device, dummy xen-vIOMMU does translation in the
> Qemu via new hypercall.
> 
> For physical PCI device, vIOMMU in hypervisor shadows IO page table from
> IOVA->GPA to IOVA->HPA and load page table to physical IOMMU.
> 
> The following diagram shows 2th level translation architecture.
> +-+
> |Qemu++   |
> || Virtual|   |
> ||   PCI device   |   |
> |||   |
> |++   |
> ||DMA |
> |V|
> |  ++   Request  ++   |
> |  |+<---+|   |
> |  |  Dummy xen vIOMMU  | Target GPA |  Memory region |   |
> |  |+--->+|   |
> |  +-+--++---++   |
> ||   ||
> ||Hypercall  ||
> +++
> |Hypervisor  |   ||
> ||   ||
> |v   ||
> | +--+--+||
> | |   vIOMMU|||
> | +--+--+|  

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Wed, Nov 23, 2016 at 06:26:06PM +0200, Artem Mygaiev wrote:
> On 23.11.16 17:19, Julien Grall wrote:
> > There is a couple of usecase where we cannot use PV console:
> > - Running unmodified baremetal application as guest. Those
> > applications will not have PV driver.
> Well, I somehow missed that requirement is to run *unmodified*
> applications... I think it will be pretty hard to provide all the HW
> infrastructure expected and UARTs as I know them have all possible
> issues & workarounds.

Hi Artem,

The way I see this is not for Xen to provide support for any HW for any
platform on all platforms.

The way I see it is that on a given platform, for example the ZynqMP.
You can run Xen as a partitioning Hypervisor essentially using device
passthrough to assign various real HW devices to each VM.

The various guests can run unmodified because they each see a subset
of the platform.

It is useful to have vuarts that go are shared into a single one.

Cheers,
Edgar



> > - Guest compliant with the VM system specification for ARM [1]. A
> > pl011 is required.
> Yes, indeed, a subset of PL011 registers is required, unfortunately I
> was not familiar with that spec, thanks.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Mon, Nov 21, 2016 at 09:24:27PM +, Julien Grall wrote:
> 
> 
> On 21/11/2016 21:13, Edgar E. Iglesias wrote:
> >On Mon, Nov 21, 2016 at 01:01:15PM -0800, Stefano Stabellini wrote:
> >>On Mon, 21 Nov 2016, Edgar E. Iglesias wrote:
> >>>On Fri, Nov 18, 2016 at 10:58:42AM -0800, Stefano Stabellini wrote:
> >>>>On Thu, 17 Nov 2016, Julien Grall wrote:
> >>>>>Hi Stefano,
> >>>>>
> >>>>>On 17/11/2016 11:26, Stefano Stabellini wrote:
> >>>>>>On Mon, 14 Nov 2016, Julien Grall wrote:
> >>>>>>>On 11/11/2016 13:55, Stefano Stabellini wrote:
> >>>>>>>>On Fri, 11 Nov 2016, Julien Grall wrote:
> >>>>>>>>>On 11/11/2016 02:24, Stefano Stabellini wrote:
> >>>>>>>>>>On Thu, 10 Nov 2016, Julien Grall wrote:
> >>>>>>>>>>>(CC Stefano and change the title)
> >>>>>>>>>>>On 10/11/16 12:13, casionwoo wrote:
> >>>>>>>>>>>>I’m pleased about your reply and you have a lot of code to
> >>>>>>>>>>>>clean-up.
> >>>>>>>>>>>>
> >>>>>>>>>>>>As you mentioned, It’s really huge to digest at once. Thank you
> >>>>>>>>>>>>for
> >>>>>>>>>>>>understanding me.
> >>>>>>>>>>>>
> >>>>>>>>>>>>And that’s why i need a small fix up and todo list.
> >>>>>>>>>>>>
> >>>>>>>>>>>>I feel familiar with ARM and c language and there’s no specific
> >>>>>>>>>>>>area
> >>>>>>>>>>>>yet.
> >>>>>>>>>>>>
> >>>>>>>>>>>>I think that i can find interesting area with following up the
> >>>>>>>>>>>>codes.
> >>>>>>>>>>>>
> >>>>>>>>>>>>I’m looking forward to being stuck on Xen.
> >>>>>>>>>>>>
> >>>>>>>>>>>>Then it would be easier for me to understand about Xen on ARM.
> >>>>>>>>>>>>
> >>>>>>>>>>>>Please let me know the TODO and bug-fix lists.
> >>>>>>>>>>>
> >>>>>>>>>>>Stefano, before giving a list of code clean-up, do you have any
> >>>>>>>>>>>small
> >>>>>>>>>>>TODO
> >>>>>>>>>>>on
> >>>>>>>>>>>ARM in mind?
> >>>>>>>>>>
> >>>>>>>>>>A simple task we talked about recently is to enable the vuart
> >>>>>>>>>>(xen/arch/arm/vuart.c) for all guests. At the moment it is only
> >>>>>>>>>>emulated
> >>>>>>>>>>for Dom0, but some guests, in particular BareMetal guests
> >>>>>>>>>>(https://en.wikipedia.org/wiki/BareMetal), would benefit from it.
> >>>>>>>>>>
> >>>>>>>>>>It would be best to introduce an option in libxl to explicitly
> >>>>>>>>>>enable/disable the vuart for DomUs. Something like vuart=1 in the VM
> >>>>>>>>>>config file.
> >>>>>>>>>
> >>>>>>>>>The vuart has not been enabled for DomU because it the UART region 
> >>>>>>>>>may
> >>>>>>>>>clash
> >>>>>>>>>with the guest memory layout (which is static).
> >>>>>>>>>
> >>>>>>>>>I don't think this option should be available until we allow the 
> >>>>>>>>>guest
> >>>>>>>>>to
> >>>>>>>>>use
> >>>>>>>>>the same memory layout as the host (see e820_host parameter for x86).
> >>>>>>>>
> >>>>>>>>Actually there is no reason for the vuart to use the same address as
> >>>>>>>>the physical uart on the platform, is there?
> >>>>>>>>In fact it doesn't eve

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Mon, Nov 21, 2016 at 01:01:15PM -0800, Stefano Stabellini wrote:
> On Mon, 21 Nov 2016, Edgar E. Iglesias wrote:
> > On Fri, Nov 18, 2016 at 10:58:42AM -0800, Stefano Stabellini wrote:
> > > On Thu, 17 Nov 2016, Julien Grall wrote:
> > > > Hi Stefano,
> > > > 
> > > > On 17/11/2016 11:26, Stefano Stabellini wrote:
> > > > > On Mon, 14 Nov 2016, Julien Grall wrote:
> > > > > > On 11/11/2016 13:55, Stefano Stabellini wrote:
> > > > > > > On Fri, 11 Nov 2016, Julien Grall wrote:
> > > > > > > > On 11/11/2016 02:24, Stefano Stabellini wrote:
> > > > > > > > > On Thu, 10 Nov 2016, Julien Grall wrote:
> > > > > > > > > > (CC Stefano and change the title)
> > > > > > > > > > On 10/11/16 12:13, casionwoo wrote:
> > > > > > > > > > > I’m pleased about your reply and you have a lot of code to
> > > > > > > > > > > clean-up.
> > > > > > > > > > > 
> > > > > > > > > > > As you mentioned, It’s really huge to digest at once. 
> > > > > > > > > > > Thank you
> > > > > > > > > > > for
> > > > > > > > > > > understanding me.
> > > > > > > > > > > 
> > > > > > > > > > > And that’s why i need a small fix up and todo list.
> > > > > > > > > > > 
> > > > > > > > > > > I feel familiar with ARM and c language and there’s no 
> > > > > > > > > > > specific
> > > > > > > > > > > area
> > > > > > > > > > > yet.
> > > > > > > > > > > 
> > > > > > > > > > > I think that i can find interesting area with following 
> > > > > > > > > > > up the
> > > > > > > > > > > codes.
> > > > > > > > > > > 
> > > > > > > > > > > I’m looking forward to being stuck on Xen.
> > > > > > > > > > > 
> > > > > > > > > > > Then it would be easier for me to understand about Xen on 
> > > > > > > > > > > ARM.
> > > > > > > > > > > 
> > > > > > > > > > > Please let me know the TODO and bug-fix lists.
> > > > > > > > > > 
> > > > > > > > > > Stefano, before giving a list of code clean-up, do you have 
> > > > > > > > > > any
> > > > > > > > > > small
> > > > > > > > > > TODO
> > > > > > > > > > on
> > > > > > > > > > ARM in mind?
> > > > > > > > > 
> > > > > > > > > A simple task we talked about recently is to enable the vuart
> > > > > > > > > (xen/arch/arm/vuart.c) for all guests. At the moment it is 
> > > > > > > > > only
> > > > > > > > > emulated
> > > > > > > > > for Dom0, but some guests, in particular BareMetal guests
> > > > > > > > > (https://en.wikipedia.org/wiki/BareMetal), would benefit from 
> > > > > > > > > it.
> > > > > > > > > 
> > > > > > > > > It would be best to introduce an option in libxl to explicitly
> > > > > > > > > enable/disable the vuart for DomUs. Something like vuart=1 in 
> > > > > > > > > the VM
> > > > > > > > > config file.
> > > > > > > > 
> > > > > > > > The vuart has not been enabled for DomU because it the UART 
> > > > > > > > region may
> > > > > > > > clash
> > > > > > > > with the guest memory layout (which is static).
> > > > > > > > 
> > > > > > > > I don't think this option should be available until we allow 
> > > > > > > > the guest
> > > > > > > > to
> > > > > > > > use
> > > > > > > > the same memory layout as the host (see e820_host parameter for 
> > > > >

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Fri, Nov 18, 2016 at 10:58:42AM -0800, Stefano Stabellini wrote:
> On Thu, 17 Nov 2016, Julien Grall wrote:
> > Hi Stefano,
> > 
> > On 17/11/2016 11:26, Stefano Stabellini wrote:
> > > On Mon, 14 Nov 2016, Julien Grall wrote:
> > > > On 11/11/2016 13:55, Stefano Stabellini wrote:
> > > > > On Fri, 11 Nov 2016, Julien Grall wrote:
> > > > > > On 11/11/2016 02:24, Stefano Stabellini wrote:
> > > > > > > On Thu, 10 Nov 2016, Julien Grall wrote:
> > > > > > > > (CC Stefano and change the title)
> > > > > > > > On 10/11/16 12:13, casionwoo wrote:
> > > > > > > > > I’m pleased about your reply and you have a lot of code to
> > > > > > > > > clean-up.
> > > > > > > > > 
> > > > > > > > > As you mentioned, It’s really huge to digest at once. Thank 
> > > > > > > > > you
> > > > > > > > > for
> > > > > > > > > understanding me.
> > > > > > > > > 
> > > > > > > > > And that’s why i need a small fix up and todo list.
> > > > > > > > > 
> > > > > > > > > I feel familiar with ARM and c language and there’s no 
> > > > > > > > > specific
> > > > > > > > > area
> > > > > > > > > yet.
> > > > > > > > > 
> > > > > > > > > I think that i can find interesting area with following up the
> > > > > > > > > codes.
> > > > > > > > > 
> > > > > > > > > I’m looking forward to being stuck on Xen.
> > > > > > > > > 
> > > > > > > > > Then it would be easier for me to understand about Xen on ARM.
> > > > > > > > > 
> > > > > > > > > Please let me know the TODO and bug-fix lists.
> > > > > > > > 
> > > > > > > > Stefano, before giving a list of code clean-up, do you have any
> > > > > > > > small
> > > > > > > > TODO
> > > > > > > > on
> > > > > > > > ARM in mind?
> > > > > > > 
> > > > > > > A simple task we talked about recently is to enable the vuart
> > > > > > > (xen/arch/arm/vuart.c) for all guests. At the moment it is only
> > > > > > > emulated
> > > > > > > for Dom0, but some guests, in particular BareMetal guests
> > > > > > > (https://en.wikipedia.org/wiki/BareMetal), would benefit from it.
> > > > > > > 
> > > > > > > It would be best to introduce an option in libxl to explicitly
> > > > > > > enable/disable the vuart for DomUs. Something like vuart=1 in the 
> > > > > > > VM
> > > > > > > config file.
> > > > > > 
> > > > > > The vuart has not been enabled for DomU because it the UART region 
> > > > > > may
> > > > > > clash
> > > > > > with the guest memory layout (which is static).
> > > > > > 
> > > > > > I don't think this option should be available until we allow the 
> > > > > > guest
> > > > > > to
> > > > > > use
> > > > > > the same memory layout as the host (see e820_host parameter for 
> > > > > > x86).
> > > > > 
> > > > > Actually there is no reason for the vuart to use the same address as
> > > > > the physical uart on the platform, is there?
> > > > > In fact it doesn't even
> > > > > have to prentend to be the same uart as the one on the board, right?
> > > > > The vuart MMIO address could be completely configurable and 
> > > > > independent
> > > > > from the one of the physical uart.
> > > > 
> > > > There is no reason to use the same information as the physical UART.
> > > > 
> > > > However, the vuart requires quite a few information (e.g base address,
> > > > offset
> > > > of different register... see vuart_info structure in 
> > > > include/xen/serial.h
> > > > for
> > > > more details) in order to fully work.
> > > > 
> > > > IHMO this is a lot of works for the user to configure. I would much 
> > > > prefer
> > > > to
> > > > see a PL011 emulated at a specific base address and let the user select
> > > > whether he wants a UART to debug or not.
> > > 
> > > So you envision the configuration of the MMIO base address to be done as
> > > part of a new dynamic guest memory map?
> > 
> > For guest using dynamic memory map, I would expect to expose an uncompleted
> > emulation of the physical UART (e.g it would only be possible to write) at 
> > the
> > exact same address as on the host.
> 
> Why? Is this a requirement for baremetal guests?
> 
> I would have actually opted for always emulating a PL011 even for guests
> using a dynamic memory map (which of course one way or another need to
> be able to choose the address of the UART, the memory and the rest).

I guess it's not black and white but trying to reduce the gap towards
being able to run unmodified SW for a given platform as a guest would
be nice.

Some times things are quite relaxed and we can recompile the SW for Xen,
add new drivers etc. Other times perhaps SW has been certified and users
may not be able to change anything.

For apps where the UARTs are only used for console data, vuarts at
configurable locations would be nice IMO.

Cheers,
Edgar

> 
> 
> > For guest using static memory map (i.e the current approach), I would 
> > envision
> > to always emulate a PL011 (or at least giving this option) and not the
> > physical UART.
> > 
> > This has a better fit with the support of SBSA/VM Spec compliant guest and
> > still allow a user 

Re: [Xen-devel] Xen ARM small task (WAS: Re: [Xen Question])

[Edgar E. Iglesias]

On Fri, Nov 11, 2016 at 09:46:56AM +, Julien Grall wrote:
> Hi Stefano,
> 
> On 11/11/2016 02:24, Stefano Stabellini wrote:
> >On Thu, 10 Nov 2016, Julien Grall wrote:
> >>(CC Stefano and change the title)
> >>
> >>Hello,
> >>
> >>On 10/11/16 12:13, casionwoo wrote:
> >>>I’m pleased about your reply and you have a lot of code to clean-up.
> >>>
> >>>As you mentioned, It’s really huge to digest at once. Thank you for
> >>>understanding me.
> >>>
> >>>And that’s why i need a small fix up and todo list.
> >>>
> >>>I feel familiar with ARM and c language and there’s no specific area yet.
> >>>
> >>>I think that i can find interesting area with following up the codes.
> >>>
> >>>I’m looking forward to being stuck on Xen.
> >>>
> >>>Then it would be easier for me to understand about Xen on ARM.
> >>>
> >>>Please let me know the TODO and bug-fix lists.
> >>
> >>Stefano, before giving a list of code clean-up, do you have any small TODO 
> >>on
> >>ARM in mind?
> >
> >A simple task we talked about recently is to enable the vuart
> >(xen/arch/arm/vuart.c) for all guests. At the moment it is only emulated
> >for Dom0, but some guests, in particular BareMetal guests
> >(https://en.wikipedia.org/wiki/BareMetal), would benefit from it.
> >
> >It would be best to introduce an option in libxl to explicitly
> >enable/disable the vuart for DomUs. Something like vuart=1 in the VM
> >config file.
> 
> The vuart has not been enabled for DomU because it the UART region may clash
> with the guest memory layout (which is static).
> 
> I don't think this option should be available until we allow the guest to
> use the same memory layout as the host (see e820_host parameter for x86).

Yes, we were thinking to give the mem layout one a try in 4.9 hopefully.

Another task that is not too huge is the one to allow Xen to map in
arbitrary memory regions as Normal Memory into domUs.
We discussed the possibility of having additional arch specific arguments
to the iomem settings.

This is for example useful to give DomU guests direct access to on chip
memories or to specific ranges of DRAM that may be needed to communicate
with devices.


> 
> >
> >BTW we should keep this up to date:
> >
> >https://wiki.xenproject.org/wiki/Xen_ARM_TODO
> 
> You are right, although it might be better to use the bug tracker [1] to
> stay aligned with the rest of the hypervisor.
> 
> Note that I have got a list of TODO/bugs I track myself but never updated
> the wiki.

Good idea, I'll add some stuff into that list!

Best regards,
Edgar



> 
> Cheers,
> 
> [1] https://bugs.xenproject.org/xen/
> 
> -- 
> Julien Grall
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> https://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen ARM community call

[Edgar E. Iglesias]

Hi Julien,

I'm interested in joining. I'm in Sweden GMT+1.

Thanks!
Edgar


On Tue, Nov 08, 2016 at 12:19:52PM +, Julien Grall wrote:
> Hi all,
> 
> I would like to start organizing a recurring community call to discuss and
> sync-up on upcoming features for Xen ARM.
> 
> Example of features that could be discussed:
>   - Sharing co-processor between guests
>   - PCI passthrough
> 
> I would suggest to start with a 1 hour meeting on the Wednesday 23rd
> November. I know that people are spread across different timezones, so I
> would like to gather thought before choosing a time.
> 
> Your sincerely,
> 
> -- 
> Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v1 1/1] xen/arm: Disable the Cortex-a53-edac

[Edgar E. Iglesias]

On Fri, Oct 07, 2016 at 07:22:33PM +0100, Wei Liu wrote:
> On Fri, Oct 07, 2016 at 10:34:15AM -0700, Stefano Stabellini wrote:
> > On Fri, 7 Oct 2016, Alistair Francis wrote:
> > > On Thu, Oct 6, 2016 at 9:36 AM, Edgar E. Iglesias
> > > <edgar.igles...@gmail.com> wrote:
> > > > From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>
> > > >
> > > > Disable the Cortex-a53-edac. Xen currently does not yet
> > > > handle reads/writes to the implementation defined CPUMERRSR
> > > > register.
> > > >
> > > > Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
> > > 
> > > This solution looks fine to me and everything boots on ZynqMP as
> > > expected with this patch.
> > > 
> > > Acked-by: Alistair Francis <alistair.fran...@xilinx.com>
> > 
> > Reviewed-by: Stefano Stabellini <sstabell...@kernel.org>
> > 
> > Wei, can we have this in 4.8? See the 0/1 email for an explanation of
> > why this change is needed. The patch just adds the troublesome
> > cortex-a15-pmu to the "skip_matches" array to remove it from dom0's
> > device tree.
> > 
> 
> Sure.


Hi Wei,

Just a friendly reminder:
This is still missing from master, staging and 4.8.0-rc2.

Best regards,
Edgar

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 1/1] xen/arm: Disable the Cortex-a53-edac

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Disable the Cortex-a53-edac. Xen currently does not yet
handle reads/writes to the implementation defined CPUMERRSR
register.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/domain_build.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/xen/arch/arm/domain_build.c b/xen/arch/arm/domain_build.c
index ce97359..e8a400c 100644
--- a/xen/arch/arm/domain_build.c
+++ b/xen/arch/arm/domain_build.c
@@ -1188,6 +1188,7 @@ static int handle_node(struct domain *d, struct 
kernel_info *kinfo,
 DT_MATCH_COMPATIBLE("arm,psci-1.0"),
 DT_MATCH_COMPATIBLE("arm,cortex-a7-pmu"),
 DT_MATCH_COMPATIBLE("arm,cortex-a15-pmu"),
+DT_MATCH_COMPATIBLE("arm,cortex-a53-edac"),
 DT_MATCH_COMPATIBLE("arm,armv8-pmuv3"),
 DT_MATCH_PATH("/cpus"),
 DT_MATCH_TYPE("memory"),
-- 
2.5.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v1 0/1] xen/arm: Disable the Cortex-a53-edac

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

The Cortex-A53 EDAC (Error Detection And Correction) Linux driver
uses the implementation defined CPUMERRSR register.
Xen traps but does not yet handle accesses to this reg so dom0 panics
on an undefined instruction at boot.

For Xen-4.8, this patchs disables the A53 EDAC module by removing it from
the device tree.

For Xen-4.9 we may want to allow dom0 to access these regs.

Comments welcome!

Best regards,
Edgar

Edgar E. Iglesias (1):
  xen/arm: Disable the Cortex-a53-edac

 xen/arch/arm/domain_build.c | 1 +
 1 file changed, 1 insertion(+)

-- 
2.5.0


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 0/7] xen/arm: Add support for mapping mmio-sram nodes into dom0

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

This series adds support for mapping mmio-sram nodes into dom0
as normal uncached MEMORY with RW perms.

If the no-memory-wc prop is present in the DTB node, we create
DEVICE RW mappings instead.

Comments welcome!

Best regards,
Edgar

ChangeLog:

v3 -> v4:
* Rename p2m_mmio_direct_nc -> p2m_mmio_direct_dev
* Rename p2m_mem_nc p2m_mmio_direct_nc
* Make p2m_mmio_direct_nc non-executable to match p2m_mmio_direct_c
* Fix typos in comments regarding shareability
* Add reference to ARMv8 specs for outer sharability attr
* Add comment describing map_regions_p2mt
* Mention the p2m type inheritance in the commit msg of path #6

v2 -> v3:
* Drop RFC
* Add comment on outer-shareable choice for non-cached mem
* Spellfix existance -> existence
* Add comment on props usage
* Props matching now only supports a single property
* Dropped p2m_access in plumbing for mapping attributes
* Rename un/map_regions to un/map_regions_p2mt
* Add path to mmio-sram device-tree bindings docs in commit msg
* Add a comment on inheriting parent mem attributes
* Dropped the mmio-sram bus

v1 -> v2
* Replace the .map method with a list of match -> map attributes
* Handle no-memory-wc by mapping as DEVICE RW
* Generalize map_regions_rw_cache instead of adding new functions

Edgar E. Iglesias (7):
  xen/arm: p2m: Rename p2m_mmio_direct_nc -> p2m_mmio_direct_dev
  xen/arm: p2m: Add support for normal non-cacheable memory
  xen/arm: Rename and generalize un/map_regions_rw_cache
  xen/device-tree: Add __DT_MATCH macros without braces
  xen/device-tree: Make dt_match_node match props
  xen/arm: domain_build: Plumb for different mapping attributes
  xen/arm: Map mmio-sram nodes as un-cached memory

 xen/arch/arm/domain_build.c   | 90 +--
 xen/arch/arm/p2m.c| 42 ++--
 xen/common/device_tree.c  |  5 ++-
 xen/include/asm-arm/p2m.h | 24 +++-
 xen/include/asm-arm/page.h|  1 +
 xen/include/xen/device_tree.h | 20 --
 6 files changed, 136 insertions(+), 46 deletions(-)

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 2/7] xen/arm: p2m: Add support for normal non-cacheable memory

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Add support for describing normal non-cacheable memory.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/p2m.c | 19 +++
 xen/include/asm-arm/p2m.h  |  1 +
 xen/include/asm-arm/page.h |  1 +
 3 files changed, 21 insertions(+)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index c3b1233..9912658 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -296,6 +296,7 @@ static void p2m_set_permission(lpae_t *e, p2m_type_t t, 
p2m_access_t a)
 case p2m_map_foreign:
 case p2m_grant_map_rw:
 case p2m_mmio_direct_dev:
+case p2m_mmio_direct_nc:
 case p2m_mmio_direct_c:
 e->p2m.xn = 1;
 e->p2m.write = 1;
@@ -376,6 +377,24 @@ static lpae_t mfn_to_p2m_entry(mfn_t mfn, p2m_type_t t, 
p2m_access_t a)
 e.p2m.sh = LPAE_SH_OUTER;
 break;
 
+/*
+ * ARM ARM: Overlaying the shareability attribute (DDI
+ * 0406C.b B3-1376 to 1377)
+ *
+ * A memory region with a resultant memory type attribute of Normal,
+ * and a resultant cacheability attribute of Inner Non-cacheable,
+ * Outer Non-cacheable, must have a resultant shareability attribute
+ * of Outer Shareable, otherwise shareability is UNPREDICTABLE.
+ *
+ * On ARMv8 shareability is ignored and explicitly treated as Outer
+ * Shareable for Normal Inner Non_cacheable, Outer Non-cacheable.
+ * See the note for table D4-40, in page 1788 of the ARM DDI 0487A.j.
+ */
+case p2m_mmio_direct_nc:
+e.p2m.mattr = MATTR_MEM_NC;
+e.p2m.sh = LPAE_SH_OUTER;
+break;
+
 default:
 e.p2m.mattr = MATTR_MEM;
 e.p2m.sh = LPAE_SH_INNER;
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index b342122..9708fdc 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -92,6 +92,7 @@ typedef enum {
 p2m_ram_rw, /* Normal read/write guest RAM */
 p2m_ram_ro, /* Read-only; writes are silently dropped */
 p2m_mmio_direct_dev,/* Read/write mapping of genuine Device MMIO area */
+p2m_mmio_direct_nc, /* Read/write mapping of genuine MMIO area 
non-cacheable */
 p2m_mmio_direct_c,  /* Read/write mapping of genuine MMIO area cacheable */
 p2m_map_foreign,/* Ram pages from foreign domain */
 p2m_grant_map_rw,   /* Read/write grant mapping */
diff --git a/xen/include/asm-arm/page.h b/xen/include/asm-arm/page.h
index 015ed63..f50fe60 100644
--- a/xen/include/asm-arm/page.h
+++ b/xen/include/asm-arm/page.h
@@ -84,6 +84,7 @@
  *
  */
 #define MATTR_DEV 0x1
+#define MATTR_MEM_NC  0x5
 #define MATTR_MEM 0xf
 
 /* Flags for get_page_from_gva, gvirt_to_maddr etc */
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 1/7] xen/arm: p2m: Rename p2m_mmio_direct_nc -> p2m_mmio_direct_dev

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Rename p2m_mmio_direct_nc to p2m_mmio_direct_dev to better
express that we are mapping device memory. This will allow us
to use p2m_mmio_direct_nc for Normal Non-Cached mappings.

No functional change.

Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/arch/arm/p2m.c| 6 +++---
 xen/include/asm-arm/p2m.h | 2 +-
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 5c5049f..c3b1233 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -295,7 +295,7 @@ static void p2m_set_permission(lpae_t *e, p2m_type_t t, 
p2m_access_t a)
 case p2m_iommu_map_rw:
 case p2m_map_foreign:
 case p2m_grant_map_rw:
-case p2m_mmio_direct_nc:
+case p2m_mmio_direct_dev:
 case p2m_mmio_direct_c:
 e->p2m.xn = 1;
 e->p2m.write = 1;
@@ -366,7 +366,7 @@ static lpae_t mfn_to_p2m_entry(mfn_t mfn, p2m_type_t t, 
p2m_access_t a)
 
 switch ( t )
 {
-case p2m_mmio_direct_nc:
+case p2m_mmio_direct_dev:
 e.p2m.mattr = MATTR_DEV;
 e.p2m.sh = LPAE_SH_OUTER;
 break;
@@ -1237,7 +1237,7 @@ int map_mmio_regions(struct domain *d,
  unsigned long nr,
  mfn_t mfn)
 {
-return p2m_insert_mapping(d, start_gfn, nr, mfn, p2m_mmio_direct_nc);
+return p2m_insert_mapping(d, start_gfn, nr, mfn, p2m_mmio_direct_dev);
 }
 
 int unmap_mmio_regions(struct domain *d,
diff --git a/xen/include/asm-arm/p2m.h b/xen/include/asm-arm/p2m.h
index 6251b37..b342122 100644
--- a/xen/include/asm-arm/p2m.h
+++ b/xen/include/asm-arm/p2m.h
@@ -91,7 +91,7 @@ typedef enum {
 p2m_invalid = 0,/* Nothing mapped here */
 p2m_ram_rw, /* Normal read/write guest RAM */
 p2m_ram_ro, /* Read-only; writes are silently dropped */
-p2m_mmio_direct_nc, /* Read/write mapping of genuine MMIO area 
non-cacheable */
+p2m_mmio_direct_dev,/* Read/write mapping of genuine Device MMIO area */
 p2m_mmio_direct_c,  /* Read/write mapping of genuine MMIO area cacheable */
 p2m_map_foreign,/* Ram pages from foreign domain */
 p2m_grant_map_rw,   /* Read/write grant mapping */
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH v4 5/7] xen/device-tree: Make dt_match_node match props

[Edgar E. Iglesias]

From: "Edgar E. Iglesias" <edgar.igles...@xilinx.com>

Make dt_match_node match for a single existing property.
We only search for the existence of the property, not
for specific values.

Acked-by: Julien Grall <julien.gr...@arm.com>
Signed-off-by: Edgar E. Iglesias <edgar.igles...@xilinx.com>
---
 xen/common/device_tree.c  | 5 -
 xen/include/xen/device_tree.h | 7 +++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/xen/common/device_tree.c b/xen/common/device_tree.c
index b39c8ca..1be074b 100644
--- a/xen/common/device_tree.c
+++ b/xen/common/device_tree.c
@@ -319,7 +319,7 @@ dt_match_node(const struct dt_device_match *matches,
 return NULL;
 
 while ( matches->path || matches->type ||
-matches->compatible || matches->not_available )
+matches->compatible || matches->not_available || matches->prop)
 {
 bool_t match = 1;
 
@@ -335,6 +335,9 @@ dt_match_node(const struct dt_device_match *matches,
 if ( matches->not_available )
 match &= !dt_device_is_available(node);
 
+if ( matches->prop )
+match &= dt_find_property(node, matches->prop, NULL) != NULL;
+
 if ( match )
 return matches;
 matches++;
diff --git a/xen/include/xen/device_tree.h b/xen/include/xen/device_tree.h
index da153a5..0aecbe0 100644
--- a/xen/include/xen/device_tree.h
+++ b/xen/include/xen/device_tree.h
@@ -29,6 +29,11 @@ struct dt_device_match {
 const char *type;
 const char *compatible;
 const bool_t not_available;
+/*
+ * Property name to search for. We only search for the property's
+ * existence.
+ */
+const char *prop;
 const void *data;
 };
 
@@ -36,11 +41,13 @@ struct dt_device_match {
 #define __DT_MATCH_TYPE(typ).type = typ
 #define __DT_MATCH_COMPATIBLE(compat)   .compatible = compat
 #define __DT_MATCH_NOT_AVAILABLE()  .not_available = 1
+#define __DT_MATCH_PROP(p)  .prop = p
 
 #define DT_MATCH_PATH(p){ __DT_MATCH_PATH(p) }
 #define DT_MATCH_TYPE(typ)  { __DT_MATCH_TYPE(typ) }
 #define DT_MATCH_COMPATIBLE(compat) { __DT_MATCH_COMPATIBLE(compat) }
 #define DT_MATCH_NOT_AVAILABLE(){ __DT_MATCH_NOT_AVAILABLE() }
+#define DT_MATCH_PROP(p){ __DT_MATCH_PROP(p) }
 
 typedef u32 dt_phandle;
 
-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel