Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

2017-08-08 Thread Stefano Stabellini 
On Tue, 8 Aug 2017, Julien Grall wrote:
> On 01/08/17 18:13, Oleksandr Tyshchenko wrote:
> > Hi, Julien
> > 
> > On Tue, Aug 1, 2017 at 3:27 PM, Julien Grall  wrote:
> > > On 26/07/17 16:09, Oleksandr Tyshchenko wrote:
> > > > 
> > > > From: Oleksandr Tyshchenko 
> > > > 
> > > > Hi, all.
> > > 
> > > 
> > > Hi,
> > > 
> > > Please CC maintainers and any relevant person on the cover letter. This is
> > > quite useful to have in the inbox.
> > Yes. I CCed guys who, I think, are/were involved in IPMMU-VMSA
> > development from Linux side +
> > IOMMU maintainers (mostly ARM). Sorry, if I missed someone or mistakenly
> > added.
> > 
> > > 
> > > > The purpose of this patch series is to add IPMMU-VMSA support to Xen on
> > > > ARM.
> > > > It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car
> > > > Gen3 SoCs (ARM).
> > > > And this IOMMU can't share the page table with the CPU since it doesn't
> > > > use the same page-table format
> > > > as the CPU on ARM therefore I name it "Non-shared" IOMMU.
> > > > This all means that current patch series must be based on "Non-shared"
> > > > IOMMU support [1]
> > > > for the IPMMU-VMSA to be functional inside Xen.
> > > > 
> > > > The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly
> > > > ported from BSP for Linux the vendor provides:
> > > > git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
> > > > rcar-3.5.3
> > > 
> > > 
> > > I think this is probably a good starting point to discuss about IOMMU
> > > support in Xen. I skimmed through the patches and saw the words "rfc" and
> > > "ported from BSP".
> > Well, at the time of porting IPMMU-VMSA driver, BSP [1] had more
> > complete support than mainline [2]
> > and seems to have at the moment.
> > For example, mainline driver still has single IPMMU context while BSP
> > driver can have up to 8 contexts,
> > the maximum uTLBs mainline driver can handle is 32, but for BSP driver
> > this value was increased to 48, etc.
> > But, I see attempts to get all required support in [3]. So, when this
> > support reaches upsteam, I hope,
> > it won't be a big problem to rebase on mainline driver if we decide to
> > align with it.
> 
> My main concern here is this driver haven't had a thorough review by the Linux
> community.
> 
> When we ported the SMMUv{1,2} driver we knew the Linux community was happy
> with it and hence adapting for Xen was not a big deal. There are only few
> limited changes in the code from Linux.
> 
> Looking at patch #2, #6, #7, the changes don't seem limited in the code from
> Linux + it is a driver from a BSP. The code really needs to be very close to
> make the port from Linux really worth it.
> 
> Stefano do you have any opinion?

Yes, the fact that a driver is already in Linux gives us a guarantee
that it had been reviewed before. This driver doesn't come with that
guarantee, that means that one of us would have to review it. Otherwise,
we could take the contribution and mark it as "unsupported/experimental"
like we do for many new features until we think it is stable enough.

Regardless, I think you are right that there is no gain in trying to
make this patch series "compatible" with the original Linux driver,
because the driver isn't in Linux. We might as well take a clean
contribution to Xen without any #if 0.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

2017-08-08 Thread Julien Grall 



On 01/08/17 18:13, Oleksandr Tyshchenko wrote:

Hi, Julien

On Tue, Aug 1, 2017 at 3:27 PM, Julien Grall  wrote:

On 26/07/17 16:09, Oleksandr Tyshchenko wrote:


From: Oleksandr Tyshchenko 

Hi, all.



Hi,

Please CC maintainers and any relevant person on the cover letter. This is
quite useful to have in the inbox.

Yes. I CCed guys who, I think, are/were involved in IPMMU-VMSA
development from Linux side +
IOMMU maintainers (mostly ARM). Sorry, if I missed someone or mistakenly added.




The purpose of this patch series is to add IPMMU-VMSA support to Xen on
ARM.
It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car
Gen3 SoCs (ARM).
And this IOMMU can't share the page table with the CPU since it doesn't
use the same page-table format
as the CPU on ARM therefore I name it "Non-shared" IOMMU.
This all means that current patch series must be based on "Non-shared"
IOMMU support [1]
for the IPMMU-VMSA to be functional inside Xen.

The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly
ported from BSP for Linux the vendor provides:
git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
rcar-3.5.3



I think this is probably a good starting point to discuss about IOMMU
support in Xen. I skimmed through the patches and saw the words "rfc" and
"ported from BSP".

Well, at the time of porting IPMMU-VMSA driver, BSP [1] had more
complete support than mainline [2]
and seems to have at the moment.
For example, mainline driver still has single IPMMU context while BSP
driver can have up to 8 contexts,
the maximum uTLBs mainline driver can handle is 32, but for BSP driver
this value was increased to 48, etc.
But, I see attempts to get all required support in [3]. So, when this
support reaches upsteam, I hope,
it won't be a big problem to rebase on mainline driver if we decide to
align with it.


My main concern here is this driver haven't had a thorough review by the 
Linux community.


When we ported the SMMUv{1,2} driver we knew the Linux community was 
happy with it and hence adapting for Xen was not a big deal. There are 
only few limited changes in the code from Linux.


Looking at patch #2, #6, #7, the changes don't seem limited in the code 
from Linux + it is a driver from a BSP. The code really needs to be very 
close to make the port from Linux really worth it.


Stefano do you have any opinion?





At the moment for IOMMU we rely on the Linux community to do the review, but
this is not the case here as it is an RFC.  I can definitely help to check
if it comply for Xen,

yes, please.


but I don't have the competence to tell whether it is
valid for the hardware.

We may want to find a compromise to get it merged in Xen, but surely we
don't want to build it by default at least until we had feedback from the
community about the validity of the code here.

agree.



As I mentioned above, we are currently borrowing drivers from Linux and
adapting for Xen. Today we support SMMUv{1,2} (we need to resync it) and
there are plan to add IPMMU-VMSA (this series) and SMMUv3.

It would be really nice to have IPMMU-VMSA support in Xen. Without
this support the SCF [4] we are developing right now
and even the Passthrough feature won't be fully functional on R-Car
Gen3 based boards powered by Xen hypervisor.


As said in the previous e-mail. This would be nice enhancement for Xen, 
but we need to decide in which form it will be upstreamed.






I am aware that Linux IOMMU subsystem has growing quite a lot making more
tricky to get support in Xen. I wanted to get feedback how complex from you
and Sameer how complex it was and whether we should consider doing our own.


Yes, the IPMMU-VMSA Linux driver relies on some Linux functional
(IOMMU/DMA/io-pgtable frameworks) the Xen doesn't have (it is
expected). So, it took *some time*
to make Linux driver happy inside Xen). Moreover, this all resulted in
the fact that the driver looks complicated a bit).
A lot of different wrappers, #if 0, code style, etc.
On the other hand, I think, I will be able to fairly quickly align
with new BSP, etc.

But, I really don't know should we continue to follow this direction
or not, perhaps it will depend on
how complex the entity is and how much things we must pull together
with it to make it happy.



Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

2017-08-01 Thread Oleksandr Tyshchenko 
Hi, Julien

On Tue, Aug 1, 2017 at 3:27 PM, Julien Grall  wrote:
> On 26/07/17 16:09, Oleksandr Tyshchenko wrote:
>>
>> From: Oleksandr Tyshchenko 
>>
>> Hi, all.
>
>
> Hi,
>
> Please CC maintainers and any relevant person on the cover letter. This is
> quite useful to have in the inbox.
Yes. I CCed guys who, I think, are/were involved in IPMMU-VMSA
development from Linux side +
IOMMU maintainers (mostly ARM). Sorry, if I missed someone or mistakenly added.

>
>> The purpose of this patch series is to add IPMMU-VMSA support to Xen on
>> ARM.
>> It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car
>> Gen3 SoCs (ARM).
>> And this IOMMU can't share the page table with the CPU since it doesn't
>> use the same page-table format
>> as the CPU on ARM therefore I name it "Non-shared" IOMMU.
>> This all means that current patch series must be based on "Non-shared"
>> IOMMU support [1]
>> for the IPMMU-VMSA to be functional inside Xen.
>>
>> The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly
>> ported from BSP for Linux the vendor provides:
>> git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git
>> rcar-3.5.3
>
>
> I think this is probably a good starting point to discuss about IOMMU
> support in Xen. I skimmed through the patches and saw the words "rfc" and
> "ported from BSP".
Well, at the time of porting IPMMU-VMSA driver, BSP [1] had more
complete support than mainline [2]
and seems to have at the moment.
For example, mainline driver still has single IPMMU context while BSP
driver can have up to 8 contexts,
the maximum uTLBs mainline driver can handle is 32, but for BSP driver
this value was increased to 48, etc.
But, I see attempts to get all required support in [3]. So, when this
support reaches upsteam, I hope,
it won't be a big problem to rebase on mainline driver if we decide to
align with it.

>
> At the moment for IOMMU we rely on the Linux community to do the review, but
> this is not the case here as it is an RFC.  I can definitely help to check
> if it comply for Xen,
yes, please.

> but I don't have the competence to tell whether it is
> valid for the hardware.
>
> We may want to find a compromise to get it merged in Xen, but surely we
> don't want to build it by default at least until we had feedback from the
> community about the validity of the code here.
agree.

>
> As I mentioned above, we are currently borrowing drivers from Linux and
> adapting for Xen. Today we support SMMUv{1,2} (we need to resync it) and
> there are plan to add IPMMU-VMSA (this series) and SMMUv3.
It would be really nice to have IPMMU-VMSA support in Xen. Without
this support the SCF [4] we are developing right now
and even the Passthrough feature won't be fully functional on R-Car
Gen3 based boards powered by Xen hypervisor.

>
> I am aware that Linux IOMMU subsystem has growing quite a lot making more
> tricky to get support in Xen. I wanted to get feedback how complex from you
> and Sameer how complex it was and whether we should consider doing our own.

Yes, the IPMMU-VMSA Linux driver relies on some Linux functional
(IOMMU/DMA/io-pgtable frameworks) the Xen doesn't have (it is
expected). So, it took *some time*
to make Linux driver happy inside Xen). Moreover, this all resulted in
the fact that the driver looks complicated a bit).
A lot of different wrappers, #if 0, code style, etc.
On the other hand, I think, I will be able to fairly quickly align
with new BSP, etc.

But, I really don't know should we continue to follow this direction
or not, perhaps it will depend on
how complex the entity is and how much things we must pull together
with it to make it happy.

>
>> Patch series was rebased on Xen 4.9.0 release and tested on Renesas R-Car
>> Gen3 H3 ES2.0/M3 based boards
>> with devices assigned to different domains.
>>
>> You can find patch series here:
>> repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_v2
>>
>> P.S. There is one more patch which needs to be brought back to life [2]
>> Any reasons why this patch hasn't been upstremed yet?
>
>
> The series didn't make it upstream. Feel free to resend it separately.
ok.

>
>>
>> Thank you.
>>
>> [1] [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM
>> https://www.mail-archive.com/xen-devel@lists.xen.org/msg115901.html
>>
>> [2] [Xen-devel] [PATCH v8 02/28] xen: Add log2 functionality
>> https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg00031.html
>>
>> Oleksandr Tyshchenko (7):
>>   iommu/arm: ipmmu-vmsa: Add IPMMU-VMSA support
>>   iommu/arm: ipmmu-vmsa: Add Xen changes for main driver
>>   iommu/arm: ipmmu-vmsa: Add io-pgtables support
>>   iommu/arm: ipmmu-vmsa: Add Xen changes for io-pgtables
>>   iommu/arm: Build IPMMU-VMSA related stuff
>>   iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously
>>   iommu/arm: ipmmu-vmsa: Enable VMSAv8-64 mode if IPMMU HW supports it
>>
>>  xen/drivers/passthrough/arm/Makefile

Re: [Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

2017-08-01 Thread Julien Grall 

On 26/07/17 16:09, Oleksandr Tyshchenko wrote:

From: Oleksandr Tyshchenko 

Hi, all.


Hi,

Please CC maintainers and any relevant person on the cover letter. This 
is quite useful to have in the inbox.



The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.
It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car Gen3 
SoCs (ARM).
And this IOMMU can't share the page table with the CPU since it doesn't use the 
same page-table format
as the CPU on ARM therefore I name it "Non-shared" IOMMU.
This all means that current patch series must be based on "Non-shared" IOMMU 
support [1]
for the IPMMU-VMSA to be functional inside Xen.

The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly ported 
from BSP for Linux the vendor provides:
git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git rcar-3.5.3


I think this is probably a good starting point to discuss about IOMMU 
support in Xen. I skimmed through the patches and saw the words "rfc" 
and "ported from BSP".


At the moment for IOMMU we rely on the Linux community to do the review, 
but this is not the case here as it is an RFC.  I can definitely help to 
check if it comply for Xen, but I don't have the competence to tell 
whether it is valid for the hardware.


We may want to find a compromise to get it merged in Xen, but surely we 
don't want to build it by default at least until we had feedback from 
the community about the validity of the code here.


As I mentioned above, we are currently borrowing drivers from Linux and 
adapting for Xen. Today we support SMMUv{1,2} (we need to resync it) and 
there are plan to add IPMMU-VMSA (this series) and SMMUv3.


I am aware that Linux IOMMU subsystem has growing quite a lot making 
more tricky to get support in Xen. I wanted to get feedback how complex 
from you and Sameer how complex it was and whether we should consider 
doing our own.



Patch series was rebased on Xen 4.9.0 release and tested on Renesas R-Car Gen3 
H3 ES2.0/M3 based boards
with devices assigned to different domains.

You can find patch series here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_v2

P.S. There is one more patch which needs to be brought back to life [2]
Any reasons why this patch hasn't been upstremed yet?


The series didn't make it upstream. Feel free to resend it separately.



Thank you.

[1] [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM
https://www.mail-archive.com/xen-devel@lists.xen.org/msg115901.html

[2] [Xen-devel] [PATCH v8 02/28] xen: Add log2 functionality
https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg00031.html

Oleksandr Tyshchenko (7):
  iommu/arm: ipmmu-vmsa: Add IPMMU-VMSA support
  iommu/arm: ipmmu-vmsa: Add Xen changes for main driver
  iommu/arm: ipmmu-vmsa: Add io-pgtables support
  iommu/arm: ipmmu-vmsa: Add Xen changes for io-pgtables
  iommu/arm: Build IPMMU-VMSA related stuff
  iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously
  iommu/arm: ipmmu-vmsa: Enable VMSAv8-64 mode if IPMMU HW supports it

 xen/drivers/passthrough/arm/Makefile |3 +
 xen/drivers/passthrough/arm/io-pgtable-arm.c | 1331 +
 xen/drivers/passthrough/arm/io-pgtable.c |   91 +
 xen/drivers/passthrough/arm/io-pgtable.h |  220 +++
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 2611 ++
 5 files changed, 4256 insertions(+)
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable-arm.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.h
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c



Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [RFC PATCH v1 0/7] IPMMU-VMSA support on ARM

2017-07-26 Thread Oleksandr Tyshchenko 
From: Oleksandr Tyshchenko 

Hi, all.

The purpose of this patch series is to add IPMMU-VMSA support to Xen on ARM.
It is VMSA-compatible IOMMU that integrated in the newest Renesas R-Car Gen3 
SoCs (ARM).
And this IOMMU can't share the page table with the CPU since it doesn't use the 
same page-table format
as the CPU on ARM therefore I name it "Non-shared" IOMMU.
This all means that current patch series must be based on "Non-shared" IOMMU 
support [1]
for the IPMMU-VMSA to be functional inside Xen.

The IPMMU-VMSA driver as well as the ARM LPAE allocator were directly ported 
from BSP for Linux the vendor provides:
git://git.kernel.org/pub/scm/linux/kernel/git/horms/renesas-bsp.git rcar-3.5.3

Patch series was rebased on Xen 4.9.0 release and tested on Renesas R-Car Gen3 
H3 ES2.0/M3 based boards
with devices assigned to different domains.

You can find patch series here:
repo: https://github.com/otyshchenko1/xen.git branch: ipmmu_v2

P.S. There is one more patch which needs to be brought back to life [2]
Any reasons why this patch hasn't been upstremed yet?

Thank you.

[1] [Xen-devel] [PATCH v2 00/13] "Non-shared" IOMMU support on ARM
https://www.mail-archive.com/xen-devel@lists.xen.org/msg115901.html

[2] [Xen-devel] [PATCH v8 02/28] xen: Add log2 functionality
https://lists.xenproject.org/archives/html/xen-devel/2016-02/msg00031.html

Oleksandr Tyshchenko (7):
  iommu/arm: ipmmu-vmsa: Add IPMMU-VMSA support
  iommu/arm: ipmmu-vmsa: Add Xen changes for main driver
  iommu/arm: ipmmu-vmsa: Add io-pgtables support
  iommu/arm: ipmmu-vmsa: Add Xen changes for io-pgtables
  iommu/arm: Build IPMMU-VMSA related stuff
  iommu/arm: ipmmu-vmsa: Deallocate page table asynchronously
  iommu/arm: ipmmu-vmsa: Enable VMSAv8-64 mode if IPMMU HW supports it

 xen/drivers/passthrough/arm/Makefile |3 +
 xen/drivers/passthrough/arm/io-pgtable-arm.c | 1331 +
 xen/drivers/passthrough/arm/io-pgtable.c |   91 +
 xen/drivers/passthrough/arm/io-pgtable.h |  220 +++
 xen/drivers/passthrough/arm/ipmmu-vmsa.c | 2611 ++
 5 files changed, 4256 insertions(+)
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable-arm.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.c
 create mode 100644 xen/drivers/passthrough/arm/io-pgtable.h
 create mode 100644 xen/drivers/passthrough/arm/ipmmu-vmsa.c

-- 
2.7.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel