Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Thu, Jul 10, 2014 at 11:32:16PM +0100, Olav Haugan wrote:
 On 7/9/2014 3:54 AM, Will Deacon wrote:
  On Wed, Jul 09, 2014 at 02:07:38AM +0100, Olav Haugan wrote:
  So how does an algorithm figure this out in both my examples? The
  algorithm would have to know about both (all) bus masters and their
  stream IDs for a specific SMMU. If the algorithm operates on the set of
  stream IDs for one bus master at a time the algorithm has no way of
  knowing which bits can be ignored since it doesn't know the value of the
  other stream IDs for the other bus masters and thus could potentially
  create a mask that could cause a stream ID to match in two different
  entries.
  
  Complete knowledge of the system topology (i.e. all bus masters) is a
  requirement for being able to configure the SMMU correctly if you want to
  guarantee that you don't have SMR aliasing issues.
 
 So you agree that an algorithm needs to know about all the bus
 masters/stream IDs for a specific IOMMU before it can figure out the
 StreamID masks and how many SMRs can be allocated to a specific bus
 master? Andreas's algorithm does not know about the other bus
 masters/stream IDs. It operates on one bus master at a time.

Right, but it can certainly be improved. There are certain things you can do
without complete knowledge, as I mentioned previously (if you can have
densely packed power-of-2 aligned/sized regions).

  I am not familiar with Andreas's proposal. Do you have a link?
 
http://marc.info/?l=linux-arm-kernelm=139110598005846w=2
 
  Unless I am mistaken the algorithm works on one bus master at a time. I
  don't think that will work.
  
  IIRC, it works for densely packed SIDs on the master, so it tries to build
  up power-of-2 sized groups for that master then mops up the rest with
  individual entries.
 
 I ran the algorithm through a few trivial cases:
 
 1)
 Stream IDs: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Number of SMRs: 9
 
 In this case the algorithm decided to set mask to 0 for all entries
 using up 8 of the SMRs.

Well think about what it's doing... we don't know about SID 0x20, for
example so there's not much it can do.

 2) Same Stream IDs but only 2 SMRs.
 The algorithm gave an error saying I did not have enough SMRs.

There's a reason this didn't get merged, and it would be great if you could
try to improve the situation ;).

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Olav Haugan]

On 7/9/2014 3:54 AM, Will Deacon wrote:
 On Wed, Jul 09, 2014 at 02:07:38AM +0100, Olav Haugan wrote:
 On 6/30/2014 2:52 AM, Will Deacon wrote:
 On Fri, Jun 27, 2014 at 11:23:27PM +0100, Olav Haugan wrote:
 Lets say I have an IOMMU with 2 masters and 2 SMRn slots with the
 following stream IDs coming from the masters:

 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x30

 To make this work I would program SMR[0] with StreamID 0x20 and mask 0xF
 to ignore lower 4 bits. SMR[1] would just be StreamID 0x30 with mask 0x0.

 However, I could also have an IOMMU with 2 masters and 9 SMRn slots with
 the following stream IDs:

 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x29

 Here I would program all SMRn and leave the mask to be 0 for all SMRn's.
 So how do I detect when to apply a mask or not?

 You would aim to use the smallest number of SMRs per master possible.
 You could probably use:

   Master 1: SMR[0].id == 0x20, SMR[0].mask = 0x07
 SMR[1].id == 0x28, SMR[1].mask = 0x00

   Master 2: SMR[2].id == 0x29, SMR[2].mask = 0x00

 So how does an algorithm figure this out in both my examples? The
 algorithm would have to know about both (all) bus masters and their
 stream IDs for a specific SMMU. If the algorithm operates on the set of
 stream IDs for one bus master at a time the algorithm has no way of
 knowing which bits can be ignored since it doesn't know the value of the
 other stream IDs for the other bus masters and thus could potentially
 create a mask that could cause a stream ID to match in two different
 entries.
 
 Complete knowledge of the system topology (i.e. all bus masters) is a
 requirement for being able to configure the SMMU correctly if you want to
 guarantee that you don't have SMR aliasing issues.

So you agree that an algorithm needs to know about all the bus
masters/stream IDs for a specific IOMMU before it can figure out the
StreamID masks and how many SMRs can be allocated to a specific bus
master? Andreas's algorithm does not know about the other bus
masters/stream IDs. It operates on one bus master at a time.

 I am not familiar with Andreas's proposal. Do you have a link?

   http://marc.info/?l=linux-arm-kernelm=139110598005846w=2

 Unless I am mistaken the algorithm works on one bus master at a time. I
 don't think that will work.
 
 IIRC, it works for densely packed SIDs on the master, so it tries to build
 up power-of-2 sized groups for that master then mops up the rest with
 individual entries.

I ran the algorithm through a few trivial cases:

1)
Stream IDs: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
Number of SMRs: 9

In this case the algorithm decided to set mask to 0 for all entries
using up 8 of the SMRs.

2) Same Stream IDs but only 2 SMRs.
The algorithm gave an error saying I did not have enough SMRs.

Thanks,

Olav

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jul 09, 2014 at 02:07:38AM +0100, Olav Haugan wrote:
 On 6/30/2014 2:52 AM, Will Deacon wrote:
  On Fri, Jun 27, 2014 at 11:23:27PM +0100, Olav Haugan wrote:
  Lets say I have an IOMMU with 2 masters and 2 SMRn slots with the
  following stream IDs coming from the masters:
 
  Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
  Master 2: 0x30
 
  To make this work I would program SMR[0] with StreamID 0x20 and mask 0xF
  to ignore lower 4 bits. SMR[1] would just be StreamID 0x30 with mask 0x0.
 
  However, I could also have an IOMMU with 2 masters and 9 SMRn slots with
  the following stream IDs:
 
  Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
  Master 2: 0x29
 
  Here I would program all SMRn and leave the mask to be 0 for all SMRn's.
  So how do I detect when to apply a mask or not?
  
  You would aim to use the smallest number of SMRs per master possible.
  You could probably use:
  
Master 1: SMR[0].id == 0x20, SMR[0].mask = 0x07
  SMR[1].id == 0x28, SMR[1].mask = 0x00
  
Master 2: SMR[2].id == 0x29, SMR[2].mask = 0x00
 
 So how does an algorithm figure this out in both my examples? The
 algorithm would have to know about both (all) bus masters and their
 stream IDs for a specific SMMU. If the algorithm operates on the set of
 stream IDs for one bus master at a time the algorithm has no way of
 knowing which bits can be ignored since it doesn't know the value of the
 other stream IDs for the other bus masters and thus could potentially
 create a mask that could cause a stream ID to match in two different
 entries.

Complete knowledge of the system topology (i.e. all bus masters) is a
requirement for being able to configure the SMMU correctly if you want to
guarantee that you don't have SMR aliasing issues.

  I am not familiar with Andreas's proposal. Do you have a link?
  
http://marc.info/?l=linux-arm-kernelm=139110598005846w=2
 
 Unless I am mistaken the algorithm works on one bus master at a time. I
 don't think that will work.

IIRC, it works for densely packed SIDs on the master, so it tries to build
up power-of-2 sized groups for that master then mops up the rest with
individual entries.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Olav Haugan]

On 6/30/2014 2:52 AM, Will Deacon wrote:
 Hi Olav,
 
 On Fri, Jun 27, 2014 at 11:23:27PM +0100, Olav Haugan wrote:
 On 6/25/2014 2:18 AM, Will Deacon wrote:
 Why can't it be dynamically detected? Whilst the StreamIDs are fixed in
 hardware (from the SMMU architecture perspective), the SMRs are completely
 programmable. Why doesn't something like Andreas's proposal work for you?
 The idea there was to find the constant bits among the StreamIDs for a
 master and create the mask accordingly.

 Lets say I have an IOMMU with 2 masters and 2 SMRn slots with the
 following stream IDs coming from the masters:

 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x30

 To make this work I would program SMR[0] with StreamID 0x20 and mask 0xF
 to ignore lower 4 bits. SMR[1] would just be StreamID 0x30 with mask 0x0.

 However, I could also have an IOMMU with 2 masters and 9 SMRn slots with
 the following stream IDs:

 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x29

 Here I would program all SMRn and leave the mask to be 0 for all SMRn's.
 So how do I detect when to apply a mask or not?
 
 You would aim to use the smallest number of SMRs per master possible.
 You could probably use:
 
   Master 1: SMR[0].id == 0x20, SMR[0].mask = 0x07
 SMR[1].id == 0x28, SMR[1].mask = 0x00
 
   Master 2: SMR[2].id == 0x29, SMR[2].mask = 0x00

So how does an algorithm figure this out in both my examples? The
algorithm would have to know about both (all) bus masters and their
stream IDs for a specific SMMU. If the algorithm operates on the set of
stream IDs for one bus master at a time the algorithm has no way of
knowing which bits can be ignored since it doesn't know the value of the
other stream IDs for the other bus masters and thus could potentially
create a mask that could cause a stream ID to match in two different
entries.

 I am not familiar with Andreas's proposal. Do you have a link?
 
   http://marc.info/?l=linux-arm-kernelm=139110598005846w=2

Unless I am mistaken the algorithm works on one bus master at a time. I
don't think that will work.

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

Hi Olav,

On Fri, Jun 27, 2014 at 11:23:27PM +0100, Olav Haugan wrote:
 On 6/25/2014 2:18 AM, Will Deacon wrote:
  Why can't it be dynamically detected? Whilst the StreamIDs are fixed in
  hardware (from the SMMU architecture perspective), the SMRs are completely
  programmable. Why doesn't something like Andreas's proposal work for you?
  The idea there was to find the constant bits among the StreamIDs for a
  master and create the mask accordingly.
  
 Lets say I have an IOMMU with 2 masters and 2 SMRn slots with the
 following stream IDs coming from the masters:
 
 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x30
 
 To make this work I would program SMR[0] with StreamID 0x20 and mask 0xF
 to ignore lower 4 bits. SMR[1] would just be StreamID 0x30 with mask 0x0.
 
 However, I could also have an IOMMU with 2 masters and 9 SMRn slots with
 the following stream IDs:
 
 Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
 Master 2: 0x29
 
 Here I would program all SMRn and leave the mask to be 0 for all SMRn's.
 So how do I detect when to apply a mask or not?

You would aim to use the smallest number of SMRs per master possible.
You could probably use:

  Master 1: SMR[0].id == 0x20, SMR[0].mask = 0x07
SMR[1].id == 0x28, SMR[1].mask = 0x00

  Master 2: SMR[2].id == 0x29, SMR[2].mask = 0x00

 I am not familiar with Andreas's proposal. Do you have a link?

  http://marc.info/?l=linux-arm-kernelm=139110598005846w=2

Note that since Calxeda went under, Andreas is unfortunately no longer
working on this.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Olav Haugan]

On 6/25/2014 2:18 AM, Will Deacon wrote:
 On Tue, Jun 24, 2014 at 10:35:54PM +0100, Olav Haugan wrote:
 On 6/24/2014 11:11 AM, Will Deacon wrote:
 On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
 On 6/24/2014 2:18 AM, Will Deacon wrote:
 On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
 We have multiple-master SMMUs and each master emits a variable number of
 StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
 for this) to the StreamIDs due to limited number of StreamID 2 Context
 Bank entries in the SMMU. If my understanding is correct we would
 represent this in the DT like this:

  iommu {
  #address-cells = 2;
  #size-cells = 0;
  };

  master@a {
  ...
  iommus = iommu StreamID0 MASK0,
   iommu StreamID1 MASK1,
   iommu StreamID2 MASK2;
  };

 Stupid question, but why not simply describe the masked IDs? What use does
 the `raw' ID have to Linux?

 We do describe the masked StreamID (SID) but we need to specify the mask
 that the SMMU should apply to the incoming SIDs, right?

 We have a bus master that emits 43 unique SIDs. However, we have only 40
 SMMU_SMRn registers in the SMMU. So we need to mask out some of the
 incoming SID bits so that the 43 SIDs can match one of 40 entries in the
 SMR.

 Hmm, so you're talking about stream matching, right? That doesn't belong in
 the device-tree. I appreciate that the current driver does a terrible job at
 allocating the SMRs (it's bloody difficult!), but we should try to improve
 the dynamic behaviour instead of moving configuration of the SMMU out into
 device-tree, where it's inflexible at best.

 I am talking about SMMU_SMRn[MASK] register bits. This is not something
 that can be dynamically detected at run-time. It is configuration at the
 same level as the actual StreamIDs.
 
 Why can't it be dynamically detected? Whilst the StreamIDs are fixed in
 hardware (from the SMMU architecture perspective), the SMRs are completely
 programmable. Why doesn't something like Andreas's proposal work for you?
 The idea there was to find the constant bits among the StreamIDs for a
 master and create the mask accordingly.
 
Lets say I have an IOMMU with 2 masters and 2 SMRn slots with the
following stream IDs coming from the masters:

Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
Master 2: 0x30

To make this work I would program SMR[0] with StreamID 0x20 and mask 0xF
to ignore lower 4 bits. SMR[1] would just be StreamID 0x30 with mask 0x0.

However, I could also have an IOMMU with 2 masters and 9 SMRn slots with
the following stream IDs:

Master 1: 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28
Master 2: 0x29

Here I would program all SMRn and leave the mask to be 0 for all SMRn's.
So how do I detect when to apply a mask or not?

I am not familiar with Andreas's proposal. Do you have a link?

Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 24, 2014 at 07:20:56PM +0100, Arnd Bergmann wrote:
 On Tuesday 24 June 2014 19:11:50 Will Deacon wrote:
  On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
   We do describe the masked StreamID (SID) but we need to specify the mask
   that the SMMU should apply to the incoming SIDs, right?
   
   We have a bus master that emits 43 unique SIDs. However, we have only 40
   SMMU_SMRn registers in the SMMU. So we need to mask out some of the
   incoming SID bits so that the 43 SIDs can match one of 40 entries in the
   SMR.
  
  Hmm, so you're talking about stream matching, right? That doesn't belong in
  the device-tree. I appreciate that the current driver does a terrible job at
  allocating the SMRs (it's bloody difficult!), but we should try to improve
  the dynamic behaviour instead of moving configuration of the SMMU out into
  device-tree, where it's inflexible at best.
  
  There have been patches previously posted by Andreas Herrmann helping here.
  I'd be glad to see them revived.
 
 Note that there are areas where we have in the past decided that dynamic
 configuration is just too hard for the kernel to do and that we're better
 off putting the configuration into DT. Pinctrl and clocks are at least
 partially in that category.
 
 It's always best if you can get the kernel to do things in the ideal
 way where that is possible, but getting there may be just not worth it.
 
 I have no idea where it should be for SMMU, but it's something to consider:
 if you can take reasonable shortcuts by reading parts of the configuration
 from DT, you may just as well do that.

I treat this in the same manner as the topology bindings we discussed
previously; we should do a best-effort to configure things dynamically and
solve corner-cases and quirks as special cases.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 24, 2014 at 10:35:54PM +0100, Olav Haugan wrote:
 On 6/24/2014 11:11 AM, Will Deacon wrote:
  On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
  On 6/24/2014 2:18 AM, Will Deacon wrote:
  On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
  We have multiple-master SMMUs and each master emits a variable number of
  StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
  for this) to the StreamIDs due to limited number of StreamID 2 Context
  Bank entries in the SMMU. If my understanding is correct we would
  represent this in the DT like this:
 
   iommu {
   #address-cells = 2;
   #size-cells = 0;
   };
 
   master@a {
   ...
   iommus = iommu StreamID0 MASK0,
iommu StreamID1 MASK1,
iommu StreamID2 MASK2;
   };
 
  Stupid question, but why not simply describe the masked IDs? What use does
  the `raw' ID have to Linux?
 
  We do describe the masked StreamID (SID) but we need to specify the mask
  that the SMMU should apply to the incoming SIDs, right?
 
  We have a bus master that emits 43 unique SIDs. However, we have only 40
  SMMU_SMRn registers in the SMMU. So we need to mask out some of the
  incoming SID bits so that the 43 SIDs can match one of 40 entries in the
  SMR.
  
  Hmm, so you're talking about stream matching, right? That doesn't belong in
  the device-tree. I appreciate that the current driver does a terrible job at
  allocating the SMRs (it's bloody difficult!), but we should try to improve
  the dynamic behaviour instead of moving configuration of the SMMU out into
  device-tree, where it's inflexible at best.
 
 I am talking about SMMU_SMRn[MASK] register bits. This is not something
 that can be dynamically detected at run-time. It is configuration at the
 same level as the actual StreamIDs.

Why can't it be dynamically detected? Whilst the StreamIDs are fixed in
hardware (from the SMMU architecture perspective), the SMRs are completely
programmable. Why doesn't something like Andreas's proposal work for you?
The idea there was to find the constant bits among the StreamIDs for a
master and create the mask accordingly.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 25 June 2014 10:17:02 Will Deacon wrote:
 On Tue, Jun 24, 2014 at 07:20:56PM +0100, Arnd Bergmann wrote:
  On Tuesday 24 June 2014 19:11:50 Will Deacon wrote:
   On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
We do describe the masked StreamID (SID) but we need to specify the mask
that the SMMU should apply to the incoming SIDs, right?

We have a bus master that emits 43 unique SIDs. However, we have only 40
SMMU_SMRn registers in the SMMU. So we need to mask out some of the
incoming SID bits so that the 43 SIDs can match one of 40 entries in the
SMR.
   
   Hmm, so you're talking about stream matching, right? That doesn't belong 
   in
   the device-tree. I appreciate that the current driver does a terrible job 
   at
   allocating the SMRs (it's bloody difficult!), but we should try to improve
   the dynamic behaviour instead of moving configuration of the SMMU out into
   device-tree, where it's inflexible at best.
   
   There have been patches previously posted by Andreas Herrmann helping 
   here.
   I'd be glad to see them revived.
  
  Note that there are areas where we have in the past decided that dynamic
  configuration is just too hard for the kernel to do and that we're better
  off putting the configuration into DT. Pinctrl and clocks are at least
  partially in that category.
  
  It's always best if you can get the kernel to do things in the ideal
  way where that is possible, but getting there may be just not worth it.
  
  I have no idea where it should be for SMMU, but it's something to consider:
  if you can take reasonable shortcuts by reading parts of the configuration
  from DT, you may just as well do that.
 
 I treat this in the same manner as the topology bindings we discussed
 previously; we should do a best-effort to configure things dynamically and
 solve corner-cases and quirks as special cases.

I think the situation is a bit different here: It's less about the corner
cases for the SMMU, but about the question whether it makes more sense to
have the kernel figure out the settings, or have them come from DT
all the time.

As I said, I can't tell which approach is best here, but it sounds to
me we should either do dynamic configuration and get it right, or
hardcode the configuration it all the time if we can't.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 25, 2014 at 10:27:50AM +0100, Arnd Bergmann wrote:
 On Wednesday 25 June 2014 10:17:02 Will Deacon wrote:
  On Tue, Jun 24, 2014 at 07:20:56PM +0100, Arnd Bergmann wrote:
   On Tuesday 24 June 2014 19:11:50 Will Deacon wrote:
On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
 We do describe the masked StreamID (SID) but we need to specify the 
 mask
 that the SMMU should apply to the incoming SIDs, right?
 
 We have a bus master that emits 43 unique SIDs. However, we have only 
 40
 SMMU_SMRn registers in the SMMU. So we need to mask out some of the
 incoming SID bits so that the 43 SIDs can match one of 40 entries in 
 the
 SMR.

Hmm, so you're talking about stream matching, right? That doesn't 
belong in
the device-tree. I appreciate that the current driver does a terrible 
job at
allocating the SMRs (it's bloody difficult!), but we should try to 
improve
the dynamic behaviour instead of moving configuration of the SMMU out 
into
device-tree, where it's inflexible at best.

There have been patches previously posted by Andreas Herrmann helping 
here.
I'd be glad to see them revived.
   
   Note that there are areas where we have in the past decided that dynamic
   configuration is just too hard for the kernel to do and that we're better
   off putting the configuration into DT. Pinctrl and clocks are at least
   partially in that category.
   
   It's always best if you can get the kernel to do things in the ideal
   way where that is possible, but getting there may be just not worth it.
   
   I have no idea where it should be for SMMU, but it's something to 
   consider:
   if you can take reasonable shortcuts by reading parts of the configuration
   from DT, you may just as well do that.
  
  I treat this in the same manner as the topology bindings we discussed
  previously; we should do a best-effort to configure things dynamically and
  solve corner-cases and quirks as special cases.
 
 I think the situation is a bit different here: It's less about the corner
 cases for the SMMU, but about the question whether it makes more sense to
 have the kernel figure out the settings, or have them come from DT
 all the time.

But, as far as I can tell, this setting is basically `which bits are
constant among this set of IDs'.

 As I said, I can't tell which approach is best here, but it sounds to
 me we should either do dynamic configuration and get it right, or
 hardcode the configuration it all the time if we can't.

I disagree. If you have `sensible' StreamID allocations, doing this
dynamically should be straight-forward and gives the driver more flexibility
(e.g. we then have the option of combining SMR entries for different masters
if they are in the same domain). The dynamic approach also lends itself to
sanity-checking (it is IMPLEMENTATION DEFINED whether the SMMU detects SMR
aliases) and helps with virtualisation (forcing QEMU to generate these masks
in a device-tree for a guest using a virtual SMMU interface is very painful).

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 25 June 2014 10:38:25 Will Deacon wrote:
 On Wed, Jun 25, 2014 at 10:27:50AM +0100, Arnd Bergmann wrote:
  On Wednesday 25 June 2014 10:17:02 Will Deacon wrote:
   On Tue, Jun 24, 2014 at 07:20:56PM +0100, Arnd Bergmann wrote:
On Tuesday 24 June 2014 19:11:50 Will Deacon wrote:
 On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
  We do describe the masked StreamID (SID) but we need to specify the 
  mask
  that the SMMU should apply to the incoming SIDs, right?
  
  We have a bus master that emits 43 unique SIDs. However, we have 
  only 40
  SMMU_SMRn registers in the SMMU. So we need to mask out some of the
  incoming SID bits so that the 43 SIDs can match one of 40 entries 
  in the
  SMR.
 
 Hmm, so you're talking about stream matching, right? That doesn't 
 belong in
 the device-tree. I appreciate that the current driver does a terrible 
 job at
 allocating the SMRs (it's bloody difficult!), but we should try to 
 improve
 the dynamic behaviour instead of moving configuration of the SMMU out 
 into
 device-tree, where it's inflexible at best.
 
 There have been patches previously posted by Andreas Herrmann helping 
 here.
 I'd be glad to see them revived.

Note that there are areas where we have in the past decided that dynamic
configuration is just too hard for the kernel to do and that we're 
better
off putting the configuration into DT. Pinctrl and clocks are at least
partially in that category.

It's always best if you can get the kernel to do things in the ideal
way where that is possible, but getting there may be just not worth it.

I have no idea where it should be for SMMU, but it's something to 
consider:
if you can take reasonable shortcuts by reading parts of the 
configuration
from DT, you may just as well do that.
   
   I treat this in the same manner as the topology bindings we discussed
   previously; we should do a best-effort to configure things dynamically and
   solve corner-cases and quirks as special cases.
  
  I think the situation is a bit different here: It's less about the corner
  cases for the SMMU, but about the question whether it makes more sense to
  have the kernel figure out the settings, or have them come from DT
  all the time.
 
 But, as far as I can tell, this setting is basically `which bits are
 constant among this set of IDs'.
 
  As I said, I can't tell which approach is best here, but it sounds to
  me we should either do dynamic configuration and get it right, or
  hardcode the configuration it all the time if we can't.
 
 I disagree. If you have `sensible' StreamID allocations, doing this
 dynamically should be straight-forward and gives the driver more flexibility
 (e.g. we then have the option of combining SMR entries for different masters
 if they are in the same domain). The dynamic approach also lends itself to
 sanity-checking (it is IMPLEMENTATION DEFINED whether the SMMU detects SMR
 aliases) and helps with virtualisation (forcing QEMU to generate these masks
 in a device-tree for a guest using a virtual SMMU interface is very painful).

In which case do you think hardcoding is needed then? I'm still confused
why you think we need both, as your arguments seem to all be in favor of
dynamic configuration.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 25, 2014 at 10:48:31AM +0100, Arnd Bergmann wrote:
 On Wednesday 25 June 2014 10:38:25 Will Deacon wrote:
  On Wed, Jun 25, 2014 at 10:27:50AM +0100, Arnd Bergmann wrote:
   I think the situation is a bit different here: It's less about the corner
   cases for the SMMU, but about the question whether it makes more sense to
   have the kernel figure out the settings, or have them come from DT
   all the time.
  
  But, as far as I can tell, this setting is basically `which bits are
  constant among this set of IDs'.
  
   As I said, I can't tell which approach is best here, but it sounds to
   me we should either do dynamic configuration and get it right, or
   hardcode the configuration it all the time if we can't.
  
  I disagree. If you have `sensible' StreamID allocations, doing this
  dynamically should be straight-forward and gives the driver more flexibility
  (e.g. we then have the option of combining SMR entries for different masters
  if they are in the same domain). The dynamic approach also lends itself to
  sanity-checking (it is IMPLEMENTATION DEFINED whether the SMMU detects SMR
  aliases) and helps with virtualisation (forcing QEMU to generate these masks
  in a device-tree for a guest using a virtual SMMU interface is very 
  painful).
 
 In which case do you think hardcoding is needed then? I'm still confused
 why you think we need both, as your arguments seem to all be in favor of
 dynamic configuration.

So far, I've been avoiding the hardcoding. However, you could potentially
build a system with a small number of SMRs (compared to the number of
StreamIDs) and allocate the StreamIDs in such a way that I think the dynamic
configuration would be NP complete if we require an optimal SMR allocation.

However:

  (1) I don't know of a system where this is the case
  (2) Not much work has been done on improving the dynamic allocator yet

which is why I'm still favouring dynamic configuration in the driver.

The other thing I forgot to mention earlier is that we need to support
device hotplug in the future, so some level of dynamic configuration
will always be required.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 25 June 2014 10:57:36 Will Deacon wrote:
 So far, I've been avoiding the hardcoding. However, you could potentially
 build a system with a small number of SMRs (compared to the number of
 StreamIDs) and allocate the StreamIDs in such a way that I think the dynamic
 configuration would be NP complete if we require an optimal SMR allocation.
 
 However:
 
   (1) I don't know of a system where this is the case
   (2) Not much work has been done on improving the dynamic allocator yet
 
 which is why I'm still favouring dynamic configuration in the driver.
 
 The other thing I forgot to mention earlier is that we need to support
 device hotplug in the future, so some level of dynamic configuration
 will always be required.

Ok, got it. So we just hope that we can make dynamic configuration
work all the time, but if it all fails, then we come up with a
hardcoded configuration method.

I guess this could be done similarly to how we handle clocks on
a lot of systems: generally these are dynamic, but you have the
option to provide hints in the clock controller node about how
you expect things to be configured.

For the SMMU that could mean that (if we get into the situation you
describe), we add optional properties to the SMMU node itself
describing how we expect the SMRs to be used.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 25, 2014 at 11:12:13AM +0100, Arnd Bergmann wrote:
 On Wednesday 25 June 2014 10:57:36 Will Deacon wrote:
  So far, I've been avoiding the hardcoding. However, you could potentially
  build a system with a small number of SMRs (compared to the number of
  StreamIDs) and allocate the StreamIDs in such a way that I think the dynamic
  configuration would be NP complete if we require an optimal SMR allocation.
  
  However:
  
(1) I don't know of a system where this is the case
(2) Not much work has been done on improving the dynamic allocator yet
  
  which is why I'm still favouring dynamic configuration in the driver.
  
  The other thing I forgot to mention earlier is that we need to support
  device hotplug in the future, so some level of dynamic configuration
  will always be required.
 
 Ok, got it. So we just hope that we can make dynamic configuration
 work all the time, but if it all fails, then we come up with a
 hardcoded configuration method.
 
 I guess this could be done similarly to how we handle clocks on
 a lot of systems: generally these are dynamic, but you have the
 option to provide hints in the clock controller node about how
 you expect things to be configured.
 
 For the SMMU that could mean that (if we get into the situation you
 describe), we add optional properties to the SMMU node itself
 describing how we expect the SMRs to be used.

That sounds good to me! Thanks for the discussion.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
 On 5/30/2014 12:06 PM, Arnd Bergmann wrote:
  On Friday 30 May 2014 08:16:05 Rob Herring wrote:
  Presumably the ID would be the streamID on ARM's SMMU. How would a
  master with 8 streamIDs be described? This is what Calxeda midway has
  for SATA and I would expect that to be somewhat common. Either you
  need some ID masking or you'll have lots of duplication when you have
  windows.
  
  I don't understand the problem. If you have stream IDs 0 through 7,
  you would have
  
  master@a {
  ...
  iommus = smmu 0;
  };
  
  master@b {
  ...
  iommus = smmu 1;
  };
  
  ...
  
  master@12 {
  ...
  iommus = smmu 7;
  };
  
  and you don't need a window at all. Why would you need a mask of
  some sort?
 
 We have multiple-master SMMUs and each master emits a variable number of
 StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
 for this) to the StreamIDs due to limited number of StreamID 2 Context
 Bank entries in the SMMU. If my understanding is correct we would
 represent this in the DT like this:
 
   iommu {
   #address-cells = 2;
   #size-cells = 0;
   };
 
   master@a {
   ...
   iommus = iommu StreamID0 MASK0,
iommu StreamID1 MASK1,
iommu StreamID2 MASK2;
   };

Stupid question, but why not simply describe the masked IDs? What use does
the `raw' ID have to Linux?

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
 On 6/24/2014 2:18 AM, Will Deacon wrote:
  On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
  We have multiple-master SMMUs and each master emits a variable number of
  StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
  for this) to the StreamIDs due to limited number of StreamID 2 Context
  Bank entries in the SMMU. If my understanding is correct we would
  represent this in the DT like this:
 
 iommu {
 #address-cells = 2;
 #size-cells = 0;
 };
 
 master@a {
 ...
 iommus = iommu StreamID0 MASK0,
  iommu StreamID1 MASK1,
  iommu StreamID2 MASK2;
 };
  
  Stupid question, but why not simply describe the masked IDs? What use does
  the `raw' ID have to Linux?
 
 We do describe the masked StreamID (SID) but we need to specify the mask
 that the SMMU should apply to the incoming SIDs, right?
 
 We have a bus master that emits 43 unique SIDs. However, we have only 40
 SMMU_SMRn registers in the SMMU. So we need to mask out some of the
 incoming SID bits so that the 43 SIDs can match one of 40 entries in the
 SMR.

Hmm, so you're talking about stream matching, right? That doesn't belong in
the device-tree. I appreciate that the current driver does a terrible job at
allocating the SMRs (it's bloody difficult!), but we should try to improve
the dynamic behaviour instead of moving configuration of the SMMU out into
device-tree, where it's inflexible at best.

There have been patches previously posted by Andreas Herrmann helping here.
I'd be glad to see them revived.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Tuesday 24 June 2014 19:11:50 Will Deacon wrote:
 On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
  On 6/24/2014 2:18 AM, Will Deacon wrote:
   On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
   We have multiple-master SMMUs and each master emits a variable number of
   StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
   for this) to the StreamIDs due to limited number of StreamID 2 Context
   Bank entries in the SMMU. If my understanding is correct we would
   represent this in the DT like this:
  
  iommu {
  #address-cells = 2;
  #size-cells = 0;
  };
  
  master@a {
  ...
  iommus = iommu StreamID0 MASK0,
   iommu StreamID1 MASK1,
   iommu StreamID2 MASK2;
  };
   
   Stupid question, but why not simply describe the masked IDs? What use does
   the `raw' ID have to Linux?
  
  We do describe the masked StreamID (SID) but we need to specify the mask
  that the SMMU should apply to the incoming SIDs, right?
  
  We have a bus master that emits 43 unique SIDs. However, we have only 40
  SMMU_SMRn registers in the SMMU. So we need to mask out some of the
  incoming SID bits so that the 43 SIDs can match one of 40 entries in the
  SMR.
 
 Hmm, so you're talking about stream matching, right? That doesn't belong in
 the device-tree. I appreciate that the current driver does a terrible job at
 allocating the SMRs (it's bloody difficult!), but we should try to improve
 the dynamic behaviour instead of moving configuration of the SMMU out into
 device-tree, where it's inflexible at best.
 
 There have been patches previously posted by Andreas Herrmann helping here.
 I'd be glad to see them revived.

Note that there are areas where we have in the past decided that dynamic
configuration is just too hard for the kernel to do and that we're better
off putting the configuration into DT. Pinctrl and clocks are at least
partially in that category.

It's always best if you can get the kernel to do things in the ideal
way where that is possible, but getting there may be just not worth it.

I have no idea where it should be for SMMU, but it's something to consider:
if you can take reasonable shortcuts by reading parts of the configuration
from DT, you may just as well do that.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Olav Haugan]

On 6/24/2014 11:11 AM, Will Deacon wrote:
 On Tue, Jun 24, 2014 at 06:57:44PM +0100, Olav Haugan wrote:
 On 6/24/2014 2:18 AM, Will Deacon wrote:
 On Sat, Jun 21, 2014 at 12:16:25AM +0100, Olav Haugan wrote:
 We have multiple-master SMMUs and each master emits a variable number of
 StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
 for this) to the StreamIDs due to limited number of StreamID 2 Context
 Bank entries in the SMMU. If my understanding is correct we would
 represent this in the DT like this:

iommu {
#address-cells = 2;
#size-cells = 0;
};

master@a {
...
iommus = iommu StreamID0 MASK0,
 iommu StreamID1 MASK1,
 iommu StreamID2 MASK2;
};

 Stupid question, but why not simply describe the masked IDs? What use does
 the `raw' ID have to Linux?

 We do describe the masked StreamID (SID) but we need to specify the mask
 that the SMMU should apply to the incoming SIDs, right?

 We have a bus master that emits 43 unique SIDs. However, we have only 40
 SMMU_SMRn registers in the SMMU. So we need to mask out some of the
 incoming SID bits so that the 43 SIDs can match one of 40 entries in the
 SMR.
 
 Hmm, so you're talking about stream matching, right? That doesn't belong in
 the device-tree. I appreciate that the current driver does a terrible job at
 allocating the SMRs (it's bloody difficult!), but we should try to improve
 the dynamic behaviour instead of moving configuration of the SMMU out into
 device-tree, where it's inflexible at best.

I am talking about SMMU_SMRn[MASK] register bits. This is not something
that can be dynamically detected at run-time. It is configuration at the
same level as the actual StreamIDs.

Thanks,


Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
 On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
   - Each master has a set of fixed StreamIDs
   - StreamIDs can be remastered by adding a constant offset (this could also
 be used to describe RequesterID - StreamID mapping)
 
 I'd hope this would be sufficient for most people. Dynamic ID assignment can
 be worked out later (I'm not even sure it belongs in this binding) and any
 mappings other than `add a constant offset' can be treated on a case-by-case
 basis. We don't want to throw the kitchen sink at a language for describing
 arbitrary transformations!
 
  We've had similar discussions before (power sequences anyone?) where we
  tried to come up with a generic way to describe something in device tree
  that just didn't work out too well. Some things are better done in code,
  so I think we should at least consider that possibility rather than
  blindly try and force everything into device tree.
 
 If we can support 90% of SoCs with a simple DT-based description, we can
 address the corner cases as they arise. I'm not ruling our hardcoding
 topology if we have no choice, but I don't think that's a healthy place to
 start from.

So we could use the arm,gicv3 comaptible string for all those that
have a relatively simple mapping, and describe that mapping entirely
in DT properties, but use a different compatible string for those
SoCs that have a mapping which we can't easily describe, and then
put that into code?

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Fri, Jun 20, 2014 at 04:53:08PM +0100, Arnd Bergmann wrote:
 On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
  On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
- Each master has a set of fixed StreamIDs
- StreamIDs can be remastered by adding a constant offset (this could also
  be used to describe RequesterID - StreamID mapping)
  
  I'd hope this would be sufficient for most people. Dynamic ID assignment can
  be worked out later (I'm not even sure it belongs in this binding) and any
  mappings other than `add a constant offset' can be treated on a case-by-case
  basis. We don't want to throw the kitchen sink at a language for describing
  arbitrary transformations!
  
   We've had similar discussions before (power sequences anyone?) where we
   tried to come up with a generic way to describe something in device tree
   that just didn't work out too well. Some things are better done in code,
   so I think we should at least consider that possibility rather than
   blindly try and force everything into device tree.
  
  If we can support 90% of SoCs with a simple DT-based description, we can
  address the corner cases as they arise. I'm not ruling our hardcoding
  topology if we have no choice, but I don't think that's a healthy place to
  start from.
 
 So we could use the arm,gicv3 comaptible string for all those that
 have a relatively simple mapping, and describe that mapping entirely
 in DT properties, but use a different compatible string for those
 SoCs that have a mapping which we can't easily describe, and then
 put that into code?

That doesn't sound unreasonable, but I don't think we should commit to
putting things into code until they come along and we can't describe them.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 20 June 2014 18:50:51 Will Deacon wrote:
 On Fri, Jun 20, 2014 at 04:53:08PM +0100, Arnd Bergmann wrote:
  On Wednesday 18 June 2014 11:14:39 Will Deacon wrote:
   On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
 - Each master has a set of fixed StreamIDs
 - StreamIDs can be remastered by adding a constant offset (this could 
   also
   be used to describe RequesterID - StreamID mapping)
   
   I'd hope this would be sufficient for most people. Dynamic ID assignment 
   can
   be worked out later (I'm not even sure it belongs in this binding) and any
   mappings other than `add a constant offset' can be treated on a 
   case-by-case
   basis. We don't want to throw the kitchen sink at a language for 
   describing
   arbitrary transformations!
   
We've had similar discussions before (power sequences anyone?) where we
tried to come up with a generic way to describe something in device tree
that just didn't work out too well. Some things are better done in code,
so I think we should at least consider that possibility rather than
blindly try and force everything into device tree.
   
   If we can support 90% of SoCs with a simple DT-based description, we can
   address the corner cases as they arise. I'm not ruling our hardcoding
   topology if we have no choice, but I don't think that's a healthy place to
   start from.
  
  So we could use the arm,gicv3 comaptible string for all those that
  have a relatively simple mapping, and describe that mapping entirely
  in DT properties, but use a different compatible string for those
  SoCs that have a mapping which we can't easily describe, and then
  put that into code?
 
 That doesn't sound unreasonable, but I don't think we should commit to
 putting things into code until they come along and we can't describe them.

Yes, that makes sense. But it's good to know that we have the option so
we don't have to cover all corner cases with the binding.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Olav Haugan]

On 5/30/2014 12:06 PM, Arnd Bergmann wrote:
 On Friday 30 May 2014 08:16:05 Rob Herring wrote:
 On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
 thierry.red...@gmail.com wrote:
 From: Thierry Reding tred...@nvidia.com
 +IOMMU master node:
 +==
 +
 +Devices that access memory through an IOMMU are called masters. A device 
 can
 +have multiple master interfaces (to one or more IOMMU devices).
 +
 +Required properties:
 +
 +- iommus: A list of phandle and IOMMU specifier pairs that describe the 
 IOMMU
 +  master interfaces of the device. One entry in the list describes one 
 master
 +  interface of the device.
 +
 +When an iommus property is specified in a device tree node, the IOMMU 
 will
 +be used for address translation. If a dma-ranges property exists in the
 +device's parent node it will be ignored. An exception to this rule is if 
 the
 +referenced IOMMU is disabled, in which case the dma-ranges property of 
 the
 +parent shall take effect.

 Just thinking out loud, could you have dma-ranges in the iommu node
 for the case when the iommu is enabled rather than putting the DMA
 window information into the iommus property?

 This would probably mean that you need both #iommu-cells and #address-cells.
 
 The reason for doing like this was that you may need a different window
 for each device, while there can only be one dma-ranges property in 
 an iommu node.
 
 +
 +Optional properties:
 +
 +- iommu-names: A list of names identifying each entry in the iommus
 +  property.

 Do we really need a name here? I would not expect that you have
 clearly documented names here from the datasheet like you would for
 interrupts or clocks, so you'd just be making up names. Sorry, but I'm
 not a fan of names properties in general.
 
 Good point, this was really overdesign by modeling it after other
 subsystems that can have a use for names.
  
 +Multiple-master IOMMU:
 +--
 +
 +   iommu {
 +   /* the specifier represents the ID of the master */
 +   #address-cells = 1;
 +   #size-cells = 0;
 +   };
 +
 +   master {
 +   /* device has master ID 42 in the IOMMU */
 +   iommus = /iommu 42;
 +   };

 Presumably the ID would be the streamID on ARM's SMMU. How would a
 master with 8 streamIDs be described? This is what Calxeda midway has
 for SATA and I would expect that to be somewhat common. Either you
 need some ID masking or you'll have lots of duplication when you have
 windows.
 
 I don't understand the problem. If you have stream IDs 0 through 7,
 you would have
 
   master@a {
   ...
   iommus = smmu 0;
   };
 
   master@b {
   ...
   iommus = smmu 1;
   };
 
   ...
 
   master@12 {
   ...
   iommus = smmu 7;
   };
 
 and you don't need a window at all. Why would you need a mask of
 some sort?

We have multiple-master SMMUs and each master emits a variable number of
StreamIDs. However, we have to apply a mask (the ARM SMMU spec allows
for this) to the StreamIDs due to limited number of StreamID 2 Context
Bank entries in the SMMU. If my understanding is correct we would
represent this in the DT like this:

iommu {
#address-cells = 2;
#size-cells = 0;
};

master@a {
...
iommus = iommu StreamID0 MASK0,
 iommu StreamID1 MASK1,
 iommu StreamID2 MASK2;
};

master@b {
...
iommus = iommu StreamID3 MASK3,
 iommu StreamID4 MASK4;
};


Thanks,

Olav Haugan

-- 
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 17, 2014 at 03:50:25PM +0100, Stuart Yoder wrote:
   I think for simple masters (i.e. those that have all their StreamIDs
   under control of one driver), then setting something during attach (or
   add?) based on the DT could work pretty well. The other case is when we
   have masters behind a bridge (such as a PCI RC). In this case, it might
   actually be better to ask the bridge for the IDs and let it sort out
  the
   allocation itself. That would also move the RequesterID - StreamID
   mapping out of the SMMU code.
  
   What do you think?
  The PCI bus iommu group creation code would be part of the SMMU driver
  (it is handled by other IOMMU drivers as well). My understanding is that
  there would be one is to one correspondence between the requestor ID and
  the iommu group. May be, we can have an API provided by the PCI bridge
  (architecture specific) for setting the stream ID.
 
 I think Will is suggesting something along those lines-- I think it's a
 question of where the streamID allocation happens.  You could
 either do something like the following in the SMMU driver when attaching
 a PCI device:
 
 id = alloc_stream_id(...);
 pci_set_streamid(pci-dev, id);
 
 or
 
 id = pci_get_streamid(pci-dev);
 
 ...i.e the PCI RC could allocate (from some TBD
 allocator) and set the stream ID itself.
 
 Not sure how big a deal it is to extend PCI RC interfaces for 
 something like that.

I think both interfaces have their place; the latter for hotpluggable buses
and the former for static allocation (i.e. set once). The only way to see
how well it works is to try implementing it, I guess.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 18, 2014 at 12:37:16AM +0100, Thierry Reding wrote:
 On Tue, Jun 17, 2014 at 01:18:11PM +0100, Will Deacon wrote:
  On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
   On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
 It can easily be argued that if the algorithm used to remap the ID
 varies, the compatibility of the device changes. Therefore I would
 expect any variant of the GICv3 that deviates from the standard
 mapping (if there is such a thing) to have its own compatible string.

There is no standard mapping; it's a property defined at system 
integration
time. I fully expect different SoCs to do different things here.
   
   My point was that the mapping itself seems to be fundamental enough to
   make devices with different mappings incompatible. Therefore I think
   this could probably be handled by using different compatible values,
   something along the lines of this:
   
 compatible = vendor,soc-gicv3, arm,gicv3;
   
   Then the mapping can be described in code, which should be a whole lot
   easier and more flexible than a more or less generic notation in device
   tree.
  
  I don't think that scales well beyond a handful of unique mappings, and I
  really anticipate everybody doing something different based on their
  integration constraints.
  
  You'd very quickly end up with sets of tables for each SoC, describing the
  topology and associated IDs in the kernel source, which feels like a giant
  step backwards from where we are today with device tree.
 
 Well, today we don't have a generic binding at all, so anything will
 really be a giant step forward in my opinion.

Oh, I'm not disputing that at all. I just think it's worth considering how
we can extend the binding in future to describe some of the ID routing and
remapping that we've discussed.

 But seriously, from what you said earlier I got the impression that some
 of the mappings may not be easy or possible to represent in DT, which is
 why I proposed to encode it into the compatible property so that it can
 be handled in code instead.

For any old arbitrary mappings, we're going to have a rough time, but that
doesn't mean we need to support that until we're forced to (and then we can
consider our options). What I *do* think we need to describe is:

  - Each master has a set of fixed StreamIDs
  - StreamIDs can be remastered by adding a constant offset (this could also
be used to describe RequesterID - StreamID mapping)

I'd hope this would be sufficient for most people. Dynamic ID assignment can
be worked out later (I'm not even sure it belongs in this binding) and any
mappings other than `add a constant offset' can be treated on a case-by-case
basis. We don't want to throw the kitchen sink at a language for describing
arbitrary transformations!

 We've had similar discussions before (power sequences anyone?) where we
 tried to come up with a generic way to describe something in device tree
 that just didn't work out too well. Some things are better done in code,
 so I think we should at least consider that possibility rather than
 blindly try and force everything into device tree.

If we can support 90% of SoCs with a simple DT-based description, we can
address the corner cases as they arise. I'm not ruling our hardcoding
topology if we have no choice, but I don't think that's a healthy place to
start from.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Varun Sethi]

 -Original Message-
 From: Yoder Stuart-B08248
 Sent: Tuesday, June 17, 2014 12:24 AM
 To: Will Deacon
 Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
 devicet...@vger.kernel.org; linux-samsung-soc@vger.kernel.org; Pawel
 Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren; linux-
 ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob Herring; Kumar
 Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org
 Subject: RE: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 
 
  -Original Message-
  From: Will Deacon [mailto:will.dea...@arm.com]
  Sent: Monday, June 16, 2014 12:04 PM
  To: Yoder Stuart-B08248
  Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
  devicet...@vger.kernel.org; linux-samsung-soc@vger.kernel.org; Pawel
  Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren;
  linux- ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob Herring;
  Kumar Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin;
  linux-arm- ker...@lists.infradead.org
  Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
  bindings
 
  Hi Stuart,
 
  On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
Do you have use-cases where you really need to change these
mappings dynamically?
  
   Yes.  In the case of a PCI bus-- you may not know in advance how many
   PCI devices there are until you probe the bus.   We have another FSL
   proprietary bus we call the fsl-mc bus that is similar.
 
  For that case, though, you could still describe an algorithmic
  transformation from RequesterID to StreamID which corresponds to a
  fixed mapping.
 
   Another thing to consider-- starting with SMMUv2, as you know, there
   is a new distributed architecture with multiple TBUs and a
   centralized TCU that walks the SMMU page tables.  So instead of
   sprinkling multiple SMMUs all over an SoC you now have the option a
   1 central TCU and
  sprinkling
   multiple TBUs around.   However, this means that the stream ID
  namespace
   is now global and can be pretty limited.  In the SMMU implementation
   we have there are only 64 stream ID total for our Soc.  But we have
   many
  more
   masters than that.
  
   So we look at stream IDs as really corresponding to an 'isolation
  context'
   and not to a bus master.  An isolation context is the domain you are
   trying to isolate with the SMMU.  Devices that all belong to the
   same 'isolation context' can share the same stream ID, since they
   share the same domain and page tables.
 
  Ok, this is more compelling.
 
   So, perhaps by default some/most SMMU masters may have a default
   stream
  ID
   of 0x0 that is used by the host...and that could be represented
   statically in the device tree.
  
   But, we absolutely will need to dynamically set new stream IDs into
   masters when a new IOMMU 'domain' is created and devices
   are added to it.   All the devices in a domain will share
   the same stream ID.
  
   So whatever we do, let's please have an architecture flexible enough
   to allow for this.
 
  What is the software interface to the logic that assigns the StreamIDs?
  Is
  it part of the SMMU, or a separate device (or set of devices)?
 
 For us at the hardware level there are a few different ways that the
 streamIDs can be set.  It is not part of the SMMU.  In the cases where
 there is simply
 1 device to 1 streamID (e.g. USB controller) there is an SoC register
 where
 you just program the stream ID.   In the case of PCI, our PCI controller
 has a RequesterID-to-streamID table that you set via some PCI controller
 registers.
 
 The way we generally thought it would work was something like
 this:
-u-boot/bootloader makes any static streamID allocation if needed,
 sets a default streamID  (e.g. 0x0) in device and expresses
 that in the device tree
-device tree would express relationship between devices
 (including bus controllers) and the SMMU through mmu-masters
 property
-u-boot would express the range of unused (or used) streamIDs via a
 new
 device tree property so the kernel SMMU driver knows what streamIDs
 are
 free
-in the SMMU driver a different vendor specific 'add_device' callback
 could be used to handle our special cases where we need to set/change
 the stream ID for devices added to a domain

Another possibility, could be to program the stream Id in the device registers 
(reference for the stream ID register can be obtained from the device tree) 
during device attach. This could be relevant in case of VFIO, when we are 
assigning multiple devices to a single VM. All the devices can share the same 
stream ID.

-Varun
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 17, 2014 at 11:26:48AM +0100, Varun Sethi wrote:
  The way we generally thought it would work was something like
  this:
 -u-boot/bootloader makes any static streamID allocation if needed,
  sets a default streamID  (e.g. 0x0) in device and expresses
  that in the device tree
 -device tree would express relationship between devices
  (including bus controllers) and the SMMU through mmu-masters
  property
 -u-boot would express the range of unused (or used) streamIDs via a
  new
  device tree property so the kernel SMMU driver knows what streamIDs
  are
  free
 -in the SMMU driver a different vendor specific 'add_device' callback
  could be used to handle our special cases where we need to set/change
  the stream ID for devices added to a domain
 
 Another possibility, could be to program the stream Id in the device
 registers (reference for the stream ID register can be obtained from the
 device tree) during device attach. This could be relevant in case of VFIO,
 when we are assigning multiple devices to a single VM. All the devices can
 share the same stream ID.

I think for simple masters (i.e. those that have all their StreamIDs under
control of one driver), then setting something during attach (or add?)
based on the DT could work pretty well. The other case is when we have
masters behind a bridge (such as a PCI RC). In this case, it might actually
be better to ask the bridge for the IDs and let it sort out the allocation
itself. That would also move the RequesterID - StreamID mapping out of
the SMMU code.

What do you think?

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Varun Sethi]

 -Original Message-
 From: iommu-boun...@lists.linux-foundation.org [mailto:iommu-
 boun...@lists.linux-foundation.org] On Behalf Of Will Deacon
 Sent: Tuesday, June 17, 2014 4:13 PM
 To: Sethi Varun-B16395
 Cc: Mark Rutland; devicet...@vger.kernel.org; linux-samsung-
 s...@vger.kernel.org; Arnd Bergmann; Pawel Moll; Ian Campbell; Grant
 Grundler; Stephen Warren; Yoder Stuart-B08248; Rob Herring; linux-
 ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Thierry Reding; Kumar
 Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org
 Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 On Tue, Jun 17, 2014 at 11:26:48AM +0100, Varun Sethi wrote:
   The way we generally thought it would work was something like
   this:
  -u-boot/bootloader makes any static streamID allocation if needed,
   sets a default streamID  (e.g. 0x0) in device and expresses
   that in the device tree
  -device tree would express relationship between devices
   (including bus controllers) and the SMMU through mmu-masters
   property
  -u-boot would express the range of unused (or used) streamIDs via
   a new
   device tree property so the kernel SMMU driver knows what
   streamIDs are
   free
  -in the SMMU driver a different vendor specific 'add_device'
 callback
   could be used to handle our special cases where we need to
 set/change
   the stream ID for devices added to a domain
 
  Another possibility, could be to program the stream Id in the device
  registers (reference for the stream ID register can be obtained from
  the device tree) during device attach. This could be relevant in case
  of VFIO, when we are assigning multiple devices to a single VM. All
  the devices can share the same stream ID.
 
 I think for simple masters (i.e. those that have all their StreamIDs
 under control of one driver), then setting something during attach (or
 add?) based on the DT could work pretty well. The other case is when we
 have masters behind a bridge (such as a PCI RC). In this case, it might
 actually be better to ask the bridge for the IDs and let it sort out the
 allocation itself. That would also move the RequesterID - StreamID
 mapping out of the SMMU code.
 
 What do you think?
The PCI bus iommu group creation code would be part of the SMMU driver (it is 
handled by other IOMMU drivers as well). My understanding is that there would 
be one is to one correspondence between the requestor ID and the iommu group. 
May be, we can have an API provided by the PCI bridge (architecture specific) 
for setting the stream ID.

-Varun
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
 On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
  On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
   On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
  [...]
Arnd, can you take another look at this binding and see if there's
anything else missing? If not I'll go through the document again and
update all #address-cells/#size-cells references with #iommu-cells as
appropriate and submit v3.
   
   How do you envisage propagation of the master ID bits downstream of the
   IOMMU would be described?
   
   We will definitely need a way to describe this for GICv3.  How those
   values are propagated is likely to vary between related SoCs and doesn't
   feel like it should be baked into a driver, especially for the ARM SMMU
   which may get reused in radically different SoC families from different
   vendors.
  
  Well, we've had cases like these in the past (power sequences come to
  mind). Some concepts are just too difficult or unwieldy to be put into
  device tree. I think that this is one of them.
  
   The most likely types of remapping are the adding of a base offset or
   some extra bits to the ID -- because not all MSIs to the GIC will
   necessarily pass through the IOMMU.  It's also possible that we might
   see ID squashing or folding in some systems.
  
  It can easily be argued that if the algorithm used to remap the ID
  varies, the compatibility of the device changes. Therefore I would
  expect any variant of the GICv3 that deviates from the standard
  mapping (if there is such a thing) to have its own compatible string.
 
 There is no standard mapping; it's a property defined at system integration
 time. I fully expect different SoCs to do different things here.

My point was that the mapping itself seems to be fundamental enough to
make devices with different mappings incompatible. Therefore I think
this could probably be handled by using different compatible values,
something along the lines of this:

compatible = vendor,soc-gicv3, arm,gicv3;

Then the mapping can be described in code, which should be a whole lot
easier and more flexible than a more or less generic notation in device
tree.

Thierry


pgpWIyVt9WhEY.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
 On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
  On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
   It can easily be argued that if the algorithm used to remap the ID
   varies, the compatibility of the device changes. Therefore I would
   expect any variant of the GICv3 that deviates from the standard
   mapping (if there is such a thing) to have its own compatible string.
  
  There is no standard mapping; it's a property defined at system integration
  time. I fully expect different SoCs to do different things here.
 
 My point was that the mapping itself seems to be fundamental enough to
 make devices with different mappings incompatible. Therefore I think
 this could probably be handled by using different compatible values,
 something along the lines of this:
 
   compatible = vendor,soc-gicv3, arm,gicv3;
 
 Then the mapping can be described in code, which should be a whole lot
 easier and more flexible than a more or less generic notation in device
 tree.

I don't think that scales well beyond a handful of unique mappings, and I
really anticipate everybody doing something different based on their
integration constraints.

You'd very quickly end up with sets of tables for each SoC, describing the
topology and associated IDs in the kernel source, which feels like a giant
step backwards from where we are today with device tree.

If, for example, the GIC architecture prescribed a fixed set of Device IDs
and an algorithm for converting from Stream IDs then your approach may have
some merits, but that's not where we are today (and I also don't think it's
practical to try and enforce such system-wide properties into an interrupt
controller architecture).

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stuart Yoder]

 -Original Message-
 From: Sethi Varun-B16395
 Sent: Tuesday, June 17, 2014 5:27 AM
 To: Yoder Stuart-B08248; Will Deacon
 Cc: Thierry Reding; Mark Rutland; devicet...@vger.kernel.org; linux-
 samsung-...@vger.kernel.org; Pawel Moll; Arnd Bergmann; Ian Campbell;
 Grant Grundler; Stephen Warren; linux-ker...@vger.kernel.org; Marc
 Zyngier; Linux IOMMU; Rob Herring; Kumar Gala; linux-
 te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org
 Subject: RE: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 
 
  -Original Message-
  From: Yoder Stuart-B08248
  Sent: Tuesday, June 17, 2014 12:24 AM
  To: Will Deacon
  Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
  devicet...@vger.kernel.org; linux-samsung-soc@vger.kernel.org; Pawel
  Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren;
 linux-
  ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob Herring; Kumar
  Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-
 arm-
  ker...@lists.infradead.org
  Subject: RE: [PATCH v2] devicetree: Add generic IOMMU device tree
  bindings
 
 
 
   -Original Message-
   From: Will Deacon [mailto:will.dea...@arm.com]
   Sent: Monday, June 16, 2014 12:04 PM
   To: Yoder Stuart-B08248
   Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
   devicet...@vger.kernel.org; linux-samsung-soc@vger.kernel.org; Pawel
   Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren;
   linux- ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob
 Herring;
   Kumar Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin;
   linux-arm- ker...@lists.infradead.org
   Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
   bindings
  
   Hi Stuart,
  
   On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
 Do you have use-cases where you really need to change these
 mappings dynamically?
   
Yes.  In the case of a PCI bus-- you may not know in advance how
 many
PCI devices there are until you probe the bus.   We have another
 FSL
proprietary bus we call the fsl-mc bus that is similar.
  
   For that case, though, you could still describe an algorithmic
   transformation from RequesterID to StreamID which corresponds to a
   fixed mapping.
  
Another thing to consider-- starting with SMMUv2, as you know,
 there
is a new distributed architecture with multiple TBUs and a
centralized TCU that walks the SMMU page tables.  So instead of
sprinkling multiple SMMUs all over an SoC you now have the option a
1 central TCU and
   sprinkling
multiple TBUs around.   However, this means that the stream ID
   namespace
is now global and can be pretty limited.  In the SMMU
 implementation
we have there are only 64 stream ID total for our Soc.  But we have
many
   more
masters than that.
   
So we look at stream IDs as really corresponding to an 'isolation
   context'
and not to a bus master.  An isolation context is the domain you
 are
trying to isolate with the SMMU.  Devices that all belong to the
same 'isolation context' can share the same stream ID, since they
share the same domain and page tables.
  
   Ok, this is more compelling.
  
So, perhaps by default some/most SMMU masters may have a default
stream
   ID
of 0x0 that is used by the host...and that could be represented
statically in the device tree.
   
But, we absolutely will need to dynamically set new stream IDs into
masters when a new IOMMU 'domain' is created and devices
are added to it.   All the devices in a domain will share
the same stream ID.
   
So whatever we do, let's please have an architecture flexible
 enough
to allow for this.
  
   What is the software interface to the logic that assigns the
 StreamIDs?
   Is
   it part of the SMMU, or a separate device (or set of devices)?
 
  For us at the hardware level there are a few different ways that the
  streamIDs can be set.  It is not part of the SMMU.  In the cases where
  there is simply
  1 device to 1 streamID (e.g. USB controller) there is an SoC register
  where
  you just program the stream ID.   In the case of PCI, our PCI
 controller
  has a RequesterID-to-streamID table that you set via some PCI
 controller
  registers.
 
  The way we generally thought it would work was something like
  this:
 -u-boot/bootloader makes any static streamID allocation if needed,
  sets a default streamID  (e.g. 0x0) in device and expresses
  that in the device tree
 -device tree would express relationship between devices
  (including bus controllers) and the SMMU through mmu-masters
  property
 -u-boot would express the range of unused (or used) streamIDs via a
  new
  device tree property so the kernel SMMU driver knows what streamIDs
  are
  free
 -in the SMMU driver a different vendor specific 'add_device'
 callback
  could

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stuart Yoder]

 -Original Message-
 From: Sethi Varun-B16395
 Sent: Tuesday, June 17, 2014 6:22 AM
 To: Will Deacon
 Cc: Mark Rutland; devicet...@vger.kernel.org; linux-samsung-
 s...@vger.kernel.org; Arnd Bergmann; Pawel Moll; Ian Campbell; Grant
 Grundler; Stephen Warren; Yoder Stuart-B08248; Rob Herring; linux-
 ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Thierry Reding; Kumar
 Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org
 Subject: RE: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 
 
  -Original Message-
  From: iommu-boun...@lists.linux-foundation.org [mailto:iommu-
  boun...@lists.linux-foundation.org] On Behalf Of Will Deacon
  Sent: Tuesday, June 17, 2014 4:13 PM
  To: Sethi Varun-B16395
  Cc: Mark Rutland; devicet...@vger.kernel.org; linux-samsung-
  s...@vger.kernel.org; Arnd Bergmann; Pawel Moll; Ian Campbell; Grant
  Grundler; Stephen Warren; Yoder Stuart-B08248; Rob Herring; linux-
  ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Thierry Reding;
 Kumar
  Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-
 arm-
  ker...@lists.infradead.org
  Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
  bindings
 
  On Tue, Jun 17, 2014 at 11:26:48AM +0100, Varun Sethi wrote:
The way we generally thought it would work was something like
this:
   -u-boot/bootloader makes any static streamID allocation if
 needed,
sets a default streamID  (e.g. 0x0) in device and expresses
that in the device tree
   -device tree would express relationship between devices
(including bus controllers) and the SMMU through mmu-masters
property
   -u-boot would express the range of unused (or used) streamIDs
 via
a new
device tree property so the kernel SMMU driver knows what
streamIDs are
free
   -in the SMMU driver a different vendor specific 'add_device'
  callback
could be used to handle our special cases where we need to
  set/change
the stream ID for devices added to a domain
  
   Another possibility, could be to program the stream Id in the device
   registers (reference for the stream ID register can be obtained from
   the device tree) during device attach. This could be relevant in case
   of VFIO, when we are assigning multiple devices to a single VM. All
   the devices can share the same stream ID.
 
  I think for simple masters (i.e. those that have all their StreamIDs
  under control of one driver), then setting something during attach (or
  add?) based on the DT could work pretty well. The other case is when we
  have masters behind a bridge (such as a PCI RC). In this case, it might
  actually be better to ask the bridge for the IDs and let it sort out
 the
  allocation itself. That would also move the RequesterID - StreamID
  mapping out of the SMMU code.
 
  What do you think?
 The PCI bus iommu group creation code would be part of the SMMU driver
 (it is handled by other IOMMU drivers as well). My understanding is that
 there would be one is to one correspondence between the requestor ID and
 the iommu group. May be, we can have an API provided by the PCI bridge
 (architecture specific) for setting the stream ID.

I think Will is suggesting something along those lines-- I think it's a
question of where the streamID allocation happens.  You could
either do something like the following in the SMMU driver when attaching
a PCI device:

id = alloc_stream_id(...);
pci_set_streamid(pci-dev, id);

or

id = pci_get_streamid(pci-dev);

...i.e the PCI RC could allocate (from some TBD
allocator) and set the stream ID itself.

Not sure how big a deal it is to extend PCI RC interfaces for 
something like that.

Stuart
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Tue, Jun 17, 2014 at 01:18:11PM +0100, Will Deacon wrote:
 On Tue, Jun 17, 2014 at 12:58:30PM +0100, Thierry Reding wrote:
  On Mon, Jun 16, 2014 at 01:57:04PM +0100, Will Deacon wrote:
   On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
It can easily be argued that if the algorithm used to remap the ID
varies, the compatibility of the device changes. Therefore I would
expect any variant of the GICv3 that deviates from the standard
mapping (if there is such a thing) to have its own compatible string.
   
   There is no standard mapping; it's a property defined at system 
   integration
   time. I fully expect different SoCs to do different things here.
  
  My point was that the mapping itself seems to be fundamental enough to
  make devices with different mappings incompatible. Therefore I think
  this could probably be handled by using different compatible values,
  something along the lines of this:
  
  compatible = vendor,soc-gicv3, arm,gicv3;
  
  Then the mapping can be described in code, which should be a whole lot
  easier and more flexible than a more or less generic notation in device
  tree.
 
 I don't think that scales well beyond a handful of unique mappings, and I
 really anticipate everybody doing something different based on their
 integration constraints.
 
 You'd very quickly end up with sets of tables for each SoC, describing the
 topology and associated IDs in the kernel source, which feels like a giant
 step backwards from where we are today with device tree.

Well, today we don't have a generic binding at all, so anything will
really be a giant step forward in my opinion.

But seriously, from what you said earlier I got the impression that some
of the mappings may not be easy or possible to represent in DT, which is
why I proposed to encode it into the compatible property so that it can
be handled in code instead.

We've had similar discussions before (power sequences anyone?) where we
tried to come up with a generic way to describe something in device tree
that just didn't work out too well. Some things are better done in code,
so I think we should at least consider that possibility rather than
blindly try and force everything into device tree.

Thierry


pgpINMokP3WHH.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 04, 2014 at 10:12:38PM +0100, Thierry Reding wrote:
 On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
  On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
 [...]
   Arnd, can you take another look at this binding and see if there's
   anything else missing? If not I'll go through the document again and
   update all #address-cells/#size-cells references with #iommu-cells as
   appropriate and submit v3.
  
  How do you envisage propagation of the master ID bits downstream of the
  IOMMU would be described?
  
  We will definitely need a way to describe this for GICv3.  How those
  values are propagated is likely to vary between related SoCs and doesn't
  feel like it should be baked into a driver, especially for the ARM SMMU
  which may get reused in radically different SoC families from different
  vendors.
 
 Well, we've had cases like these in the past (power sequences come to
 mind). Some concepts are just too difficult or unwieldy to be put into
 device tree. I think that this is one of them.
 
  The most likely types of remapping are the adding of a base offset or
  some extra bits to the ID -- because not all MSIs to the GIC will
  necessarily pass through the IOMMU.  It's also possible that we might
  see ID squashing or folding in some systems.
 
 It can easily be argued that if the algorithm used to remap the ID
 varies, the compatibility of the device changes. Therefore I would
 expect any variant of the GICv3 that deviates from the standard
 mapping (if there is such a thing) to have its own compatible string.

There is no standard mapping; it's a property defined at system integration
time. I fully expect different SoCs to do different things here.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

Hi Varun,

On Thu, Jun 05, 2014 at 08:10:19PM +0100, Varun Sethi wrote:
  The set of StreamIDs that can be generated by a master is fixed in the
  hardware. The SMMU can then be programmed to map these incoming IDs onto
  a context ID (or a set of context IDs), which are the IDs used internally
  by the SMMU to find the page tables etc.
  
  The StreamID - ContextID mapping is dynamic and controlled by software.
  The Master - StreamIDs mapping is fixed in the hardware.
 The Master - StreamIDs mapping may not always be fixed in the hardware.
 At, least in our case we plan to program these via software. PCI devices
 is one place where this mapping would have to be dynamic (based on the
 topology i.e. if the devices are connected to a bridge etc). Also, we have
 a hot plug device architecture where the stream ID is software
 programmable.
 
 Other than that, based on the isolation requirements (iommu domain)
 software programmability offers greater flexibility.

For the sake of consistency, I'd really like to treat the master -
streamIDs mapping as fixed. If that means setting it once during boot in
your case, then so be it (ideally in the firmware). That way, we just
describe the fixed mapping to the kernel and the driver doesn't have to
worry about changing the mapping, especially given that that's likely to be
highly error-prone once the SMMU is in us.

Do you have use-cases where you really need to change these mappings
dynamically?

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stuart Yoder]

 -Original Message-
 From: Will Deacon [mailto:will.dea...@arm.com]
 Sent: Monday, June 16, 2014 10:28 AM
 To: Sethi Varun-B16395
 Cc: Thierry Reding; Mark Rutland; devicet...@vger.kernel.org; linux-
 samsung-...@vger.kernel.org; Pawel Moll; Arnd Bergmann; Ian Campbell;
 Grant Grundler; Stephen Warren; linux-ker...@vger.kernel.org; Marc
 Zyngier; Linux IOMMU; Rob Herring; Kumar Gala; linux-
 te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org; Yoder Stuart-B08248
 Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 Hi Varun,
 
 On Thu, Jun 05, 2014 at 08:10:19PM +0100, Varun Sethi wrote:
   The set of StreamIDs that can be generated by a master is fixed in
 the
   hardware. The SMMU can then be programmed to map these incoming IDs
 onto
   a context ID (or a set of context IDs), which are the IDs used
 internally
   by the SMMU to find the page tables etc.
  
   The StreamID - ContextID mapping is dynamic and controlled by
 software.
   The Master - StreamIDs mapping is fixed in the hardware.
  The Master - StreamIDs mapping may not always be fixed in the
 hardware.
  At, least in our case we plan to program these via software. PCI
 devices
  is one place where this mapping would have to be dynamic (based on the
  topology i.e. if the devices are connected to a bridge etc). Also, we
 have
  a hot plug device architecture where the stream ID is software
  programmable.
 
  Other than that, based on the isolation requirements (iommu domain)
  software programmability offers greater flexibility.
 
 For the sake of consistency, I'd really like to treat the master -
 streamIDs mapping as fixed. If that means setting it once during boot in
 your case, then so be it (ideally in the firmware). That way, we just
 describe the fixed mapping to the kernel and the driver doesn't have to
 worry about changing the mapping, especially given that that's likely to
 be
 highly error-prone once the SMMU is in us.
 
 Do you have use-cases where you really need to change these mappings
 dynamically?

Yes.  In the case of a PCI bus-- you may not know in advance how many
PCI devices there are until you probe the bus.   We have another FSL
proprietary bus we call the fsl-mc bus that is similar.

Another thing to consider-- starting with SMMUv2, as you know, there
is a new distributed architecture with multiple TBUs and a centralized
TCU that walks the SMMU page tables.  So instead of sprinkling multiple
SMMUs all over an SoC you now have the option a 1 central TCU and sprinkling
multiple TBUs around.   However, this means that the stream ID namespace
is now global and can be pretty limited.  In the SMMU implementation we 
have there are only 64 stream ID total for our Soc.  But we have many more
masters than that.

So we look at stream IDs as really corresponding to an 'isolation context'
and not to a bus master.  An isolation context is the domain you are
trying to isolate with the SMMU.  Devices that all belong to the same
'isolation context' can share the same stream ID, since they share
the same domain and page tables.

So, perhaps by default some/most SMMU masters may have a default stream ID
of 0x0 that is used by the host...and that could be represented
statically in the device tree.

But, we absolutely will need to dynamically set new stream IDs
into masters when a new IOMMU 'domain' is created and devices
are added to it.   All the devices in a domain will share
the same stream ID.

So whatever we do, let's please have an architecture flexible enough
to allow for this.

Thanks,
Stuart






--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

Hi Stuart,

On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
  Do you have use-cases where you really need to change these mappings
  dynamically?
 
 Yes.  In the case of a PCI bus-- you may not know in advance how many
 PCI devices there are until you probe the bus.   We have another FSL
 proprietary bus we call the fsl-mc bus that is similar.

For that case, though, you could still describe an algorithmic
transformation from RequesterID to StreamID which corresponds to a fixed
mapping.

 Another thing to consider-- starting with SMMUv2, as you know, there
 is a new distributed architecture with multiple TBUs and a centralized
 TCU that walks the SMMU page tables.  So instead of sprinkling multiple
 SMMUs all over an SoC you now have the option a 1 central TCU and sprinkling
 multiple TBUs around.   However, this means that the stream ID namespace
 is now global and can be pretty limited.  In the SMMU implementation we 
 have there are only 64 stream ID total for our Soc.  But we have many more
 masters than that.
 
 So we look at stream IDs as really corresponding to an 'isolation context'
 and not to a bus master.  An isolation context is the domain you are
 trying to isolate with the SMMU.  Devices that all belong to the same
 'isolation context' can share the same stream ID, since they share
 the same domain and page tables.

Ok, this is more compelling.

 So, perhaps by default some/most SMMU masters may have a default stream ID
 of 0x0 that is used by the host...and that could be represented
 statically in the device tree.
 
 But, we absolutely will need to dynamically set new stream IDs
 into masters when a new IOMMU 'domain' is created and devices
 are added to it.   All the devices in a domain will share
 the same stream ID.
 
 So whatever we do, let's please have an architecture flexible enough
 to allow for this.

What is the software interface to the logic that assigns the StreamIDs? Is
it part of the SMMU, or a separate device (or set of devices)?

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Monday 16 June 2014 18:04:16 Will Deacon wrote:
 
 On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
   Do you have use-cases where you really need to change these mappings
   dynamically?
  
  Yes.  In the case of a PCI bus-- you may not know in advance how many
  PCI devices there are until you probe the bus.   We have another FSL
  proprietary bus we call the fsl-mc bus that is similar.
 
 For that case, though, you could still describe an algorithmic
 transformation from RequesterID to StreamID which corresponds to a fixed
 mapping.

It sounds to me like the best option here would be to have only the
RequesterID passed in the iommus property and have the StreamID
dynamically assigned. This would mean we treat the StreamID as
the context.

  Another thing to consider-- starting with SMMUv2, as you know, there
  is a new distributed architecture with multiple TBUs and a centralized
  TCU that walks the SMMU page tables.  So instead of sprinkling multiple
  SMMUs all over an SoC you now have the option a 1 central TCU and sprinkling
  multiple TBUs around.   However, this means that the stream ID namespace
  is now global and can be pretty limited.  In the SMMU implementation we 
  have there are only 64 stream ID total for our Soc.  But we have many more
  masters than that.
  
  So we look at stream IDs as really corresponding to an 'isolation context'
  and not to a bus master.  An isolation context is the domain you are
  trying to isolate with the SMMU.  Devices that all belong to the same
  'isolation context' can share the same stream ID, since they share
  the same domain and page tables.
 
 Ok, this is more compelling.

This also makes sense from the kernel's perspective: The shared Stream ID
is the one that is used by the dma-mapping API here, while all other
Stream IDs would only be used if you have things like PCI device assignment
or GPU contexts that map into IOMMU contexts using the Linux IOMMU
abstraction, which is a lower-level interface than the dma-mapping
abstraction.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stuart Yoder]

 -Original Message-
 From: Will Deacon [mailto:will.dea...@arm.com]
 Sent: Monday, June 16, 2014 12:04 PM
 To: Yoder Stuart-B08248
 Cc: Sethi Varun-B16395; Thierry Reding; Mark Rutland;
 devicet...@vger.kernel.org; linux-samsung-soc@vger.kernel.org; Pawel
 Moll; Arnd Bergmann; Ian Campbell; Grant Grundler; Stephen Warren; linux-
 ker...@vger.kernel.org; Marc Zyngier; Linux IOMMU; Rob Herring; Kumar
 Gala; linux-te...@vger.kernel.org; Cho KyongHo; Dave P Martin; linux-arm-
 ker...@lists.infradead.org
 Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 Hi Stuart,
 
 On Mon, Jun 16, 2014 at 05:56:32PM +0100, Stuart Yoder wrote:
   Do you have use-cases where you really need to change these mappings
   dynamically?
 
  Yes.  In the case of a PCI bus-- you may not know in advance how many
  PCI devices there are until you probe the bus.   We have another FSL
  proprietary bus we call the fsl-mc bus that is similar.
 
 For that case, though, you could still describe an algorithmic
 transformation from RequesterID to StreamID which corresponds to a fixed
 mapping.
 
  Another thing to consider-- starting with SMMUv2, as you know, there
  is a new distributed architecture with multiple TBUs and a centralized
  TCU that walks the SMMU page tables.  So instead of sprinkling multiple
  SMMUs all over an SoC you now have the option a 1 central TCU and
 sprinkling
  multiple TBUs around.   However, this means that the stream ID
 namespace
  is now global and can be pretty limited.  In the SMMU implementation we
  have there are only 64 stream ID total for our Soc.  But we have many
 more
  masters than that.
 
  So we look at stream IDs as really corresponding to an 'isolation
 context'
  and not to a bus master.  An isolation context is the domain you are
  trying to isolate with the SMMU.  Devices that all belong to the same
  'isolation context' can share the same stream ID, since they share
  the same domain and page tables.
 
 Ok, this is more compelling.
 
  So, perhaps by default some/most SMMU masters may have a default stream
 ID
  of 0x0 that is used by the host...and that could be represented
  statically in the device tree.
 
  But, we absolutely will need to dynamically set new stream IDs
  into masters when a new IOMMU 'domain' is created and devices
  are added to it.   All the devices in a domain will share
  the same stream ID.
 
  So whatever we do, let's please have an architecture flexible enough
  to allow for this.
 
 What is the software interface to the logic that assigns the StreamIDs?
 Is
 it part of the SMMU, or a separate device (or set of devices)?

For us at the hardware level there are a few different ways that the streamIDs
can be set.  It is not part of the SMMU.  In the cases where there is simply
1 device to 1 streamID (e.g. USB controller) there is an SoC register where
you just program the stream ID.   In the case of PCI, our PCI controller
has a RequesterID-to-streamID table that you set via some PCI controller
registers.

The way we generally thought it would work was something like
this:
   -u-boot/bootloader makes any static streamID allocation if needed,
sets a default streamID  (e.g. 0x0) in device and expresses
that in the device tree
   -device tree would express relationship between devices
(including bus controllers) and the SMMU through mmu-masters
property
   -u-boot would express the range of unused (or used) streamIDs via a new
device tree property so the kernel SMMU driver knows what streamIDs are
free
   -in the SMMU driver a different vendor specific 'add_device' callback
could be used to handle our special cases where we need to set/change
the stream ID for devices added to a domain

Thanks,
Stuart
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Sat, Jun 07, 2014 at 03:22:13PM +0200, Arnd Bergmann wrote:
 On Saturday 07 June 2014 00:45:45 Thierry Reding wrote:
  This is somewhat off-topic, but given the various concepts discussed in
  this thread I'm beginning to wonder how they will be implemented.
 
 I think it's good you raised the question.
 
  The
  current implementation hooks the IOMMU API into the DMA mapping API, and
  the way this is done is by setting a single IOMMU (or rather a set of
  IOMMU operations) globally per bus type.
 
 I hadn't realized that we have a per-bus iommu_ops pointer. I agree
 this will become a limitation as soon as we have a soc with two different
 IOMMUs that have platform devices attached, and it has to be moved into
 the device or a structure related to that.
 
 If that turns out controversial, we can probably have a set of pseudo
 iommu ops that just call into dev-archdata-iommu_ops-function()
 for ARM.
 
  There are two issues that I can see with that: one is that we can't
  support multiple IOMMUs in the system at all, and the other is that
  there is no context associated with the IOMMU ops, and therefore there
  is no way to differentiate between two instances of the same IOMMU. A
  few drivers use global variables to keep track of context information
  but that won't work with multiple instances, unless they keep a global
  list of all instances and then require explicit lookup in each of the
  IOMMU operations. That sounds more like a workaround rather than a
  proper solution to me.
 
 Supporting multiple iommus that share one iommu driver should work
 without such hacks, as you can put the per-device information into
 dev-device_dma_parameters (this works only for very simple IOMMUs)
 or dev-archdata-iommu

I was talking about the lack of a place to store context for the IOMMU
itself. Currently none of the functions in iommu_ops have a way to get
access to the IOMMU context itself. In fact there's not even a common
structure that could be used for this purpose. I have a couple of local
patches that try to add something like that, along with functions to
more explicitly hook up a device with it's IOMMU(s). It looks somewhat
like this:

struct iommu {
struct device *dev;

struct list_head list;

const struct iommu_ops *ops;
};

/* register and unregister IOMMU device with core */
int iommu_add(struct iommu *iommu);
void iommu_remove(struct iommu *iommu);

/*
 * Attach a device to one or more IOMMUs (according to the
 * iommus property).
 */
int iommu_attach(struct device *dev);
int iommu_detach(struct device *dev);

Does that look like a direction that we would want to pursue?

 (we may want to generalize that, I think someone just posted patches
 for it).

Perhaps you mean this:

[PATCHv3 1/3] device.h: arm dma-iommu: Move out dma_iommu_mapping struct

? From a quick glance that indeed looks like a promising step towards
unifying this across architectures.

  Discussion in this thread indicates that both of the above will be
  required at some point. Have I completely missed something or will we
  have to rework (parts of) the IOMMU API to make this work?
  
  One other thing that I have some difficulty understanding is how we can
  support things like process isolation using the current IOMMU API. Since
  a device will be statically assigned to one IOMMU domain at probe time
  there is no way we can change the address space upon a context switch.
 
 We have just introduced a way to parse dma-ranges in of_dma_configure().
 
 The only way I see this done for platform devices is to do the IOMMU
 configuration in the same place: if an iommus property is found there,
 we call out to the iommu driver that matches the respective iommu device
 and let it configure the master device.
 
 The device already has multiple properties related to iommus:
 'struct device_dma_parameters', 'archdata', 'iommu_group', and
 pdev_archdata for platform devices. This should be enough to set up
 the default iommu dma_map_ops so we can have non-isolated DMA using
 dma_map_* and dma_alloc_coherent.

Right, I think up to that point things should be fine with the existing
IOMMU API and using only DMA mapping functions.

 I haven't given much thought to devices that want to use the IOMMU
 API directly so they can have multiple domains rather than rely on
 the dma-mapping abstraction.

I'm specifically thinking about cases where we want to use the IOMMU to
isolate processes from each other. This is probably most relevant for
GPUs, since they are driven to a large degree from userspace. Other
peripherals are mostly services in kernel space exclusively, so I don't
think the issue is as relevant there.

On Tegra there's two IOMMUs, one system-wide and another one directly
used by the GPU (and programmed by the GPU driver (nouveau)). For the
latter it probably doesn't make sense to expose it 

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Saturday 07 June 2014 00:45:45 Thierry Reding wrote:
 This is somewhat off-topic, but given the various concepts discussed in
 this thread I'm beginning to wonder how they will be implemented.

I think it's good you raised the question.

 The
 current implementation hooks the IOMMU API into the DMA mapping API, and
 the way this is done is by setting a single IOMMU (or rather a set of
 IOMMU operations) globally per bus type.

I hadn't realized that we have a per-bus iommu_ops pointer. I agree
this will become a limitation as soon as we have a soc with two different
IOMMUs that have platform devices attached, and it has to be moved into
the device or a structure related to that.

If that turns out controversial, we can probably have a set of pseudo
iommu ops that just call into dev-archdata-iommu_ops-function()
for ARM.

 There are two issues that I can see with that: one is that we can't
 support multiple IOMMUs in the system at all, and the other is that
 there is no context associated with the IOMMU ops, and therefore there
 is no way to differentiate between two instances of the same IOMMU. A
 few drivers use global variables to keep track of context information
 but that won't work with multiple instances, unless they keep a global
 list of all instances and then require explicit lookup in each of the
 IOMMU operations. That sounds more like a workaround rather than a
 proper solution to me.

Supporting multiple iommus that share one iommu driver should work
without such hacks, as you can put the per-device information into
dev-device_dma_parameters (this works only for very simple IOMMUs)
or dev-archdata-iommu (we may want to generalize that, I think
someone just posted patches for it).

 Discussion in this thread indicates that both of the above will be
 required at some point. Have I completely missed something or will we
 have to rework (parts of) the IOMMU API to make this work?
 
 One other thing that I have some difficulty understanding is how we can
 support things like process isolation using the current IOMMU API. Since
 a device will be statically assigned to one IOMMU domain at probe time
 there is no way we can change the address space upon a context switch.

We have just introduced a way to parse dma-ranges in of_dma_configure().

The only way I see this done for platform devices is to do the IOMMU
configuration in the same place: if an iommus property is found there,
we call out to the iommu driver that matches the respective iommu device
and let it configure the master device.

The device already has multiple properties related to iommus:
'struct device_dma_parameters', 'archdata', 'iommu_group', and
pdev_archdata for platform devices. This should be enough to set up
the default iommu dma_map_ops so we can have non-isolated DMA using
dma_map_* and dma_alloc_coherent.

I haven't given much thought to devices that want to use the IOMMU
API directly so they can have multiple domains rather than rely on
the dma-mapping abstraction.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

This is somewhat off-topic, but given the various concepts discussed in
this thread I'm beginning to wonder how they will be implemented. The
current implementation hooks the IOMMU API into the DMA mapping API, and
the way this is done is by setting a single IOMMU (or rather a set of
IOMMU operations) globally per bus type.

There are two issues that I can see with that: one is that we can't
support multiple IOMMUs in the system at all, and the other is that
there is no context associated with the IOMMU ops, and therefore there
is no way to differentiate between two instances of the same IOMMU. A
few drivers use global variables to keep track of context information
but that won't work with multiple instances, unless they keep a global
list of all instances and then require explicit lookup in each of the
IOMMU operations. That sounds more like a workaround rather than a
proper solution to me.

Discussion in this thread indicates that both of the above will be
required at some point. Have I completely missed something or will we
have to rework (parts of) the IOMMU API to make this work?

One other thing that I have some difficulty understanding is how we can
support things like process isolation using the current IOMMU API. Since
a device will be statically assigned to one IOMMU domain at probe time
there is no way we can change the address space upon a context switch.

Thierry


pgpgifk5_e0To.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 04 June 2014 23:32:00 Thierry Reding wrote:
 On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
  On Friday 30 May 2014 12:22:32 Dave Martin wrote:
+
+Examples:
+=
+
+Single-master IOMMU:
+
+
+ iommu {
+ #address-cells = 0;
+ #size-cells = 0;
+ };
+
+ master {
+ iommus = /iommu;
+ };
+
+Multiple-master IOMMU with fixed associations:
+--
+
+ /* multiple-master IOMMU */
+ iommu {
+ /*
+  * Masters are statically associated with this IOMMU and
+  * address translation is always enabled.
+  */
+ #address-cells = 0;
+ #size-cells = 0;
   
   In this example, can different translations be set up for the different
   masters?
   
   With no cells available to contain any sort of ID, it looks like this
   is not possible.
  
  Correct, this example is for an IOMMU that does not use IDs but has a
  shared address space for all devices.
 
 Couldn't these device all still have separate address spaces?

No. If they had separate address spaces, they would require a more
sophisticated IOMMU. A simple IOMMU without IDs can only be used
for overcoming address space limits (e.g. for 32-bit DMA masters on
systems with more than 4GB RAM) but not for strict isolation.

You basically have one page table shared across all devices connected
to the IOMMU, and every call to dma_alloc_coherent or dma_map_*
allocates a new IOVA that isn't used by any of the other devices
already, but you can't prevent a malicious user from getting a device
to do DMA to an IOVA that has been set up for another device.

You could have one such IOMMU per device of course, but I guess that's
not what you mean.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Varun Sethi]

 -Original Message-
 From: iommu-boun...@lists.linux-foundation.org [mailto:iommu-
 boun...@lists.linux-foundation.org] On Behalf Of Will Deacon
 Sent: Wednesday, June 04, 2014 10:12 PM
 To: Thierry Reding
 Cc: Mark Rutland; devicet...@vger.kernel.org; linux-samsung-
 s...@vger.kernel.org; Pawel Moll; Arnd Bergmann; Ian Campbell; Grant
 Grundler; Stephen Warren; linux-ker...@vger.kernel.org; Marc Zyngier;
 Linux IOMMU; Rob Herring; Kumar Gala; linux-te...@vger.kernel.org; Cho
 KyongHo; Dave P Martin; linux-arm-ker...@lists.infradead.org
 Subject: Re: [PATCH v2] devicetree: Add generic IOMMU device tree
 bindings
 
 On Wed, Jun 04, 2014 at 03:35:10PM +0100, Thierry Reding wrote:
  On Mon, Jun 02, 2014 at 11:41:04AM +0100, Dave Martin wrote:
   In the strictest sense, no.
  
   But for a large set of sane configurations, this probably works.
  
   Small sets of randomly-assigned IDs can just be enumerated one by
 one.
  
   We wouldn't be able to describe folding and bit shuffling, but we
   probably don't want to encourage that anyway.
 
  I'm having some difficulty understanding this. You make it sound like
  there's a fairly arbitrary number of IDs that the SMMU can handle. So
  how is the mapping to devices defined? If you say encourage that does
  make it sound like the assignment of IDs is purely defined by some
  mechanism in software rather than in hardware. Or they are more or
  less randomly picked by someone. If that's the case, is that not
  something that should be dynamically allocated by the kernel rather
  than put into the device tree?
 
 The set of StreamIDs that can be generated by a master is fixed in the
 hardware. The SMMU can then be programmed to map these incoming IDs onto
 a context ID (or a set of context IDs), which are the IDs used internally
 by the SMMU to find the page tables etc.
 
 The StreamID - ContextID mapping is dynamic and controlled by software.
 The Master - StreamIDs mapping is fixed in the hardware.
The Master - StreamIDs mapping may not always be fixed in the hardware. At, 
least in our case we plan to program these via software. PCI devices is one 
place where this mapping would have to be dynamic (based on the topology i.e. 
if the devices are connected to a bridge etc). Also, we have a hot plug device 
architecture where the stream ID is software programmable.

Other than that, based on the isolation requirements (iommu domain) software 
programmability offers greater flexibility.

-Varun


--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Sun, Jun 01, 2014 at 10:55:46AM +0100, Will Deacon wrote:
 On Fri, May 30, 2014 at 08:54:37PM +0100, Arnd Bergmann wrote:
  On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
   Tegra,SMMU has a similar problem and we have used a fixed size bitmap(64
   bit) to afford 64 stream IDs so that a single device can hold multiple
   IDs. If we apply the same bitmap to the above exmaple:
   
  iommu {
  /* the specifier represents the ID of the master */
  #address-cells = 1;
  #size-cells = 0;
  };
   
   master@a {
   ...
   iommus = smmu (BIT(1) | BIT(2) | BIT(3)); # IDs 1 2 3
   };
   
   master@b {
   ...
   iommus = smmu BIT(4); # ID 4
   };
   
   The disadvantage of this is that this limits the max number of streamIDs
   to support. If # of streamID is increased later more than 64, this
   format cannot cover any more. You have to predict the max # of streamIDs
   in advance if steamID is statically assigned.
   
  
  Well, the iommu specific binding could allow a variable #address-cells.
  That way, you just need to know the number of stream IDs for that instance
  of the iommu.
 
 In general, though, the SMMU will be able to support a large number of
 stream IDs (say a 16-bit space). The restriction we're interested in here is
 how many different stream IDs can be emitted by a single master device
 coming into the SMMU. *That* is a property of the master, not the SMMU.
 
 In the current arm,smmu binding I have a #stream-id-cells property in each
 master. I can then feed that straight into of_parse_phandle_with_args to
 enumerate the IDs for that master. The problem with that is we're
 artificially restricted by MAX_PHANDLE_ARGS.

Maybe I don't fully understand, but since we leave it up to the IOMMU
binding to define the exact meaning of #iommu-cells, can't you simply
use that to your advantage and define something like:

iommus = smmu 0 7;

to mean IDs 0 to 7 for this particular IOMMU type?

Thierry


pgpheD_P9pXp8.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Fri, May 30, 2014 at 09:54:37PM +0200, Arnd Bergmann wrote:
 On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
  
  IIUC the original problem, a master with 8 streamIDs means something
  like below, where some devices have multiple IDs but some have a
  single. A sinle #address-cells cannot afford those 2 masters at once.
  
 iommu {
 /* the specifier represents the ID of the master */
 #address-cells = 1;
 #size-cells = 0;
 };
  
  master@a {
  ...
  iommus = smmu 1 2 3; # 3 IDs
  };
  
  master@b {
  ...
  iommus = smmu 4; # 1 ID
  };
 
 This would not be the usual format really. It should instead be
 
   iommus = smmu 1, smmu 2, smmu 3;
 
 which can be tedious to type.

Tedious to type doesn't sound like a good argument to me. I don't see
why the above would necessarily be a bad notation. It's very much up to
the point and very explicit. This very obviously translates to:

This device has three master interfaces, one for smmu ID 1, one
for smmu ID 2 and one for smmu ID 3.

  Tegra,SMMU has a similar problem and we have used a fixed size bitmap(64
  bit) to afford 64 stream IDs so that a single device can hold multiple
  IDs. If we apply the same bitmap to the above exmaple:
  
 iommu {
 /* the specifier represents the ID of the master */
 #address-cells = 1;
 #size-cells = 0;
 };
  
  master@a {
  ...
  iommus = smmu (BIT(1) | BIT(2) | BIT(3)); # IDs 1 2 3
  };
  
  master@b {
  ...
  iommus = smmu BIT(4); # ID 4
  };
  
  The disadvantage of this is that this limits the max number of streamIDs
  to support. If # of streamID is increased later more than 64, this
  format cannot cover any more. You have to predict the max # of streamIDs
  in advance if steamID is statically assigned.
  
 
 Well, the iommu specific binding could allow a variable #address-cells.
 That way, you just need to know the number of stream IDs for that instance
 of the iommu.

That sounds fairly complicated to me. I don't see what that buys us over
the clarity and simplicity that the above explicit notation gives us. Is
it not more common for a device to have a single master rather than a
whole bunch of them?

Thierry


pgpyUy1Lx2PG8.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 04 June 2014 15:44:03 Thierry Reding wrote:
  Well, the iommu specific binding could allow a variable #address-cells.
  That way, you just need to know the number of stream IDs for that instance
  of the iommu.
 
 That sounds fairly complicated to me. I don't see what that buys us over
 the clarity and simplicity that the above explicit notation gives us. Is
 it not more common for a device to have a single master rather than a
 whole bunch of them?

If I understood the problem right, the case that people want to handle
is not actually multiple masters but instead multiple IOMMU contexts, which
you can have in a GPU to handle multiple processes drawing on different
parts of the screen, or in a NIC with virtual functions assigned to KVM
guests or user processes. I don't know what the number of contexts per
device would be here, but I assume it may be large enough to become
a nuisance to list them individually.

Then again, I also don't know how IOMMUs would show those. In the cases
I've seen before, you actually have an ID for the master that is separate
from the context ID, but apparently SMMU isn't that smart.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 04, 2014 at 02:44:03PM +0100, Thierry Reding wrote:
 On Fri, May 30, 2014 at 09:54:37PM +0200, Arnd Bergmann wrote:
  On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
   The disadvantage of this is that this limits the max number of streamIDs
   to support. If # of streamID is increased later more than 64, this
   format cannot cover any more. You have to predict the max # of streamIDs
   in advance if steamID is statically assigned.
   
  
  Well, the iommu specific binding could allow a variable #address-cells.
  That way, you just need to know the number of stream IDs for that instance
  of the iommu.
 
 That sounds fairly complicated to me. I don't see what that buys us over
 the clarity and simplicity that the above explicit notation gives us. Is
 it not more common for a device to have a single master rather than a
 whole bunch of them?

I've never seen a device upstream of an ARM SMMU with a single stream-id;
they always seem to have a whole bunch of them. Calxeda's SATA controller
had 10 IDs, for example, and a PL330 DMA controller tends to have at least
3.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Wednesday 04 June 2014 14:56:01 Will Deacon wrote:
 On Wed, Jun 04, 2014 at 02:44:03PM +0100, Thierry Reding wrote:
  On Fri, May 30, 2014 at 09:54:37PM +0200, Arnd Bergmann wrote:
   On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
The disadvantage of this is that this limits the max number of streamIDs
to support. If # of streamID is increased later more than 64, this
format cannot cover any more. You have to predict the max # of streamIDs
in advance if steamID is statically assigned.

   
   Well, the iommu specific binding could allow a variable #address-cells.
   That way, you just need to know the number of stream IDs for that instance
   of the iommu.
  
  That sounds fairly complicated to me. I don't see what that buys us over
  the clarity and simplicity that the above explicit notation gives us. Is
  it not more common for a device to have a single master rather than a
  whole bunch of them?
 
 I've never seen a device upstream of an ARM SMMU with a single stream-id;
 they always seem to have a whole bunch of them. Calxeda's SATA controller
 had 10 IDs, for example, and a PL330 DMA controller tends to have at least
 3.

What are those good for? Would we just always use the first one?

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Mon, Jun 02, 2014 at 11:41:04AM +0100, Dave Martin wrote:
 On Fri, May 30, 2014 at 09:49:59PM +0200, Arnd Bergmann wrote:
  On Friday 30 May 2014 14:31:55 Rob Herring wrote:
   On Fri, May 30, 2014 at 2:06 PM, Arnd Bergmann a...@arndb.de wrote:
On Friday 30 May 2014 08:16:05 Rob Herring wrote:
On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
thierry.red...@gmail.com wrote:
 From: Thierry Reding tred...@nvidia.com
 +IOMMU master node:
 +==
 +
 +Devices that access memory through an IOMMU are called masters. A 
 device can
 +have multiple master interfaces (to one or more IOMMU devices).
 +
 +Required properties:
 +
 +- iommus: A list of phandle and IOMMU specifier pairs that describe 
 the IOMMU
 +  master interfaces of the device. One entry in the list describes 
 one master
 +  interface of the device.
 +
 +When an iommus property is specified in a device tree node, the 
 IOMMU will
 +be used for address translation. If a dma-ranges property exists 
 in the
 +device's parent node it will be ignored. An exception to this rule 
 is if the
 +referenced IOMMU is disabled, in which case the dma-ranges 
 property of the
 +parent shall take effect.
   
Just thinking out loud, could you have dma-ranges in the iommu node
for the case when the iommu is enabled rather than putting the DMA
window information into the iommus property?
   
This would probably mean that you need both #iommu-cells and 
#address-cells.
   
The reason for doing like this was that you may need a different window
for each device, while there can only be one dma-ranges property in
an iommu node.
   
   My suggestion was that you also put the IDs in the dma-ranges as the
   first cell much as ranges for PCI encodes other information in the
   first cell. Then you can have an entry for each ID. The downside is
   another special case like PCI.
   
   The argument for using #address-cells and #size-cells seems to be to
   align with how ranges work. If that's the route we want to go, then I
   say we should not stop there and actually use dma-ranges as well.
   Otherwise, I don't see the advantage over #iommu-cells.
  
  I can see how dma-ranges in bus nodes work, it just doesn't seem to
  have any reasonable meaning in the iommu node itself.
 
 dma-ranges defines a static mapping for mastering through the bus node.
 
 The whole point of an IOMMU is that it maps dynamically, so I agree:
 I'm unclear on what dma-ranges should mean in the IOMMU node itself
 (if anything).
 
  
I don't understand the problem. If you have stream IDs 0 through 7,
you would have
   
master@a {
...
iommus = smmu 0;
};
   
master@b {
...
iommus = smmu 1;
};
   
...
   
master@12 {
...
iommus = smmu 7;
};
   
and you don't need a window at all. Why would you need a mask of
some sort?
   
   1 master with 7 IDs like this:
   
master@a {
...
iommus = smmu 0 smmu 1 smmu 2 smmu 3
   smmu 4 smmu 5 smmu 6 smmu 7;
};
   
   If there was any sort of window, then it is almost certainly the same
   window for each ID.
 
 Do we have real examples of using a window *and* an ID?  I thought the
 windowed-IOMMU concept was partly a way of encoding the ID in some
 real address bits on the bus.  If you're doing that, it seems less likely
 that there is a true ID as such (though it is possible).
 
  Ok, I see. In that case you'd probably want to have #address-cells = 1
  and #size-cells = 1 and give a range of IDs like
  
  iommus = smmu 0 8;
  
  Do you think that ranges can have a meaningful definition with the ARM
  SMMU stream IDs?
 
 In the strictest sense, no.
 
 But for a large set of sane configurations, this probably works.
 
 Small sets of randomly-assigned IDs can just be enumerated one by one.
 
 We wouldn't be able to describe folding and bit shuffling, but we
 probably don't want to encourage that anyway.

I'm having some difficulty understanding this. You make it sound like
there's a fairly arbitrary number of IDs that the SMMU can handle. So
how is the mapping to devices defined? If you say encourage that does
make it sound like the assignment of IDs is purely defined by some
mechanism in software rather than in hardware. Or they are more or less
randomly picked by someone. If that's the case, is that not something
that should be dynamically allocated by the kernel rather than put into
the device tree?

Maybe in the above case all you really need to know is how many IDs a
device needs to be assigned so that they can be properly allocated,
rather than the device exactly specifying which ones.

Thierry

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 04, 2014 at 03:01:15PM +0100, Arnd Bergmann wrote:
 On Wednesday 04 June 2014 14:56:01 Will Deacon wrote:
  On Wed, Jun 04, 2014 at 02:44:03PM +0100, Thierry Reding wrote:
   On Fri, May 30, 2014 at 09:54:37PM +0200, Arnd Bergmann wrote:
On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
 The disadvantage of this is that this limits the max number of 
 streamIDs
 to support. If # of streamID is increased later more than 64, this
 format cannot cover any more. You have to predict the max # of 
 streamIDs
 in advance if steamID is statically assigned.
 

Well, the iommu specific binding could allow a variable #address-cells.
That way, you just need to know the number of stream IDs for that 
instance
of the iommu.
   
   That sounds fairly complicated to me. I don't see what that buys us over
   the clarity and simplicity that the above explicit notation gives us. Is
   it not more common for a device to have a single master rather than a
   whole bunch of them?
  
  I've never seen a device upstream of an ARM SMMU with a single stream-id;
  they always seem to have a whole bunch of them. Calxeda's SATA controller
  had 10 IDs, for example, and a PL330 DMA controller tends to have at least
  3.
 
 What are those good for? Would we just always use the first one?

I think It's just an artifact of how these systems tend to be built. The
StreamID is constructed out of bits on the bus (AXI and sideband), so a
DMA controller could easily have one ID for reads, one for writes,
one for execute. Masters capable of multiple outstanding transactions
could also have sets of IDs too.

The SMMU driver would point them all at the same context bank, as far as I
can tell, but that means the driver does need to be aware of the incoming
IDs.

Rob might know how they constructed the IDs on the Calxeda SATA controller?

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Wed, Jun 04, 2014 at 03:35:10PM +0100, Thierry Reding wrote:
 On Mon, Jun 02, 2014 at 11:41:04AM +0100, Dave Martin wrote:
  In the strictest sense, no.
  
  But for a large set of sane configurations, this probably works.
  
  Small sets of randomly-assigned IDs can just be enumerated one by one.
  
  We wouldn't be able to describe folding and bit shuffling, but we
  probably don't want to encourage that anyway.
 
 I'm having some difficulty understanding this. You make it sound like
 there's a fairly arbitrary number of IDs that the SMMU can handle. So
 how is the mapping to devices defined? If you say encourage that does
 make it sound like the assignment of IDs is purely defined by some
 mechanism in software rather than in hardware. Or they are more or less
 randomly picked by someone. If that's the case, is that not something
 that should be dynamically allocated by the kernel rather than put into
 the device tree?

The set of StreamIDs that can be generated by a master is fixed in the
hardware. The SMMU can then be programmed to map these incoming IDs onto
a context ID (or a set of context IDs), which are the IDs used internally
by the SMMU to find the page tables etc.

The StreamID - ContextID mapping is dynamic and controlled by software.
The Master - StreamIDs mapping is fixed in the hardware.

Will

--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Wed, Jun 04, 2014 at 05:41:32PM +0100, Will Deacon wrote:
 On Wed, Jun 04, 2014 at 03:35:10PM +0100, Thierry Reding wrote:
  On Mon, Jun 02, 2014 at 11:41:04AM +0100, Dave Martin wrote:
   In the strictest sense, no.
   
   But for a large set of sane configurations, this probably works.
   
   Small sets of randomly-assigned IDs can just be enumerated one by one.
   
   We wouldn't be able to describe folding and bit shuffling, but we
   probably don't want to encourage that anyway.
  
  I'm having some difficulty understanding this. You make it sound like
  there's a fairly arbitrary number of IDs that the SMMU can handle. So
  how is the mapping to devices defined? If you say encourage that does
  make it sound like the assignment of IDs is purely defined by some
  mechanism in software rather than in hardware. Or they are more or less
  randomly picked by someone. If that's the case, is that not something
  that should be dynamically allocated by the kernel rather than put into
  the device tree?
 
 The set of StreamIDs that can be generated by a master is fixed in the
 hardware. The SMMU can then be programmed to map these incoming IDs onto
 a context ID (or a set of context IDs), which are the IDs used internally
 by the SMMU to find the page tables etc.
 
 The StreamID - ContextID mapping is dynamic and controlled by software.
 The Master - StreamIDs mapping is fixed in the hardware.

Okay, that sounds similar to what the Tegra SMMU does. The naming is
slightly differently. Master is called memory client, StreamID
would be SWGROUP and ContextID I guess would map to what's called an
ASID (Address Space ID) on Tegra.

I'm not sure about the amount of leverage that we have to encourage the
mappings that we think are easy to describe in DT and discourage those
that we don't want. Last time I checked we were still playing catch up
rather than being able to give recommendations to hardware engineers
about what will result in sane DT content and what won't.

Irrespective of the above, I still think that a generic binding based on
an #iommu-cells and iommus property gives us the most flexibility. For
easy cases, most of which are listed in the current proposal could
easily be dealt with in convenience helpers. For the more complex cases
drivers for those IOMMUs can define a specifier that make sense for the
mappings they need to be able to describe.

Also we're talking about a generic IOMMU binding here. That means that
if some hardware comes along that's not generic and doesn't fit into
the constraints set by this binding, then we still have the option of
defining a completely different one for that particular case.

Thierry


pgpgeCZKqAdrG.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Fri, May 30, 2014 at 12:27:28PM +0100, Dave Martin wrote:
 On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
[...]
  Arnd, can you take another look at this binding and see if there's
  anything else missing? If not I'll go through the document again and
  update all #address-cells/#size-cells references with #iommu-cells as
  appropriate and submit v3.
 
 How do you envisage propagation of the master ID bits downstream of the
 IOMMU would be described?
 
 We will definitely need a way to describe this for GICv3.  How those
 values are propagated is likely to vary between related SoCs and doesn't
 feel like it should be baked into a driver, especially for the ARM SMMU
 which may get reused in radically different SoC families from different
 vendors.

Well, we've had cases like these in the past (power sequences come to
mind). Some concepts are just too difficult or unwieldy to be put into
device tree. I think that this is one of them.

 The most likely types of remapping are the adding of a base offset or
 some extra bits to the ID -- because not all MSIs to the GIC will
 necessarily pass through the IOMMU.  It's also possible that we might
 see ID squashing or folding in some systems.

It can easily be argued that if the algorithm used to remap the ID
varies, the compatibility of the device changes. Therefore I would
expect any variant of the GICv3 that deviates from the standard
mapping (if there is such a thing) to have its own compatible string.

At that point there are two possibilities: a) handle the remapping in
code hooked into the driver via matching of the compatible string or
b) specifying a different specifier for each of these and keep the
description in DT. From an implementation point of view, variant b) is
likely going to be very similar to a), though (I'm thinking .of_xlate
type of functions here).

Thierry


pgp_inyCP5rEo.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
 On Friday 30 May 2014 12:22:32 Dave Martin wrote:
   +
   +Examples:
   +=
   +
   +Single-master IOMMU:
   +
   +
   + iommu {
   + #address-cells = 0;
   + #size-cells = 0;
   + };
   +
   + master {
   + iommus = /iommu;
   + };
   +
   +Multiple-master IOMMU with fixed associations:
   +--
   +
   + /* multiple-master IOMMU */
   + iommu {
   + /*
   +  * Masters are statically associated with this IOMMU and
   +  * address translation is always enabled.
   +  */
   + #address-cells = 0;
   + #size-cells = 0;
  
  In this example, can different translations be set up for the different
  masters?
  
  With no cells available to contain any sort of ID, it looks like this
  is not possible.
 
 Correct, this example is for an IOMMU that does not use IDs but has a
 shared address space for all devices.

Couldn't these device all still have separate address spaces?

Thierry


pgpswaTg3Z3lu.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Dave Martin]

On Fri, May 30, 2014 at 09:49:59PM +0200, Arnd Bergmann wrote:
 On Friday 30 May 2014 14:31:55 Rob Herring wrote:
  On Fri, May 30, 2014 at 2:06 PM, Arnd Bergmann a...@arndb.de wrote:
   On Friday 30 May 2014 08:16:05 Rob Herring wrote:
   On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
   thierry.red...@gmail.com wrote:
From: Thierry Reding tred...@nvidia.com
+IOMMU master node:
+==
+
+Devices that access memory through an IOMMU are called masters. A 
device can
+have multiple master interfaces (to one or more IOMMU devices).
+
+Required properties:
+
+- iommus: A list of phandle and IOMMU specifier pairs that describe 
the IOMMU
+  master interfaces of the device. One entry in the list describes 
one master
+  interface of the device.
+
+When an iommus property is specified in a device tree node, the 
IOMMU will
+be used for address translation. If a dma-ranges property exists in 
the
+device's parent node it will be ignored. An exception to this rule is 
if the
+referenced IOMMU is disabled, in which case the dma-ranges property 
of the
+parent shall take effect.
  
   Just thinking out loud, could you have dma-ranges in the iommu node
   for the case when the iommu is enabled rather than putting the DMA
   window information into the iommus property?
  
   This would probably mean that you need both #iommu-cells and 
   #address-cells.
  
   The reason for doing like this was that you may need a different window
   for each device, while there can only be one dma-ranges property in
   an iommu node.
  
  My suggestion was that you also put the IDs in the dma-ranges as the
  first cell much as ranges for PCI encodes other information in the
  first cell. Then you can have an entry for each ID. The downside is
  another special case like PCI.
  
  The argument for using #address-cells and #size-cells seems to be to
  align with how ranges work. If that's the route we want to go, then I
  say we should not stop there and actually use dma-ranges as well.
  Otherwise, I don't see the advantage over #iommu-cells.
 
 I can see how dma-ranges in bus nodes work, it just doesn't seem to
 have any reasonable meaning in the iommu node itself.

dma-ranges defines a static mapping for mastering through the bus node.

The whole point of an IOMMU is that it maps dynamically, so I agree:
I'm unclear on what dma-ranges should mean in the IOMMU node itself
(if anything).

 
   I don't understand the problem. If you have stream IDs 0 through 7,
   you would have
  
   master@a {
   ...
   iommus = smmu 0;
   };
  
   master@b {
   ...
   iommus = smmu 1;
   };
  
   ...
  
   master@12 {
   ...
   iommus = smmu 7;
   };
  
   and you don't need a window at all. Why would you need a mask of
   some sort?
  
  1 master with 7 IDs like this:
  
   master@a {
   ...
   iommus = smmu 0 smmu 1 smmu 2 smmu 3
  smmu 4 smmu 5 smmu 6 smmu 7;
   };
  
  If there was any sort of window, then it is almost certainly the same
  window for each ID.

Do we have real examples of using a window *and* an ID?  I thought the
windowed-IOMMU concept was partly a way of encoding the ID in some
real address bits on the bus.  If you're doing that, it seems less likely
that there is a true ID as such (though it is possible).

 Ok, I see. In that case you'd probably want to have #address-cells = 1
 and #size-cells = 1 and give a range of IDs like
 
   iommus = smmu 0 8;
 
 Do you think that ranges can have a meaningful definition with the ARM
 SMMU stream IDs?

In the strictest sense, no.

But for a large set of sane configurations, this probably works.

Small sets of randomly-assigned IDs can just be enumerated one by one.

We wouldn't be able to describe folding and bit shuffling, but we
probably don't want to encourage that anyway.

Cheers
---Dave
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Dave Martin]

On Fri, May 30, 2014 at 09:11:07PM +0200, Arnd Bergmann wrote:
 On Friday 30 May 2014 12:27:28 Dave Martin wrote:
  On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
   On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
On 05/23/2014 02:36 PM, Thierry Reding wrote:
I think this is a mistake. address-cells/size-cells are for transactions
flowing down the bus (from the CPU to date). Describing a connection
from a device to an IOMMU is something completely different, and should
therefore simply use an iommu-cells property to describe any necessary
information. If we start re-using properties for different things in
different contexts, how is anyone going to know what they mean, and how
will conflicts be resolved. For example, what if there's a single HW
module that both acts as a regular register bus with children (where
address-cells/size-cells defines how transactions reach the children
from the parent), and is also an IOMMU (where according to this binding
proposal, address-cells/size-cells represent some aspect of the IOMMU
feature). Using different properties for different things is the only
sane way to keep different concepts separate. Another alternative would
be to represent the single HW module as separate nodes in DT, but I
think that will only make our lives harder, and where I've done that in
the past, I've regretted it.
   
   There was some back-and-forth on this topic and the latest concensus
   when I wrote the second version was that #address-cells and #size-cells
   were to be used.
   
   But there was some bore back-and-forth after that, and it seems like
   Arnd no longer thinks that using #address-cells and #size-cells is a
   good idea either[0].
  
  Mistake or not, ePAPR already (ab)uses the address concept for PCI.
  Unless ePAPR is wrong, I don't think it makes sense to argue that
  the address concept cannot be repurposed.
  
  The reason why this is abused for PCI is the same as our reason here:
  different masters really are treated as distinct even when accessing
  the same destination address, and DT has no general native way to
  describe that.
  
  One clear advantage of using #address-cells etc. is that ePAPR already
  has very clear and well-defined ways of how to specify range mappings.
  This gives us a ready-made way to describe windowed IOMMUs and 1:1
  remappings of whole blocks of master IDs, which are the most obvious
  cases other than having a small-integer set of explicit IDs.
  
  That said, the PCI pseudo-address thing is not something we _necessarily_
  want to repeat.
 
 I'm really impartial to the question of whether to use #address-cells
 or #iommus now. It's possible that we can use the addresses in some
 meaningful way, but it's not clear to me that this will help make the
 representation cleaner or that we will actually want it for SMMU.
 
   Arnd, can you take another look at this binding and see if there's
   anything else missing? If not I'll go through the document again and
   update all #address-cells/#size-cells references with #iommu-cells as
   appropriate and submit v3.
  
  How do you envisage propagation of the master ID bits downstream of the
  IOMMU would be described?
  
  We will definitely need a way to describe this for GICv3.  How those
  values are propagated is likely to vary between related SoCs and doesn't
  feel like it should be baked into a driver, especially for the ARM SMMU
  which may get reused in radically different SoC families from different
  vendors.
  
  The most likely types of remapping are the adding of a base offset or
  some extra bits to the ID -- because not all MSIs to the GIC will
  necessarily pass through the IOMMU.  It's also possible that we might
  see ID squashing or folding in some systems.
  
  
  For types of remapping which mix the ID and address together, I now
  do tend to agree that any flexibility arising from describing that
  in a general way that is unlikely to repay the cost of trying to
  interpret and analyse the DT.  Defining a few sterotypical kinds
  of mapping explicitly, as needed, looks more sensible for now.  The
  windowed-IOMMU case is one example.
 
 For MSI, my feeling is that we'd just be best off doing an end-to-end
 description. Any device using an MSI would have an interrupt specifier
 that is sufficient for the interrupt controller to understand where
 the IRQ comes from, and return an address/value pair back to the driver
 to program into some register.

This is OK so long as the number of endpoint pairings is trivial.

If there are multiple GICv3 ITSes addressable by a master, then we
might have to describe each one per master, or impose policy on the
DT whereby we choose one or two arbitrary or preferred ITSes for a
particular master and describe just those.

It's worth nothing that ePAPR strenuously avoids such an approach
for describing DMA: device-device DMA would require 

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Dave Martin]

On Fri, May 30, 2014 at 09:01:19PM +0200, Arnd Bergmann wrote:
 On Friday 30 May 2014 12:22:32 Dave Martin wrote:
   +
   +Examples:
   +=
   +
   +Single-master IOMMU:
   +
   +
   + iommu {
   + #address-cells = 0;
   + #size-cells = 0;
   + };
   +
   + master {
   + iommus = /iommu;
   + };
   +
   +Multiple-master IOMMU with fixed associations:
   +--
   +
   + /* multiple-master IOMMU */
   + iommu {
   + /*
   +  * Masters are statically associated with this IOMMU and
   +  * address translation is always enabled.
   +  */
   + #address-cells = 0;
   + #size-cells = 0;
  
  In this example, can different translations be set up for the different
  masters?
  
  With no cells available to contain any sort of ID, it looks like this
  is not possible.
 
 Correct, this example is for an IOMMU that does not use IDs but has a
 shared address space for all devices.
 
   +Multiple-master IOMMU with configurable DMA window:
   +---
   +
   + / {
   + #address-cells = 1;
   + #size-cells = 1;
   +
   + iommu {
   + /* master ID, address of DMA window */
   + #address-cells = 2;
   + #size-cells = 2;
   + };
   +
   + master {
   + /* master ID 42, 4 GiB DMA window starting at 0 */
   + iommus = /iommu  42 0  0x1 0x0;
  
  I'm still concerned that in order to deal with future cases we will have
  to invent multiple ways to parse the iommus property.  For example, if
  we have a PCEe RC mastering through an IOMMU, it will pass a huge set
  of possible master IDs to the IOMMU, not just noe or two.
  
  Do you have a solution in mind for that which doesn't break backwards
  compatibility?
 
 I think we can treat PCI as a special case here and have an interface
 that gets used by the PCI core code to talk to the IOMMU core code
 when setting up a the dma_map_ops for a PCI function. As long as the
 IOMMU driver understands what PCI is, we don't have to describe the
 mapping in detail.

PCI is only an example, but I admit it's likely to be the most important
example of a peripheral using a huge ID space.

There may still be integration-specific parameters which would need to
be fed in via DT regarding how IDs coming out of the RC map onto the
IOMMU and GIC.

 
  One option is to include an extra cell to the IOMMUs property
  that indicates how to parse it.  For now, only a single value would
  be defined.  For example:
  
  iommus = /iommu IOMMU_SIMPLE 42;
  
  Then maybe later
  
  iommus = /iommu IOMMU_RANGE 0x1 0x1;
  
  (I'm not suggesting what IOMMU_RANGE might mean.)
  
 
 This can really be left up to the specific IOMMU driver itself.
 We can have drivers that support both #address-cells=1
 and #address-cells=2 and behave differently based on that.
 I don't see a reason to define that across IOMMU implementations.

There's a risk we have to come up with multiple solutions to the same
problem there -- i.e., one solution per IOMMU implementation and then
another solution for the GIC.

Ideally that would be avoided, but the number of times this problem
would need to be solved is probably not that large.

Cheers
---Dave
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Will Deacon]

On Fri, May 30, 2014 at 08:54:37PM +0100, Arnd Bergmann wrote:
 On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
  Tegra,SMMU has a similar problem and we have used a fixed size bitmap(64
  bit) to afford 64 stream IDs so that a single device can hold multiple
  IDs. If we apply the same bitmap to the above exmaple:
  
 iommu {
 /* the specifier represents the ID of the master */
 #address-cells = 1;
 #size-cells = 0;
 };
  
  master@a {
  ...
  iommus = smmu (BIT(1) | BIT(2) | BIT(3)); # IDs 1 2 3
  };
  
  master@b {
  ...
  iommus = smmu BIT(4); # ID 4
  };
  
  The disadvantage of this is that this limits the max number of streamIDs
  to support. If # of streamID is increased later more than 64, this
  format cannot cover any more. You have to predict the max # of streamIDs
  in advance if steamID is statically assigned.
  
 
 Well, the iommu specific binding could allow a variable #address-cells.
 That way, you just need to know the number of stream IDs for that instance
 of the iommu.

In general, though, the SMMU will be able to support a large number of
stream IDs (say a 16-bit space). The restriction we're interested in here is
how many different stream IDs can be emitted by a single master device
coming into the SMMU. *That* is a property of the master, not the SMMU.

In the current arm,smmu binding I have a #stream-id-cells property in each
master. I can then feed that straight into of_parse_phandle_with_args to
enumerate the IDs for that master. The problem with that is we're
artificially restricted by MAX_PHANDLE_ARGS.

Will
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Thierry Reding]

On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
 On 05/23/2014 02:36 PM, Thierry Reding wrote:
  From: Thierry Reding tred...@nvidia.com
  
  This commit introduces a generic device tree binding for IOMMU devices.
  Only a very minimal subset is described here, but it is enough to cover
  the requirements of both the Exynos System MMU and Tegra SMMU as
  discussed here:
  
  https://lkml.org/lkml/2014/4/27/346
  
  Signed-off-by: Thierry Reding tred...@nvidia.com
  ---
  Apologies for the noise, but apparently I mistyped one of the email
  addresses, should be fixed now.
  
  Changes in v2:
  - add notes about dma-ranges property (drop note from commit message)
  - document priorities of iommus property vs. dma-ranges property
  - drop #iommu-cells in favour of #address-cells and #size-cells
 
 I think this is a mistake. address-cells/size-cells are for transactions
 flowing down the bus (from the CPU to date). Describing a connection
 from a device to an IOMMU is something completely different, and should
 therefore simply use an iommu-cells property to describe any necessary
 information. If we start re-using properties for different things in
 different contexts, how is anyone going to know what they mean, and how
 will conflicts be resolved. For example, what if there's a single HW
 module that both acts as a regular register bus with children (where
 address-cells/size-cells defines how transactions reach the children
 from the parent), and is also an IOMMU (where according to this binding
 proposal, address-cells/size-cells represent some aspect of the IOMMU
 feature). Using different properties for different things is the only
 sane way to keep different concepts separate. Another alternative would
 be to represent the single HW module as separate nodes in DT, but I
 think that will only make our lives harder, and where I've done that in
 the past, I've regretted it.

There was some back-and-forth on this topic and the latest concensus
when I wrote the second version was that #address-cells and #size-cells
were to be used.

But there was some bore back-and-forth after that, and it seems like
Arnd no longer thinks that using #address-cells and #size-cells is a
good idea either[0].

Arnd, can you take another look at this binding and see if there's
anything else missing? If not I'll go through the document again and
update all #address-cells/#size-cells references with #iommu-cells as
appropriate and submit v3.

Thanks,
Thierry

[0]: https://lkml.org/lkml/2014/5/20/609


pgpRKsl1aLJUT.pgp
Description: PGP signature

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Dave Martin]

On Fri, May 23, 2014 at 10:36:35PM +0200, Thierry Reding wrote:
 From: Thierry Reding tred...@nvidia.com
 
 This commit introduces a generic device tree binding for IOMMU devices.
 Only a very minimal subset is described here, but it is enough to cover
 the requirements of both the Exynos System MMU and Tegra SMMU as
 discussed here:
 
 https://lkml.org/lkml/2014/4/27/346
 
 Signed-off-by: Thierry Reding tred...@nvidia.com
 ---
 Apologies for the noise, but apparently I mistyped one of the email
 addresses, should be fixed now.
 
 Changes in v2:
 - add notes about dma-ranges property (drop note from commit message)
 - document priorities of iommus property vs. dma-ranges property
 - drop #iommu-cells in favour of #address-cells and #size-cells
 - remove multiple-master device example
 
  Documentation/devicetree/bindings/iommu/iommu.txt | 167 
 ++
  1 file changed, 167 insertions(+)
  create mode 100644 Documentation/devicetree/bindings/iommu/iommu.txt
 
 diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt 
 b/Documentation/devicetree/bindings/iommu/iommu.txt
 new file mode 100644
 index ..6ce759afcc94
 --- /dev/null
 +++ b/Documentation/devicetree/bindings/iommu/iommu.txt
 @@ -0,0 +1,167 @@
 +This document describes the generic device tree binding for IOMMUs and their
 +master(s).
 +
 +
 +IOMMU device node:
 +==
 +
 +An IOMMU can provide the following services:
 +
 +* Remap address space to allow devices to access physical memory ranges that
 +  they otherwise wouldn't be capable of accessing.
 +
 +  Example: 32-bit DMA to 64-bit physical addresses
 +
 +* Implement scatter-gather at page level granularity so that the device does
 +  not have to.
 +
 +* Provide system protection against rogue DMA by forcing all accesses to go
 +  through the IOMMU and faulting when encountering accesses to unmapped
 +  address regions.
 +
 +* Provide address space isolation between multiple contexts.
 +
 +  Example: Virtualization
 +
 +Device nodes compatible with this binding represent hardware with some of the
 +above capabilities.
 +
 +IOMMUs can be single-master or multiple-master. Single-master IOMMU devices
 +typically have a fixed association to the master device, whereas multiple-
 +master IOMMU devices can translate accesses from more than one master.
 +
 +The device tree node of the IOMMU device's parent bus must contain a valid
 +dma-ranges property that describes how the physical address space of the
 +IOMMU maps to memory. An empty dma-ranges property means that there is a
 +1:1 mapping from IOMMU to memory.
 +
 +Required properties:
 +
 +- #address-cells: The number of cells in an IOMMU specifier needed to encode
 +  an address.
 +- #size-cells: The number of cells in an IOMMU specifier needed to represent
 +  the length of an address range.
 +
 +Typical values for the above include:
 +- #address-cells = 0, size-cells = 0: Single master IOMMU devices are not
 +  configurable and therefore no additional information needs to be encoded in
 +  the specifier. This may also apply to multiple master IOMMU devices that do
 +  not allow the association of masters to be configured.
 +- #address-cells = 1, size-cells = 0: Multiple master IOMMU devices may
 +  need to be configured in order to enable translation for a given master. In
 +  such cases the single address cell corresponds to the master device's ID.
 +- #address-cells = 2, size-cells = 2: Some IOMMU devices allow the DMA
 +  window for masters to be configured. The first cell of the address in this
 +  may contain the master device's ID for example, while the second cell could
 +  contain the start of the DMA window for the given device. The length of the
 +  DMA window is specified by two additional cells.
 +
 +
 +IOMMU master node:
 +==
 +
 +Devices that access memory through an IOMMU are called masters. A device can
 +have multiple master interfaces (to one or more IOMMU devices).
 +
 +Required properties:
 +
 +- iommus: A list of phandle and IOMMU specifier pairs that describe the IOMMU
 +  master interfaces of the device. One entry in the list describes one master
 +  interface of the device.
 +
 +When an iommus property is specified in a device tree node, the IOMMU will
 +be used for address translation. If a dma-ranges property exists in the
 +device's parent node it will be ignored. An exception to this rule is if the
 +referenced IOMMU is disabled, in which case the dma-ranges property of the
 +parent shall take effect.
 +
 +Optional properties:
 +
 +- iommu-names: A list of names identifying each entry in the iommus
 +  property.
 +
 +
 +Notes:
 +==
 +
 +One possible extension to the above is to use an iommus property along with
 +a dma-ranges property in a bus device node (such as PCI host bridges). This
 +can be useful to describe how children on the bus relate to the IOMMU if they
 +are not explicitly 

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Dave Martin]

On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
 On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
  On 05/23/2014 02:36 PM, Thierry Reding wrote:
   From: Thierry Reding tred...@nvidia.com
   
   This commit introduces a generic device tree binding for IOMMU devices.
   Only a very minimal subset is described here, but it is enough to cover
   the requirements of both the Exynos System MMU and Tegra SMMU as
   discussed here:
   
   https://lkml.org/lkml/2014/4/27/346
   
   Signed-off-by: Thierry Reding tred...@nvidia.com
   ---
   Apologies for the noise, but apparently I mistyped one of the email
   addresses, should be fixed now.
   
   Changes in v2:
   - add notes about dma-ranges property (drop note from commit message)
   - document priorities of iommus property vs. dma-ranges property
   - drop #iommu-cells in favour of #address-cells and #size-cells
  
  I think this is a mistake. address-cells/size-cells are for transactions
  flowing down the bus (from the CPU to date). Describing a connection
  from a device to an IOMMU is something completely different, and should
  therefore simply use an iommu-cells property to describe any necessary
  information. If we start re-using properties for different things in
  different contexts, how is anyone going to know what they mean, and how
  will conflicts be resolved. For example, what if there's a single HW
  module that both acts as a regular register bus with children (where
  address-cells/size-cells defines how transactions reach the children
  from the parent), and is also an IOMMU (where according to this binding
  proposal, address-cells/size-cells represent some aspect of the IOMMU
  feature). Using different properties for different things is the only
  sane way to keep different concepts separate. Another alternative would
  be to represent the single HW module as separate nodes in DT, but I
  think that will only make our lives harder, and where I've done that in
  the past, I've regretted it.
 
 There was some back-and-forth on this topic and the latest concensus
 when I wrote the second version was that #address-cells and #size-cells
 were to be used.
 
 But there was some bore back-and-forth after that, and it seems like
 Arnd no longer thinks that using #address-cells and #size-cells is a
 good idea either[0].

Mistake or not, ePAPR already (ab)uses the address concept for PCI.
Unless ePAPR is wrong, I don't think it makes sense to argue that
the address concept cannot be repurposed.

The reason why this is abused for PCI is the same as our reason here:
different masters really are treated as distinct even when accessing
the same destination address, and DT has no general native way to
describe that.

One clear advantage of using #address-cells etc. is that ePAPR already
has very clear and well-defined ways of how to specify range mappings.
This gives us a ready-made way to describe windowed IOMMUs and 1:1
remappings of whole blocks of master IDs, which are the most obvious
cases other than having a small-integer set of explicit IDs.

That said, the PCI pseudo-address thing is not something we _necessarily_
want to repeat.

 Arnd, can you take another look at this binding and see if there's
 anything else missing? If not I'll go through the document again and
 update all #address-cells/#size-cells references with #iommu-cells as
 appropriate and submit v3.

How do you envisage propagation of the master ID bits downstream of the
IOMMU would be described?

We will definitely need a way to describe this for GICv3.  How those
values are propagated is likely to vary between related SoCs and doesn't
feel like it should be baked into a driver, especially for the ARM SMMU
which may get reused in radically different SoC families from different
vendors.

The most likely types of remapping are the adding of a base offset or
some extra bits to the ID -- because not all MSIs to the GIC will
necessarily pass through the IOMMU.  It's also possible that we might
see ID squashing or folding in some systems.


For types of remapping which mix the ID and address together, I now
do tend to agree that any flexibility arising from describing that
in a general way that is unlikely to repay the cost of trying to
interpret and analyse the DT.  Defining a few sterotypical kinds
of mapping explicitly, as needed, looks more sensible for now.  The
windowed-IOMMU case is one example.

Cheers
---Dave
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Rob Herring]

On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
thierry.red...@gmail.com wrote:
 From: Thierry Reding tred...@nvidia.com

 This commit introduces a generic device tree binding for IOMMU devices.
 Only a very minimal subset is described here, but it is enough to cover
 the requirements of both the Exynos System MMU and Tegra SMMU as
 discussed here:

 https://lkml.org/lkml/2014/4/27/346

 Signed-off-by: Thierry Reding tred...@nvidia.com
 ---
 Changes in v2:
 - add notes about dma-ranges property (drop note from commit message)
 - document priorities of iommus property vs. dma-ranges property
 - drop #iommu-cells in favour of #address-cells and #size-cells
 - remove multiple-master device example

  Documentation/devicetree/bindings/iommu/iommu.txt | 167 
 ++
  1 file changed, 167 insertions(+)
  create mode 100644 Documentation/devicetree/bindings/iommu/iommu.txt

 diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt 
 b/Documentation/devicetree/bindings/iommu/iommu.txt
 new file mode 100644
 index ..6ce759afcc94
 --- /dev/null
 +++ b/Documentation/devicetree/bindings/iommu/iommu.txt
 @@ -0,0 +1,167 @@
 +This document describes the generic device tree binding for IOMMUs and their
 +master(s).
 +
 +
 +IOMMU device node:
 +==
 +
 +An IOMMU can provide the following services:
 +
 +* Remap address space to allow devices to access physical memory ranges that
 +  they otherwise wouldn't be capable of accessing.
 +
 +  Example: 32-bit DMA to 64-bit physical addresses
 +
 +* Implement scatter-gather at page level granularity so that the device does
 +  not have to.
 +
 +* Provide system protection against rogue DMA by forcing all accesses to go
 +  through the IOMMU and faulting when encountering accesses to unmapped
 +  address regions.
 +
 +* Provide address space isolation between multiple contexts.
 +
 +  Example: Virtualization
 +
 +Device nodes compatible with this binding represent hardware with some of the
 +above capabilities.
 +
 +IOMMUs can be single-master or multiple-master. Single-master IOMMU devices
 +typically have a fixed association to the master device, whereas multiple-
 +master IOMMU devices can translate accesses from more than one master.
 +
 +The device tree node of the IOMMU device's parent bus must contain a valid
 +dma-ranges property that describes how the physical address space of the
 +IOMMU maps to memory. An empty dma-ranges property means that there is a
 +1:1 mapping from IOMMU to memory.
 +
 +Required properties:
 +
 +- #address-cells: The number of cells in an IOMMU specifier needed to encode
 +  an address.
 +- #size-cells: The number of cells in an IOMMU specifier needed to represent
 +  the length of an address range.
 +
 +Typical values for the above include:
 +- #address-cells = 0, size-cells = 0: Single master IOMMU devices are not
 +  configurable and therefore no additional information needs to be encoded in
 +  the specifier. This may also apply to multiple master IOMMU devices that do
 +  not allow the association of masters to be configured.
 +- #address-cells = 1, size-cells = 0: Multiple master IOMMU devices may
 +  need to be configured in order to enable translation for a given master. In
 +  such cases the single address cell corresponds to the master device's ID.
 +- #address-cells = 2, size-cells = 2: Some IOMMU devices allow the DMA
 +  window for masters to be configured. The first cell of the address in this
 +  may contain the master device's ID for example, while the second cell could
 +  contain the start of the DMA window for the given device. The length of the
 +  DMA window is specified by two additional cells.
 +
 +
 +IOMMU master node:
 +==
 +
 +Devices that access memory through an IOMMU are called masters. A device can
 +have multiple master interfaces (to one or more IOMMU devices).
 +
 +Required properties:
 +
 +- iommus: A list of phandle and IOMMU specifier pairs that describe the IOMMU
 +  master interfaces of the device. One entry in the list describes one master
 +  interface of the device.
 +
 +When an iommus property is specified in a device tree node, the IOMMU will
 +be used for address translation. If a dma-ranges property exists in the
 +device's parent node it will be ignored. An exception to this rule is if the
 +referenced IOMMU is disabled, in which case the dma-ranges property of the
 +parent shall take effect.

Just thinking out loud, could you have dma-ranges in the iommu node
for the case when the iommu is enabled rather than putting the DMA
window information into the iommus property?

This would probably mean that you need both #iommu-cells and #address-cells.

 +
 +Optional properties:
 +
 +- iommu-names: A list of names identifying each entry in the iommus
 +  property.

Do we really need a name here? I would not expect that you have
clearly documented names here from the datasheet like 

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 30 May 2014 12:22:32 Dave Martin wrote:
  +
  +Examples:
  +=
  +
  +Single-master IOMMU:
  +
  +
  +   iommu {
  +   #address-cells = 0;
  +   #size-cells = 0;
  +   };
  +
  +   master {
  +   iommus = /iommu;
  +   };
  +
  +Multiple-master IOMMU with fixed associations:
  +--
  +
  +   /* multiple-master IOMMU */
  +   iommu {
  +   /*
  +* Masters are statically associated with this IOMMU and
  +* address translation is always enabled.
  +*/
  +   #address-cells = 0;
  +   #size-cells = 0;
 
 In this example, can different translations be set up for the different
 masters?
 
 With no cells available to contain any sort of ID, it looks like this
 is not possible.

Correct, this example is for an IOMMU that does not use IDs but has a
shared address space for all devices.

  +Multiple-master IOMMU with configurable DMA window:
  +---
  +
  +   / {
  +   #address-cells = 1;
  +   #size-cells = 1;
  +
  +   iommu {
  +   /* master ID, address of DMA window */
  +   #address-cells = 2;
  +   #size-cells = 2;
  +   };
  +
  +   master {
  +   /* master ID 42, 4 GiB DMA window starting at 0 */
  +   iommus = /iommu  42 0  0x1 0x0;
 
 I'm still concerned that in order to deal with future cases we will have
 to invent multiple ways to parse the iommus property.  For example, if
 we have a PCEe RC mastering through an IOMMU, it will pass a huge set
 of possible master IDs to the IOMMU, not just noe or two.
 
 Do you have a solution in mind for that which doesn't break backwards
 compatibility?

I think we can treat PCI as a special case here and have an interface
that gets used by the PCI core code to talk to the IOMMU core code
when setting up a the dma_map_ops for a PCI function. As long as the
IOMMU driver understands what PCI is, we don't have to describe the
mapping in detail.

 One option is to include an extra cell to the IOMMUs property
 that indicates how to parse it.  For now, only a single value would
 be defined.  For example:
 
   iommus = /iommu IOMMU_SIMPLE 42;
 
 Then maybe later
 
   iommus = /iommu IOMMU_RANGE 0x1 0x1;
 
 (I'm not suggesting what IOMMU_RANGE might mean.)
 

This can really be left up to the specific IOMMU driver itself.
We can have drivers that support both #address-cells=1
and #address-cells=2 and behave differently based on that.
I don't see a reason to define that across IOMMU implementations.

 Other options are to introduce a new property name
 
   range-iommus = /iommu 0x1 0x1;
 
 or control the parsing of incompatible iommus properties via a compatible
 string somewhere.

Introducing a new compatible string is always an option as the last resort.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 30 May 2014 08:16:05 Rob Herring wrote:
 On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
 thierry.red...@gmail.com wrote:
  From: Thierry Reding tred...@nvidia.com
  +IOMMU master node:
  +==
  +
  +Devices that access memory through an IOMMU are called masters. A device 
  can
  +have multiple master interfaces (to one or more IOMMU devices).
  +
  +Required properties:
  +
  +- iommus: A list of phandle and IOMMU specifier pairs that describe the 
  IOMMU
  +  master interfaces of the device. One entry in the list describes one 
  master
  +  interface of the device.
  +
  +When an iommus property is specified in a device tree node, the IOMMU 
  will
  +be used for address translation. If a dma-ranges property exists in the
  +device's parent node it will be ignored. An exception to this rule is if 
  the
  +referenced IOMMU is disabled, in which case the dma-ranges property of 
  the
  +parent shall take effect.
 
 Just thinking out loud, could you have dma-ranges in the iommu node
 for the case when the iommu is enabled rather than putting the DMA
 window information into the iommus property?
 
 This would probably mean that you need both #iommu-cells and #address-cells.

The reason for doing like this was that you may need a different window
for each device, while there can only be one dma-ranges property in 
an iommu node.

  +
  +Optional properties:
  +
  +- iommu-names: A list of names identifying each entry in the iommus
  +  property.
 
 Do we really need a name here? I would not expect that you have
 clearly documented names here from the datasheet like you would for
 interrupts or clocks, so you'd just be making up names. Sorry, but I'm
 not a fan of names properties in general.

Good point, this was really overdesign by modeling it after other
subsystems that can have a use for names.
 
  +Multiple-master IOMMU:
  +--
  +
  +   iommu {
  +   /* the specifier represents the ID of the master */
  +   #address-cells = 1;
  +   #size-cells = 0;
  +   };
  +
  +   master {
  +   /* device has master ID 42 in the IOMMU */
  +   iommus = /iommu 42;
  +   };
 
 Presumably the ID would be the streamID on ARM's SMMU. How would a
 master with 8 streamIDs be described? This is what Calxeda midway has
 for SATA and I would expect that to be somewhat common. Either you
 need some ID masking or you'll have lots of duplication when you have
 windows.

I don't understand the problem. If you have stream IDs 0 through 7,
you would have

master@a {
...
iommus = smmu 0;
};

master@b {
...
iommus = smmu 1;
};

...

master@12 {
...
iommus = smmu 7;
};

and you don't need a window at all. Why would you need a mask of
some sort?

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 30 May 2014 12:27:28 Dave Martin wrote:
 On Fri, May 30, 2014 at 08:30:08AM +0100, Thierry Reding wrote:
  On Thu, May 29, 2014 at 09:52:22AM -0600, Stephen Warren wrote:
   On 05/23/2014 02:36 PM, Thierry Reding wrote:
   I think this is a mistake. address-cells/size-cells are for transactions
   flowing down the bus (from the CPU to date). Describing a connection
   from a device to an IOMMU is something completely different, and should
   therefore simply use an iommu-cells property to describe any necessary
   information. If we start re-using properties for different things in
   different contexts, how is anyone going to know what they mean, and how
   will conflicts be resolved. For example, what if there's a single HW
   module that both acts as a regular register bus with children (where
   address-cells/size-cells defines how transactions reach the children
   from the parent), and is also an IOMMU (where according to this binding
   proposal, address-cells/size-cells represent some aspect of the IOMMU
   feature). Using different properties for different things is the only
   sane way to keep different concepts separate. Another alternative would
   be to represent the single HW module as separate nodes in DT, but I
   think that will only make our lives harder, and where I've done that in
   the past, I've regretted it.
  
  There was some back-and-forth on this topic and the latest concensus
  when I wrote the second version was that #address-cells and #size-cells
  were to be used.
  
  But there was some bore back-and-forth after that, and it seems like
  Arnd no longer thinks that using #address-cells and #size-cells is a
  good idea either[0].
 
 Mistake or not, ePAPR already (ab)uses the address concept for PCI.
 Unless ePAPR is wrong, I don't think it makes sense to argue that
 the address concept cannot be repurposed.
 
 The reason why this is abused for PCI is the same as our reason here:
 different masters really are treated as distinct even when accessing
 the same destination address, and DT has no general native way to
 describe that.
 
 One clear advantage of using #address-cells etc. is that ePAPR already
 has very clear and well-defined ways of how to specify range mappings.
 This gives us a ready-made way to describe windowed IOMMUs and 1:1
 remappings of whole blocks of master IDs, which are the most obvious
 cases other than having a small-integer set of explicit IDs.
 
 That said, the PCI pseudo-address thing is not something we _necessarily_
 want to repeat.

I'm really impartial to the question of whether to use #address-cells
or #iommus now. It's possible that we can use the addresses in some
meaningful way, but it's not clear to me that this will help make the
representation cleaner or that we will actually want it for SMMU.

  Arnd, can you take another look at this binding and see if there's
  anything else missing? If not I'll go through the document again and
  update all #address-cells/#size-cells references with #iommu-cells as
  appropriate and submit v3.
 
 How do you envisage propagation of the master ID bits downstream of the
 IOMMU would be described?
 
 We will definitely need a way to describe this for GICv3.  How those
 values are propagated is likely to vary between related SoCs and doesn't
 feel like it should be baked into a driver, especially for the ARM SMMU
 which may get reused in radically different SoC families from different
 vendors.
 
 The most likely types of remapping are the adding of a base offset or
 some extra bits to the ID -- because not all MSIs to the GIC will
 necessarily pass through the IOMMU.  It's also possible that we might
 see ID squashing or folding in some systems.
 
 
 For types of remapping which mix the ID and address together, I now
 do tend to agree that any flexibility arising from describing that
 in a general way that is unlikely to repay the cost of trying to
 interpret and analyse the DT.  Defining a few sterotypical kinds
 of mapping explicitly, as needed, looks more sensible for now.  The
 windowed-IOMMU case is one example.

For MSI, my feeling is that we'd just be best off doing an end-to-end
description. Any device using an MSI would have an interrupt specifier
that is sufficient for the interrupt controller to understand where
the IRQ comes from, and return an address/value pair back to the driver
to program into some register.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Hiroshi Doyu]

Arnd Bergmann a...@arndb.de writes:

  +Multiple-master IOMMU:
  +--
  +
  +   iommu {
  +   /* the specifier represents the ID of the master */
  +   #address-cells = 1;
  +   #size-cells = 0;
  +   };
  +
  +   master {
  +   /* device has master ID 42 in the IOMMU */
  +   iommus = /iommu 42;
  +   };
 
 Presumably the ID would be the streamID on ARM's SMMU. How would a
 master with 8 streamIDs be described? This is what Calxeda midway has
 for SATA and I would expect that to be somewhat common. Either you
 need some ID masking or you'll have lots of duplication when you have
 windows.

 I don't understand the problem. If you have stream IDs 0 through 7,
 you would have

   master@a {
   ...
   iommus = smmu 0;
   };

   master@b {
   ...
   iommus = smmu 1;
   };

   ...

   master@12 {
   ...
   iommus = smmu 7;
   };

 and you don't need a window at all. Why would you need a mask of
 some sort?

IIUC the original problem, a master with 8 streamIDs means something
like below, where some devices have multiple IDs but some have a
single. A sinle #address-cells cannot afford those 2 masters at once.

   iommu {
   /* the specifier represents the ID of the master */
   #address-cells = 1;
   #size-cells = 0;
   };

master@a {
...
iommus = smmu 1 2 3; # 3 IDs
};

master@b {
...
iommus = smmu 4; # 1 ID
};

Tegra,SMMU has a similar problem and we have used a fixed size bitmap(64
bit) to afford 64 stream IDs so that a single device can hold multiple
IDs. If we apply the same bitmap to the above exmaple:

   iommu {
   /* the specifier represents the ID of the master */
   #address-cells = 1;
   #size-cells = 0;
   };

master@a {
...
iommus = smmu (BIT(1) | BIT(2) | BIT(3)); # IDs 1 2 3
};

master@b {
...
iommus = smmu BIT(4); # ID 4
};

The disadvantage of this is that this limits the max number of streamIDs
to support. If # of streamID is increased later more than 64, this
format cannot cover any more. You have to predict the max # of streamIDs
in advance if steamID is statically assigned.
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Rob Herring]

On Fri, May 30, 2014 at 2:06 PM, Arnd Bergmann a...@arndb.de wrote:
 On Friday 30 May 2014 08:16:05 Rob Herring wrote:
 On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
 thierry.red...@gmail.com wrote:
  From: Thierry Reding tred...@nvidia.com
  +IOMMU master node:
  +==
  +
  +Devices that access memory through an IOMMU are called masters. A device 
  can
  +have multiple master interfaces (to one or more IOMMU devices).
  +
  +Required properties:
  +
  +- iommus: A list of phandle and IOMMU specifier pairs that describe the 
  IOMMU
  +  master interfaces of the device. One entry in the list describes one 
  master
  +  interface of the device.
  +
  +When an iommus property is specified in a device tree node, the IOMMU 
  will
  +be used for address translation. If a dma-ranges property exists in the
  +device's parent node it will be ignored. An exception to this rule is if 
  the
  +referenced IOMMU is disabled, in which case the dma-ranges property of 
  the
  +parent shall take effect.

 Just thinking out loud, could you have dma-ranges in the iommu node
 for the case when the iommu is enabled rather than putting the DMA
 window information into the iommus property?

 This would probably mean that you need both #iommu-cells and #address-cells.

 The reason for doing like this was that you may need a different window
 for each device, while there can only be one dma-ranges property in
 an iommu node.

My suggestion was that you also put the IDs in the dma-ranges as the
first cell much as ranges for PCI encodes other information in the
first cell. Then you can have an entry for each ID. The downside is
another special case like PCI.

The argument for using #address-cells and #size-cells seems to be to
align with how ranges work. If that's the route we want to go, then I
say we should not stop there and actually use dma-ranges as well.
Otherwise, I don't see the advantage over #iommu-cells.

  +
  +Optional properties:
  +
  +- iommu-names: A list of names identifying each entry in the iommus
  +  property.

 Do we really need a name here? I would not expect that you have
 clearly documented names here from the datasheet like you would for
 interrupts or clocks, so you'd just be making up names. Sorry, but I'm
 not a fan of names properties in general.

 Good point, this was really overdesign by modeling it after other
 subsystems that can have a use for names.

  +Multiple-master IOMMU:
  +--
  +
  +   iommu {
  +   /* the specifier represents the ID of the master */
  +   #address-cells = 1;
  +   #size-cells = 0;
  +   };
  +
  +   master {
  +   /* device has master ID 42 in the IOMMU */
  +   iommus = /iommu 42;
  +   };

 Presumably the ID would be the streamID on ARM's SMMU. How would a
 master with 8 streamIDs be described? This is what Calxeda midway has
 for SATA and I would expect that to be somewhat common. Either you
 need some ID masking or you'll have lots of duplication when you have
 windows.

 I don't understand the problem. If you have stream IDs 0 through 7,
 you would have

 master@a {
 ...
 iommus = smmu 0;
 };

 master@b {
 ...
 iommus = smmu 1;
 };

 ...

 master@12 {
 ...
 iommus = smmu 7;
 };

 and you don't need a window at all. Why would you need a mask of
 some sort?

1 master with 7 IDs like this:

 master@a {
 ...
 iommus = smmu 0 smmu 1 smmu 2 smmu 3
smmu 4 smmu 5 smmu 6 smmu 7;
 };

If there was any sort of window, then it is almost certainly the same
window for each ID.

Rob
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 30 May 2014 14:31:55 Rob Herring wrote:
 On Fri, May 30, 2014 at 2:06 PM, Arnd Bergmann a...@arndb.de wrote:
  On Friday 30 May 2014 08:16:05 Rob Herring wrote:
  On Fri, May 23, 2014 at 3:33 PM, Thierry Reding
  thierry.red...@gmail.com wrote:
   From: Thierry Reding tred...@nvidia.com
   +IOMMU master node:
   +==
   +
   +Devices that access memory through an IOMMU are called masters. A 
   device can
   +have multiple master interfaces (to one or more IOMMU devices).
   +
   +Required properties:
   +
   +- iommus: A list of phandle and IOMMU specifier pairs that describe the 
   IOMMU
   +  master interfaces of the device. One entry in the list describes one 
   master
   +  interface of the device.
   +
   +When an iommus property is specified in a device tree node, the IOMMU 
   will
   +be used for address translation. If a dma-ranges property exists in 
   the
   +device's parent node it will be ignored. An exception to this rule is 
   if the
   +referenced IOMMU is disabled, in which case the dma-ranges property 
   of the
   +parent shall take effect.
 
  Just thinking out loud, could you have dma-ranges in the iommu node
  for the case when the iommu is enabled rather than putting the DMA
  window information into the iommus property?
 
  This would probably mean that you need both #iommu-cells and 
  #address-cells.
 
  The reason for doing like this was that you may need a different window
  for each device, while there can only be one dma-ranges property in
  an iommu node.
 
 My suggestion was that you also put the IDs in the dma-ranges as the
 first cell much as ranges for PCI encodes other information in the
 first cell. Then you can have an entry for each ID. The downside is
 another special case like PCI.
 
 The argument for using #address-cells and #size-cells seems to be to
 align with how ranges work. If that's the route we want to go, then I
 say we should not stop there and actually use dma-ranges as well.
 Otherwise, I don't see the advantage over #iommu-cells.

I can see how dma-ranges in bus nodes work, it just doesn't seem to
have any reasonable meaning in the iommu node itself.

  I don't understand the problem. If you have stream IDs 0 through 7,
  you would have
 
  master@a {
  ...
  iommus = smmu 0;
  };
 
  master@b {
  ...
  iommus = smmu 1;
  };
 
  ...
 
  master@12 {
  ...
  iommus = smmu 7;
  };
 
  and you don't need a window at all. Why would you need a mask of
  some sort?
 
 1 master with 7 IDs like this:
 
  master@a {
  ...
  iommus = smmu 0 smmu 1 smmu 2 smmu 3
 smmu 4 smmu 5 smmu 6 smmu 7;
  };
 
 If there was any sort of window, then it is almost certainly the same
 window for each ID.

Ok, I see. In that case you'd probably want to have #address-cells = 1
and #size-cells = 1 and give a range of IDs like

iommus = smmu 0 8;

Do you think that ranges can have a meaningful definition with the ARM
SMMU stream IDs?

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Arnd Bergmann]

On Friday 30 May 2014 22:29:13 Hiroshi Doyu wrote:
 
 IIUC the original problem, a master with 8 streamIDs means something
 like below, where some devices have multiple IDs but some have a
 single. A sinle #address-cells cannot afford those 2 masters at once.
 
iommu {
/* the specifier represents the ID of the master */
#address-cells = 1;
#size-cells = 0;
};
 
 master@a {
 ...
 iommus = smmu 1 2 3; # 3 IDs
 };
 
 master@b {
 ...
 iommus = smmu 4; # 1 ID
 };

This would not be the usual format really. It should instead be

iommus = smmu 1, smmu 2, smmu 3;

which can be tedious to type.

 Tegra,SMMU has a similar problem and we have used a fixed size bitmap(64
 bit) to afford 64 stream IDs so that a single device can hold multiple
 IDs. If we apply the same bitmap to the above exmaple:
 
iommu {
/* the specifier represents the ID of the master */
#address-cells = 1;
#size-cells = 0;
};
 
 master@a {
 ...
 iommus = smmu (BIT(1) | BIT(2) | BIT(3)); # IDs 1 2 3
 };
 
 master@b {
 ...
 iommus = smmu BIT(4); # ID 4
 };
 
 The disadvantage of this is that this limits the max number of streamIDs
 to support. If # of streamID is increased later more than 64, this
 format cannot cover any more. You have to predict the max # of streamIDs
 in advance if steamID is statically assigned.
 

Well, the iommu specific binding could allow a variable #address-cells.
That way, you just need to know the number of stream IDs for that instance
of the iommu.

Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] devicetree: Add generic IOMMU device tree bindings

[Stephen Warren]

On 05/23/2014 02:36 PM, Thierry Reding wrote:
 From: Thierry Reding tred...@nvidia.com
 
 This commit introduces a generic device tree binding for IOMMU devices.
 Only a very minimal subset is described here, but it is enough to cover
 the requirements of both the Exynos System MMU and Tegra SMMU as
 discussed here:
 
 https://lkml.org/lkml/2014/4/27/346
 
 Signed-off-by: Thierry Reding tred...@nvidia.com
 ---
 Apologies for the noise, but apparently I mistyped one of the email
 addresses, should be fixed now.
 
 Changes in v2:
 - add notes about dma-ranges property (drop note from commit message)
 - document priorities of iommus property vs. dma-ranges property
 - drop #iommu-cells in favour of #address-cells and #size-cells

I think this is a mistake. address-cells/size-cells are for transactions
flowing down the bus (from the CPU to date). Describing a connection
from a device to an IOMMU is something completely different, and should
therefore simply use an iommu-cells property to describe any necessary
information. If we start re-using properties for different things in
different contexts, how is anyone going to know what they mean, and how
will conflicts be resolved. For example, what if there's a single HW
module that both acts as a regular register bus with children (where
address-cells/size-cells defines how transactions reach the children
from the parent), and is also an IOMMU (where according to this binding
proposal, address-cells/size-cells represent some aspect of the IOMMU
feature). Using different properties for different things is the only
sane way to keep different concepts separate. Another alternative would
be to represent the single HW module as separate nodes in DT, but I
think that will only make our lives harder, and where I've done that in
the past, I've regretted it.
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html