Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
[...]
let me clarify by example:
iommu@1 {
compatible = some,simple-iommu;
reg = 1;
#iommu-cells = 0; /* supports only one master */
};
iommu@2 {
compatible = some,other-iommu;
reg = 3;
#iommu-cells = 1; /* contains master ID */
};
iommu@3 {
compatible = some,windowed-iommu;
reg = 2;
#iommu-cells = 2; /* contains dma-window */
};
device@4 {
compatible = some,ethernet;
iommus = /iommu@1;
};
device@5 {
compatible = some,dmaengine;
iommus = /iommu@2 0x4000 0x100,
/iommu@3 0x101;
};
The device at address 4 has a one-one relationship with iommu@1, so there
is no need for any data. device@5 has two master ports. One is connected to
an IOMMU that has a per-device aperture, device@5 can only issue transfers
to the 256MB area at 0x4000, and the IOMMU will have to put entries for
this device into that address. The second master port is connected to
iommu@3, which uses a master ID that gets passed along with each transfer,
so that needs to be put into the IOTLBs.
iommu@3 and the second port of device@5 seem to match what we need for
Tegra (and as I understand also Exynos). Can we settle on this for now
so that Hiroshi and Cho can go update their drivers for this binding?
A variation would be to not use #iommu-cells at all, but provide a
#address-cells / #size-cells pair in the IOMMU, and have a translation
as we do for dma-ranges. This is probably most flexible.
The remainder of this discussion seems to indicate that #iommu-cells and
dma-ranges don't have to be mutually exclusive. For some IOMMUs it might
make sense to use both.
In fact perhaps we should require every IOMMU user to also specify a
dma-ranges property, even if for some cases the range would be simply
the complete physical address space. Perhaps in analogy to the ranges
property an empty dma-ranges property could be taken to mean all of the
physical address space.
I'm aware that this doesn't cover any of the more exotic cases out
there, but the fact is that we have real devices out there that ship
with some variations of these simple IOMMUs and I don't think we're
doing ourselves a favour by blocking support for these to be added on
the hope of merging the perfect solution that covers all use-cases.
Patches for Tegra have already been around for close to half a year.
Thierry
pgpuNbRYhHRuo.pgp
Description: PGP signature
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Thu, 15 May 2014 22:37:31 +0200, Thierry Reding wrote:
On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
[...]
let me clarify by example:
iommu@1 {
compatible = some,simple-iommu;
reg = 1;
#iommu-cells = 0; /* supports only one master */
};
iommu@2 {
compatible = some,other-iommu;
reg = 3;
#iommu-cells = 1; /* contains master ID */
};
iommu@3 {
compatible = some,windowed-iommu;
reg = 2;
#iommu-cells = 2; /* contains dma-window */
};
device@4 {
compatible = some,ethernet;
iommus = /iommu@1;
};
device@5 {
compatible = some,dmaengine;
iommus = /iommu@2 0x4000 0x100,
/iommu@3 0x101;
};
The device at address 4 has a one-one relationship with iommu@1, so there
is no need for any data. device@5 has two master ports. One is connected to
an IOMMU that has a per-device aperture, device@5 can only issue transfers
to the 256MB area at 0x4000, and the IOMMU will have to put entries for
this device into that address. The second master port is connected to
iommu@3, which uses a master ID that gets passed along with each transfer,
so that needs to be put into the IOTLBs.
iommu@3 and the second port of device@5 seem to match what we need for
Tegra (and as I understand also Exynos). Can we settle on this for now
so that Hiroshi and Cho can go update their drivers for this binding?
Currently, Exynos IOMMU is the case of iommu@1.
But in the near future, it will support multiple masters with a single context
that means all masters that shares a single System MMU also views the same
address space.
For some cases, we may need iommu@3 that supports dma-window.
So, I have no other opinion.
By the way, iommu framework should allow to process the parameters
to 'iommus' property in the master nodes by iommu driver implementations
because it is depended on implementations.
A variation would be to not use #iommu-cells at all, but provide a
#address-cells / #size-cells pair in the IOMMU, and have a translation
as we do for dma-ranges. This is probably most flexible.
The remainder of this discussion seems to indicate that #iommu-cells and
dma-ranges don't have to be mutually exclusive. For some IOMMUs it might
make sense to use both.
In fact perhaps we should require every IOMMU user to also specify a
dma-ranges property, even if for some cases the range would be simply
the complete physical address space. Perhaps in analogy to the ranges
property an empty dma-ranges property could be taken to mean all of the
physical address space.
I'm aware that this doesn't cover any of the more exotic cases out
there, but the fact is that we have real devices out there that ship
with some variations of these simple IOMMUs and I don't think we're
doing ourselves a favour by blocking support for these to be added on
the hope of merging the perfect solution that covers all use-cases.
Patches for Tegra have already been around for close to half a year.
Thierry
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
+- mmu-masters: A phandle to device nodes representing the master for which
+ the System MMU can provide a translation. Any additional
values
+ after the phandle will be ignored because a System MMU never
+ have two or more masters. #stream-id-cells specified in the
+ master's node will be also ignored.
+ If more than one phandle is specified, only the first phandle
+ will be treated.
This seems completely backwards: Why would you list the masters for an IOMMU
in the IOMMU node?
The master should have a standard property pointing to the IOMMU instead.
We don't have a generic binding for IOMMUs yet it seems, but the time is
overdue to make one.
Consider this NAKed until there is a generic binding for IOMMUs that all
relevant developers have agreed to.
I'd like to take this opportunity and revive one of the hibernating
patch sets that we have for Tegra. The last effort to get things merged
was back in January I think. I haven't bothered to look up the reference
since it's probably good to start from scratch anyway.
The latest version of the binding that was under discussion back then I
think looked something like this:
device@... {
iommus = iommu [spec][, other_iommu [other_spec]...];
};
And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.
On Tegra the specifier would be used to encode a memory controller's
client ID. One discussion point back at the time was to encode the ID as
a bitmask to allow more than a single master per entry. Another solution
which I think is a little cleaner and more generic, would be to use one
entry per master and use a single cell to encode the client ID. Devices
with multiple clients to the same IOMMU could then use multiple entries
referencing the same IOMMU.
I've added Hiroshi Doyu on Cc since he knows the Tegra IOMMU best.
Hiroshi, can you summarize exactly what the proposed bindings were. If
my memory serves me well they were mostly along the lines of what Arnd
proposes here, and perhaps they are something that can also be used for
Exynos.
Will Deacon (I think) had some comments on the earlier discussion as
well, so I've added him on Cc for visibility. Sorry if I'm confusing you
with someone else, Will. In that case perhaps you know who to include in
the discussion from the ARM side.
Also adding Stephen Warren for visibility.
Thierry
pgpNkQwva4gNA.pgp
Description: PGP signature
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
+- mmu-masters: A phandle to device nodes representing the master for
which
+ the System MMU can provide a translation. Any additional
values
+ after the phandle will be ignored because a System MMU
never
+ have two or more masters. #stream-id-cells specified in
the
+ master's node will be also ignored.
+ If more than one phandle is specified, only the first
phandle
+ will be treated.
This seems completely backwards: Why would you list the masters for an IOMMU
in the IOMMU node?
The master should have a standard property pointing to the IOMMU instead.
We don't have a generic binding for IOMMUs yet it seems, but the time is
overdue to make one.
Consider this NAKed until there is a generic binding for IOMMUs that all
relevant developers have agreed to.
I'd like to take this opportunity and revive one of the hibernating
patch sets that we have for Tegra. The last effort to get things merged
was back in January I think. I haven't bothered to look up the reference
since it's probably good to start from scratch anyway.
The latest version of the binding that was under discussion back then I
think looked something like this:
device@... {
iommus = iommu [spec][, other_iommu [other_spec]...];
};
And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.
Yes, that seems reasonable. Just one question: How would you represent a
device that has multiple masters, with at least one connected to an IOMMU
and another one connected to memory directly, without going to the IOMMU?
On Tegra the specifier would be used to encode a memory controller's
client ID. One discussion point back at the time was to encode the ID as
a bitmask to allow more than a single master per entry. Another solution
which I think is a little cleaner and more generic, would be to use one
entry per master and use a single cell to encode the client ID. Devices
with multiple clients to the same IOMMU could then use multiple entries
referencing the same IOMMU.
I'm not completely following here. Are you talking about the generic
binding, or the part that is tegra specific for the specifier?
My first impression is that the generic binding should just allow an
arbitrary specifier with a variable #iommu-cells, and leave the format
up to the IOMMU driver. A lot of drivers probably only support one
master, so they can just set #iommu-cells=0, others might require
IDs that do not fit into one cell.
Arnd
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
On Sun, Apr 27, 2014 at 08:23:06PM +0200, Arnd Bergmann wrote:
On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote:
+- mmu-masters: A phandle to device nodes representing the master for
which
+ the System MMU can provide a translation. Any
additional values
+ after the phandle will be ignored because a System MMU
never
+ have two or more masters. #stream-id-cells specified
in the
+ master's node will be also ignored.
+ If more than one phandle is specified, only the first
phandle
+ will be treated.
This seems completely backwards: Why would you list the masters for an
IOMMU
in the IOMMU node?
The master should have a standard property pointing to the IOMMU instead.
We don't have a generic binding for IOMMUs yet it seems, but the time is
overdue to make one.
Consider this NAKed until there is a generic binding for IOMMUs that all
relevant developers have agreed to.
I'd like to take this opportunity and revive one of the hibernating
patch sets that we have for Tegra. The last effort to get things merged
was back in January I think. I haven't bothered to look up the reference
since it's probably good to start from scratch anyway.
The latest version of the binding that was under discussion back then I
think looked something like this:
device@... {
iommus = iommu [spec][, other_iommu [other_spec]...];
};
And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.
Yes, that seems reasonable. Just one question: How would you represent a
device that has multiple masters, with at least one connected to an IOMMU
and another one connected to memory directly, without going to the IOMMU?
Heh, I don't think I've ever thought about that use-case. I guess I was
always assuming that in the absence of an IOMMU the device would simply
access memory directly. From what I can tell that's how Tegra works at
least. If the IOMMU is not enabled for a given client, that client will
access physical memory untranslated.
I suppose if that really must be represented then a global dummy IOMMU
could be introduced to help with these cases.
On Tegra the specifier would be used to encode a memory controller's
client ID. One discussion point back at the time was to encode the ID as
a bitmask to allow more than a single master per entry. Another solution
which I think is a little cleaner and more generic, would be to use one
entry per master and use a single cell to encode the client ID. Devices
with multiple clients to the same IOMMU could then use multiple entries
referencing the same IOMMU.
I'm not completely following here. Are you talking about the generic
binding, or the part that is tegra specific for the specifier?
My first impression is that the generic binding should just allow an
arbitrary specifier with a variable #iommu-cells, and leave the format
up to the IOMMU driver.
Yes, I was getting ahead of myself. The idea was to have #iommu-cells
and allow the specifier to be IOMMU-specific. On Tegra that would
translate to the memory controller client ID, on other devices I suspect
something similar might exist, but for the generic binding it should be
completely opaque and hence irrelevant.
Really just like any of the other bindings that have foos and #foo-cells
properties.
A lot of drivers probably only support one
master, so they can just set #iommu-cells=0, others might require
IDs that do not fit into one cell.
You mean #iommu-cells = 1 for devices that only require one master?
There still has to be one cell to specify which master. Unless perhaps
if they can be arbitrarily assigned. I guess even if there's a fixed
mapping that applies to one SoC generation, it might be good to still
employ a specifier and have the mapping in DT for flexibility.
Thierry
pgpADPyGxVp3v.pgp
Description: PGP signature
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.
Yes, that seems reasonable. Just one question: How would you represent a
device that has multiple masters, with at least one connected to an IOMMU
and another one connected to memory directly, without going to the IOMMU?
Heh, I don't think I've ever thought about that use-case. I guess I was
always assuming that in the absence of an IOMMU the device would simply
access memory directly. From what I can tell that's how Tegra works at
least. If the IOMMU is not enabled for a given client, that client will
access physical memory untranslated.
I suppose if that really must be represented then a global dummy IOMMU
could be introduced to help with these cases.
It's actually not too uncommon: you can have e.g. the lower 2GB mapped
directly from the device address space into the host memory, but have
an iommu that translates accesses from some range in the upper 2GB of
the 32-bit address space into full 64-bit addresses.
This use case makes no sense if you use the IOMMU for isolation
or virtualization, but it gives better performance for lowmem access
when the only reason to have the IOMMU is to map highmem addresses.
On Tegra the specifier would be used to encode a memory controller's
client ID. One discussion point back at the time was to encode the ID as
a bitmask to allow more than a single master per entry. Another solution
which I think is a little cleaner and more generic, would be to use one
entry per master and use a single cell to encode the client ID. Devices
with multiple clients to the same IOMMU could then use multiple entries
referencing the same IOMMU.
I'm not completely following here. Are you talking about the generic
binding, or the part that is tegra specific for the specifier?
My first impression is that the generic binding should just allow an
arbitrary specifier with a variable #iommu-cells, and leave the format
up to the IOMMU driver.
Yes, I was getting ahead of myself. The idea was to have #iommu-cells
and allow the specifier to be IOMMU-specific. On Tegra that would
translate to the memory controller client ID, on other devices I suspect
something similar might exist, but for the generic binding it should be
completely opaque and hence irrelevant.
Really just like any of the other bindings that have foos and #foo-cells
properties.
Ok.
A lot of drivers probably only support one
master, so they can just set #iommu-cells=0, others might require
IDs that do not fit into one cell.
You mean #iommu-cells = 1 for devices that only require one master?
I meant an IOMMU device that acts as the slave for exactly one device,
even if that device has multiple master ports.
There still has to be one cell to specify which master. Unless perhaps
if they can be arbitrarily assigned. I guess even if there's a fixed
mapping that applies to one SoC generation, it might be good to still
employ a specifier and have the mapping in DT for flexibility.
let me clarify by example:
iommu@1 {
compatible = some,simple-iommu;
reg = 1;
#iommu-cells = 0; /* supports only one master */
};
iommu@2 {
compatible = some,other-iommu;
reg = 3;
#iommu-cells = 1; /* contains master ID */
};
iommu@3 {
compatible = some,windowed-iommu;
reg = 2;
#iommu-cells = 2; /* contains dma-window */
};
device@4 {
compatible = some,ethernet;
iommus = /iommu@1;
};
device@5 {
compatible = some,dmaengine;
iommus = /iommu@2 0x4000 0x100,
/iommu@3 0x101;
};
The device at address 4 has a one-one relationship with iommu@1, so there
is no need for any data. device@5 has two master ports. One is connected to
an IOMMU that has a per-device aperture, device@5 can only issue transfers
to the 256MB area at 0x4000, and the IOMMU will have to put entries for
this device into that address. The second master port is connected to
iommu@3, which uses a master ID that gets passed along with each transfer,
so that needs to be put into the IOTLBs.
A variation would be to not use #iommu-cells at all, but provide a
#address-cells / #size-cells pair in the IOMMU, and have a translation
as we do for dma-ranges. This is probably most flexible.
One completely open question that I just noticed is how the kernel should
deal with the case of
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Mon, Apr 28, 2014 at 02:05:30PM +0200, Arnd Bergmann wrote:
On Monday 28 April 2014 13:18:03 Thierry Reding wrote:
On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote:
On Monday 28 April 2014 12:39:20 Thierry Reding wrote:
And possibly with a iommu-names property to go along with that. The idea
being that a device can be a master on possibly multiple IOMMUs. Using
the above it would also be possible to have one device be multiple
masters on the same IOMMU.
Yes, that seems reasonable. Just one question: How would you represent a
device that has multiple masters, with at least one connected to an IOMMU
and another one connected to memory directly, without going to the IOMMU?
Heh, I don't think I've ever thought about that use-case. I guess I was
always assuming that in the absence of an IOMMU the device would simply
access memory directly. From what I can tell that's how Tegra works at
least. If the IOMMU is not enabled for a given client, that client will
access physical memory untranslated.
I suppose if that really must be represented then a global dummy IOMMU
could be introduced to help with these cases.
It's actually not too uncommon: you can have e.g. the lower 2GB mapped
directly from the device address space into the host memory, but have
an iommu that translates accesses from some range in the upper 2GB of
the 32-bit address space into full 64-bit addresses.
This use case makes no sense if you use the IOMMU for isolation
or virtualization, but it gives better performance for lowmem access
when the only reason to have the IOMMU is to map highmem addresses.
Thinking about this some more, isn't the non-IOMMU master something we
can completely ignore in the DT? Or at least it shouldn't be handled by
the IOMMU bindings because, well, it's not an IOMMU to begin with.
Perhaps it's something that should be described using dma-ranges?
A lot of drivers probably only support one
master, so they can just set #iommu-cells=0, others might require
IDs that do not fit into one cell.
You mean #iommu-cells = 1 for devices that only require one master?
I meant an IOMMU device that acts as the slave for exactly one device,
even if that device has multiple master ports.
Okay, makes sense. I guess depending on the nature of the IOMMU it might
make sense not to expose it as an IOMMU at all. For example if it lives
completely within the register space of its master device. In that case
it could be directly programmed from the device's driver.
There still has to be one cell to specify which master. Unless perhaps
if they can be arbitrarily assigned. I guess even if there's a fixed
mapping that applies to one SoC generation, it might be good to still
employ a specifier and have the mapping in DT for flexibility.
let me clarify by example:
iommu@1 {
compatible = some,simple-iommu;
reg = 1;
#iommu-cells = 0; /* supports only one master */
};
iommu@2 {
compatible = some,other-iommu;
reg = 3;
#iommu-cells = 1; /* contains master ID */
};
iommu@3 {
compatible = some,windowed-iommu;
reg = 2;
#iommu-cells = 2; /* contains dma-window */
};
device@4 {
compatible = some,ethernet;
iommus = /iommu@1;
};
device@5 {
compatible = some,dmaengine;
iommus = /iommu@2 0x4000 0x100,
/iommu@3 0x101;
};
The device at address 4 has a one-one relationship with iommu@1, so there
is no need for any data. device@5 has two master ports. One is connected to
an IOMMU that has a per-device aperture, device@5 can only issue transfers
to the 256MB area at 0x4000, and the IOMMU will have to put entries for
this device into that address. The second master port is connected to
iommu@3, which uses a master ID that gets passed along with each transfer,
so that needs to be put into the IOTLBs.
The above sounds reasonable to me with the exception of the DMA window
specifier. Isn't that precisely the information that we currently
describe using the dma-ranges property?
A variation would be to not use #iommu-cells at all, but provide a
#address-cells / #size-cells pair in the IOMMU, and have a translation
as we do for dma-ranges. This is probably most flexible.
I'm not sure I follow. Wouldn't that require masters to be children of
the IOMMU DT nodes for that to work out? Also how would that work for
cases where more data than the address ranges (such as the master ID) is
needed to operate the IOMMU?
One completely open question that I just noticed is how the kernel should
deal with the case of multiple IOMMUs attached to one master: the
data structures we have assume that we know exactly how to do DMA by
setting the per-device
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On 04/28/2014 05:18 AM, Thierry Reding wrote: On Mon, Apr 28, 2014 at 12:56:03PM +0200, Arnd Bergmann wrote: ... A lot of drivers probably only support one master, so they can just set #iommu-cells=0, others might require IDs that do not fit into one cell. You mean #iommu-cells = 1 for devices that only require one master? There still has to be one cell to specify which master. Unless perhaps if they can be arbitrarily assigned. I guess even if there's a fixed mapping that applies to one SoC generation, it might be good to still employ a specifier and have the mapping in DT for flexibility. #iommu-cells doesn't include the phandle, so if you want the client references to be: property = iommu; then that's #iommu-cells=0, whereas: property = iommu N; is #iommu-cells=1. -- To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v12 11/31] documentation: iommu: add binding document of Exynos System MMU
On Sunday 27 April 2014 13:07:43 Shaik Ameer Basha wrote: +- mmu-masters: A phandle to device nodes representing the master for which + the System MMU can provide a translation. Any additional values + after the phandle will be ignored because a System MMU never + have two or more masters. #stream-id-cells specified in the + master's node will be also ignored. + If more than one phandle is specified, only the first phandle + will be treated. This seems completely backwards: Why would you list the masters for an IOMMU in the IOMMU node? The master should have a standard property pointing to the IOMMU instead. We don't have a generic binding for IOMMUs yet it seems, but the time is overdue to make one. Consider this NAKed until there is a generic binding for IOMMUs that all relevant developers have agreed to. Arnd -- To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
