Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
04.10.2021 22:23, Thierry Reding пишет: > On Sun, Oct 03, 2021 at 04:09:56AM +0300, Dmitry Osipenko wrote: >> 23.04.2021 19:32, Thierry Reding пишет: >>> I've made corresponding changes in the proprietary bootloader, added a >>> compatibility shim in U-Boot (which forwards information created by the >>> proprietary bootloader to the kernel) and the attached patches to test >>> this on Jetson TX1, Jetson TX2 and Jetson AGX Xavier. >> >> Could you please tell what downstream kernel does for coping with the >> identity mappings in conjunction with the original proprietary bootloader? >> >> If there is some other method of passing mappings to kernel, could it be >> supported by upstream? Putting burden on users to upgrade bootloader >> feels a bit inconvenient. > > It depends on the chip generation. As far as I know there have been > several iterations. The earliest was to pass this information via a > command-line option, but more recent versions use device tree to pass > this information in a similar way as described here. However, these > use non-standard DT bindings, so I don't think we can just implement > them as-is. Is it possible to boot upstream kernel with that original bootloader? I remember seeing other platforms, like QCOM, supporting downstream quirks in upstream kernel on a side, i.e. they are undocumented, but the additional support code is there. That is what "normal" people want. You should consider doing that for Tegra too, if possible. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
On Sun, Oct 03, 2021 at 04:09:56AM +0300, Dmitry Osipenko wrote: > 23.04.2021 19:32, Thierry Reding пишет: > > I've made corresponding changes in the proprietary bootloader, added a > > compatibility shim in U-Boot (which forwards information created by the > > proprietary bootloader to the kernel) and the attached patches to test > > this on Jetson TX1, Jetson TX2 and Jetson AGX Xavier. > > Could you please tell what downstream kernel does for coping with the > identity mappings in conjunction with the original proprietary bootloader? > > If there is some other method of passing mappings to kernel, could it be > supported by upstream? Putting burden on users to upgrade bootloader > feels a bit inconvenient. It depends on the chip generation. As far as I know there have been several iterations. The earliest was to pass this information via a command-line option, but more recent versions use device tree to pass this information in a similar way as described here. However, these use non-standard DT bindings, so I don't think we can just implement them as-is. Thierry signature.asc Description: PGP signature ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
23.04.2021 19:32, Thierry Reding пишет: > I've made corresponding changes in the proprietary bootloader, added a > compatibility shim in U-Boot (which forwards information created by the > proprietary bootloader to the kernel) and the attached patches to test > this on Jetson TX1, Jetson TX2 and Jetson AGX Xavier. Could you please tell what downstream kernel does for coping with the identity mappings in conjunction with the original proprietary bootloader? If there is some other method of passing mappings to kernel, could it be supported by upstream? Putting burden on users to upgrade bootloader feels a bit inconvenient. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
29.04.2021 08:51, Krishna Reddy пишет: > Hi Dmitry, > >> Thank you for the answer. Could you please give more information about: >> 1) Are you on software or hardware team, or both? > > I am in the software team and has contributed to initial Tegra SMMU driver in > the downstream along with earlier team member Hiroshi Doyu. > >> 2) Is SMMU a third party IP or developed in-house? > > Tegra SMMU is developed in-house. > >> 3) Do you have a direct access to HDL sources? Are you 100% sure that >> hardware does what you say? > > It was discussed with Hardware team before and again today as well. > Enabling ASID for display engine while it continues to access the buffer > memory is a safe operation. > As per HW team, The only side-effect that can happen is additional latency to > transaction as SMMU caches get warmed up. > >> 4) What happens when CPU writes to ASID register? Does SMMU state machine >> latch ASID status (making it visible) only at a single "safe" point? > > MC makes a decision on routing transaction through either SMMU page tables or > bypassing based on the ASID register value. It > checks the ASID register only once per transaction in the pipeline. Thank you very much for the clarification. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
Hi Dmitry, > Thank you for the answer. Could you please give more information about: > 1) Are you on software or hardware team, or both? I am in the software team and has contributed to initial Tegra SMMU driver in the downstream along with earlier team member Hiroshi Doyu. > 2) Is SMMU a third party IP or developed in-house? Tegra SMMU is developed in-house. > 3) Do you have a direct access to HDL sources? Are you 100% sure that > hardware does what you say? It was discussed with Hardware team before and again today as well. Enabling ASID for display engine while it continues to access the buffer memory is a safe operation. As per HW team, The only side-effect that can happen is additional latency to transaction as SMMU caches get warmed up. > 4) What happens when CPU writes to ASID register? Does SMMU state machine > latch ASID status (making it visible) only at a single "safe" point? MC makes a decision on routing transaction through either SMMU page tables or bypassing based on the ASID register value. It checks the ASID register only once per transaction in the pipeline. -KR ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
28.04.2021 08:57, Mikko Perttunen пишет: > On 4/28/21 8:51 AM, Dmitry Osipenko wrote: >> 23.04.2021 19:32, Thierry Reding пишет: >>> Note that there will be no new releases of the bootloader for earlier >>> devices, so adding support for these new DT bindings will not be >>> practical. The bootloaders on those devices do pass information about >>> the active framebuffer via the kernel command-line, so we may want to >>> add code to create reserved regions in the IOMMU based on that. >> >> Since this change requires a bootloader update anyways, why it's not >> possible to fix the bootloader properly, making it to stop all the DMA >> activity before jumping into kernel? >> > > That is not desirable, as then we couldn't have seamless > bootloader-kernel boot splash transition. The seamless transition should be more complicated since it should require to read out the hardware state in order to convert it into DRM state + display panel needs to stay ON. It's a bit questionable whether this is really needed, so far this is not achievable in mainline. Nevertheless, it will be good to have an early simple-framebuffer, which I realized only after sending out the message. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
On 4/28/21 8:51 AM, Dmitry Osipenko wrote: 23.04.2021 19:32, Thierry Reding пишет: Note that there will be no new releases of the bootloader for earlier devices, so adding support for these new DT bindings will not be practical. The bootloaders on those devices do pass information about the active framebuffer via the kernel command-line, so we may want to add code to create reserved regions in the IOMMU based on that. Since this change requires a bootloader update anyways, why it's not possible to fix the bootloader properly, making it to stop all the DMA activity before jumping into kernel? That is not desirable, as then we couldn't have seamless bootloader-kernel boot splash transition. Mikko ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
23.04.2021 19:32, Thierry Reding пишет: > Note that an earlier proposal was to use the existing simple-framebuffer > device tree bindings to transport this information. Unfortunately there > are cases where this is not enough. On Tegra SoCs, for example, the > bootloader will also set up a color space correction lookup table in the > system memory that the display controller will access during boot, > alongside the framebuffer. The simple-framebuffer DT bindings have no > way of describing this (and I guess one could argue that this particular > setup no longer is a "simple" framebuffer), so the above, more flexible > proposal was implemented. Will simple-framebuffer be able to use that reserved region transparently? Or will it require a custom simple-framebuffer driver? Could we make simple-framebuffer support a part of this series? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
23.04.2021 19:32, Thierry Reding пишет: > Note that there will be no new releases of the bootloader for earlier > devices, so adding support for these new DT bindings will not be > practical. The bootloaders on those devices do pass information about > the active framebuffer via the kernel command-line, so we may want to > add code to create reserved regions in the IOMMU based on that. Since this change requires a bootloader update anyways, why it's not possible to fix the bootloader properly, making it to stop all the DMA activity before jumping into kernel? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
27.04.2021 21:30, Krishna Reddy пишет: >> Is it always safe to enable SMMU ASID in a middle of a DMA request made by a >> memory client? > > From MC point of view, It is safe to enable and has been this way from many > years in downstream code for display engine. > It doesn't impact the transactions that have already bypassed SMMU before > enabling SMMU ASID. > Transactions that are yet to pass SMMU stage would go through SMMU once SMMU > ASID is enabled and visible. Hello, Thank you for the answer. Could you please give more information about: 1) Are you on software or hardware team, or both? 2) Is SMMU a third party IP or developed in-house? 3) Do you have a direct access to HDL sources? Are you 100% sure that hardware does what you say? 4) What happens when CPU writes to ASID register? Does SMMU state machine latch ASID status (making it visible) only at a single "safe" point? ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
> Is it always safe to enable SMMU ASID in a middle of a DMA request made by a > memory client? >From MC point of view, It is safe to enable and has been this way from many >years in downstream code for display engine. It doesn't impact the transactions that have already bypassed SMMU before enabling SMMU ASID. Transactions that are yet to pass SMMU stage would go through SMMU once SMMU ASID is enabled and visible. -KR ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
23.04.2021 19:32, Thierry Reding пишет: > Hi, > > this is an updated proposal to solve the problem of passing memory > regions that are actively being accessed during boot. The particular > use-case that I need this for is when the bootloader has set up the > display controller to scan out a boot splash screen. During boot the > DMA/IOMMU glue code will attach devices to an IOMMU domain and by > doing so enable IOMMU translations. Typically this will be before a > device driver has had a chance to either disable the display > controller or set up a new framebuffer and map it to the IOMMU. Hello Thierry, Is it always safe to enable SMMU ASID in a middle of a DMA request made by a memory client? The memory controller supports blocking DMA requests, which we are already using for the memory hot-resetting. A block could be needed before ASID is toggled. This needs to be clarified. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 0/5] iommu: Support identity mappings of reserved-memory regions
From: Thierry Reding
Hi,
this is an updated proposal to solve the problem of passing memory
regions that are actively being accessed during boot. The particular
use-case that I need this for is when the bootloader has set up the
display controller to scan out a boot splash screen. During boot the
DMA/IOMMU glue code will attach devices to an IOMMU domain and by
doing so enable IOMMU translations. Typically this will be before a
device driver has had a chance to either disable the display
controller or set up a new framebuffer and map it to the IOMMU.
In that case, the IOMMU will start to fault because the accesses of
the display controller will be for memory addresses that are not mapped
in the IOMMU. The solution is obviously to create identity mappings for
such memory regions. From a device tree point of view, these memory
regions can be described using the reserved-memory device tree bindings
and hooked up to the consumer devices using the "memory-region"
property. On the kernel side, the IOMMU framework already supports the
concept of reserved regions, as well as a way of marking these regions
as requiring identity (a.k.a. direct) mappings.
Unfortunately, the current reserved-memory region bindings only allow
properties of the regions themselves to be described (such as whether a
kernel virtual mapping of the region is needed or not), but it doesn't
provide a way of associating extra information with any particular
reference to these regions. However, that's exactly what's needed for
this case because a given region may need to be identity mapped for a
specific device (such as the display controller scanning out from the
region) but referenced by multiple devices (e.g. if the memory is some
special carveout memory reserved for display purposes).
This series of patches proposes a simple solution: extend memory-region
properties to use an optional specifier, such as the ones already
commonly used for things like GPIOs or interrupts. The specifier needs
to be provided if the reserved-memory region has a non-zero
#memory-region-cells property (if the property is not present, zero is
the assumed default value). The specifier contains flags that specify
how the reference is to be treated. This series of patches introduces
the MEMORY_REGION_IDENTITY_MAPPING flag (value: 0x1) that marks the
specific reference to the memory region to require an identity mapping.
In practice, a device tree would look like this:
reserved-memory {
#address-cells = <2>;
#size-cells = <2>;
fb: framebuffer@92cb2000 {
reg = <0 0x92cb2000 0 0x0080>;
#memory-region-cells = <1>;
};
};
...
display@5240 {
...
memory-region = <&fb MEMORY_REGION_IDENTITY_MAPPING>;
...
};
Note: While the above would be valid DTS content, it's more likely that
in practice this content would be dynamically generated by the
bootloader using runtime information (such as the framebuffer memory
location).
An operating system can derive from that pair that
the 8 MiB of memory at physical address 0x92cb2000 need to be identity
mapped to the same IO virtual address if the device is attached to an
IOMMU. If no IOMMU is enabled in the system, obviously no identity
mapping needs to be created, but the operating system may still use the
reference to transition to its own framebuffer using the existing memory
region.
Note that an earlier proposal was to use the existing simple-framebuffer
device tree bindings to transport this information. Unfortunately there
are cases where this is not enough. On Tegra SoCs, for example, the
bootloader will also set up a color space correction lookup table in the
system memory that the display controller will access during boot,
alongside the framebuffer. The simple-framebuffer DT bindings have no
way of describing this (and I guess one could argue that this particular
setup no longer is a "simple" framebuffer), so the above, more flexible
proposal was implemented.
I've made corresponding changes in the proprietary bootloader, added a
compatibility shim in U-Boot (which forwards information created by the
proprietary bootloader to the kernel) and the attached patches to test
this on Jetson TX1, Jetson TX2 and Jetson AGX Xavier.
Note that there will be no new releases of the bootloader for earlier
devices, so adding support for these new DT bindings will not be
practical. The bootloaders on those devices do pass information about
the active framebuffer via the kernel command-line, so we may want to
add code to create reserved regions in the IOMMU based on that.
Thierry
Navneet Kumar (1):
iommu/tegra-smmu: Support managed domains
Thierry Reding (4):
dt-bindings: reserved-memory: Document memory region specifier
iommu: Implement of_iommu_get_resv_regions()
iommu: dma: Use of_iommu_get_resv_regions()
iommu/te
