Re: [PATCH v2 3/3] mmc: sdhci-msm: Request non-strict IOMMU mode
Hi, On Thu, Jun 24, 2021 at 10:18 AM Douglas Anderson wrote: > > The concept of IOMMUs being in strict vs. non-strict mode is a > pre-existing Linux concept. I've included a rough summary here to help > evaluate this patch. > > IOMMUs can be run in "strict" mode or in "non-strict" mode. The > quick-summary difference between the two is that in "strict" mode we > wait until everything is flushed out when we unmap DMA memory. In > "non-strict" we don't. > > Using the IOMMU in "strict" mode is more secure/safer but slower > because we have to sit and wait for flushes while we're unmapping. To > explain a bit why "non-strict" mode is unsafe, let's imagine two > examples. > > An example of "non-strict" being insecure when reading from a device: > a) Linux driver maps memory for DMA. > b) Linux driver starts DMA on the device. > c) Device write to RAM subject to bounds checking done by IOMMU. > d) Device finishes writing to RAM and signals transfer is finished. > e) Linux driver starts unmapping DMA memory but doesn't wait for the >unmap to finish (the definition of non-strict). At this point, >though, the Linux APIs say that the driver owns the memory and >shouldn't expect any more scribbling from the DMA device. > f) Linux driver validates that the data in memory looks sane and that >accessing it won't cause the driver to, for instance, overflow a >buffer. > g) Device takes advantage of knowledge of how the Linux driver works >and sneaks in a modification to the data after the validation but >before the IOMMU unmap flush finishes. > h) Device has now caused the Linux driver to access memory it >shouldn't. > > An example of "non-strict" being insecure when writing to a device: > a) Linux driver writes data intended for the device to RAM. > b) Linux driver maps memory for DMA. > c) Linux driver starts DMA on the device. > d) Device reads from RAM subject to bounds checking done by IOMMU. > e) Device finishes reading from RAM and signals transfer is finished. > f) Linux driver starts unmapping DMA memory but doesn't wait for the >unmap to finish (the definition of non-strict) > g) Linux driver frees memory and returns it to the pool of memory >available for other users to allocate. > h) Memory is allocated for another purpose since it was free memory. > i) Device takes advantage of the period of time before IOMMU flush to >read memory that it shouldn't have had access to. What exactly the >memory could contain depends on the randomness of who allocated >next, though exploits have been built on flimisier holes. > > As you can see from the above examples, using the iommu in > "non-strict" mode might not sound _too_ scary (the window of badness > is small and the exposed memory is small) but there is certainly > risk. Let's evaluate the risk by breaking it down into two problems > that IOMMUs are supposed to be protecting us against: > > Case 1: IOMMUs prevent malicious code running on the peripheral (maybe > a malicious peripheral or maybe someone exploited a benign peripheral) > from turning into an exploit of the Linux kernel. This is particularly > important if the peripheral has loadable / updatable firmware or if > the peripheral has some type of general purpose processor and is > processing untrusted inputs. It's also important if the device is > something that can be easily plugged into the host and the device has > direct DMA access itself, like a PCIe device. > > Case 2: IOMMUs limit the severity of a class of software bugs in the > kernel. If we misconfigure a peripheral by accident then instead of > the peripheral clobbering random memory due to a bug we might get an > IOMMU error. > > Now that we understand the issue and the risks, let's evaluate whether > we really need "strict" mode for the Qualcomm SDHCI controllers. I > will make the argument that we don't _need_ strict mode for them. Why? > * The SDHCI controller on Qualcomm SoCs doesn't appear to have > loadable / updatable firmware and, assuming it's got some firmware > baked into it, I see no evidence that the firmware could be > compromised. > * Even though, for external SD cards in particular, the controller is > dealing with "untrusted" inputs, it's dealing with them in a very > controlled way. It seems unlikely that a rogue SD card would be > able to present something to the SDHCI controller that would cause > it to DMA to/from an address other than one the kernel told it > about. > * Although it would be nice to catch more software bugs, once the > Linux driver has been debugged and stressed the value is not very > high. If the IOMMU caught something like this the system would be in > a pretty bad shape anyway (we don't really recover from IOMMU > errors) and the only benefit would be a better spotlight on what > went wrong. > > Now we have a good understanding of the benefits of "strict" mode for > our SDHCI controllers, let's look at some performance
[PATCH v2 3/3] mmc: sdhci-msm: Request non-strict IOMMU mode
The concept of IOMMUs being in strict vs. non-strict mode is a pre-existing Linux concept. I've included a rough summary here to help evaluate this patch. IOMMUs can be run in "strict" mode or in "non-strict" mode. The quick-summary difference between the two is that in "strict" mode we wait until everything is flushed out when we unmap DMA memory. In "non-strict" we don't. Using the IOMMU in "strict" mode is more secure/safer but slower because we have to sit and wait for flushes while we're unmapping. To explain a bit why "non-strict" mode is unsafe, let's imagine two examples. An example of "non-strict" being insecure when reading from a device: a) Linux driver maps memory for DMA. b) Linux driver starts DMA on the device. c) Device write to RAM subject to bounds checking done by IOMMU. d) Device finishes writing to RAM and signals transfer is finished. e) Linux driver starts unmapping DMA memory but doesn't wait for the unmap to finish (the definition of non-strict). At this point, though, the Linux APIs say that the driver owns the memory and shouldn't expect any more scribbling from the DMA device. f) Linux driver validates that the data in memory looks sane and that accessing it won't cause the driver to, for instance, overflow a buffer. g) Device takes advantage of knowledge of how the Linux driver works and sneaks in a modification to the data after the validation but before the IOMMU unmap flush finishes. h) Device has now caused the Linux driver to access memory it shouldn't. An example of "non-strict" being insecure when writing to a device: a) Linux driver writes data intended for the device to RAM. b) Linux driver maps memory for DMA. c) Linux driver starts DMA on the device. d) Device reads from RAM subject to bounds checking done by IOMMU. e) Device finishes reading from RAM and signals transfer is finished. f) Linux driver starts unmapping DMA memory but doesn't wait for the unmap to finish (the definition of non-strict) g) Linux driver frees memory and returns it to the pool of memory available for other users to allocate. h) Memory is allocated for another purpose since it was free memory. i) Device takes advantage of the period of time before IOMMU flush to read memory that it shouldn't have had access to. What exactly the memory could contain depends on the randomness of who allocated next, though exploits have been built on flimisier holes. As you can see from the above examples, using the iommu in "non-strict" mode might not sound _too_ scary (the window of badness is small and the exposed memory is small) but there is certainly risk. Let's evaluate the risk by breaking it down into two problems that IOMMUs are supposed to be protecting us against: Case 1: IOMMUs prevent malicious code running on the peripheral (maybe a malicious peripheral or maybe someone exploited a benign peripheral) from turning into an exploit of the Linux kernel. This is particularly important if the peripheral has loadable / updatable firmware or if the peripheral has some type of general purpose processor and is processing untrusted inputs. It's also important if the device is something that can be easily plugged into the host and the device has direct DMA access itself, like a PCIe device. Case 2: IOMMUs limit the severity of a class of software bugs in the kernel. If we misconfigure a peripheral by accident then instead of the peripheral clobbering random memory due to a bug we might get an IOMMU error. Now that we understand the issue and the risks, let's evaluate whether we really need "strict" mode for the Qualcomm SDHCI controllers. I will make the argument that we don't _need_ strict mode for them. Why? * The SDHCI controller on Qualcomm SoCs doesn't appear to have loadable / updatable firmware and, assuming it's got some firmware baked into it, I see no evidence that the firmware could be compromised. * Even though, for external SD cards in particular, the controller is dealing with "untrusted" inputs, it's dealing with them in a very controlled way. It seems unlikely that a rogue SD card would be able to present something to the SDHCI controller that would cause it to DMA to/from an address other than one the kernel told it about. * Although it would be nice to catch more software bugs, once the Linux driver has been debugged and stressed the value is not very high. If the IOMMU caught something like this the system would be in a pretty bad shape anyway (we don't really recover from IOMMU errors) and the only benefit would be a better spotlight on what went wrong. Now we have a good understanding of the benefits of "strict" mode for our SDHCI controllers, let's look at some performance numbers. I used "dd" to measure read speeds from eMMC on a sc7180-trogdor-lazor board. Basic test command (while booted from USB): echo 3 > /proc/sys/vm/drop_caches dd if=/dev/mmcblk1 of=/dev/null bs=4M count=512 I attempted to run my
