From: Christian Gmeiner <[email protected]>
On MMUv1, all clients share one address space and the allocator
always picks the lowest free IOVA. When a process frees a range,
it can immediately be reused by another process.
The GL.FLUSH_MMU write before the next submit does not reliably
invalidate stale TLB entries for that range. The GPU may therefore
continue translating accesses to the old pages. Writes through
these stale entries are lost for the new owner and may corrupt
memory that has already been freed.
The vendor driver works around this in a similar way. It does not
return freed MMUv1 ranges to its free list until the address space
is exhausted.
Restore the address space cycling for MMUv1 that was previously
removed as no longer needed. Start searching after the last
allocation and only wrap around to the beginning before trying
to evict existing mappings. This keeps freed ranges in the
allocator but delays their reuse until the search wraps around.
MMUv2 has a proper TLB flush and keeps the current behavior.
Fixes: ccae45928fc4 ("drm/etnaviv: remove cycling through MMU address space")
Cc: [email protected]
Signed-off-by: Christian Gmeiner <[email protected]>
---
drivers/gpu/drm/etnaviv/etnaviv_mmu.c | 21 ++++++++++++++++++++-
drivers/gpu/drm/etnaviv/etnaviv_mmu.h | 1 +
2 files changed, 21 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
index e3572461b599..c4e4dd3d4235 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
+++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.c
@@ -141,10 +141,20 @@ static int etnaviv_iommu_find_iova(struct
etnaviv_iommu_context *context,
{
struct etnaviv_vram_mapping *free = NULL;
enum drm_mm_insert_mode mode = DRM_MM_INSERT_LOW;
+ u64 start = 0;
int ret;
lockdep_assert_held(&context->lock);
+ /*
+ * MMUv1 does not reliably drop stale TLB entries on a flush, so a
+ * freed IOVA that is handed out again right away can still be
+ * translated to the old pages. Cycle through the address space so
+ * recently freed IOVAs are only reused once the search wraps around.
+ */
+ if (context->global->version == ETNAVIV_IOMMU_V1)
+ start = context->last_iova;
+
while (1) {
struct etnaviv_vram_mapping *m, *n;
struct drm_mm_scan scan;
@@ -152,10 +162,16 @@ static int etnaviv_iommu_find_iova(struct
etnaviv_iommu_context *context,
bool found;
ret = drm_mm_insert_node_in_range(&context->mm, node,
- size, 0, 0, 0, U64_MAX, mode);
+ size, 0, 0, start, U64_MAX,
mode);
if (ret != -ENOSPC)
break;
+ /* Retry from the start of the address space before evicting. */
+ if (start) {
+ start = 0;
+ continue;
+ }
+
/* Try to retire some entries */
drm_mm_scan_init(&scan, &context->mm, size, 0, 0, mode);
@@ -215,6 +231,9 @@ static int etnaviv_iommu_find_iova(struct
etnaviv_iommu_context *context,
*/
}
+ if (!ret && context->global->version == ETNAVIV_IOMMU_V1)
+ context->last_iova = node->start + node->size;
+
return ret;
}
diff --git a/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
b/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
index 7f8ac0178547..7fa6e3ab427a 100644
--- a/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
+++ b/drivers/gpu/drm/etnaviv/etnaviv_mmu.h
@@ -75,6 +75,7 @@ struct etnaviv_iommu_context {
struct list_head mappings;
struct drm_mm mm;
unsigned int flush_seq;
+ u64 last_iova;
/* Not part of the context, but needs to have the same lifetime */
struct etnaviv_vram_mapping cmdbuf_mapping;
---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
change-id: 20261102-etna-mmuv1-iova-cycle-2fc0e6200fa6
Best regards,
--
Christian Gmeiner <[email protected]>
