On 28/04/2025 14:15, Christian König wrote:
On 4/24/25 09:07, Tvrtko Ursulin wrote:
On 23/04/2025 14:12, Christian König wrote:
On 4/18/25 18:42, Tvrtko Ursulin wrote:
Hi all,
Recently I mentioned to Danilo about some fence lifetime issues so here is a
rough series, more than anything intended to start the discussion.
Most of the problem statement can be found in the first patch but to briefly
summarise - because sched fence can outlive the scheduler, we can trivially
engineer an use after free with xe and possibly other drivers. All that is
needed is to convert a syncobj into a sync file behind drivers back, and I don't
see what the driver can do about it.
Yeah that topic again :) The problem here is that this is not a bug, it is a
feature!
IIRC it was Alex who pointed that issue out on the very first fence patch set,
and we already discussed what to do back then.
The problem with grabbing module references for fences is that you get
trivially into circle references and so basically always preventing the module
from unloading.
Where "always" is only "while there are active objects from that module", no?
The problem is that dma_fences stay around after they are signaled. And
basically all drivers keep some dma_fence around for their resource management.
E.g. amdgpu for the VMIDs.
This means that some dma_fence is referenced by the module and the module
referenced by some dma_fence. E.g. you are never able to unload the module.
Are you thinking truly never or for as long someone has a reference?
For example while userspace has a reference to dma_fence via sync_file
fence owning module would not unloadable. One would have to terminate
the process, which granted wouldn't be easy to see which process
prevents the unload, before driver could be unloaded.
For the foreign fences kept around in kernel space, that would be
solvable by some periodic house keeping at worst.
Also, about the use cases for module unload. Since you and Brost
especially seem to be expressing a hard no to module references, what
are the use cases you are concerned about?
The decision was made to postpone this and live with the potential use after
free on module unload until somebody has time to fix it. Well that was +10
years ago :)
I discussed this with Sima again last year and we came to the conclusion that
the easiest way forward would be to decouple the dma_fence implementation from
the driver or component issuing the fence.
I then came up with the following steps to allow this:
1. Decouple the lock used for protecting the dma_fence callback list from the
caller.
2. Stop calling enable_signaling with the lock held.
3. Nuke all those kmem_cache implementations and force drivers to always
allocate fences using kvmalloc().
4. Nuke the release callback (or maybe move it directly after signaling) and set
fence->ops to NULL after signaling the fence.
I already send patches out for #1 and #2, but don't have enough time to
actually finish the work.
If you want take a look at nuking all those kmem_cache implementations for
allocating the fence memory. I think that can be completed completely separate
to everything else.
So enabling dma fence "revoke" so to say.
Just to check we are on the same page, it is not just about the module
references, but also use after frees which can happen even if module is still
loaded but any memory reachable via dma fence entry points has been freed.
Yeah, that came much later when people started to use the scheduler
dynamically. Basically the sched pointer in the drm_sched_fence implementation
becomes invalid as soon as the fence signals.
In that case, as Matt has already asked, if you could dig up your unfinished
work it would be interesting to see.
This is what I already send out:
https://gitlab.freedesktop.org/ckoenig/linux-drm/-/commits/dma-fence-rework-enable-signaling
A bunch of the cleanup patches in that branch have already been applied, only
the last one is missing IIRC.
And here is a WIP patch to decouple the lock I wrote halve a year ago or so:
https://gitlab.freedesktop.org/ckoenig/linux-drm/-/commits/dma-fence-rework-locking
Thanks!
My concern here is that to me it appears the whole premise is to leave
fences dangling in memory and somehow make them safe to be accessed by
importers.
For starters this can create permanent memory leaks. Or at least for the
same window of duration as would the exporters be not unloadable with
the reference counting alternative. So we would not a strong argument
for why poorly bound memory leaks are better than poorly bound
unloadable modules.
It is also a question how to "revoke" fences safely (race free). It
sounds hard to me. It does not seem you got to this last problem in the
above branches so I don't know if you had some elegant ideas for that.
Maybe first to ask if anyone is aware of a precedent where something in
the kernel already uses this design pattern?
Regards,
Tvrtko
