Hi Christian, thank you for the pointers. [Note: your reply (Message-ID <[email protected]>) is on an earlier duplicate posting of this RFC (<by5pr02mb6866e35a2b4241cf3a728a99d3...@by5pr02mb6866.namprd02.prod.outlook.com>) that went out with mangled formatting. Continuing the discussion here on this thread instead, quoting your message below.]
On 8/6/26 13:52, Christian König wrote: > Oh, well history repeats itself. > > Well to make a longer story short what you describe here was the > original dma_fence proposal from >10 years ago. We fortunately > rejected it already back then. > First of all some of the stuff you suggest here go strictly against > documented dma_fence restrictions, especially no point to point > remote signaling! dma_fences have a huge interaction with Linux core > memory management which makes stuff like that impossible. Sure we'll need to be careful to not repeat seqno_fence. Synx can be used standalone without the optional dma-fence interop. It could have the properties of memory fences and need not satisfy the dma-fence contracts. Here remote-to-remote signalling doesn't wake-up the CPU. But we have use cases with synchronization points in the synx and dma-fence ecosystem for which we could provide synx<->dma-fence interop support "conditionally" while meeting the dma-fence contracts. What we would like to keep on the table is a narrower class (1) condition to assure point-to-point signal completion : remoteproc's path to signal must not depend on Linux memory management i.e. every buffer the remote touches in order to reach the signal is committed before the fence is published, with no on-demand faulting and no allocation from a Linux managed pool. Wherever this does not hold, the handle must not carry a dma_fence. This condition is what makes the amdgpu user-queue fence driver (97ff19462539) legal: the memory firmware writes into a pinned kernel BO allocated once at init. (2) the condition wants to be enforced in code so that reviewer and lockdep can check. We would make it a per-remoteproc capability and refuse to attach a dma_fence to a handle from a remote that does not declare it. This is how Xe refuses out-fences backed by dma-fence for a long-running VM. Composition would follow the same rule: a merged handle is dma_fence-capable only if every constituent is. (3) on top of the point-to-point signal, the remoteproc's Linux driver has the obligation to complete the dma-fence within a reasonable time. Interop fences can't avoid CPU wake-up but standalone synx handles can. Interop fences benefit from the low latency in point-to-point signalling. (4) to handle remote hangs that aren't related to memory, we need a watchdog armed at the dma-fence initialization which upon bite prevents further submission and triggers remoteproc recovery. Please let me know the issues in the model above. And to extend beyond the above conditions, the following proposal from you (which hit a deadend) seems very relevant. "Tackling the indefinite/user DMA fence problem" ([email protected]) Were/Are there any efforts to resolve the challenges that are open in this thread? > What you can do is to do this in userspace with DMA-buf and user > space fences, signaling points etc... Take a look at how user space > waits in XE work, how eventfd works and ROCm events work for an > overview and maybe some starting ideas. > > We could certainly build a framework around that, especially > eventfd sounds like a common ground to most drivers already, but > please stay away from dma_fences when you want to do something like > that. These pointers are helpful and we can start looking into these. We could keep the dma-fence interop discussion open and move ahead in the standalone and memory fence direction. Will share the high-level design. Please let me know if you need to focus on anything in particular. Thanks again, Pravin Kumar Ravi
