Hi Ravi, On 8/20/26 19:59, Pravin Kumar Ravi wrote: ... > (1) condition to assure point-to-point signal completion : remoteproc's path > to signal must not depend on Linux memory management i.e. every buffer the > remote touches in order to reach the signal is committed before the fence is > published, with no on-demand faulting and no allocation from a Linux managed > pool. Wherever this does not hold, the handle must not carry a dma_fence. > This condition is what makes the amdgpu user-queue fence driver > (97ff19462539) legal: the memory firmware writes into a pinned kernel > BO allocated once at init. > > (2) the condition wants to be enforced in code so that reviewer and > lockdep can check. We would make it a per-remoteproc capability and > refuse to attach a dma_fence to a handle from a remote that does not > declare it. This is how Xe refuses out-fences backed by dma-fence > for a long-running VM. Composition would follow the same rule: a merged > handle is dma_fence-capable only if every constituent is. > > (3) on top of the point-to-point signal, the remoteproc's Linux driver > has the obligation to complete the dma-fence within a reasonable time. > Interop fences can't avoid CPU wake-up but standalone synx handles can. > Interop fences benefit from the low latency in point-to-point signalling. > > (4) to handle remote hangs that aren't related to memory, we need a > watchdog armed at the dma-fence initialization which upon bite prevents > further submission and triggers remoteproc recovery. > > Please let me know the issues in the model above.
Well to make it clear I would NAK that approach from my side as DMA-buf maintainer. We have tried that multiple times with the same extremely disappointing results. > And to extend beyond the above conditions, the following proposal > from you (which hit a deadend) seems very relevant. > "Tackling the indefinite/user DMA fence problem" > ([email protected]) > Were/Are there any efforts to resolve the challenges that are open in > this thread? That is simply not possible. It is basically a re-occurring topic on every XDC conference of the last 10 years or so. >> What you can do is to do this in userspace with DMA-buf and user>> space >> fences, signaling points etc... Take a look at how user space >> waits in XE work, how eventfd works and ROCm events work for an >> overview and maybe some starting ideas. >> >> We could certainly build a framework around that, especially >> eventfd sounds like a common ground to most drivers already, but >> please stay away from dma_fences when you want to do something like >> that. > > These pointers are helpful and we can start looking into these. > We could keep the dma-fence interop discussion open and move ahead in the > standalone and memory fence direction. Will share the high-level design. > Please let me know if you need to focus on anything in particular. Essentially you can have two different modes of operation: 1. Preallocate resources on work submission to the HW. This is what the dma_fence framework implements. The restriction here is that you don't have any dependency to userspace or other device drivers, e.g. no page faulting, waiting for memory values etc... Background is that the core memory management depends so hard on the execution properties of the dma_fence that none of those advanced features are possible without breaking core assumptions. Inter device operation is only allowed if the device drivers "know" about each other, for example between rendering and displaying pipelines. But we have intentionally rejected any approach of generalizing device to device signaling in a framework because it makes dma_fence practically a recipe for trouble and we have more than enough evidence to confirm this now. 2. On demand resource allocation using PCIe functionalities like IOMMUv2/ATC/ATS/PRI or device specific page fault handling. The point is that you can do things like generating commands on the fly, long running jobs, device to device signaling etc... The only restriction is that you can't create a dma_fence with this as output, you can only consume them as input to the operation. There was also an absolutely great presentation on the topic from Faith on LPC a few years ago. Regards, Christian. > > Thanks again, > Pravin Kumar Ravi
