On 10/19/26 15:30, Jason Gunthorpe wrote:
> On Mon, Oct 19, 2026 at 02:43:29PM +0200, Thomas Hellström wrote:
>> On Mon, 2026-10-19 at 10:35 +0200, Christian König wrote:
>>> On 10/16/26 22:26, Karl Mehltretter wrote:
>>>> On Tue, Feb 17, 2026 at 08:11:34PM -0400, Jason Gunthorpe wrote:
>>>>> + match_args.attach = attach;
>>>>> + if (dmabuf->ops->match_mapping) {
>>>>> + ret = dmabuf->ops->match_mapping(&match_args);
>>>>> + }
>>>>> +
>>>>> attach->dmabuf = dmabuf;
>>>>> attach->importer_ops = importer_ops;
>>>>> attach->importer_priv = importer_priv;
>>>>
>>>> I may misunderstand the intended contract. match_mapping() runs
>>>> while the
>>>> attachment is still zeroed and without dmabuf->resv held.
>>>>
>>>> Patch 11's amdgpu callback needs importer_ops and importer_priv, so
>>>> its XGMI
>>>> match cannot work here. Patch 12's VFIO callback asserts dmabuf-
>>>>> resv and
>>>> reads priv->vdev, which cleanup changes under that lock.
>>>>
>>>> Should the core initialize the attachment and hold dmabuf->resv for
>>>> match_mapping(), or should callbacks avoid attachment and lock-
>>>> protected
>>>> state?
>>>
>>> Right. This completely misplaced.
>>>
>>> The matching can't be done at attachment time, but rather needs to be
>>> done before creating a mapping while holding the resv lock so that
>>> the matching decision can't change.
>>
>> So question is what exactly should the contract be?
>> Looking at the pcie_p2p negotiation today it looks somewhat like.
>>
>> 1) Importer and exporter agree at attach time what mappings are allowed
>> pending placement conditions and other attachers.
>> 2) At map-time the mapping is negotiated taking all constraints into
>> account.
>> 3) The negotiated mapping is valid until an invalidate_mappings()
>> callback is received.
>>
>> Following this would allow a re-implementation of pcie_p2p as a mapping
>> type and would allow a controlled link-severing to send an
>> invalidate_mappings() notifier and the mappings to be re-negotiated.
>>
>> Importers requiring an early hint as to what the actual mapping would
>> look like would need to be prepared to re-consider at map-time, with a
>> reasonable assumption that the reconsideration and re-mapping would
>> happen seldom enough.
>
> The idea was the supported mapping types are fixed properties of the
> importer and exporter.
>
> Like iommufd is not suddenly going to start understanding ualink after
> an invalidation.
Yeah but I think we can sort things out early on.
E.g. even if both devices can do PCI P2P we can't use it when they are not
under the same root complex and there is no routing between root complexes.
Or both devices have an XGMI connection with other devices but are not
physically connected to each other, etc...
That is something which doesn't change unless you physically change the system.
> Further, I had the (perhaps naive) expectation that the interconnect
> option was not something that would change during mapping. I did not
> think you would loose your PCIe option if ualink becomes available, or
> vice versa. Though on deeper reflection I suppose someone could build
> HW so you might loose your ualink option if the dmabuf is moved to
> system memory while PCIe would still work.
Yeah, that is definitely not correct.
We have the clear requirement here that when one interconnect physically fails
that we can still do things like let the work reach the next checkpoint, then
it is taken of the HW, and the HW suspended for repair, then resumed and then
work can continue....
Similarly we have on the graphics side the requirement that physical placement
can be moved to system memory because of some other device attaching, pinning
for scanout or something suddenly requires the backing store to be contiguously
allocated etc,,,
So there are vast requirements that a buffer changes location when the mapping
changes. I mean it is kind of pointless to have a feature like ODP to suspend
your traffic if the underlying physical location of the memory can't change.
Christian.
> Making the decision right before map seems like a reasonable
> enhancement, but I can no longer remember off hand if that would cause
> complications :)
>
> Jason
