On Fri, 2025-09-26 at 14:56 +0200, Christian König wrote:
>
>
> On 26.09.25 10:46, Thomas Hellström wrote:
> > Add a function to the dma_buf_attach_ops to indicate whether the
> > connection is a private interconnect. If so the function returns
> > the address to an interconnect-defined structure that can be
> > used for further negotiating.
> >
> > Also add a field to the dma_buf_attachment that indicates whether
> > a private interconnect is used by the attachment.
> >
> > Signed-off-by: Thomas Hellström <[email protected]>
> > ---
> > include/linux/dma-buf.h | 51
> > +++++++++++++++++++++++++++++++++++++++++
> > 1 file changed, 51 insertions(+)
> >
> > diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
> > index d58e329ac0e7..25dbf1fea09a 100644
> > --- a/include/linux/dma-buf.h
> > +++ b/include/linux/dma-buf.h
> > @@ -442,6 +442,39 @@ struct dma_buf {
> > #endif
> > };
> >
> > +/* RFC: Separate header for the interconnect defines? */
> > +
> > +/**
> > + * struct dma_buf_interconnect - Private interconnect
> > + * @name: The name of the interconnect
> > + */
> > +struct dma_buf_interconnect {
> > + const char *name;
> > +};
> > +
> > +/**
> > + * struct dma_buf_interconnect_attach_ops - Interconnect attach
> > ops base-class
> > + *
> > + * Declared for type-safety. Interconnect implementations should
> > subclass to
> > + * implement negotiation-specific ops.
> > + */
> > +struct dma_buf_interconnect_attach_ops {
> > +};
> > +
> > +/**
> > + * struct dma_buf_interconnect_attach - Interconnect state
> > + * @interconnect: The struct dma_buf_interconnect identifying the
> > interconnect
> > + *
> > + * Interconnect implementations subclass as needed for attachment
> > state
> > + * that can't be stored elsewhere. It could, for example, hold a
> > pointer
> > + * to a replacement of the sg-list after the attachment has been
> > mapped.
> > + * If no additional state is needed, an exporter could define a
> > single
> > + * static instance of this struct.
> > + */
> > +struct dma_buf_interconnect_attach {
> > + const struct dma_buf_interconnect *interconnect;
> > +};
> > +
> > /**
> > * struct dma_buf_attach_ops - importer operations for an
> > attachment
> > *
> > @@ -475,6 +508,21 @@ struct dma_buf_attach_ops {
> > * point to the new location of the DMA-buf.
> > */
> > void (*move_notify)(struct dma_buf_attachment *attach);
> > +
> > + /**
> > + * @supports_interconnect: [optional] - Does the driver
> > support a local interconnect?
> > + *
> > + * Does the importer support a private interconnect? The
> > interconnect is
> > + * identified using a unique address defined instantiated
> > either by the driver
> > + * if the interconnect is driver-private or globally
> > + * (RFC added to the dma-buf-interconnect.c file) if
> > cross-driver.
> > + *
> > + * Return: A pointer to the interconnect-private
> > attach_ops structure if supported,
> > + * %NULL otherwise.
> > + */
> > + const struct dma_buf_interconnect_attach_ops *
> > + (*supports_interconnect)(struct dma_buf_attachment
> > *attach,
> > + const struct dma_buf_interconnect
> > *interconnect);
>
> This looks like it sits in the wrong structure. The
> dma_buf_attach_ops are the operations provided by the importer, e.g.
> move notification.
>
> When we want to check if using an interconnect is possible we need to
> do that on the exporter, e.g. dma_buf_ops().
Well both exporter and exporter has specific information WRT this. The
ultimate decision is done in the exporter attach() callback, just like
pcie_p2p. And the exporter acknowledges that by setting the
dma_buf_attachment::interconnect_attach field. In analogy with the
dma_buf_attachment::peer2peer member.
So the above function mimics the dma_buf_attach_ops::allow_peer2peer
bool, except it's not a single interconnect so we'd either use a set of
bools, one for each potential interconnect, or a function like this.
A function has the benefit that it can also provide any additional
attach ops the interconnect might need.
So the flow becomes:
1) Importer calls exporter attach() with a non-NULL
supports_interconnect() to signal that it supports some additional
interconnects.
2) exporter calls supports_interconnect(my_interconnect) to figure out
whether the importer supports a specific interconnect it wants to try.
This is similar to the exporter checking "allow_peer2peer" (or rather
the core checking "allow_peer2peer" on the behalf of the exporter).
3) Importer finds it supports the interconnect and provides additional
dma_buf_interconnect_attach_ops.
4) Now the exporter checks that the interconnect is indeed possible.
This is similar to calling pci_p2p_distance(), but interconnect-
specific. This might involve querying the importer, for example if the
importer feels like the exporting device:bar pair does indeed have an
implicit VF_PF connection. This can be done if needed using the
dma_buf_interconnect_attach_ops.
5) Exporter is happy, and sets the
dma_buf_attachment::interconnect_attach field. This is similar to
setting the dma_buf_attachment::peer2peer field.
So basically this is the pcie peer2peer negotiation flow generalized.
It would be trivial to implement the pcie peer2peer negotiation as a
private protocol using the above.
>
> I think we should have an map_interconnect(connector type descriptor)
> that the importer can use to establish a mapping for itself.
>
> Additional to that we need an unmap_interconnect() to let the
> exporter know that an importer doesn't need a specific mapping any
> more.
Is this to not overload the map_attachment() and unmap_attachment()
functions that otherwise could be used? Is it because they return an
sg_table? Yeah, that could make sense but not for the interconnect
negotiation itself, right? That happens during attach time like
pcie_p2p?
>
> > };
> >
> > /**
> > @@ -484,6 +532,8 @@ struct dma_buf_attach_ops {
> > * @node: list of dma_buf_attachment, protected by dma_resv lock
> > of the dmabuf.
> > * @peer2peer: true if the importer can handle peer resources
> > without pages.
> > * @priv: exporter specific attachment data.
> > + * @interconnect_attach: Private interconnect state for the
> > connection if used,
> > + * NULL otherwise.
> > * @importer_ops: importer operations for this attachment, if
> > provided
> > * dma_buf_map/unmap_attachment() must be called with the dma_resv
> > lock held.
> > * @importer_priv: importer specific attachment data.
> > @@ -503,6 +553,7 @@ struct dma_buf_attachment {
> > struct list_head node;
> > bool peer2peer;
> > const struct dma_buf_attach_ops *importer_ops;
> > + struct dma_buf_interconnect_attach *interconnect_attach;
>
> We already have an importer and an exporter private void *. Do we
> really need that?
See above. It looks like the exporter private is largely unused in
xekmd at least but the importer would want to inspect that as well, to
find out whether the attachment indeed is an interconnect attachment.
And I'm not sure whether a driver that already uses the exporter priv
would ever want to use a private interconnect like this.
Thanks,
Thomas
>
> Regards,
> Christian.
>
> > void *importer_priv;
> > void *priv;
> > };
>
