On 10/1/2025 1:57 PM, Jerin Jacob wrote:
>
>
>> -----Original Message-----
>> From: Vamsi Krishna Attunuru <[email protected]>
>> Sent: Wednesday, September 24, 2025 9:45 AM
>> To: fengchengwen <[email protected]>; [email protected];
>> [email protected]
>> Cc: [email protected]; [email protected];
>> [email protected]; [email protected];
>> [email protected]; Jerin Jacob <[email protected]>
>> Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process and
>> inter-OS
>> DMA
>>
>> Hi Feng, Anatoly,
>>
>> Gentle ping for below review.
>>
>>> -----Original Message-----
>>> From: Vamsi Krishna Attunuru
>>> Sent: Monday, September 22, 2025 5:19 PM
>>> To: fengchengwen <[email protected]>; [email protected];
>>> [email protected]
>>> Cc: [email protected]; [email protected];
>>> [email protected]; [email protected];
>>> [email protected]; Jerin Jacob <[email protected]>
>>> Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process and
>>> inter- OS DMA
>>>
>>> Hi Feng,
>>>
>>>> Hi Vamsi, This commit change is more than discussed, it add control
>>>> API which for group management. 1. Control API: I check this commit
>>>> and Intel commit [1], it seem has a quite difference. I hope Intel
>>>> guys can express views
>>>> ZjQcmQRYFpfptBannerEnd
>>>> Hi Vamsi,
>>>>
>>>> This commit change is more than discussed, it add control API which
>>>> for group management.
>>>>
>>>> 1. Control API: I check this commit and Intel commit [1], it seem has
>>>> a quite difference.
>>>> I hope Intel guys can express views. I prefer not add this part if
>>>> no
>>> response.
>>>
>>> This new feature needs to be securely managed through control APIs. It
>>> would be extremely helpful if the folks at Intel and you as well could
>>> provide support or inputs on this.
>
>
> Beyond adding Intel folks to this thread, I don't see any further steps we
> can take to drive review at this stage.
>
> That said, the table-based concept used in the current API may not be
> portable, and we may need improvements here.
>
> Based on my understanding, DMA devices used for inter process copy can be
> classified into three categories:
>
> Class A: Requires a pair of DMA devices (one on each end of process/domain)
> for data transfer. Marvell DMA devices fall into this category.
> Class B: Requires only a single DMA device (one process/domain has a DMA
> device, the other process does not). Intel DMA devices fall here.
> Class C: Other types of devices that we are not yet aware of.
>
> Abstracting all of these under a single API will be challenging. Linux and
> other OSes do not provide control-plane APIs for this,
> so DPDK must provide control plane mechanisms to support Class A, Class B,
> and Class C devices.
>
> Proposal: Split development into separate sets:
> -----------------------------------------------
> Set A: Focus only on the datapath. Assume uint16_t *src_handle and uint16_t
> *dst_handle come from elsewhere (Class C).
> Set B: Introduce capabilities for Class A devices with portable APIs
> (proposal below, without table concept).
> Set C: Introduce capabilities for Class B devices and relevant APIs, to be
> added when needed.
>
> We can merge Set A in the current release and move Set B to a next release
> _if_ review or support for Class A devices requires more time.
>
> @fengchengwen Thoughts?
okay
>
> Class A API Proposal:
> ---------------------
> These APIs are based on a new capability flag for inter-process or inter-OS
> DMA transfers for Class A devices.
>
>
> /** Creates an access group for pair-type inter-process or inter-OS DMA
> transfers. */
> int rte_dma_access_pair_group_create(const struct rte_dma_dev *dev,
> rte_uuid_t process_id,
> rte_uuid_t token,
> uint16_t *group_id);
how about rte_dma_access_group_create(), and add pair-type as one parameter,
and also rename process_id as domain_id
>
> /** Destroys an access group once all participating devices have exited. */
> int rte_dma_access_pair_group_destroy(const struct rte_dma_dev *dev,
> uint16_t group_id);
rte_dma_access_group_destroy()
>
> /** Allows a device to join an existing access group using a device handle
> and token. */
> int rte_dma_access_pair_group_join(const struct rte_dma_dev *dev,
> uint16_t group_id,
> rte_uuid_t process_id,
> rte_uuid_t token,
> rte_dma_access_pair_leave_cb_t leave_cb);
rte_dma_access_group_join()
>
> /** Removes a device from an access group. */
> int rte_dma_access_pair_group_leave(const struct rte_dma_dev *dev,
> uint16_t group_id);
>
> /** Retrieves the source and destination handles for a given device within
> the group. */
> int rte_dma_access_pair_gorup_src_dst_handles_get(const struct rte_dma_dev
> *dev,
> uint16_t group_id,
> rte_uuid_t src_process_id,
> rte_uuid_t dst_process_id,
> uint16_t *src_handle,
> uint16_t *dst_handle);
rte_dma_access_group_handle_get(const struct rte_dma_dev *dev,
uint16_t group_id,
rte_uuid_t domain_id,
uint16_t *handle);
so user could invoke multiple time if they want to get differ domain_id's
handle.
>
>
> Parameters that need explanation:
> --------------------------------
> process_id: Unique ID for the process, generated via rte_uuid_ APIs
> token: Provided by an administrative actor to grant access, similar to VFIO
> VF token creation used in VFIO PF driver.
> leave_cb: Callback to notify when another side leaves the group
>
>
> Example Workflow for Class A Inter-Domain DMA Transfer:
> -------------------------------------------------------
>
> This example demonstrates how three processes — p0, p1, and p2 — coordinate
> inter-domain DMA transfers using pair-type(Class A) DMA devices.
>
> Step 1: Group Creation (p0)
> Process p0 calls rte_dma_access_pair_group_create() with a unique process
> handle and token. A group_id is returned.
I prefer the group_id is int type so it could hold such like file descriptor.
>
> Step 2: Group Sharing
> group_id and token are shared with p1 and p2 via IPC or shared memory.
>
> Step 3: Group Joining (p1 & p2)
> Processes p1 and p2 call rte_dma_access_pair_group_join() with their process
> id and the shared token from admin
>
> Step 4: Handle Discovery
> Each process uses rte_dma_access_pair_gorup_src_dst_handles_get() to retrieve
> source and destination handles for other processes.
>
> Step 5: Transfer Coordination
> Using the handles, each process configures a virtual channel (vchan) and
> initiates DMA transfers.
>
> Step 6: Group Teardown
> When a process no longer needs to participate, it calls
> rte_dma_access_pair_group_leave(). Other processes are notified via the
> registered callback with rte_dma_access_pair_group_join().
> Once all devices have exited, p0 calls rte_dma_access_pair_group_destroy() to
> clean up.
>
>
> For Class B: We can add new capability flag and have new set of APIs
> rte_dma_access_master_ or so. When such devices comes or when intel wants to
> add it
>
>
>
>
>
