Hi Feng, Anatoly Could please review below proposal and share your thoughts.
Regards Vamsi >> -----Original Message----- >> From: Vamsi Krishna Attunuru <[email protected]> >> >From: Vamsi Krishna Attunuru >> >Sent: Monday, September 22, 2025 5:19 PM >> >To: fengchengwen <[email protected]>; [email protected]; >> >[email protected] >> >Cc: [email protected]; [email protected]; >> >[email protected]; [email protected]; >> >[email protected]; Jerin Jacob <[email protected]> >> >Subject: RE: [EXTERNAL] Re: [RFC] lib/dma: introduce inter-process >> >and >> >inter- OS DMA >> > >> >Hi Feng, >> > >> >>Hi Vamsi, This commit change is more than discussed, it add control >> >>API which for group management. 1. Control API: I check this commit >> >>and Intel commit [1], it seem has a quite difference. I hope Intel >> >>guys can express views ZjQcmQRYFpfptBannerEnd Hi Vamsi, >> >> >> >>This commit change is more than discussed, it add control API which >> >>for group management. >> >> >> >>1. Control API: I check this commit and Intel commit [1], it seem >> >>has a quite difference. >> >> I hope Intel guys can express views. I prefer not add this part >> >>if no >> >response. >> > >> >This new feature needs to be securely managed through control APIs. >> >It would be extremely helpful if the folks at Intel and you as well >> >could provide support or inputs on this. > > >Beyond adding Intel folks to this thread, I don't see any further steps we can >take to drive review at this stage. > >That said, the table-based concept used in the current API may not be >portable, and we may need improvements here. > >Based on my understanding, DMA devices used for inter process copy can be >classified into three categories: > >Class A: Requires a pair of DMA devices (one on each end of process/domain) >for data transfer. Marvell DMA devices fall into this category. >Class B: Requires only a single DMA device (one process/domain has a DMA >device, the other process does not). Intel DMA devices fall here. >Class C: Other types of devices that we are not yet aware of. > >Abstracting all of these under a single API will be challenging. Linux and >other >OSes do not provide control-plane APIs for this, so DPDK must provide control >plane mechanisms to support Class A, Class B, and Class C devices. > >Proposal: Split development into separate sets: >----------------------------------------------- >Set A: Focus only on the datapath. Assume uint16_t *src_handle and uint16_t >*dst_handle come from elsewhere (Class C). >Set B: Introduce capabilities for Class A devices with portable APIs (proposal >below, without table concept). >Set C: Introduce capabilities for Class B devices and relevant APIs, to be >added >when needed. > >We can merge Set A in the current release and move Set B to a next release >_if_ review or support for Class A devices requires more time. > >@fengchengwen Thoughts? > >Class A API Proposal: >--------------------- >These APIs are based on a new capability flag for inter-process or inter-OS >DMA transfers for Class A devices. > > >/** Creates an access group for pair-type inter-process or inter-OS DMA >transfers. */ int rte_dma_access_pair_group_create(const struct >rte_dma_dev *dev, > rte_uuid_t process_id, > rte_uuid_t token, > uint16_t *group_id); > >/** Destroys an access group once all participating devices have exited. */ int >rte_dma_access_pair_group_destroy(const struct rte_dma_dev *dev, > uint16_t group_id); > >/** Allows a device to join an existing access group using a device handle and >token. */ int rte_dma_access_pair_group_join(const struct rte_dma_dev >*dev, > uint16_t group_id, > rte_uuid_t process_id, > rte_uuid_t token, > rte_dma_access_pair_leave_cb_t leave_cb); > >/** Removes a device from an access group. */ int >rte_dma_access_pair_group_leave(const struct rte_dma_dev *dev, > uint16_t group_id); > >/** Retrieves the source and destination handles for a given device within the >group. */ int rte_dma_access_pair_gorup_src_dst_handles_get(const struct >rte_dma_dev *dev, > uint16_t group_id, > rte_uuid_t src_process_id, > rte_uuid_t dst_process_id, > uint16_t *src_handle, > uint16_t *dst_handle); > > >Parameters that need explanation: >-------------------------------- >process_id: Unique ID for the process, generated via rte_uuid_ APIs >token: Provided by an administrative actor to grant access, similar to VFIO VF >token creation used in VFIO PF driver. >leave_cb: Callback to notify when another side leaves the group > > >Example Workflow for Class A Inter-Domain DMA Transfer: >------------------------------------------------------- > >This example demonstrates how three processes — p0, p1, and p2 — >coordinate inter-domain DMA transfers using pair-type(Class A) DMA devices. > >Step 1: Group Creation (p0) >Process p0 calls rte_dma_access_pair_group_create() with a unique process >handle and token. A group_id is returned. > >Step 2: Group Sharing >group_id and token are shared with p1 and p2 via IPC or shared memory. > >Step 3: Group Joining (p1 & p2) >Processes p1 and p2 call rte_dma_access_pair_group_join() with their process >id and the shared token from admin > >Step 4: Handle Discovery >Each process uses rte_dma_access_pair_gorup_src_dst_handles_get() to >retrieve source and destination handles for other processes. > >Step 5: Transfer Coordination >Using the handles, each process configures a virtual channel (vchan) and >initiates DMA transfers. > >Step 6: Group Teardown >When a process no longer needs to participate, it calls >rte_dma_access_pair_group_leave(). Other processes are notified via the >registered callback with rte_dma_access_pair_group_join(). >Once all devices have exited, p0 calls rte_dma_access_pair_group_destroy() >to clean up. > > >For Class B: We can add new capability flag and have new set of APIs >rte_dma_access_master_ or so. When such devices comes or when intel >wants to add it > > > >
