Hi, Few points from my side,
1. We are creating pktio and then assigning the created queue to the pktio. IMO we should abstract the PKTIO, since it is up to the implementation to either support IPC through dummy interface or shared memory. Lets say if an implementation does IPC through shared memory there is no need for any PKTIO initialization. 2. We need to think about the design of how many receive queues are there per process, do we follow the design of one receive queue per process which means all the processes which wants to send message to process-A always enqueues into a queue named queue-process-A 3. Regarding synchronization, IMO if we agree that there will be only one receive queue per process, we can create IPC queues based on standard names, and when a process which needs IPC mechanism registers we can create the receive queue for that specific process and the process can issue find_IPC_queue() function using the standardised process name and get the queue handle for process to which it wants to send the message. Regards, Bala On 26 August 2014 17:37, Ola Liljedahl <[email protected]> wrote: > Sharing queues between threads that share memory is not a big problem. But > sharing queues between different programs/processes which do not want to > share memory (e.g. between a control plane program and one or several > dataplane applications) is a problem. It is not just the lookup of the > queue identifier. Depending on the ODP implementation, some memory might > have to be shared between these different processes even if the user code > is (and wants to be) unaware of this. An implementation might get away from > sharing memory in this case if either the HW can do the copy or if the ODP > implementation calls the Linux kernel to perform the copy of buffers > (messages). > > > On 26 August 2014 14:03, Maxim Uvarov <[email protected]> wrote: > >> On 08/26/2014 03:13 PM, Ola Liljedahl wrote: >> >>> Possibly we are missing a public API for looking up a queue (or perhaps >>> specifically an IPC queue) by name. >>> >>> If you have two threads (in same or different processes) that first try >>> to look up an (IPC or global) queue by name and if this fails then creates >>> said IPC queue, you have introduced a race condition. Possibly the second >>> IPC queue create will fail because the named queue exists (EEXIST) and the >>> caller can then understand it does not have to create the queue. >>> >>> So I think we have some synchronization issues to understand and solve >>> when it comes to IPC. Because the primary use case for IPC will be for >>> communication between different programs (processes) so they have no means >>> for synchronization (I would rather not have to use other Linux mechanisms >>> for setting up an IPC channel). >>> >>> Ola, I agree with that. We have function to look up for pool, but don't >> have that function to loop up for queue. But it might be common case where >> different threads want to reference to single queue. >> >> odp_queue_t odp_queue_lookup(const char *name, odp_queue_type_t type) >> >> which will do: >> - walk over queue_tbl and compare name with argument, if name match >> return it; >> - if type ipc, search for shared memory object, if it's exist connect to >> it. >> >> Maxim. >> >> >> >>> On 26 August 2014 11:48, Maxim Uvarov <[email protected] <mailto: >>> [email protected]>> wrote: >>> >>> On 08/26/2014 12:56 PM, Taras Kondratiuk wrote: >>> >>> On 08/25/2014 05:37 PM, Maxim Uvarov wrote: >>> >>> I pushed here some raw code: >>> https://git.linaro.org/people/maxim.uvarov/odp.git/shortlog/ >>> refs/heads/odp_ipc2 >>> >>> >>> >>> Example here app here: >>> https://git.linaro.org/people/maxim.uvarov/odp.git/blob/ >>> refs/heads/odp_ipc2:/example/fork/odp_fork.c >>> >>> >>> >>> pktio_queue_thread() takes packets from inq queue and put >>> them to ipc >>> queue. >>> >>> ring_thread() takes packets from ipc queue and free this >>> buffer. >>> >>> >>> I have a few things I didn't understand: >>> 1. It is not clear how IPC pktio and its pool is used. >>> Especially on >>> 'sender' side. >>> >>> >>> We create pool somewhere. Then do loop up for it: >>> pkt_pool = odp_buffer_pool_lookup("packet_pool"); >>> >>> Then we want that IPC queue can work with that pool. So we open >>> pktio: >>> >>> pktio_ipc_params.type = ODP_PKTIO_TYPE_IPC; >>> pktio_ipc = odp_pktio_open(NULL, pkt_pool, &pktio_ipc_params); >>> >>> And then create queue: >>> >>> ipcq_def = odp_queue_create("shared-queue", ODP_QUEUE_TYPE_IPC, >>> &qparam); >>> >>> when we place packet to this queue with: >>> odp_queue_enq(ipcq_def, buf); >>> >>> Than everything depend on implementation. So linux-generic >>> (software), packet data >>> still will be in pool. odp_buffer_t (uint32_t) value will be added >>> to ring. >>> >>> Other process gets odp_buffer_t, and find pointer to that buffer >>> (odp_buffer_hdr_t*). >>> >>> On hw implementation you just place odp_buffer_t to the queue. And >>> if ipc queue has the same >>> pool than packet can be delivered to software. If there are >>> different pools, then packet should be >>> copied to ipc pool, and after that delivered to hardware. >>> >>> The main goal here is we can say that ipc pool can be the same as >>> ingress pool or it can be different. >>> Depends on hw configuration. >>> >>> >>> 2. On both sides IPC queue is created. My impression was that >>> it should >>> be looked up on one side via pktio or directly via queue name. >>> >>> >>> Is there way to ask hardware which queues were already created? >>> >>> Actually I do look up, but it's hidden in implementation. Idea is >>> that if queue type is IPC, >>> then I do loop up if that queue already exist. If exist then I >>> connect to that queue. If it does >>> not exist - I create it. >>> >>> static void queue_init(queue_entry_t *queue, const char *name, >>> odp_queue_type_t type, odp_queue_param_t *param) >>> >>> case ODP_QUEUE_TYPE_IPC: >>> queue->s.r = odp_ring_lookup(name); >>> >>> if (!queue->s.r) >>> { >>> printf("creating new odp ring: %s\n", name); >>> queue->s.r = odp_ring_create(name, RING_SIZE, >>> ODP_RING_SHM_PROC); >>> if (queue->s.r == NULL) { >>> ODP_ERR("ring create failed\n"); >>> } >>> } else >>> printf("odp ring found!!!\n"); >>> >>> Because of we can say that several processes can connect to one >>> queue, there is no difference >>> which exactly process will create this queue. So my idea was to >>> stay with original odp_queue_create() app API. >>> >>> Thanks, >>> Maxim. >>> >>> >>> >>> >>> >>> >>> _______________________________________________ >>> lng-odp mailing list >>> [email protected] <mailto:[email protected]> >>> http://lists.linaro.org/mailman/listinfo/lng-odp >>> >>> >>> >> > > _______________________________________________ > lng-odp mailing list > [email protected] > http://lists.linaro.org/mailman/listinfo/lng-odp > >
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
