Thanks for bringing it up Mike. One queue per process is still kept as a question in the document. If we agree we need to update the document.
Regards, Bala On 26 August 2014 17:58, Mike Holmes <[email protected]> wrote: > > > > On 26 August 2014 08:21, Bala Manoharan <[email protected]> wrote: > >> Hi, >> >> Few points from my side, >> >> 1. We are creating pktio and then assigning the created queue to the >> pktio. IMO we should abstract the PKTIO, since it is up to the >> implementation to either support IPC through dummy interface or shared >> memory. Lets say if an implementation does IPC through shared memory there >> is no need for any PKTIO initialization. >> >> 2. We need to think about the design of how many receive queues are there >> per process, do we follow the design of one receive queue per process which >> means all the processes which wants to send message to process-A always >> enqueues into a queue named queue-process-A >> > I thought we had already settled on one queue per process, do we need to > clarify the requirements in > https://docs.google.com/a/linaro.org/document/d/1bC0XHNGQMMtZq6aYaUaWU-tYi_DgOl4ePQ65XRxDOx8/edit#heading=h.kskoz435lvo2 > ? > >> >> 3. Regarding synchronization, IMO if we agree that there will be only one >> receive queue per process, we can create IPC queues based on standard >> names, and when a process which needs IPC mechanism registers we can create >> the receive queue for that specific process and the process can issue >> find_IPC_queue() function using the standardised process name and get the >> queue handle for process to which it wants to send the message. >> >> Regards, >> Bala >> >> >> On 26 August 2014 17:37, Ola Liljedahl <[email protected]> wrote: >> >>> Sharing queues between threads that share memory is not a big problem. >>> But sharing queues between different programs/processes which do not want >>> to share memory (e.g. between a control plane program and one or several >>> dataplane applications) is a problem. It is not just the lookup of the >>> queue identifier. Depending on the ODP implementation, some memory might >>> have to be shared between these different processes even if the user code >>> is (and wants to be) unaware of this. An implementation might get away from >>> sharing memory in this case if either the HW can do the copy or if the ODP >>> implementation calls the Linux kernel to perform the copy of buffers >>> (messages). >>> >>> >>> On 26 August 2014 14:03, Maxim Uvarov <[email protected]> wrote: >>> >>>> On 08/26/2014 03:13 PM, Ola Liljedahl wrote: >>>> >>>>> Possibly we are missing a public API for looking up a queue (or >>>>> perhaps specifically an IPC queue) by name. >>>>> >>>>> If you have two threads (in same or different processes) that first >>>>> try to look up an (IPC or global) queue by name and if this fails then >>>>> creates said IPC queue, you have introduced a race condition. Possibly the >>>>> second IPC queue create will fail because the named queue exists (EEXIST) >>>>> and the caller can then understand it does not have to create the queue. >>>>> >>>>> So I think we have some synchronization issues to understand and solve >>>>> when it comes to IPC. Because the primary use case for IPC will be for >>>>> communication between different programs (processes) so they have no means >>>>> for synchronization (I would rather not have to use other Linux mechanisms >>>>> for setting up an IPC channel). >>>>> >>>>> Ola, I agree with that. We have function to look up for pool, but >>>> don't have that function to loop up for queue. But it might be common case >>>> where different threads want to reference to single queue. >>>> >>>> odp_queue_t odp_queue_lookup(const char *name, odp_queue_type_t type) >>>> >>>> which will do: >>>> - walk over queue_tbl and compare name with argument, if name match >>>> return it; >>>> - if type ipc, search for shared memory object, if it's exist connect >>>> to it. >>>> >>>> Maxim. >>>> >>>> >>>> >>>>> On 26 August 2014 11:48, Maxim Uvarov <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> On 08/26/2014 12:56 PM, Taras Kondratiuk wrote: >>>>> >>>>> On 08/25/2014 05:37 PM, Maxim Uvarov wrote: >>>>> >>>>> I pushed here some raw code: >>>>> https://git.linaro.org/people/ >>>>> maxim.uvarov/odp.git/shortlog/refs/heads/odp_ipc2 >>>>> >>>>> >>>>> >>>>> Example here app here: >>>>> https://git.linaro.org/people/maxim.uvarov/odp.git/blob/ >>>>> refs/heads/odp_ipc2:/example/fork/odp_fork.c >>>>> >>>>> >>>>> >>>>> pktio_queue_thread() takes packets from inq queue and put >>>>> them to ipc >>>>> queue. >>>>> >>>>> ring_thread() takes packets from ipc queue and free this >>>>> buffer. >>>>> >>>>> >>>>> I have a few things I didn't understand: >>>>> 1. It is not clear how IPC pktio and its pool is used. >>>>> Especially on >>>>> 'sender' side. >>>>> >>>>> >>>>> We create pool somewhere. Then do loop up for it: >>>>> pkt_pool = odp_buffer_pool_lookup("packet_pool"); >>>>> >>>>> Then we want that IPC queue can work with that pool. So we open >>>>> pktio: >>>>> >>>>> pktio_ipc_params.type = ODP_PKTIO_TYPE_IPC; >>>>> pktio_ipc = odp_pktio_open(NULL, pkt_pool, &pktio_ipc_params); >>>>> >>>>> And then create queue: >>>>> >>>>> ipcq_def = odp_queue_create("shared-queue", ODP_QUEUE_TYPE_IPC, >>>>> &qparam); >>>>> >>>>> when we place packet to this queue with: >>>>> odp_queue_enq(ipcq_def, buf); >>>>> >>>>> Than everything depend on implementation. So linux-generic >>>>> (software), packet data >>>>> still will be in pool. odp_buffer_t (uint32_t) value will be added >>>>> to ring. >>>>> >>>>> Other process gets odp_buffer_t, and find pointer to that buffer >>>>> (odp_buffer_hdr_t*). >>>>> >>>>> On hw implementation you just place odp_buffer_t to the queue. And >>>>> if ipc queue has the same >>>>> pool than packet can be delivered to software. If there are >>>>> different pools, then packet should be >>>>> copied to ipc pool, and after that delivered to hardware. >>>>> >>>>> The main goal here is we can say that ipc pool can be the same as >>>>> ingress pool or it can be different. >>>>> Depends on hw configuration. >>>>> >>>>> >>>>> 2. On both sides IPC queue is created. My impression was that >>>>> it should >>>>> be looked up on one side via pktio or directly via queue name. >>>>> >>>>> >>>>> Is there way to ask hardware which queues were already created? >>>>> >>>>> Actually I do look up, but it's hidden in implementation. Idea is >>>>> that if queue type is IPC, >>>>> then I do loop up if that queue already exist. If exist then I >>>>> connect to that queue. If it does >>>>> not exist - I create it. >>>>> >>>>> static void queue_init(queue_entry_t *queue, const char *name, >>>>> odp_queue_type_t type, odp_queue_param_t *param) >>>>> >>>>> case ODP_QUEUE_TYPE_IPC: >>>>> queue->s.r = odp_ring_lookup(name); >>>>> >>>>> if (!queue->s.r) >>>>> { >>>>> printf("creating new odp ring: %s\n", name); >>>>> queue->s.r = odp_ring_create(name, RING_SIZE, >>>>> ODP_RING_SHM_PROC); >>>>> if (queue->s.r == NULL) { >>>>> ODP_ERR("ring create failed\n"); >>>>> } >>>>> } else >>>>> printf("odp ring found!!!\n"); >>>>> >>>>> Because of we can say that several processes can connect to one >>>>> queue, there is no difference >>>>> which exactly process will create this queue. So my idea was to >>>>> stay with original odp_queue_create() app API. >>>>> >>>>> Thanks, >>>>> Maxim. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> lng-odp mailing list >>>>> [email protected] <mailto:[email protected]> >>>>> http://lists.linaro.org/mailman/listinfo/lng-odp >>>>> >>>>> >>>>> >>>> >>> >>> _______________________________________________ >>> lng-odp mailing list >>> [email protected] >>> http://lists.linaro.org/mailman/listinfo/lng-odp >>> >>> >> >> _______________________________________________ >> lng-odp mailing list >> [email protected] >> http://lists.linaro.org/mailman/listinfo/lng-odp >> >> > > > -- > *Mike Holmes* > Linaro Technical Manager / Lead > LNG - ODP >
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
