On 08/26/2014 04:21 PM, Bala Manoharan wrote:
Hi,
Few points from my side,
1. We are creating pktio and then assigning the created queue to the
pktio. IMO we should abstract the PKTIO, since it is up to the
implementation to either support IPC through dummy interface or shared
memory. Lets say if an implementation does IPC through shared memory
there is no need for any PKTIO initialization.
pktio is needed for ipc thread as well. pkio describes where packet data
is stored, while queue has only references for buffers in the pool. In
my patch I just connect packet i/o entry with pool:
odp_pktio_t odp_pktio_open(const char *dev, odp_buffer_pool_t pool,
odp_pktio_params_t *params)
case ODP_PKTIO_TYPE_IPC:
pktio_entry->s.pool = pool;
break;
The main goal is to make queue visible by different problem. And avoid
coping packet body. So reference to original pool is needed.
2. We need to think about the design of how many receive queues are
there per process, do we follow the design of one receive queue per
process which means all the processes which wants to send message to
process-A always enqueues into a queue named queue-process-A
As Mike replied we staid on one queue per process. That is better fit to
hardware implementation. How ever in software we are more flexible
(odp_ring has multiply consumers/producers).
3. Regarding synchronization, IMO if we agree that there will be only
one receive queue per process, we can create IPC queues based on
standard names, and when a process which needs IPC mechanism registers
we can create the receive queue for that specific process and the
process can issue find_IPC_queue() function using the standardised
process name and get the queue handle for process to which it wants to
send the message.
yep, I will implement that. For now it's hidden to platform
implementation. But it's more right to use queue_lookup().
Maxim.
Regards,
Bala
On 26 August 2014 17:37, Ola Liljedahl <[email protected]
<mailto:[email protected]>> wrote:
Sharing queues between threads that share memory is not a big
problem. But sharing queues between different programs/processes
which do not want to share memory (e.g. between a control plane
program and one or several dataplane applications) is a problem.
It is not just the lookup of the queue identifier. Depending on
the ODP implementation, some memory might have to be shared
between these different processes even if the user code is (and
wants to be) unaware of this. An implementation might get away
from sharing memory in this case if either the HW can do the copy
or if the ODP implementation calls the Linux kernel to perform the
copy of buffers (messages).
On 26 August 2014 14:03, Maxim Uvarov <[email protected]
<mailto:[email protected]>> wrote:
On 08/26/2014 03:13 PM, Ola Liljedahl wrote:
Possibly we are missing a public API for looking up a
queue (or perhaps specifically an IPC queue) by name.
If you have two threads (in same or different processes)
that first try to look up an (IPC or global) queue by name
and if this fails then creates said IPC queue, you have
introduced a race condition. Possibly the second IPC queue
create will fail because the named queue exists (EEXIST)
and the caller can then understand it does not have to
create the queue.
So I think we have some synchronization issues to
understand and solve when it comes to IPC. Because the
primary use case for IPC will be for communication between
different programs (processes) so they have no means for
synchronization (I would rather not have to use other
Linux mechanisms for setting up an IPC channel).
Ola, I agree with that. We have function to look up for pool,
but don't have that function to loop up for queue. But it
might be common case where different threads want to reference
to single queue.
odp_queue_t odp_queue_lookup(const char *name,
odp_queue_type_t type)
which will do:
- walk over queue_tbl and compare name with argument, if name
match return it;
- if type ipc, search for shared memory object, if it's exist
connect to it.
Maxim.
On 26 August 2014 11:48, Maxim Uvarov
<[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>> wrote:
On 08/26/2014 12:56 PM, Taras Kondratiuk wrote:
On 08/25/2014 05:37 PM, Maxim Uvarov wrote:
I pushed here some raw code:
https://git.linaro.org/people/maxim.uvarov/odp.git/shortlog/refs/heads/odp_ipc2
Example here app here:
https://git.linaro.org/people/maxim.uvarov/odp.git/blob/refs/heads/odp_ipc2:/example/fork/odp_fork.c
pktio_queue_thread() takes packets from inq
queue and put
them to ipc
queue.
ring_thread() takes packets from ipc queue and
free this
buffer.
I have a few things I didn't understand:
1. It is not clear how IPC pktio and its pool is used.
Especially on
'sender' side.
We create pool somewhere. Then do loop up for it:
pkt_pool = odp_buffer_pool_lookup("packet_pool");
Then we want that IPC queue can work with that pool.
So we open
pktio:
pktio_ipc_params.type = ODP_PKTIO_TYPE_IPC;
pktio_ipc = odp_pktio_open(NULL, pkt_pool,
&pktio_ipc_params);
And then create queue:
ipcq_def = odp_queue_create("shared-queue",
ODP_QUEUE_TYPE_IPC,
&qparam);
when we place packet to this queue with:
odp_queue_enq(ipcq_def, buf);
Than everything depend on implementation. So linux-generic
(software), packet data
still will be in pool. odp_buffer_t (uint32_t) value
will be added
to ring.
Other process gets odp_buffer_t, and find pointer to
that buffer
(odp_buffer_hdr_t*).
On hw implementation you just place odp_buffer_t to
the queue. And
if ipc queue has the same
pool than packet can be delivered to software. If
there are
different pools, then packet should be
copied to ipc pool, and after that delivered to hardware.
The main goal here is we can say that ipc pool can be
the same as
ingress pool or it can be different.
Depends on hw configuration.
2. On both sides IPC queue is created. My
impression was that
it should
be looked up on one side via pktio or directly via
queue name.
Is there way to ask hardware which queues were already
created?
Actually I do look up, but it's hidden in
implementation. Idea is
that if queue type is IPC,
then I do loop up if that queue already exist. If
exist then I
connect to that queue. If it does
not exist - I create it.
static void queue_init(queue_entry_t *queue, const
char *name,
odp_queue_type_t type,
odp_queue_param_t *param)
case ODP_QUEUE_TYPE_IPC:
queue->s.r = odp_ring_lookup(name);
if (!queue->s.r)
{
printf("creating new odp ring: %s\n", name);
queue->s.r = odp_ring_create(name, RING_SIZE,
ODP_RING_SHM_PROC);
if (queue->s.r == NULL) {
ODP_ERR("ring create failed\n");
}
} else
printf("odp ring found!!!\n");
Because of we can say that several processes can
connect to one
queue, there is no difference
which exactly process will create this queue. So my
idea was to
stay with original odp_queue_create() app API.
Thanks,
Maxim.
_______________________________________________
lng-odp mailing list
[email protected] <mailto:[email protected]>
<mailto:[email protected]
<mailto:[email protected]>>
http://lists.linaro.org/mailman/listinfo/lng-odp
_______________________________________________
lng-odp mailing list
[email protected] <mailto:[email protected]>
http://lists.linaro.org/mailman/listinfo/lng-odp
_______________________________________________
lng-odp mailing list
[email protected]
http://lists.linaro.org/mailman/listinfo/lng-odp
