On 28 October 2014 12:20, Maxim Uvarov <[email protected]> wrote:
> Signed-off-by: Maxim Uvarov <[email protected]> > --- > ipc.dox | 219 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 219 insertions(+) > create mode 100644 ipc.dox > > diff --git a/ipc.dox b/ipc.dox > new file mode 100644 > index 0000000..93ecaa0 > --- /dev/null > +++ b/ipc.dox > @@ -0,0 +1,219 @@ > +/* Copyright (c) 2014, Linaro Limited > + * All rights reserved > + * > + * SPDX-License-Identifier: BSD-3-Clause > + */ > + > +/** > +@page ipc_design Inter Process Communication API > @page ipc_design Inter Process Communication (IPC) API Do we need to say API, our doc is starting to be full of heading that say API and I expect readers realise this already ? > + > +@tableofcontents > + > +@section ipc_intro Introduction > + This document defines different ODP applications modes such as > +multi threading and multiprocessing and IPC APIs for that modes ODP. > This document defines the two different ODP application modes multithreading and multiprocessing with respect to their impact on IPC > + > +@subsection odp_modes Application Thread/Process modes: > + ODP applications can use following programming models for multi core > support: > + -# Single application with ODP worker Threads. > + -# Multi process application with single packet I/O pool and common > initialization. > + -# Different processed communicated thought IPC API. > + > Is there any diagram that would illuminate the relationships and differences clearly ? > +@subsubsection odp_mode_threads Thread mode > + Initialization sequence in thread mode is following: > The Initialization sequence for thread mode is following: > + > +@verbatim > + main() { > + /* Init ODP before calling anything else. */ > + odp_init_global(NULL, NULL); > + > + /* Init this thread. */ > + odp_init_local(); > + > + /* Allocate memory for packets pool. That memory will be > visible for all threads.*/ > + shm = odp_shm_reserve("shm_packet_pool", > + SHM_PKT_POOL_SIZE, > ODP_CACHE_LINE_SIZE, 0); > + pool_base = odp_shm_addr(shm); > + > + /* Create pool instance with reserved shm. */ > + pool = odp_buffer_pool_create("packet_pool", pool_base, > + SHM_PKT_POOL_SIZE, > + SHM_PKT_POOL_BUF_SIZE, > + ODP_CACHE_LINE_SIZE, > + ODP_BUFFER_TYPE_PACKET); > + > + /* Create worker threads. */ > + odph_linux_pthread_create(&thread_tbl[i], 1, core, > thr_run_func, > + &args); > + } > + > + //// thread code > What does //// mean ? > + thr_run_func () { > + /* Lookup the packet pool */ > + pkt_pool = odp_buffer_pool_lookup("packet_pool"); > + > + /* Open a packet IO instance for this thread */ > + pktio = odp_pktio_open("eth0", pkt_pool); > + > + for (;;) { > + /* read buffer */ > + buf = odp_schedule(NULL, ODP_SCHED_WAIT); > + ... do something ... > + } > + } > +@endverbatim > + > Can the above be in odp/example and the relevant text imported by doxygen for inclusion above to ensure the example always works ? > +@subsubsection odp_mode_processes Processes mode with shared memory > + Initialization sequence in processes mode with shared memory is > following: > + > +@verbatim > + main() { > + /* Init ODP before calling anything else. In process mode > odp_init_global > + * function called only once in main run process. > + */ > + odp_init_global(NULL, NULL); > + > + /* Init this thread. */ > + odp_init_local(); > + > + /* Allocate memory for packets pool. That memory will be > visible for all threads.*/ > + shm = odp_shm_reserve("shm_packet_pool", > + SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, > 0); > + pool_base = odp_shm_addr(shm); > + > + /* Create pool instance with reserved shm. */ > + pool = odp_buffer_pool_create("packet_pool", pool_base, > + SHM_PKT_POOL_SIZE, > + SHM_PKT_POOL_BUF_SIZE, > + ODP_CACHE_LINE_SIZE, > + ODP_BUFFER_TYPE_PACKET); > + > + /* Call odph_linux_process_fork_n which will fork() > current process to > + * different processes. > + */ > + odph_linux_process_fork_n(proc, num_workers, first_core); > + > + /* Run same function as thread uses */ > + thr_run_func(); > + } > + > + //// thread code > + thr_run_func () { > + /* Lookup the packet pool */ > + pkt_pool = odp_buffer_pool_lookup("packet_pool"); > + > + /* Open a packet IO instance for this thread */ > + pktio = odp_pktio_open("eth0", pkt_pool); > + > + for (;;) { > + /* read buffer */ > + buf = odp_schedule(NULL, ODP_SCHED_WAIT); > + ... do something ... > + } > + } > +@endverbatim > + > > +@subsubsection odp_mode_sep_processes Separate Processes mode > + This mode differs from mode with common shared memory. Each process > is completely separate process which calls > I think you needs to not define process by saying process Each execution thread (execution object from event machine ? ) is a completely independent process which calls > +odp_init_global() and do other initialization process then open IPC pktio > interface and do packets exchange between processes. > To communicate between these independent processes the IPC pktio interface may be used to exchange packets. > +For base implementation (linux-generic) shared memory is used for IPC > mechanism to make it flexible to port to different use > For the base implementation (linux-generic) shared memory is used as the IPC mechanism to make it easy to reuse for different use > +cases. Which can be different processes, processes in different VM's or > any bare metal applications which can share memory > The use cases are process that may be spread amongst different VMs, bare metal or regular Linux user space, in fact any process that can share memory > +between each other. In hardware implementation IPC pktio can be offloaded > to HW SoC packets functions. > In hardware implementations IPC > + Initialization sequence in separate processes mode is same as for > threads or processes with shared memory but with following > The Initialization sequence in the separate thread mode model is same as it is process, both using shared memory but with following + difference: > + > +@subsubsection odp_mode_sep_processes_cons Separate Processes Sender > (linux-generic) > + -# Each process calls odp_init_global(), pool creation and etc. > + > + -# ODP_SHM_PROC flag provided to be able to map that memory from > different process. > + > + shm = odp_shm_reserve("shm_packet_pool", > + SHM_PKT_POOL_SIZE, > + ODP_CACHE_LINE_SIZE, > + ODP_SHM_PROC); > + > + pool_base = odp_shm_addr(shm); > + > + -# Worker thread (or process) creates IPC pktio, and sends buffers > to it: > + > + A) > + odp_pktio_t ipc_pktio = odp_pktio_open("ipc_pktio", 0); > + odp_pktio_send(ipc_pktio, pkt_tbl, pkts); > + > + B) instead of using packet io queue can be used in following way: > + > +@verbatim > + odp_queue_t ipcq = odp_pktio_outq_getdef(ipc_pktio); > + /* Then enqueue the packet for output queue */ > + odp_queue_enq(ipcq, buf); > +@endverbatim > + > +@subsubsection odp_mode_sep_processes_recv Separate Processes Receiver > (linux-generic) > + On the other end process also creates IPC packet I/O and receives > packets > + from it. > + > +@verbatim > + /* Create packet pool visible by only second process. We will copy > + * packets to that queue from IPC shared memory. > + */ > + shm = odp_shm_reserve("local_packet_pool", > + SHM_PKT_POOL_SIZE, ODP_CACHE_LINE_SIZE, 0); > + > + pool_base = odp_shm_addr(shm); > + pool = odp_buffer_pool_create("ipc_packet_pool", pool_base, > + SHM_PKT_POOL_SIZE, > + SHM_PKT_POOL_BUF_SIZE, > + ODP_CACHE_LINE_SIZE, > + ODP_BUFFER_TYPE_PACKET); > + > + pool_base = NULL; > + /* Loop to find remote shared pool */ > + while (1) { > + shm = odp_shm_reserve("shm_packet_pool", > + SHM_PKT_POOL_SIZE, > + ODP_CACHE_LINE_SIZE, > + ODP_SHM_PROC_NOCREAT); <- > ODP_SHM_PROC_NOCREAT flag provided to > + not create shared memory > object, do only lookup. > + pool_base = odp_shm_addr(shm); > + if (pool_base != NULL) { > + break; > + } else { > + ODP_DBG("looking up for shm_packet_pool\n"); > + sleep(1); > + } > + } > + > + > + /* Do lookup packet I/O in IPC shared memory, > + * and link it to local pool. */ > + while (1) { > + pktio = odp_pktio_lookup("ipc_pktio", pool, pool_base); > + if (pktio == ODP_PKTIO_INVALID) { > + sleep(1); > + printf("pid %d: looking for ipc_pktio\n", > getpid()); > + continue; > + } > + break; > + } > + > + /* Get packets from the IPC */ > + for (;;) { > + pkts = odp_pktio_recv(pktio, pkt_tbl, MAX_PKT_BURST); > + ... > + } > +@endverbatim > + > +@subsubsection odp_mode_sep_processes_hw Separate Processes Hardware > optimized > + Hardware SoC implementation of IPC exchange can differ. It can use > share pool > Hardware SoC implementation of IPC exchange can differ. It can use a shared pool > + or can relay on hardware for packet transmission. But the API > interface remains the > or it can rely on the hardware for packet transmission. But the API interface remains the + same: > + > + odp_pktio_open(), odp_pktio_lookup() > + > + > +@subsubsection odp_ipc_future_updates IPC future update and rework: > > + -# odp_buffer_pool_create() API will change to allocate memory for > pool inside it. + So odp_shm_reserve() for remote pool memory and odp_pktio_lookup() > can go inside > + odp_buffer_pool_create(). > + > +*/ > -- > 1.8.5.1.163.gd7aced9 > > > _______________________________________________ > lng-odp mailing list > [email protected] > http://lists.linaro.org/mailman/listinfo/lng-odp > -- *Mike Holmes* Linaro Sr Technical Manager LNG - ODP
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
