[lng-odp] [Bug 2494] Extend odp_pktio_capability_t to include minimal pool size required by pktio implementation.

[bugzilla-daemon]

https://bugs.linaro.org/show_bug.cgi?id=2494

--- Comment #8 from Maciej Czekaj  ---
(In reply to Bala Manoharan from comment #7)
> Regards,
> Bala
> 
> 
> On 3 November 2016 at 19:36,   wrote:
> > https://bugs.linaro.org/show_bug.cgi?id=2494
> >
> > --- Comment #6 from Maciej Czekaj  ---
> > (In reply to Bala Manoharan from comment #5)
> >> Application portability across multiple platforms can not be done without
> >> any modifications. However I do not have an issue is adding the minimal 
> >> pool
> >> size as capability in the capability struct but in the point 2 of your
> >> initial suggestion states that pool size be given by the implementation
> >> based on the buffer requirements, since the pools are actually created by
> >> the implementation and the application gives buffer count and pool type
> >> can't this be internally handled by the implementation without the
> >> additional API.
> >>
> >> If we are on agreement here I can send a proposal to modify the capability
> >> struct.
> >
> > Yes, application specifies and creates a pool, so user needs to manually
> > account for increased pool consumption.
> >
> > Also, it is
> > nb_rx_queues*(rx_queue_depth+alloc_queue_depth) + 
> > nb_tx_queues*tx_queue_depth.
> >
> > In case of ThunderX the formula is
> >
> > (nb_rx_queues+nb_tx_queues)*1024 + ceil(nb_rx_queues/8)*8192
> >
> >
> > Now the challenge is that the buffer consumption has to be known before
> > odp_pktio_open() since that call consumes the buffer pool... but there is no
> > call to be made on pktio before odp_pktio_open() because this is a factory
> > function for pktio objects. The best way would be to:
> >
> > 1.create pktio object without a pool
> > 2.config the number of queues
> > 3.ask about capabilities , including buffer needs
> > 4.create the pool with application specified size + internal consumption 
> > size
> > taken from capability
> > 5.start the interface
> >
> > Adding the pool in the first call to pktio complicates the whole thing, as 
> > it
> > is not possible to alter the pool size later and it is not possible to query
> > the pktio pool consumption before odp_pktio_open().
> 
> The pools are configured based on "num" packets of size "len" or
> smaller which it has to support so if a pool is configured to support
> 1024 packets of size 512K then it can only support 512 packets of size
> 1024K. The pools are linked at pktio level and not at queue level the
> maximum number of packets which it has to support across all queue in
> a pktio is "num".

> 
> Since the pktio capability comes after the pktio_open(), so couldn't
> the application expose the maximum queue which it can support based on
> pool configured with pktio interface.




What do you mean by "application exposing the maximum queue"?
Currently, application specifies how many buffers is needed by "num" as you
pointed out. Do you mean something else?


> 
> >
> >>
> >> The real question I would like to understand is the expectation of this
> >> minimal buffer size lets say if the application requires 1024
> >> packets/buffers in a pktio interface and the minimal buffer size is 512 
> >> then
> >> should be pool be configured with (1024 + 512) buffers and can the 512
> >> buffer required by pktio interface be used for storing application packets?
> >
> > Yes, the buffers are shared btw internal queues and application, because 
> > they
> > are really used for internal queuing of packets in HW, e.g. the contain:
> >  -  packets waiting for reception in RX queue
> >  -  packets waiting for transmission in TX queue
> >  -  empty packets allocated for future reception
> >
> > If the driver is under heavy load, all those queues may be filled up and 
> > lead
> > to buffer starvation that is difficult to recover from and it certainly 
> > limits
> > the performance since lack of buffers leads to dropping traffic.
> 
> Packets being dropped coz of buffer starvation is expected and
> acceptable under heavy load and usually this is handled using Random
> Early Discard when the buffer in the pool reaches a threshold limit.
>


Packet drops under heavy load are expected but a small memory pool may cause
packet drops not due to heavy load but due to buffer shortage itself. Thus
application may be half-idle and still packets may be dropped. This is an
artificial shortage caused by configuration. 

At the very least it should be possible to detect it and at best it should be
possible to prevent it by making sure the pool is big enough. 

Detection could be done by returning an error code or adding extra statistics.

Prevention is achieved if the API allows for discovering the sufficient pool
size.


> >
> > --
> > You are receiving this mail because:
> > You are on the CC list for the bug.

-- 
You are receiving this mail because:
You are on the CC list for the bug.

Re: [lng-odp] continuous memory allocation for drivers

[Maciej Czekaj]

On Fri, 11 Nov 2016 11:13:27 +0100 Francois Ozog wrote:

On 11 November 2016 at 10:10, Brian Brooks  wrote:


On 11/10 18:52:49, Christophe Milard wrote:

Hi,

My hope was that packet segments would all be smaller than one page
(either normal pages or huge pages)

When is this the case? With a 4096 byte page, a couple 1518 byte Ethernet
packets can fit. A 9038 byte Jumbo wont fit.


[FF] WHen you allocate a queue with 256 packets for Intel, Vritio, Mellanox
cards, you need a small area of 256 descriptors that fit in a page. Then
drivers allocate 256 buffers in a contiguous memory. This leads to 512K of
buffers. They may allocate this zone per packet though. But for high
performance cards such as Chelsio and Netcope: this is a strict requirement
because the obtained memory zone is managed by HW: packet allocation is not
controlled by software. You give the zone to hardware wich places packets
the way it wants in the zone. HW informs the SW where the packets are by
updating the ring. VFIO does not change the requirement of a large
contiguous area.
PCIexpress has a limitation of 36M DMA transactions per second. Which is
lower than 60Mpps required for 40Gbps and much lower than 148Mpps required
for 100Gbps. The only way to achieve line rate is to fit more than one
packet in a DMA transaction. That's what Chelsio, Netcope and others are
doing. This requires HW controlled memory allocations. This requires large
memory blocks to be supplied to HW.
As we move forawrd, I expect all cards to adopt a similar scheme and escape
the "Intel" Model of IO.
Now if we look at performance, cost of managing virt_to_phys() even in
kernel for each packet is preventing spread allocations. You amortize the
cost by getting the physical address of the 256 buffer zone, and using
offsets from that to get the physical address of an individual packet. If
you try to do that in userland, then just use linux networking stack, it
will be faster ;-)




I second Francois here. H/W requires physically contiguous memory at 
least for DMA queues.
A queue well exceeds the size of a page or even huge page in some cases. 
E.g. ThunderX has a single
DMA queue of size 512K and it may even have a 4M queue which is beyond a 
2M huge page on ARM when 4K page is default.
64K pages are preferable and they are safer as the huge page is then 
512M but it may be too much for some clients.


vfio-pci only partially solves that problem because of 2 reasons:

1. In the guest environment there is no vfio-pci, or at least there is 
no extra mapping that could be done.
   From VM perspective all DMA memory should be contiguous with respect 
to Intermediate Virtual Address (under ARM nomenclature).


2. IOMMU mappings are not for free. In synthetic benchmarks there is no 
difference in performance due to IOMMU
   but if the system is using IOMMU extensively, e.g. due to many VMs, 
it may well prove otherwise.
   IOTLB miss has similar cost to TLB miss. Ideally, a system 
integrator should have a choice to use it or not.





Or is it to ease the memory manager by having a logical array of objects
laid out in virtual memory space and depending on the number of objects
and the size of each object, a few are bound to span across 2 pages which
might not be adjacent in physical memory?

Or is it when the view of a packet is a bunch of packet segments which
may be of varying sizes and possibly scattered across in memory and the
packet needs to go out the wire?

Are 2M, 16M, 1G page size used?


to guarantee physical memory
continuity which is needed by some drivers (read non vfio drivers for
PCI).

[FF] linux kernel uses a special allocator for that, huge pages are not

the unit. As said above, some HW require large contiguous blocks and vfio
or iommu does not avoid the requirement.



If IOMMU enables IO device the same virtual addressing as the CPU by
sharing page tables, would ever a IO device or IOMMU have limitations
on the number of pages supported or other performance limitations
during the VA->PA translation?

[FF] no information on that



Does the IOMMU remap interrupts from the IO device when the vm
migrates cores? What happens when no irq remapping, does core get
irq and must interprocessorinterrupt core where vm is now running?

[FF] I hope not.



Are non vfio drivers for PCI needing contiguous physical memory the
design target?


[FF] not related to VFIO but related to HW requirements.



Francois Ozog's experience (with dpdk)shows that this hope will fail
in some case: not all platforms support the required huge page size.
And it would be nice to be able to run even in the absence of huge
pages.

I am therefore planning to expand drvshm to include a flag requesting
contiguous physical memory. But sadly, from user space, this is
nothing we can guarantee... So when this flag is set, the allocator
will allocate untill physical memory "happens to be continuous".
This is a bit like the DPDK approach (try & error), which I dislike,

[lng-odp] [Bug 2571] AES GCM decryption is broken

[bugzilla-daemon]

https://bugs.linaro.org/show_bug.cgi?id=2571

Mike Holmes  changed:

   What|Removed |Added

 Resolution|--- |FIXED
 CC||mike.hol...@linaro.org
 Status|IN_PROGRESS |RESOLVED

--- Comment #3 from Mike Holmes  ---
f09c70a

-- 
You are receiving this mail because:
You are on the CC list for the bug.

[lng-odp] Plain text test post

[Mike Holmes]

-- 
Mike Holmes
Program Manager - Linaro Networking Group
Linaro.org │ Open source software for ARM SoCs
"Work should be fun and collaborative, the rest follows"

Re: [lng-odp] Plain text test post

[Bill Fischofer]

That post showed up for me.

On Thu, Nov 17, 2016 at 8:30 AM, Mike Holmes  wrote:

> --
> Mike Holmes
> Program Manager - Linaro Networking Group
> Linaro.org │ Open source software for ARM SoCs
> "Work should be fun and collaborative, the rest follows"
>

Re: [lng-odp] concat and split semantics

[Bill Fischofer]

On Thu, Nov 17, 2016 at 9:42 AM, Joe Savage  wrote:

> Hey,
>
> I have recently been looking into the suitability of the ODP APIs for the
> fragmentation and reassembly of IP packets. Particularly, I've been focused
> on the split and concat APIs — using the former to break larger packets
> into
> smaller fragments, and the latter to glue such fragments back together.
>
> I'm not entirely sure to what the degree these APIs were designed to
> support
> this kind of use case, though. Does the specification allow, for example,
> for
> "zero-copy" implementations of split and concat, in which the operations
> act
> purely on packet metadata (segments etc.) rather than the packet data
> itself?
>

ODP specifies functional behavior of it's APIs. Implementations are free to
choose whatever techniques or other internals they wish to realize this
behavior. So things like lazy/deferred/batched operations are entirely
possible as long as the external semantics are matched. The odp-linux
reference implementation currently uses copy operations internally to
realize these operations but that's not part of the specification.


>
> And more broadly, if it is possible, is this the kind of behaviour that one
> should expect from a good implementation? I'm not sure of the exact
> purposes
> for which concat and split were designed, but it may be the case that this
> kind of zero-copy ideal is outside the primary use case for these APIs, and
> hence goes against the grain of what many implementations will do.
>

odp-linux is designed to provide a simple and clear reference
implementation of the ODP APIs. These are not necessarily optimized for
performance. One of the reasons split/concat were implemented the way they
are is that we did not have
more sophisticated internal structures available at the time these were
originally implemented. Improvement patches in this and other areas are
always welcome.

Ideally these and other packet operations would make use of HW features
present on platforms optimized for packet processing. That's why ODP
permits such implementation flexibility.


>
> It's also interesting to contrast the semantics of concat and split to the
> splice/reference API proposals, which more explicitly indicate that the
> operations need not be implemented via costly copies. Do people have a good
> idea of when it might, on a purely semantic level, be a better idea to use
> concat rather than, say, nested references?
>

The key difference between split/concat and references is that the latter
imposes usage restrictions on the end-product while the former do not. So
the "shared" portion of references should be treated as read/only by the
application. Also note that references are still under development and we
do not yet have consensus on these semantics to say that all ODP platforms
are OK with implementing them as currently defined. Feedback on that is
still ongoing.


>
> Thanks,
>
> Joe
>

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Bill Fischofer]

For this series:

Reviewed-and-tested-by: Bill Fischofer 

On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
petri.savolai...@nokia.com> wrote:

> Pool performance is optimized by using a ring as the global buffer storage.
> IPC build is disabled, since it needs large modifications due to
> dependency to
> pool internals. Old pool implementation was based on locks and linked list
> of
> buffer headers. New implementation maintain a ring of buffer handles, which
> enable fast, burst based allocs and frees. Also ring scales better with
> number
> of cpus than a list (enq and deq operations update opposite ends of the
> pool).
>
> L2fwd link rate (%), 2 x 40GE, 64 byte packets
>
> direct- parallel-   atomic-
> cpusorigdirect  difforigparall  difforigatomic
> diff
> 1   7 % 8 % 1 % 6 % 6 % 2 % 5.4 %   5.6 %   4 %
> 2   14 %15 %7 % 9 % 9 % 5 % 8 % 9 % 8 %
> 4   28 %30 %6 % 13 %14 %13 %12 %15 %19
> %
> 6   42 %44 %6 % 16 %19 %19 %8 % 20 %
> 150 %
> 8   46 %59 %28 %19 %23 %26 %18 %24 %34
> %
> 10  55 %57 %3 % 20 %27 %37 %8 % 28 %
> 264 %
> 12  56 %56 %-1 %22 %31 %43 %7 % 32 %
> 357 %
>
> Max packet rate of NICs are reached with 10-12 cpu on direct mode.
> Otherwise,
> all cases were improved. Especially, scheduler driven cases suffered on bad
> pool scalability.
>
> changed in v3:
> * rebased
> * ipc disabled with #ifdef
> * added support for multi-segment packets
> * API: added explicit limits for packet length in alloc calls
> * Corrected validation test and example application bugs found during
>   segmentation implementation
>
> changed in v2:
> * rebased to api-next branch
> * added a comment that ring size must be larger than number of items in it
> * fixed clang build issue
> * added parens in align macro
>
> v1 reviews:
> Reviewed-by: Brian Brooks 
>
>
>
> Petri Savolainen (19):
>   linux-gen: ipc: disable build of ipc pktio
>   linux-gen: pktio: do not free zero packets
>   linux-gen: ring: created common ring implementation
>   linux-gen: align: added round up power of two
>   linux-gen: pool: reimplement pool with ring
>   linux-gen: ring: added multi enq and deq
>   linux-gen: pool: use ring multi enq and deq operations
>   linux-gen: pool: optimize buffer alloc
>   linux-gen: pool: clean up pool inlines functions
>   linux-gen: pool: ptr instead of hdl in buffer_alloc_multi
>   test: validation: buf: test alignment
>   test: performance: crypto: use capability to select max packet
>   test: correctly initialize pool parameters
>   test: validation: packet: fix bugs in tailroom and concat tests
>   linux-gen: packet: added support for segmented packets
>   test: validation: packet: improved multi-segment alloc test
>   api: packet: added limits for packet len on alloc
>   linux-gen: packet: remove zero len support from alloc
>   linux-gen: packet: enable multi-segment packets
>
>  example/generator/odp_generator.c  |2 +-
>  include/odp/api/spec/packet.h  |9 +-
>  include/odp/api/spec/pool.h|6 +
>  platform/linux-generic/Makefile.am |1 +
>  .../include/odp/api/plat/packet_types.h|6 +-
>  .../include/odp/api/plat/pool_types.h  |6 -
>  .../linux-generic/include/odp_align_internal.h |   34 +-
>  .../linux-generic/include/odp_buffer_inlines.h |  167 +--
>  .../linux-generic/include/odp_buffer_internal.h|  120 +-
>  .../include/odp_classification_datamodel.h |2 +-
>  .../linux-generic/include/odp_config_internal.h|   55 +-
>  .../linux-generic/include/odp_packet_internal.h|   87 +-
>  platform/linux-generic/include/odp_pool_internal.h |  289 +---
>  platform/linux-generic/include/odp_ring_internal.h |  176 +++
>  .../linux-generic/include/odp_timer_internal.h |4 -
>  platform/linux-generic/odp_buffer.c|   22 +-
>  platform/linux-generic/odp_classification.c|   25 +-
>  platform/linux-generic/odp_crypto.c|   12 +-
>  platform/linux-generic/odp_packet.c|  717 --
>  platform/linux-generic/odp_packet_io.c |2 +-
>  platform/linux-generic/odp_pool.c  | 1440
> 
>  platform/linux-generic/odp_queue.c |4 +-
>  platform/linux-generic/odp_schedule.c  |  102 +-
>  platform/linux-generic/odp_schedule_ordered.c  |4 +-
>  platform/linux-generic/odp_timer.c |3 +-
>  platform/linux-generic/pktio/dpdk.c|   10 +-
>  platform/linux-generic/pktio/ipc.c |3 +-
>  platform/linux-generic/pktio/loop.c|2 

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Bill Fischofer]

Done. My v2 patch applies on top of your v3. Still need your review on the
APIs for packet references.

On Tue, Nov 15, 2016 at 8:50 AM, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia-bell-labs.com> wrote:

>
> <2nd reply to HTML mail>
>
> Actually, the definition of '#define ODP_UNUSED(x) (void)(x)'
> implementation was never discussed. We just wrapped
> "__attribute__((__unused__))" of GCC. That's why I'm questioning it here
> now. What if your compiler X checks for unused params but does not support
> the same or similar attribute *inside* function prototype ?
>
> We could also remove ODP_UNUSED from the API if cast to (void) is a valid
> C language feature.
>
> Anyway, for this series I'll send a v4 with ODP_UNUSED.
>
> 
>
> If you wish. Alternately I can fold that into my patch that goes on top of
> this one and I can review your series as-is since it otherwise looks fine
> and clearly is something we want.
>
> <3rd reply to HTML mail>
>
> That's OK of course :) Thanks.
>
> -Petri
>

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Francois Ozog]

ODP is about abstracting things so that applications can concentrate on the
logic.

Exposing drivers on this northbound interface contradicts the essence of
ODP.

ODP implementers will decide how they will accomodate driver "additions".
One may even decide to retrieve drivers through TFTP ;-)

If we say there are:
- a device framework
- a driver framework

You can expose driver loading interface as part of the driver framework so
that implementers have a standard way to "activate" drivers that they have
retrieved the way they think appropriate.

The device framework will leverage the driver framework to associate
devices and drivers.

FF

PS: dpdk-app  -- 

On 16 November 2016 at 12:49, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> Command line parameters will go to the application. As far as I know, we
> have no ODP parameters at this time, and they would require some command
> line processing cooperation with the application (main(argv, argc)) is in
> the app, we are just a lib). That put apps using getopts in a bad position.
> argparse is really the command line parser which thought about subparsing,
> but is not always used.
> Command line parameters also assume a command :-). If ODP comes with a
> shell-less system this becomes less obvious.
> I agree that in some cases (command line from linux) it would be clearer,
> but...
>
> Scanning a directory makes assumption on the directory structure, unless
> we can specify that directory from the north interface (and then, aren't we
> back more or less to the initial proposal?), or command line.
>
> I have hard to see the problem with enabling driver loading from the north
> interface: The apps which needs a specific driver (because they run on a
> specific HW) have a chance to load what they need.
> I agree a set of default driver would make sense, and possibly command
> line args if they come into picture. But dynamic loading make sense to me:
> both command line args and directory scan implies init-only driver loading,
> right? (of course on linux one could receive an event on directory write...)
>
> In dpdk, many drivers come with dpdk. Possibly we could do that too, and
> that could reduce the usage on the driver_load function...
>
> Christophe
>
>
>
> On 16 November 2016 at 12:20, Francois Ozog 
> wrote:
>
>> Why not just scanning a directory or give a command line parameter?
>>
>> On 16 November 2016 at 12:05, Christophe Milard <
>> christophe.mil...@linaro.org> wrote:
>>
>>> So what?
>>> If we don't allow to load a driver from the north interface, where
>>> should they be loaded from??
>>> Or are we saying that we go for a predefined list of driver only?
>>>
>>> So drivers have to be .so files (because of autotools), but we don't
>>> give the possibility to load a free driver??
>>>
>>> Christophe
>>>
>>> On 16 November 2016 at 11:45, Francois Ozog 
>>> wrote:
>>>
 Hello,

 If the north API is the one visible by ODP applications then I don't
 think it is a good idea to expose that.
 DPDK exposed it at the beginning and is now internal.

 That said there must be a standard way to manage drivers fir the
 benefit of the device framework.

 I don't think the idea of integrating it with packetio open because
 packetio is not really a device. It is an abstract way of dealing with
 ports which may be hosted on a device (NIC) or on the platform (SoC).
 It can be done as a PoC (that's what I do with virtio-net exploratiry
 project) but that is not a long term solution.

 FF


 Le mercredi 16 novembre 2016, Christophe Milard <
 christophe.mil...@linaro.org> a écrit :

> On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
>  wrote:
> >
> >>  /**
> >> + * Driver loading
> >> + *
> >> + * This function is used by the application to load NIC drivers
> into ODP.
> >> + * Calls to this function are optional, but should be performed
> (if any)
> >> + * after odp_init_global
> >> + *
> >> + *
> >> + * @param filenameDriver shared lib (dynamic library)
> >> + *
> >> + * @retval 0 on success
> >> + * @retval <0 on failure
> >> + *
> >> + */
> >> +int odp_load_driver(const char *filename);
> >
> > Why application should explicitly need to load a driver ? Why it
> cannot happen as part of odp_pktio_open() ?
> >
> > To me a better approach would be:
> > * user reads implementation documentation or uses system commands to
> find a list of available interfaces
> > * odp_global_init() finds out available pktio interfaces and drivers
> for those
> > * application opens the interface the user commands (just like today)
> > * pktio_open call loads the driver
> >
> > Or with more ODP support:
> > * odp_global_init() finds out available 

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Bill Fischofer]

Trying to reproduce this I'm seeing sporadic failures in the scheduler
validation test that don't seem to appear in the base api-next branch.
Issue seems to be failures in the ordered queue tests:

  Test: scheduler_test_multi_mq_mt_prio_n
...linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
started as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
passed
  Test: scheduler_test_multi_mq_mt_prio_a
...linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
started as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
passed
  Test: scheduler_test_multi_mq_mt_prio_o
...linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
started as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
linux.c:273:odpthread_run_start_routine():helper: ODP worker thread started
as linux pthread. (pid=6274)
FAILED
1. scheduler.c:871  - bctx->sequence == seq
2. scheduler.c:871  - bctx->sequence == seq
  Test: scheduler_test_multi_1q_mt_a_excl
...linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
started as linux pthread. (pid=6274)

We had seen these earlier but they were never consistently reproducible.
Petri: are you able to recreate this on your local systems?


On Wed, Nov 16, 2016 at 2:03 PM, Maxim Uvarov 
wrote:

> I can not test patch by patch this series because it fails (one time it
> was TM, one time kernel died, other time OOM killer killed tests then hang
> kernel).
>
> And for all patches test/common_plat/validation/api/pktio/pktio_main
> hangs forever:
>
>
> Program received signal SIGINT, Interrupt.
> 0x2afbe69ffb80 in __nanosleep_nocancel () at
> ../sysdeps/unix/syscall-template.S:81
> 81in ../sysdeps/unix/syscall-template.S
> (gdb) bt
> #0  0x2afbe69ffb80 in __nanosleep_nocancel () at
> ../sysdeps/unix/syscall-template.S:81
> #1  0x00415ced in odp_pktin_recv_tmo (queue=...,
> packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
> wait=wait@entry=18446744073709551615) at
> ../../../platform/linux-generic/odp_packet_io.c:1584
> #2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
> pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
> seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1, 
> mode=mode@entry=RECV_TMO,
> tmo=tmo@entry=18446744073709551615, ns=ns@entry=0)
> at ../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
> #3  0x004075f8 in test_recv_tmo (mode=RECV_TMO) at
> ../../../../../../test/common_plat/validation/api/pktio/pktio.c:940
> #4  0x2afbe61cc482 in run_single_test () from
> /usr/local/lib/libcunit.so.1
> #5  0x2afbe61cc0b2 in run_single_suite () from
> /usr/local/lib/libcunit.so.1
> #6  0x2afbe61c9d55 in CU_run_all_tests () from
> /usr/local/lib/libcunit.so.1
> #7  0x2afbe61ce245 in basic_run_all_tests () from
> /usr/local/lib/libcunit.so.1
> #8  0x2afbe61cdfe7 in CU_basic_run_tests () from
> /usr/local/lib/libcunit.so.1
> #9  0x00409361 in odp_cunit_run () at
> ../../../../test/common_plat/common/odp_cunit_common.c:298
> #10 0x2afbe6c2ff45 in __libc_start_main (main=0x403850 , argc=1,
> argv=0x7ffed64d9878, init=,
> fini=, rtld_fini=,
> stack_end=0x7ffed64d9868) at libc-start.c:287
> #11 0x0040387e in _start ()
> (gdb) up
> #1  0x00415ced in odp_pktin_recv_tmo (queue=...,
> packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
> wait=wait@entry=18446744073709551615) at
> ../../../platform/linux-generic/odp_packet_io.c:1584
> 1584nanosleep(, NULL);
> (gdb) p ts
> $1 = {tv_sec = 0, tv_nsec = 1000}
> (gdb) l
> 1579}
> 1580
> 1581wait--;
> 1582}
> 1583
> 1584nanosleep(, NULL);
> 1585}
> 1586}
> 1587
> 1588int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[],
> unsigned num_q,
> (gdb) up
> #2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
> pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
> seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1, 
> mode=mode@entry=RECV_TMO,
> tmo=tmo@entry=18446744073709551615, ns=ns@entry=0)
> at ../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
> 515n = 

[lng-odp] [API-NEXT PATCHv7 01/13] linux-gen: _ishm: create description file for external memory sharing

[Christophe Milard]

A new flag called _ODP_ISHM_EXPORT is added to _ishm. When this flag is
specified at reserve() time, an extra file
("/tmp/odp--shm-", where  is the process ID of the
main ODP instatiation process and  is the block name given at
reserve time) is created, describing to the underlying block attributes.
This file is meant to be used by processes external to ODP willing to
share this memory.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/_ishm.c  | 74 ++---
 platform/linux-generic/include/_ishm_internal.h |  1 +
 2 files changed, 68 insertions(+), 7 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index f4aa6d3..9018fb9 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -99,6 +99,14 @@
 #define ISHM_FILENAME_NORMAL_PAGE_DIR "/tmp"
 
 /*
+ * when the memory is to be shared with an external entity (such as another
+ * ODP instance or an OS process not part of this ODP instance) then a
+ * export file is created describing the exported memory: this defines the
+ * location and the filename format of this description file
+ */
+#define ISHM_EXPTNAME_FORMAT "/tmp/odp-%d-shm-%s"
+
+/*
  * At worse case the virtual space gets so fragmented that there is
  * a unallocated fragment between each allocated fragment:
  * In that case, the number of fragments to take care of is twice the
@@ -107,6 +115,17 @@
 #define ISHM_NB_FRAGMNTS (ISHM_MAX_NB_BLOCKS * 2 + 1)
 
 /*
+ * when a memory block is to be exported outside its ODP instance,
+ * an block 'attribute file' is created in /tmp/odp--shm-.
+ * The information given in this file is according to the following:
+ */
+#define EXPORT_FILE_LINE1_FMT "ODP exported shm block info:"
+#define EXPORT_FILE_LINE2_FMT "ishm_blockname: %s"
+#define EXPORT_FILE_LINE3_FMT "file: %s"
+#define EXPORT_FILE_LINE4_FMT "length: %" PRIu64
+#define EXPORT_FILE_LINE5_FMT "flags: %" PRIu32
+#define EXPORT_FILE_LINE6_FMT "align: %" PRIu32
+/*
  * A fragment describes a piece of the shared virtual address space,
  * and is allocated only when allocation is done with the _ODP_ISHM_SINGLE_VA
  * flag:
@@ -136,6 +155,7 @@ typedef struct ishm_fragment {
 typedef struct ishm_block {
char name[ISHM_NAME_MAXLEN];/* name for the ishm block (if any) */
char filename[ISHM_FILENAME_MAXLEN]; /* name of the .../odp-* file  */
+   char exptname[ISHM_FILENAME_MAXLEN]; /* name of the export file */
int  main_odpthread; /* The thread which did the initial reserve*/
uint32_t user_flags; /* any flags the user want to remember.*/
uint32_t flags;  /* block creation flags.   */
@@ -380,7 +400,8 @@ static void free_fragment(ishm_fragment_t *fragmnt)
  * or /mnt/huge/odp-- (for huge pages)
  * Return the new file descriptor, or -1 on error.
  */
-static int create_file(int block_index, int huge, uint64_t len)
+static int create_file(int block_index, int huge, uint64_t len,
+  uint32_t flags, uint32_t align)
 {
char *name;
int  fd;
@@ -388,6 +409,7 @@ static int create_file(int block_index, int huge, uint64_t 
len)
char seq_string[ISHM_FILENAME_MAXLEN];   /* used to construct filename*/
char filename[ISHM_FILENAME_MAXLEN];/* filename in /tmp/ or /mnt/huge */
int  oflag = O_RDWR | O_CREAT | O_TRUNC; /* flags for open*/
+   FILE *export_file;
 
new_block = _tbl->block[block_index];
name = new_block->name;
@@ -429,9 +451,48 @@ static int create_file(int block_index, int huge, uint64_t 
len)
 
strncpy(new_block->filename, filename, ISHM_FILENAME_MAXLEN - 1);
 
+   /* if _ODP_ISHM_EXPORT is set, create a description file for
+* external ref:
+*/
+   if (flags & _ODP_ISHM_EXPORT) {
+   snprintf(new_block->exptname, ISHM_FILENAME_MAXLEN,
+ISHM_EXPTNAME_FORMAT,
+odp_global_data.main_pid,
+(name && name[0]) ? name : seq_string);
+   export_file = fopen(new_block->exptname, "w");
+   if (export_file == NULL) {
+   ODP_ERR("open failed: err=%s.\n",
+   strerror(errno));
+   new_block->exptname[0] = 0;
+   } else {
+   fprintf(export_file, EXPORT_FILE_LINE1_FMT "\n");
+   fprintf(export_file, EXPORT_FILE_LINE2_FMT "\n",  name);
+   fprintf(export_file, EXPORT_FILE_LINE3_FMT "\n",
+   new_block->filename);
+   fprintf(export_file, EXPORT_FILE_LINE4_FMT "\n", len);
+   fprintf(export_file, EXPORT_FILE_LINE5_FMT "\n", flags);
+   fprintf(export_file, EXPORT_FILE_LINE6_FMT "\n", align);
+
+   fclose(export_file);
+ 

[lng-odp] [API-NEXT PATCHv7 04/13] linux-gen: Push internal flag definition

[Christophe Milard]

File platform/linux-generic/include/odp_shm_internal.h exposes shm
internals used by IPC. The bits used by the internal flags are moved
to make room for more "official" values.
The platform/linux-generic/include/odp_shm_internal.h file should really
be removed when _ishm is used, but as long as we have the current IPC,
removing the file would break compilation.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/include/odp_shm_internal.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/platform/linux-generic/include/odp_shm_internal.h 
b/platform/linux-generic/include/odp_shm_internal.h
index 30e60f7..8bd105d 100644
--- a/platform/linux-generic/include/odp_shm_internal.h
+++ b/platform/linux-generic/include/odp_shm_internal.h
@@ -16,8 +16,8 @@ extern "C" {
 #define SHM_DEVNAME_MAXLEN (ODP_SHM_NAME_LEN + 16)
 #define SHM_DEVNAME_FORMAT "/odp-%d-%s" /* /dev/shm/odp-- */
 
-#define _ODP_SHM_PROC_NOCREAT 0x4  /**< Do not create shm if not exist */
-#define _ODP_SHM_O_EXCL  0x8  /**< Do not create shm if exist */
+#define _ODP_SHM_PROC_NOCREAT 0x40  /**< Do not create shm if not exist */
+#define _ODP_SHM_O_EXCL  0x80  /**< Do not create shm if exist */
 
 #ifdef __cplusplus
 }
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv7 09/13] linux-gen: shm: add flag and function to share memory between ODP instances

[Christophe Milard]

Implemented by calling the related functions from _ishm.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/odp_shared_memory.c | 31 +++---
 1 file changed, 28 insertions(+), 3 deletions(-)

diff --git a/platform/linux-generic/odp_shared_memory.c 
b/platform/linux-generic/odp_shared_memory.c
index 2377f16..d2bb74c 100644
--- a/platform/linux-generic/odp_shared_memory.c
+++ b/platform/linux-generic/odp_shared_memory.c
@@ -24,6 +24,21 @@ static inline odp_shm_t to_handle(uint32_t index)
return _odp_cast_scalar(odp_shm_t, index + 1);
 }
 
+static uint32_t get_ishm_flags(uint32_t flags)
+{
+   uint32_t f = 0; /* internal ishm flags */
+
+   /* set internal ishm flags according to API flags:
+* note that both ODP_SHM_PROC and ODP_SHM_EXPORT maps to
+* _ODP_ISHM_LINK as in the linux-gen implementation there is
+* no difference between exporting to another ODP instance or
+* another linux process */
+   f |= (flags & (ODP_SHM_PROC | ODP_SHM_EXPORT)) ? _ODP_ISHM_EXPORT : 0;
+   f |= (flags & ODP_SHM_SINGLE_VA) ? _ODP_ISHM_SINGLE_VA : 0;
+
+   return f;
+}
+
 int odp_shm_capability(odp_shm_capability_t *capa)
 {
memset(capa, 0, sizeof(odp_shm_capability_t));
@@ -41,9 +56,7 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, 
uint64_t align,
int block_index;
int flgs = 0; /* internal ishm flags */
 
-   /* set internal ishm flags according to API flags: */
-   flgs |= (flags & ODP_SHM_PROC) ? _ODP_ISHM_EXPORT : 0;
-   flgs |= (flags & ODP_SHM_SINGLE_VA) ? _ODP_ISHM_SINGLE_VA : 0;
+   flgs = get_ishm_flags(flags);
 
/* all mem reserved through this interface is requested to be locked: */
flgs |= (flags & _ODP_ISHM_LOCK);
@@ -55,6 +68,18 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, 
uint64_t align,
return ODP_SHM_INVALID;
 }
 
+odp_shm_t odp_shm_import(const char *remote_name,
+odp_instance_t odp_inst,
+const char *local_name)
+{
+   int ret;
+
+   ret =  _odp_ishm_find_exported(remote_name, (pid_t)odp_inst,
+  local_name);
+
+   return to_handle(ret);
+}
+
 int odp_shm_free(odp_shm_t shm)
 {
return _odp_ishm_free_by_index(from_handle(shm));
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv7 07/13] test: api: shmem: new proper tests for shm API

[Christophe Milard]

The shmem "sunnydays" tests for the north interface API are replaced with
proper tests, testing memory allocation at different time (before and
after ODP thread creation, i.e. the tests make sure shmem behaves
the same regardless of fork time). The tests also include stress testing
trying to provoque race conditions. The new shmem tests do not assume
pthreads any longer and are runnable in process mode.

Signed-off-by: Christophe Milard 
---
 test/common_plat/validation/api/shmem/shmem.c | 687 --
 test/common_plat/validation/api/shmem/shmem.h |   5 +-
 2 files changed, 658 insertions(+), 34 deletions(-)

diff --git a/test/common_plat/validation/api/shmem/shmem.c 
b/test/common_plat/validation/api/shmem/shmem.c
index cbff673..6ea92d9 100644
--- a/test/common_plat/validation/api/shmem/shmem.c
+++ b/test/common_plat/validation/api/shmem/shmem.c
@@ -7,82 +7,703 @@
 #include 
 #include 
 #include "shmem.h"
+#include 
 
-#define ALIGE_SIZE  (128)
-#define TESTNAME "cunit_test_shared_data"
+#define ALIGN_SIZE  (128)
+#define MEM_NAME "test_shmem"
+#define NAME_LEN (sizeof(MEM_NAME) + 20)
 #define TEST_SHARE_FOO (0xf0f0f0f0)
 #define TEST_SHARE_BAR (0xf0f0f0f)
+#define SMALL_MEM 10
+#define MEDIUM_MEM 4096
+#define BIG_MEM 65536
+#define STRESS_SIZE 32 /* power of 2 and <=256 */
+#define STRESS_RANDOM_SZ 5
+#define STRESS_ITERATION 5000
 
-static odp_barrier_t test_barrier;
+typedef enum {
+   STRESS_FREE, /* entry is free and can be allocated */
+   STRESS_BUSY, /* entry is being processed: don't touch */
+   STRESS_ALLOC /* entry is allocated  and can be freed */
+} stress_state_t;
 
-static int run_shm_thread(void *arg ODP_UNUSED)
+typedef struct {
+   stress_state_t state;
+   odp_shm_t shm;
+   char name[NAME_LEN];
+   void *address;
+   uint32_t flags;
+   uint32_t size;
+   uint64_t align;
+   uint8_t data_val;
+} stress_data_t;
+
+typedef struct {
+   odp_barrier_t test_barrier1;
+   odp_barrier_t test_barrier2;
+   odp_barrier_t test_barrier3;
+   odp_barrier_t test_barrier4;
+   uint32_t foo;
+   uint32_t bar;
+   odp_atomic_u32_t index;
+   uint32_t nb_threads;
+   odp_shm_t shm[MAX_WORKERS];
+   void *address[MAX_WORKERS];
+   char name[MAX_WORKERS][NAME_LEN];
+   odp_spinlock_t  stress_lock;
+   stress_data_t stress[STRESS_SIZE];
+} shared_test_data_t;
+
+/* memory stuff expected to fit in a single page */
+typedef struct {
+   int data[SMALL_MEM];
+} shared_test_data_small_t;
+
+/* memory stuff expected to fit in a huge page */
+typedef struct {
+   int data[MEDIUM_MEM];
+} shared_test_data_medium_t;
+
+/* memory stuff expected to fit in many huge pages */
+typedef struct {
+   int data[BIG_MEM];
+} shared_test_data_big_t;
+
+/*
+ * thread part for the shmem_test_basic test
+ */
+static int run_test_basic_thread(void *arg ODP_UNUSED)
 {
odp_shm_info_t  info;
odp_shm_t shm;
-   test_shared_data_t *test_shared_data;
+   shared_test_data_t *shared_test_data;
int thr;
 
-   odp_barrier_wait(_barrier);
thr = odp_thread_id();
printf("Thread %i starts\n", thr);
 
-   shm = odp_shm_lookup(TESTNAME);
+   shm = odp_shm_lookup(MEM_NAME);
CU_ASSERT(ODP_SHM_INVALID != shm);
-   test_shared_data = odp_shm_addr(shm);
-   CU_ASSERT(TEST_SHARE_FOO == test_shared_data->foo);
-   CU_ASSERT(TEST_SHARE_BAR == test_shared_data->bar);
+   shared_test_data = odp_shm_addr(shm);
+   CU_ASSERT(NULL != shared_test_data);
+
+   odp_barrier_wait(_test_data->test_barrier1);
+   odp_shm_print_all();
+   CU_ASSERT(TEST_SHARE_FOO == shared_test_data->foo);
+   CU_ASSERT(TEST_SHARE_BAR == shared_test_data->bar);
CU_ASSERT(0 == odp_shm_info(shm, ));
-   CU_ASSERT(0 == strcmp(TESTNAME, info.name));
+   CU_ASSERT(0 == strcmp(MEM_NAME, info.name));
CU_ASSERT(0 == info.flags);
-   CU_ASSERT(test_shared_data == info.addr);
-   CU_ASSERT(sizeof(test_shared_data_t) <= info.size);
-#ifdef MAP_HUGETLB
-   CU_ASSERT(odp_sys_huge_page_size() == info.page_size);
-#else
-   CU_ASSERT(odp_sys_page_size() == info.page_size);
-#endif
+   CU_ASSERT(shared_test_data == info.addr);
+   CU_ASSERT(sizeof(shared_test_data_t) <= info.size);
+   CU_ASSERT((info.page_size == odp_sys_huge_page_size()) ||
+ (info.page_size == odp_sys_page_size()))
odp_shm_print_all();
 
fflush(stdout);
return CU_get_number_of_failures();
 }
 
-void shmem_test_odp_shm_sunnyday(void)
+/*
+ * test basic things: shmem creation, info, share, and free
+ */
+void shmem_test_basic(void)
 {
pthrd_arg thrdarg;
odp_shm_t shm;
-   test_shared_data_t *test_shared_data;
+   shared_test_data_t *shared_test_data;
odp_cpumask_t unused;
 
-   shm = odp_shm_reserve(TESTNAME,
-   

[lng-odp] [API-NEXT PATCHv7 13/13] doc: updating docs for the shm interface extension

[Christophe Milard]

Signed-off-by: Christophe Milard 
---
 doc/users-guide/users-guide.adoc | 68 ++--
 1 file changed, 66 insertions(+), 2 deletions(-)

diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc
index 62f5833..078dd7c 100755
--- a/doc/users-guide/users-guide.adoc
+++ b/doc/users-guide/users-guide.adoc
@@ -649,13 +649,19 @@ mapping the shared memory block. There is no 
fragmentation.
 By default ODP threads are assumed to behave as cache coherent systems:
 Any change performed on a shared memory block is guaranteed to eventually
 become visible to other ODP threads sharing this memory block.
-(this behaviour may be altered by flags to `odp_shm_reserve()` in the future).
 Nevertheless, there is no implicit memory barrier associated with any action
 on shared memories: *When* a change performed by an ODP thread becomes visible
 to another ODP thread is not known: An application using shared memory
 blocks has to use some memory barrier provided by ODP to guarantee shared data
 validity between ODP threads.
 
+The virtual address at which a given memory block is mapped in different ODP
+threads may differ from ODP thread to ODP thread, if ODP threads have separate
+virtual spaces (for instance if ODP threads are implemented as processes).
+However, the ODP_SHM_SINGLE_VA flag can be used at `odp_shm_reserve()` time
+to guarantee address uniqueness in all ODP threads, regardless of their
+implementation or creation time.
+
 === Lookup by name
 As mentioned, shared memory handles can be sent from ODP threads to ODP
 threads using any IPC mechanism, and then the block address retrieved.
@@ -698,9 +704,49 @@ if (odp_shm_free(shm) != 0) {
 }
 
 
+=== sharing memory with the external world
+ODP provides ways of sharing memory with entities located outside
+ODP instances:
+
+Sharing a block of memory with an external (non ODP) thread is achieved
+by setting the ODP_SHM_PROC flag at `odp_shm_reserve()` time.
+How the memory block is retrieved on the Operating System side is
+implementation and Operating System dependent.
+
+Sharing a block of memory with an external ODP instance (running
+on the same Operating System) is achieved
+by setting the ODP_SHM_EXPORT flag at `odp_shm_reserve()` time.
+A block of memory created with this flag in an ODP instance A, can be "mapped"
+into a remote ODP instance B (on the same OS) by using the
+`odp_shm_import()`, on ODP instance B:
+
+.sharing memory between ODP instances: instance A
+[source,c]
+
+odp_shm_t shmA;
+shmA = odp_shm_reserve("memoryA", size, 0, ODP_SHM_EXPORT);
+
+
+.sharing memory between ODP instances: instance B
+[source,c]
+
+odp_shm_t shmB;
+odp_instance_t odpA;
+
+/* get ODP A instance handle by some OS method */
+odpA = ...
+
+/* get the shared memory exported by A:
+shmB = odp_shm_import("memoryA", odpA, "memoryB", 0, 0);
+
+
+Note that the handles shmA and shmB are scoped by each ODP instance
+(you can not use them outside the ODP instance they belong to).
+Also note that both ODP instances have to call `odp_shm_free()` when done.
+
 === Memory creation flags
 The last argument to odp_shm_reserve() is a set of ORed flags.
-Two flags are supported:
+The following flags are supported:
 
  ODP_SHM_PROC
 When this flag is given, the allocated shared memory will become visible
@@ -710,6 +756,12 @@ will be able to access the memory using native (non ODP) 
OS calls such as
 Each ODP implementation should provide a description on exactly how
 this mapping should be done on that specific platform.
 
+ ODP_SHM_EXPORT
+When this flag is given, the allocated shared memory will become visible
+to other ODP instances running on the same OS.
+Other ODP instances willing to see this exported memory should use the
+`odp_shm_import()` ODP function.
+
  ODP_SHM_SW_ONLY
 This flag tells ODP that the shared memory will be used by the ODP application
 software only: no HW (such as DMA, or other accelerator) will ever
@@ -719,6 +771,18 @@ implementation), except for `odp_shm_lookup()` and 
`odp_shm_free()`.
 ODP implementations may use this flag as a hint for performance optimization,
 or may as well ignore this flag.
 
+ ODP_SHM_SINGLE_VA
+This flag is used to guarantee the uniqueness of the address at which
+the shared memory is mapped: without this flag, a given memory block may be
+mapped at different virtual addresses (assuming the target have virtual
+addresses) by different ODP threads. This means that the value returned by
+`odp_shm_addr()` would be different in different threads, in this case.
+Setting this flag guarantees that all ODP threads sharing this memory
+block will see it at the same address (`odp_shm_addr()` would return the
+same value on all ODP threads, for a given memory block, in this case)
+Note that ODP implementations may have restrictions of the amount of memory
+which can be allocated with this flag.
+
 == Queues
 Queues are the 

[lng-odp] [PATCH] test: linux-gen: mmap_vlan_ins change bin to test programs

[Maxim Uvarov]

Generated binaries should be marked as test programs to be
installed by autotools in test directory.

Signed-off-by: Maxim Uvarov 
---
 test/linux-generic/mmap_vlan_ins/Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/linux-generic/mmap_vlan_ins/Makefile.am 
b/test/linux-generic/mmap_vlan_ins/Makefile.am
index 2641556..5cac159 100644
--- a/test/linux-generic/mmap_vlan_ins/Makefile.am
+++ b/test/linux-generic/mmap_vlan_ins/Makefile.am
@@ -7,7 +7,7 @@ dist_check_SCRIPTS = vlan.pcap \
 
 test_SCRIPTS = $(dist_check_SCRIPTS)
 
-bin_PROGRAMS = plat_mmap_vlan_ins$(EXEEXT)
+test_PROGRAMS = plat_mmap_vlan_ins$(EXEEXT)
 plat_mmap_vlan_ins_LDFLAGS = $(AM_LDFLAGS) -static
 plat_mmap_vlan_ins_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/example
 
-- 
2.7.1.250.gff4ea60

Re: [lng-odp] [API-NEXT PATCHv2 2/2] linux-gen: _ishmphy: adding debug function for pysical address mapping

[Christophe Milard]

On 14 November 2016 at 06:02, Yi He  wrote:
> One comment inline:
>
> On 11 November 2016 at 22:50, Christophe Milard
>  wrote:
>>
>> _odp_ishmphy_memmap_print() prints the virtual to physical address
>> mapping of some memory region (defined by virtuall address + length).
>>
>> Signed-off-by: Christophe Milard 
>> ---
>>  platform/linux-generic/_ishmphy.c  | 58
>> ++
>>  platform/linux-generic/include/_ishmphy_internal.h |  5 ++
>>  2 files changed, 63 insertions(+)
>>
>> diff --git a/platform/linux-generic/_ishmphy.c
>> b/platform/linux-generic/_ishmphy.c
>> index 8c0f46e..21f6bd1 100644
>> --- a/platform/linux-generic/_ishmphy.c
>> +++ b/platform/linux-generic/_ishmphy.c
>> @@ -265,3 +265,61 @@ int _odp_ishmphy_can_virt_to_phys(void)
>>
>> return 1;
>>  }
>> +
>> +/*
>> + * dump the physical mapping from virtual address addr, length len.
>> + */
>> +void _odp_ishmphy_memmap_print(void *curr_addr, uint64_t len)
>> +{
>> +   int page_sz;
>> +   phys_addr_t curr_phy;
>> +   phys_addr_t start_phy;
>> +   void *start_address = 0;
>> +   uint32_t curr_len = 0;
>> +   uint32_t pfn_count = 0;
>> +
>> +   /* get normal page sizes: */
>> +   page_sz = odp_sys_page_size();
>> +
>> +   curr_phy = _odp_ishmphy_virt_to_phys(curr_addr);
>> +   if (curr_phy == PHYS_ADDR_INVALID) {
>> +   ODP_DBG("Phy Dump failed (permission?).\n");
>> +   return;
>> +   }
>> +
>> +   ODP_DBG("Phy Dump:\n");
>> +   start_address = curr_addr;
>> +   start_phy = curr_phy;
>> +
>> +   curr_len += page_sz;
>> +   curr_addr = (void *)((char *)curr_addr + page_sz);
>> +   pfn_count++;
>> +   while (curr_len < len) {
>> +   if (_odp_ishmphy_virt_to_phys(curr_addr) ==
>> +   curr_phy + page_sz) {
>> +   curr_len += page_sz;
>> +   curr_addr = (void *)((char *)curr_addr + page_sz);
>> +   curr_phy += page_sz;
>> +   pfn_count++;
>> +   continue;
>> +   }
>> +
>> +   ODP_DBG("Virtual: %08" PRIx64 " <-->  Phy: %08" PRIx64
>> +   "   %" PRIu32 " PFNs, %" PRIu32 " bytes\n",
>> +   (uint64_t)start_address, start_phy,
>> +   pfn_count, pfn_count * page_sz);
>> +   curr_phy =  _odp_ishmphy_virt_to_phys(curr_addr);
>
>
> Should here check the success or failure of the call? And break in case
> failed.

hmmm. not sure. The fact that the function is available, that is that
you have the right permission, is tested at start of function.
Now that you have the right permissions, it is likely you want to see
what the function returns (even in case of error), as this is a debug
function anyway.
If the function said just "error", I guess the next debug step would
be to see what lead to the error... i.e. see the real value.
But both behaviour make sense. so If you really prefer a test, I can
put one :-)

Christophe.

>
>>
>> +   start_address = curr_addr;
>> +   start_phy = curr_phy;
>> +   pfn_count = 0;
>> +   curr_len += page_sz;
>> +   curr_addr = (void *)((char *)curr_addr + page_sz);
>> +   pfn_count++;
>> +   }
>> +
>> +   if (pfn_count)
>> +   ODP_DBG("Virtual: %08" PRIx64 " <-->  Phy: %08" PRIx64
>> +   "   %" PRIu32 " PFNs, %" PRIu32 " bytes\n",
>> +   (uint64_t)start_address, start_phy,
>> +   pfn_count, pfn_count * page_sz);
>> +}
>> diff --git a/platform/linux-generic/include/_ishmphy_internal.h
>> b/platform/linux-generic/include/_ishmphy_internal.h
>> index 2022590..c8752c0 100644
>> --- a/platform/linux-generic/include/_ishmphy_internal.h
>> +++ b/platform/linux-generic/include/_ishmphy_internal.h
>> @@ -32,6 +32,11 @@ int _odp_ishmphy_can_virt_to_phys(void);
>>   */
>>  phys_addr_t _odp_ishmphy_virt_to_phys(const void *addr);
>>
>> +/*
>> + * dump the physical mapping from virtual address addr, length len.
>> + */
>> +void _odp_ishmphy_memmap_print(void *addr, uint64_t len);
>> +
>>  #ifdef __cplusplus
>>  }
>>  #endif
>> --
>> 2.7.4
>>
>

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Bill Fischofer]

On Tue, Nov 15, 2016 at 6:20 AM, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia-bell-labs.com> wrote:

>
>
> From: Bill Fischofer [mailto:bill.fischo...@linaro.org]
> Sent: Monday, November 14, 2016 3:37 AM
> To: Savolainen, Petri (Nokia - FI/Espoo)  labs.com>
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added
> support for segmented packets
>
>
>
> On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
> petri.savolai...@nokia.com> wrote:
> Added support for multi-segmented packets. The first segments
> is the packet descriptor, which contains all metadata and
> pointers to other segments.
>
> Signed-off-by: Petri Savolainen 
> ---
>  .../include/odp/api/plat/packet_types.h|   6 +-
>  .../linux-generic/include/odp_buffer_inlines.h |  11 -
>  .../linux-generic/include/odp_buffer_internal.h|  23 +-
>  .../linux-generic/include/odp_config_internal.h|  39 +-
>  .../linux-generic/include/odp_packet_internal.h|  80 +--
>  platform/linux-generic/include/odp_pool_internal.h |   3 -
>  platform/linux-generic/odp_buffer.c|   8 +-
>  platform/linux-generic/odp_crypto.c|   8 +-
>  platform/linux-generic/odp_packet.c| 712
> +
>  platform/linux-generic/odp_pool.c  | 123 ++--
>  platform/linux-generic/pktio/netmap.c  |   4 +-
>  platform/linux-generic/pktio/socket.c  |   3 +-
>  12 files changed, 692 insertions(+), 328 deletions(-)
>
>
>
>
>
> diff --git a/platform/linux-generic/odp_packet.c
> b/platform/linux-generic/odp_packet.c
> index 2eee775..a5c6ff4 100644
> --- a/platform/linux-generic/odp_packet.c
> +++ b/platform/linux-generic/odp_packet.c
> @@ -20,12 +20,155 @@
>  #include 
>  #include 
>
> -/*
> - *
> - * Alloc and free
> - * 
> - *
> - */
> +static inline odp_packet_t packet_handle(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return (odp_packet_t)pkt_hdr->buf_hdr.handle.handle;
> +}
> +
> +static inline odp_buffer_t buffer_handle(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return pkt_hdr->buf_hdr.handle.handle;
> +}
> +
> +static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr,
> + uint32_t seg_idx)
> +{
> +   return pkt_hdr->buf_hdr.seg[seg_idx].len;
> +}
> +
> +static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t
> seg_idx)
>
> This is more usefully typed as returning uint8_t * rather than void * to
> permit easy arithmetic on the result. The uint8_t * converts automatically
> to void * returns for the external API calls that use this.
>
>
> 
>
> This is actually the only place packet_seg_data() is called. The API
> function returns (void *), so the inline function fits that.
>
> void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg)
> {
> odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);
>
> if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount))
> return NULL;
>
> return packet_seg_data(pkt_hdr, seg);
> }
>
> Also (void *) can be assigned to any pointer type without cast. For
> example:
>
> uint8_t *data;
> data = packet_seg_data(pkt_hdr, seg);
>
>
> So, I think there's no benefit to change.
>

I do use this function in the packet reference extensions, and need to
perform arithmetic on the return, which is why having it return uint8_t *
rather than void * is more useful. I can include this change as part of my
patch series if you don't want to make it part of this base patch.


>
> 
>
>
> +{
> +   return pkt_hdr->buf_hdr.seg[seg_idx].data;
> +}
> +
> +static inline int packet_last_seg(odp_packet_hdr_t *pkt_hdr)
> +{
> +   if (CONFIG_PACKET_MAX_SEGS == 1)
> +   return 0;
> +   else
> +   return pkt_hdr->buf_hdr.segcount - 1;
> +}
> +
> +static inline uint32_t packet_first_seg_len(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return packet_seg_len(pkt_hdr, 0);
> +}
> +
> +static inline uint32_t packet_last_seg_len(odp_packet_hdr_t *pkt_hdr)
> +{
> +   int last = packet_last_seg(pkt_hdr);
> +
> +   return packet_seg_len(pkt_hdr, last);
> +}
> +
> +static inline void *packet_data(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return pkt_hdr->buf_hdr.seg[0].data;
>
>
> Given the earlier definition of the packet_seg_data helper this would more
> consistently be:
>
> return packet_seg_data(pkt_hdr, 0);
>
>
>
> 
>
>
> There's no need to change the return type of packet_seg_data() to be able
> to use it here.
>
> Anyway, I decided not to hide buf_hdr.seg[0].data reads behind a function
> call, since this way it's easy to find all reads and *writes* of it
> throughout the file.
>
>
Then I'd question why have this function at all if the intent is not to
help hide the particulars of the seg structure contained in the buf_hdr? I
thought 

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Savolainen, Petri (Nokia - FI/Espoo)]

<2nd reply to HTML mail>

Actually, the definition of '#define ODP_UNUSED(x) (void)(x)' implementation 
was never discussed. We just wrapped "__attribute__((__unused__))" of GCC. 
That's why I'm questioning it here now. What if your compiler X checks for 
unused params but does not support the same or similar attribute *inside* 
function prototype ?

We could also remove ODP_UNUSED from the API if cast to (void) is a valid C 
language feature.

Anyway, for this series I'll send a v4 with ODP_UNUSED.



If you wish. Alternately I can fold that into my patch that goes on top of this 
one and I can review your series as-is since it otherwise looks fine and 
clearly is something we want.
 
<3rd reply to HTML mail>

That's OK of course :) Thanks.

-Petri

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Francois Ozog]

Why not just scanning a directory or give a command line parameter?

On 16 November 2016 at 12:05, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> So what?
> If we don't allow to load a driver from the north interface, where should
> they be loaded from??
> Or are we saying that we go for a predefined list of driver only?
>
> So drivers have to be .so files (because of autotools), but we don't give
> the possibility to load a free driver??
>
> Christophe
>
> On 16 November 2016 at 11:45, Francois Ozog 
> wrote:
>
>> Hello,
>>
>> If the north API is the one visible by ODP applications then I don't
>> think it is a good idea to expose that.
>> DPDK exposed it at the beginning and is now internal.
>>
>> That said there must be a standard way to manage drivers fir the benefit
>> of the device framework.
>>
>> I don't think the idea of integrating it with packetio open because
>> packetio is not really a device. It is an abstract way of dealing with
>> ports which may be hosted on a device (NIC) or on the platform (SoC).
>> It can be done as a PoC (that's what I do with virtio-net exploratiry
>> project) but that is not a long term solution.
>>
>> FF
>>
>>
>> Le mercredi 16 novembre 2016, Christophe Milard <
>> christophe.mil...@linaro.org> a écrit :
>>
>>> On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
>>>  wrote:
>>> >
>>> >>  /**
>>> >> + * Driver loading
>>> >> + *
>>> >> + * This function is used by the application to load NIC drivers into
>>> ODP.
>>> >> + * Calls to this function are optional, but should be performed (if
>>> any)
>>> >> + * after odp_init_global
>>> >> + *
>>> >> + *
>>> >> + * @param filenameDriver shared lib (dynamic library)
>>> >> + *
>>> >> + * @retval 0 on success
>>> >> + * @retval <0 on failure
>>> >> + *
>>> >> + */
>>> >> +int odp_load_driver(const char *filename);
>>> >
>>> > Why application should explicitly need to load a driver ? Why it
>>> cannot happen as part of odp_pktio_open() ?
>>> >
>>> > To me a better approach would be:
>>> > * user reads implementation documentation or uses system commands to
>>> find a list of available interfaces
>>> > * odp_global_init() finds out available pktio interfaces and drivers
>>> for those
>>> > * application opens the interface the user commands (just like today)
>>> > * pktio_open call loads the driver
>>> >
>>> > Or with more ODP support:
>>> > * odp_global_init() finds out available pktio interfaces and drivers
>>> for those
>>> > * a new API call lists all available interfaces
>>> > * application chooses an interface from the list and calls open
>>> > * pktio_open call loads the driver
>>> >
>>> >
>>> > -Petri
>>>
>>>
>>> Having ODP finding the driver by itself means "hard-coding" the list
>>> of available drivers (in the sense that ODP must know in advance what
>>> drivers matches what interface and where to find these drivers).
>>>
>>> The approach above let people do their own driver and attach it to ODP
>>> whithout any pre-requirements on ODP.
>>> When we have a set of "drivers we like", nothing prevent us from
>>> loading a default set of drivers at init.
>>> Unless the driver name (and path) is given as part as the pktio device
>>> name, which would just be awfull, we need such a load function to
>>> enable private driver loading.
>>>
>>> Moreover, the driver must be loaded to be probed: The drivers must
>>> tell what they can do, not ODP.
>>>
>>> I think the application should as a little aware as possible of
>>> drivers (and what HW they match) but should be given a way to load
>>> private drivers. The approach here enable that. Load your own driver
>>> (or none if it is in the default set), then open pktios without
>>> specifying any driver (note that 1 driver may be handling many
>>> interfaces)
>>>
>>> I think having an API call to list available interfaces makes sense,
>>> and does stress my proposal: Drivers must be loaded and probed to
>>> build that list.
>>>
>>> Christophe.
>>>
>>
>>
>> --
>> [image: Linaro] 
>> François-Frédéric Ozog | *Director Linaro Networking Group*
>> T: +33.67221.6485
>> francois.o...@linaro.org | Skype: ffozog
>>
>>
>>
>


-- 
[image: Linaro] 
François-Frédéric Ozog | *Director Linaro Networking Group*
T: +33.67221.6485
francois.o...@linaro.org | Skype: ffozog

Re: [lng-odp] [API-NEXT PATCHv6 05/13] api: shm: add flags to shm_reserve and function to find external mem

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> Christophe Milard
> Sent: Monday, October 31, 2016 10:52 AM
> To: mike.hol...@linaro.org; bill.fischo...@linaro.org; lng-
> o...@lists.linaro.org
> Subject: [lng-odp] [API-NEXT PATCHv6 05/13] api: shm: add flags to
> shm_reserve and function to find external mem
> 
> The ODP_SHM_SINGLE_VA flag is created: when set (at odp_shm_reserve()),
> this flag guarantees that all ODP threads sharing this memory
> block will see the block at the same address (regadless of ODP
> thread type -pthread vs process- or fork time)
> 
> The flag ODP_SHM_EXPORT is added: when passed at odp_shm_reserve() time
> the memory block becomes visible to other ODP instances.
> The function odp_shm_find_exported() is added: this function enables to
> reserve block of memories exported by other ODP instances (using the
> ODP_SHM_EXPORT flag).
> 
> Signed-off-by: Christophe Milard 
> ---
>  include/odp/api/spec/shared_memory.h | 38
> +++-
>  1 file changed, 33 insertions(+), 5 deletions(-)
> 
> diff --git a/include/odp/api/spec/shared_memory.h
> b/include/odp/api/spec/shared_memory.h
> index 8c76807..80cc143 100644
> --- a/include/odp/api/spec/shared_memory.h
> +++ b/include/odp/api/spec/shared_memory.h
> @@ -14,6 +14,7 @@
>  #ifndef ODP_API_SHARED_MEMORY_H_
>  #define ODP_API_SHARED_MEMORY_H_
>  #include 
> +#include 
> 
>  #ifdef __cplusplus
>  extern "C" {
> @@ -43,12 +44,25 @@ extern "C" {
>  #define ODP_SHM_NAME_LEN 32
> 
>  /*
> - * Shared memory flags
> + * Shared memory flags:
>   */
> -
> -/* Share level */
> -#define ODP_SHM_SW_ONLY 0x1 /**< Application SW only, no HW access */
> -#define ODP_SHM_PROC0x2 /**< Share with external processes */
> +#define ODP_SHM_SW_ONLY  0x1 /**< Application SW only, no HW
> access   */
> +#define ODP_SHM_PROC 0x2 /**< Share with external processes
> */
> +/**
> + * Single virtual address
> + *
> + * When set, this flag guarantees that all ODP threads sharing this
> + * memory block will see the block at the same address - regardless
> + * of ODP thread type (e.g. pthread vs. process (or fork process time)).
> + */
> +#define ODP_SHM_SINGLE_VA0x4
> +/**
> + * Export memory
> + *
> + * When set, the memory block becomes visible to other ODP instances
> + * through odp_shm_find_exported().

Through odp_shm_import() see under

> + */
> +#define ODP_SHM_EXPORT   0x08
> 
>  /**
>   * Shared memory block info
> @@ -135,6 +149,20 @@ int odp_shm_free(odp_shm_t shm);
>   */
>  odp_shm_t odp_shm_lookup(const char *name);
> 
> +/**
> + * Get and reserve a block of shared memory, exported by another ODP
> instance
> + *
> + * @param remote_name  Name of the block, in the remote ODP instance
> + * @param odp_inst Remote ODP instance, as returned by
> odp_init_global()
> + * @param local_name   Name given to the block, in the local ODP instance
> + *  (or NULL, if the application doesn't care)
> + *
> + * @return A new handle to the block if it is found (must be freed when
> done).
> + * @retval ODP_SHM_INVALID on failure
> + */
> +odp_shm_t odp_shm_find_exported(const char *remote_name,
> + odp_instance_t odp_inst,
> + const char *local_name);


Patch is otherwise OK, but I think export -> import is a better naming 
convention. I also updated the spec text to be a bit more specific about the 
feature, see under.

/**
 * Import a block of shared memory that was exported by another ODP instance
 *
 * This call creates a new handle for accessing a shared memory block created
 * (with ODP_SHM_EXPORT flag) by another ODP instance. An instance may have
 * only a single handle to the same block. Application must not access the
 * block after freeing the handle. When an imported handle is freed, only
 * the calling instance is affected. The exported block may be freed only
 * after all other instances have stopped accessing the block.
 *
 * @param remote_name  Name of the block, in the remote ODP instance
 * @param odp_inst Remote ODP instance, as returned by odp_init_global()
 * @param local_name   Name given to the shm block, in the local ODP instance.
 * May be NULL, if the application doesn't need a name
 * (for a lookup).
 *
 * @return A handle to access a block exported by another ODP instance
 * @retval ODP_SHM_INVALID on failure
 */
odp_shm_t odp_shm_import(const char *remote_name, odp_instance_t odp_inst,
 const char *local_name);


-Petri

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Maxim Uvarov]

ok, looks like you changed something in the pool and long packets can 
not be allocated:



passed
  Test: pktio_test_pktio_config ...passed
  Test: pktio_test_info ...pktio 0
  name   pktiop0p1
  driver socket
pktio 1
  name   pktiop1p0
  driver socket
passed
  Test: pktio_test_pktin_queue_config_direct ...passed
  Test: pktio_test_pktin_queue_config_sched ...passed
  Test: pktio_test_pktin_queue_config_queue ...passed
  Test: pktio_test_pktout_queue_config ...passed
  Test: pktio_test_plain_queue ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts
2. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_plain_multi ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts
2. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_sched_queue ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts
2. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_sched_multi ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts
2. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_recv ...error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_recv_multi ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:635 - 
num_rx == num_pkts

  Test: pktio_test_recv_queue ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
FAILED
1. 
../../../../../../test/common_plat/validation/api/pktio/pktio.c:864 - 
num_rx == TX_BATCH_LEN

  Test: pktio_test_recv_tmo ...error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: packet length invalid: 9216 (64)
error: header magic invalid 0
error: header magic invalid 393473
error: offset invalid
error: offset invalid
error: header magic invalid 0
error: header magic invalid 0
error: header magic invalid 8650752
error: offset invalid
error: header magic invalid 0
error: header magic invalid 0
error: header magic invalid 8650752
error: offset invalid
error: header magic invalid 0
error: header magic invalid 8650752


On 11/17/16 11:36, Savolainen, Petri (Nokia - FI/Espoo) wrote:

I have reported a bug https://bugs.linaro.org/show_bug.cgi?id=2595 about the 
new VLAN test hang. Could this be the same issue? It hangs for me sometimes, 
most times not. It happens both at the tip of the master and api-next.

-Petri




-Original Message-
From: Maxim Uvarov [mailto:maxim.uva...@linaro.org]
Sent: Wednesday, November 16, 2016 10:03 PM
To: Bill Fischofer ; Savolainen, Petri (Nokia -
FI/Espoo) ; Mike Holmes

Cc: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

I can not test patch by patch this series because it fails (one time it
was TM, one time kernel died, other time OOM killer killed tests then
hang kernel).

And for all patches test/common_plat/validation/api/pktio/pktio_main
hangs forever:


Program received signal SIGINT, Interrupt.
0x2afbe69ffb80 in __nanosleep_nocancel () at
../sysdeps/unix/syscall-template.S:81
81in ../sysdeps/unix/syscall-template.S
(gdb) bt
#0  0x2afbe69ffb80 in __nanosleep_nocancel () at
../sysdeps/unix/syscall-template.S:81
#1  0x00415ced in odp_pktin_recv_tmo (queue=...,
packets=packets@entry=0x7ffed64d8bd0, 

[lng-odp] [API-NEXT PATCHv7 08/13] linux-gen: _ishm: adding function to map memory from other ODP

[Christophe Milard]

functionality to export and map memory between ODP instance is added:
This includes:
- a bit of simplification in _odp_ishm_reserve() for externaly provided
  file descriptors.
- a new function, _odp_ishm_find_exported() to map memory from
  other ODP instances (On same OS)

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/_ishm.c  | 160 +++-
 platform/linux-generic/include/_ishm_internal.h |   5 +
 2 files changed, 136 insertions(+), 29 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index a988ab9..17b186f 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -152,6 +152,7 @@ typedef struct ishm_fragment {
  * will allocate both a block and a fragment.
  * Blocks contain only global data common to all processes.
  */
+typedef enum {UNKNOWN, HUGE, NORMAL, EXTERNAL} huge_flag_t;
 typedef struct ishm_block {
char name[ISHM_NAME_MAXLEN];/* name for the ishm block (if any) */
char filename[ISHM_FILENAME_MAXLEN]; /* name of the .../odp-* file  */
@@ -162,7 +163,7 @@ typedef struct ishm_block {
void *start; /* only valid if _ODP_ISHM_SINGLE_VA is set*/
uint64_t len;/* length. multiple of page size. 0 if free*/
ishm_fragment_t *fragment; /* used when _ODP_ISHM_SINGLE_VA is used */
-   int   huge; /* true if this segment is mapped using huge pages  */
+   huge_flag_t huge;/* page type: external means unknown here. */
uint64_t seq;   /* sequence number, incremented on alloc and free   */
uint64_t refcnt;/* number of linux processes mapping this block */
 } ishm_block_t;
@@ -400,7 +401,7 @@ static void free_fragment(ishm_fragment_t *fragmnt)
  * or /mnt/huge/odp-- (for huge pages)
  * Return the new file descriptor, or -1 on error.
  */
-static int create_file(int block_index, int huge, uint64_t len,
+static int create_file(int block_index, huge_flag_t huge, uint64_t len,
   uint32_t flags, uint32_t align)
 {
char *name;
@@ -419,10 +420,11 @@ static int create_file(int block_index, int huge, 
uint64_t len,
 ishm_tbl->dev_seq++);
 
/* huge dir must be known to create files there!: */
-   if (huge && !odp_global_data.hugepage_info.default_huge_page_dir)
+   if ((huge == HUGE) &&
+   (!odp_global_data.hugepage_info.default_huge_page_dir))
return -1;
 
-   if (huge)
+   if (huge == HUGE)
snprintf(filename, ISHM_FILENAME_MAXLEN,
 ISHM_FILENAME_FORMAT,
 odp_global_data.hugepage_info.default_huge_page_dir,
@@ -502,7 +504,7 @@ static void delete_file(ishm_block_t *block)
  * Mutex must be assured by the caller.
  */
 static void *do_map(int block_index, uint64_t len, uint32_t align,
-   uint32_t flags, int huge, int *fd)
+   uint32_t flags, huge_flag_t huge, int *fd)
 {
ishm_block_t *new_block;  /* entry in the main block table   */
void *addr = NULL;
@@ -552,8 +554,6 @@ static void *do_map(int block_index, uint64_t len, uint32_t 
align,
return NULL;
}
 
-   new_block->huge = huge;
-
return mapped_addr;
 }
 
@@ -756,27 +756,21 @@ int _odp_ishm_reserve(const char *name, uint64_t size, 
int fd,
int new_index;/* index in the main block table*/
ishm_block_t *new_block;  /* entry in the main block table*/
uint64_t page_sz; /* normal page size. usually 4K*/
-   uint64_t alloc_size;  /* includes extra for alignement*/
uint64_t page_hp_size;/* huge page size */
-   uint64_t alloc_hp_size;   /* includes extra for alignement*/
uint32_t hp_align;
uint64_t len; /* mapped length */
void *addr = NULL;/* mapping address */
int new_proc_entry;
-
-   page_sz = odp_sys_page_size();
+   struct stat statbuf;
 
odp_spinlock_lock(_tbl->lock);
 
/* update this process view... */
procsync();
 
-   /* roundup to page size */
-   alloc_size = (size + (page_sz - 1)) & (-page_sz);
-
+   /* Get system page sizes: page_hp_size is 0 if no huge page available*/
+   page_sz  = odp_sys_page_size();
page_hp_size = odp_sys_huge_page_size();
-   /* roundup to page size */
-   alloc_hp_size = (size + (page_hp_size - 1)) & (-page_hp_size);
 
/* check if name already exists */
if (name && (find_block_by_name(name) >= 0)) {
@@ -809,8 +803,24 @@ int _odp_ishm_reserve(const char *name, uint64_t size, int 
fd,
else
new_block->name[0] = 0;
 
-   /* Try first huge pages when possible and needed: */
-   if (page_hp_size && (alloc_size > 

[lng-odp] [API-NEXT PATCHv7 11/13] linux-gen: _ishm: cleaning remaining block at odp_term_global

[Christophe Milard]

Remaining (forgotten, not freed) blocks are gathered and related files
cleaned when odp_term_global() is called. An error message is also issued
so the application writters get to know about these blocks

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/_ishm.c | 13 +
 1 file changed, 13 insertions(+)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index 17b186f..0586a96 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -1505,12 +1505,25 @@ int _odp_ishm_term_local(void)
 int _odp_ishm_term_global(void)
 {
int ret = 0;
+   int index;
+   ishm_block_t *block;
 
if ((getpid() != odp_global_data.main_pid) ||
(syscall(SYS_gettid) != getpid()))
ODP_ERR("odp_term_global() must be performed by the main "
"ODP process!\n.");
 
+   /* cleanup possibly non freed memory (and complain a bit): */
+   for (index = 0; index < ISHM_MAX_NB_BLOCKS; index++) {
+   block = _tbl->block[index];
+   if (block->len != 0) {
+   ODP_ERR("block '%s' (file %s) was never freed "
+   "(cleaning up...).\n",
+   block->name, block->filename);
+   delete_file(block);
+   }
+   }
+
/* perform the last thread terminate which was postponed: */
ret = do_odp_ishm_term_local();
 
-- 
2.7.4

Re: [lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Mike Holmes]

On 16 November 2016 at 22:06, Bill Fischofer  wrote:
> Signed-off-by: Bill Fischofer 
> ---
>  CHANGELOG | 211 
> ++
>  1 file changed, 211 insertions(+)
>
> diff --git a/CHANGELOG b/CHANGELOG
> index d8230cd..3ab7354 100644
> --- a/CHANGELOG
> +++ b/CHANGELOG
> @@ -1,3 +1,214 @@
> +== OpenDataPlane (1.10.2.0)

https://git.linaro.org/lng/odp.git

latest tag is v1.11.0.0_monarch  should the mainline be 1.11.1.0
since mainline is monarch + latest stuff

> +
> +=== New Features
> +
> + APIs
> +ODP v1.10.2.0 is a minor API revision level and introduces a small change to
> +the Traffic Manager API as well as some documentation clarification 
> surrounding
> +other APIs.
> +
> + Traffic Manager Egress Function
> +The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
> +(`egress_fcn_supported`) that indicates whether the TM system supports
> +a user-provided egress function. When specified this function is called to
> +"output" a packet rather than having TM transmit it directly to a PktIO
> +interface.
> +
> + Traffic Manager Coupling Change
> +The `odp_tm_egress_t` struct has been changed to associate a TM system with 
> an
> +output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`. This 
> makes
> +an explicit 1-to-1 map between a TM system and a PktIO interface.
> +
> + Default huge page size clarification
> +The documentation for the `odp_sys_huge_page_size()` API has been reworded to
> +clarify that this API refers to default huge page size.
> +
> + Strict Priority (SP) Scheduler
> +Building on the modular scheduler framework added in v1.10.1.0, an alternate
> +Strict Priority (SP) Scheduler is now available for latency-sensitive
> +workloads. Applications wishing to use the SP scheduler should specify
> +the `./configure` option `--enable-schedule-sp`. This scheduler emphasizes 
> low
> +latency processing of high priority events at the expense of throughput. This
> +alternate scheduler is considered experimental and should not be used for
> +production at this time.
> +
> + Application Binary Interface (ABI) Support
> +Support is added to enable ODP applications to be binary compatible across
> +different implementations of ODP sharing the same Instruction Set 
> Architecture
> +(ISA). This support introduces a new `configure` option:
> +
> +`--enable-abi-compat=yes`::
> +This is the default and specifies that the ODP library is to be built to
> +support ABI compatibility mode. In this mode ODP APIs are never inlined. ABI
> +compatibility ensures maximum application portability in cloud environments.
> +
> +`--enable-abi-compat=no`::
> +Specify this option to enable the inlining of ODP APIs. This may result in
> +improved performance at the cost of ABI compatibility and is suitable for
> +applications running in embedded environments.
> +
> +Note that ODP applications retain source code portability between ODP
> +implementations regardless of the ABI mode chosen. To move to a different ODP
> +application running on a different ISA, code need simply be recompiled 
> against
> +that target ODP implementation.
> +
> + SCTP Parsing Support
> +The ODP classifier adds support for recognizing Stream Control Transmission
> +Protocol (SCTP) packets. The APIs for this were previously not implemented.
> +
> +=== Packaging and Implementation Refinements
> +
> + Remove dependency on Linux headers
> +ODP no longer has a dependency on Linux headers. This will help make the
> +odp-linux reference implementation more easily portable to non-Linux
> +environments.
> +
> + Remove dependency on helpers
> +The odp-linux implementation has been made independent of the helper library
> +to avoid circular dependency issues with packaging. Helper functions may use
> +ODP APIs, however ODP implementations should not use helper functions.
> +
> + Reorganization of `test` directory
> +The `test` directory has been reorganized to better support a unified 
> approach
> +to ODP component testing. API tests now live in
> +`test/common_plat/validation/api` instead of the former
> +`test/validation`. With this change performance and validation tests, as well
> +as common and platform-specific tests can all be part of a unified test
> +hierarchy.
> +
> +The resulting test tree now looks like:
> +
> +.New `test` directory hierarchy
> +-
> +test
> +├── common_plat
> +│   ├── common
> +│   ├── m4
> +│   ├── miscellaneous
> +│   ├── performance
> +│   └── validation
> +│   └── api
> +│   ├── atomic
> +│   ├── barrier
> +│   ├── buffer
> +│   ├── classification
> +│   ├── cpumask
> +│   ├── crypto
> +│   ├── errno
> +│   ├── hash
> +│   ├── init
> +│   ├── lock
> +│   ├── packet
> +│   ├── pktio
> +│   ├── pool
> +│   ├── queue
> +│   ├── 

Re: [lng-odp] [PATCHv2 1/1] validation: pktio: fix invalid mac addr

[Bala Manoharan]

Ping. Review needed.

Regards,
Bala


On 10 November 2016 at 19:58, Balasubramanian Manoharan
 wrote:
> Fixes https://bugs.linaro.org/show_bug.cgi?id=2496
>
> Signed-off-by: Balasubramanian Manoharan 
> ---
> v2: Incorporate review comments
>  test/common_plat/validation/api/pktio/pktio.c | 24 +---
>  1 file changed, 21 insertions(+), 3 deletions(-)
>
> diff --git a/test/common_plat/validation/api/pktio/pktio.c 
> b/test/common_plat/validation/api/pktio/pktio.c
> index a6a18c3..7115def 100644
> --- a/test/common_plat/validation/api/pktio/pktio.c
> +++ b/test/common_plat/validation/api/pktio/pktio.c
> @@ -31,6 +31,8 @@
>  #define PKTIN_TS_MAX_RES   100
>  #define PKTIN_TS_CMP_RES   1
>
> +#define PKTIO_SRC_MAC  {1, 2, 3, 4, 5, 6}
> +#define PKTIO_DST_MAC  {1, 2, 3, 4, 5, 6}
>  #undef DEBUG_STATS
>
>  /** interface names used for testing */
> @@ -241,16 +243,32 @@ static uint32_t pktio_init_packet(odp_packet_t pkt)
> odph_udphdr_t *udp;
> char *buf;
> uint16_t seq;
> -   uint8_t mac[ODP_PKTIO_MACADDR_MAXSIZE] = {0};
> +   uint8_t src_mac[ODP_PKTIO_MACADDR_MAXSIZE] = PKTIO_SRC_MAC;
> +   uint8_t dst_mac[ODP_PKTIO_MACADDR_MAXSIZE] = PKTIO_DST_MAC;
> +   uint8_t src_mac_be[ODP_PKTIO_MACADDR_MAXSIZE];
> +   uint8_t dst_mac_be[ODP_PKTIO_MACADDR_MAXSIZE];
> int pkt_len = odp_packet_len(pkt);
> +   int i;
> +
> +   #if ODP_BYTE_ORDER == ODP_LITTLE_ENDIAN
> +   for (i = 0; i < ODP_PKTIO_MACADDR_MAXSIZE; i++) {
> +   src_mac_be[i] = src_mac[i];
> +   dst_mac_be[i] = dst_mac[i];
> +   }
> +   #else
> +   for (i = 0; i < ODP_PKTIO_MACADDR_MAXSIZE; i++) {
> +   src_mac_be[i] = src_mac[ODP_PKTIO_MACADDR_MAXSIZE - i];
> +   dst_mac_be[i] = dst_mac[ODP_PKTIO_MACADDR_MAXSIZE - i];
> +   }
> +   #endif
>
> buf = odp_packet_data(pkt);
>
> /* Ethernet */
> odp_packet_l2_offset_set(pkt, 0);
> eth = (odph_ethhdr_t *)buf;
> -   memcpy(eth->src.addr, mac, ODPH_ETHADDR_LEN);
> -   memcpy(eth->dst.addr, mac, ODPH_ETHADDR_LEN);
> +   memcpy(eth->src.addr, _mac_be, ODPH_ETHADDR_LEN);
> +   memcpy(eth->dst.addr, _mac_be, ODPH_ETHADDR_LEN);
> eth->type = odp_cpu_to_be_16(ODPH_ETHTYPE_IPV4);
>
> /* IP */
> --
> 1.9.1
>

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Bill Fischofer]

Reposting this since it doesn't seem to have made it to the mailing list.

For this series:

Reviewed-and-tested-by: Bill Fischofer 

On Tue, Nov 15, 2016 at 8:41 AM, Bill Fischofer 
wrote:

> For this series:
>
> Reviewed-and-tested-by: Bill Fischofer 
>
> On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
> petri.savolai...@nokia.com> wrote:
>
>> Pool performance is optimized by using a ring as the global buffer
>> storage.
>> IPC build is disabled, since it needs large modifications due to
>> dependency to
>> pool internals. Old pool implementation was based on locks and linked
>> list of
>> buffer headers. New implementation maintain a ring of buffer handles,
>> which
>> enable fast, burst based allocs and frees. Also ring scales better with
>> number
>> of cpus than a list (enq and deq operations update opposite ends of the
>> pool).
>>
>> L2fwd link rate (%), 2 x 40GE, 64 byte packets
>>
>> direct- parallel-   atomic-
>> cpusorigdirect  difforigparall  difforigatomic
>> diff
>> 1   7 % 8 % 1 % 6 % 6 % 2 % 5.4 %   5.6 %   4
>> %
>> 2   14 %15 %7 % 9 % 9 % 5 % 8 % 9 % 8
>> %
>> 4   28 %30 %6 % 13 %14 %13 %12 %15 %
>> 19 %
>> 6   42 %44 %6 % 16 %19 %19 %8 % 20 %
>> 150 %
>> 8   46 %59 %28 %19 %23 %26 %18 %24 %
>> 34 %
>> 10  55 %57 %3 % 20 %27 %37 %8 % 28 %
>> 264 %
>> 12  56 %56 %-1 %22 %31 %43 %7 % 32 %
>> 357 %
>>
>> Max packet rate of NICs are reached with 10-12 cpu on direct mode.
>> Otherwise,
>> all cases were improved. Especially, scheduler driven cases suffered on
>> bad
>> pool scalability.
>>
>> changed in v3:
>> * rebased
>> * ipc disabled with #ifdef
>> * added support for multi-segment packets
>> * API: added explicit limits for packet length in alloc calls
>> * Corrected validation test and example application bugs found during
>>   segmentation implementation
>>
>> changed in v2:
>> * rebased to api-next branch
>> * added a comment that ring size must be larger than number of items in it
>> * fixed clang build issue
>> * added parens in align macro
>>
>> v1 reviews:
>> Reviewed-by: Brian Brooks 
>>
>>
>>
>> Petri Savolainen (19):
>>   linux-gen: ipc: disable build of ipc pktio
>>   linux-gen: pktio: do not free zero packets
>>   linux-gen: ring: created common ring implementation
>>   linux-gen: align: added round up power of two
>>   linux-gen: pool: reimplement pool with ring
>>   linux-gen: ring: added multi enq and deq
>>   linux-gen: pool: use ring multi enq and deq operations
>>   linux-gen: pool: optimize buffer alloc
>>   linux-gen: pool: clean up pool inlines functions
>>   linux-gen: pool: ptr instead of hdl in buffer_alloc_multi
>>   test: validation: buf: test alignment
>>   test: performance: crypto: use capability to select max packet
>>   test: correctly initialize pool parameters
>>   test: validation: packet: fix bugs in tailroom and concat tests
>>   linux-gen: packet: added support for segmented packets
>>   test: validation: packet: improved multi-segment alloc test
>>   api: packet: added limits for packet len on alloc
>>   linux-gen: packet: remove zero len support from alloc
>>   linux-gen: packet: enable multi-segment packets
>>
>>  example/generator/odp_generator.c  |2 +-
>>  include/odp/api/spec/packet.h  |9 +-
>>  include/odp/api/spec/pool.h|6 +
>>  platform/linux-generic/Makefile.am |1 +
>>  .../include/odp/api/plat/packet_types.h|6 +-
>>  .../include/odp/api/plat/pool_types.h  |6 -
>>  .../linux-generic/include/odp_align_internal.h |   34 +-
>>  .../linux-generic/include/odp_buffer_inlines.h |  167 +--
>>  .../linux-generic/include/odp_buffer_internal.h|  120 +-
>>  .../include/odp_classification_datamodel.h |2 +-
>>  .../linux-generic/include/odp_config_internal.h|   55 +-
>>  .../linux-generic/include/odp_packet_internal.h|   87 +-
>>  platform/linux-generic/include/odp_pool_internal.h |  289 +---
>>  platform/linux-generic/include/odp_ring_internal.h |  176 +++
>>  .../linux-generic/include/odp_timer_internal.h |4 -
>>  platform/linux-generic/odp_buffer.c|   22 +-
>>  platform/linux-generic/odp_classification.c|   25 +-
>>  platform/linux-generic/odp_crypto.c|   12 +-
>>  platform/linux-generic/odp_packet.c|  717 --
>>  platform/linux-generic/odp_packet_io.c |2 +-
>>  platform/linux-generic/odp_pool.c  | 1440
>> 
>>  platform/linux-generic/odp_queue.c |4 +-
>>  

Re: [lng-odp] [PATCH v2 1/2] test: perf: add new ordered pktio application

[Elo, Matias (Nokia - FI/Espoo)]

Ping.


> On 1 Nov 2016, at 10:53, Matias Elo  wrote:
> 
> Add new test application for ordered queue functionality and performance
> validation. The application sets sequence numbers to incoming packets using
> ordered pktin queues and ordered context locks. After being tagged packets
> are enqueued to atomic queues based on flow hash (IPv4 5-tuple). In atomic
> flow processing the sequence number is validated and packet is sent to
> selected output interface.
> 
> Main options:
> -m: Input queue type can be changed to atomic or parallel to enable easy
>performance comparison. With parallel input queues the packet order is
>not maintained.
> -r: Number of input queues per interface
> -f: Number of atomic flow queues per interface
> -e: Number of extra input processing rounds. This can be used to simulate
>"fat pipe" traffic processing.
> 
> Signed-off-by: Matias Elo 
> ---
> 
> V2:
> - Fixed missing/invalid documentation (Maxim)
> - Fixed resource leak (Maxim)
> - Check odp_shm_reserve() return value (Maxim)
> - Check that CPU count command line argument is valid
> 
> test/common_plat/performance/.gitignore  |1 +
> test/common_plat/performance/Makefile.am |7 +-
> test/common_plat/performance/dummy_crc.h |  493 
> test/common_plat/performance/odp_pktio_ordered.c | 1337 ++
> 4 files changed, 1837 insertions(+), 1 deletion(-)
> create mode 100644 test/common_plat/performance/dummy_crc.h
> create mode 100644 test/common_plat/performance/odp_pktio_ordered.c
> 
> diff --git a/test/common_plat/performance/.gitignore 
> b/test/common_plat/performance/.gitignore
> index 1527d25..8bb18f5 100644
> --- a/test/common_plat/performance/.gitignore
> +++ b/test/common_plat/performance/.gitignore
> @@ -3,6 +3,7 @@
> odp_atomic
> odp_crypto
> odp_l2fwd
> +odp_pktio_ordered
> odp_pktio_perf
> odp_sched_latency
> odp_scheduling
> diff --git a/test/common_plat/performance/Makefile.am 
> b/test/common_plat/performance/Makefile.am
> index f184609..790ddae 100644
> --- a/test/common_plat/performance/Makefile.am
> +++ b/test/common_plat/performance/Makefile.am
> @@ -5,6 +5,7 @@ TESTS_ENVIRONMENT += TEST_DIR=${builddir}
> EXECUTABLES = odp_crypto$(EXEEXT) odp_pktio_perf$(EXEEXT)
> 
> COMPILE_ONLY = odp_l2fwd$(EXEEXT) \
> +odp_pktio_ordered$(EXEEXT) \
>  odp_sched_latency$(EXEEXT) \
>  odp_scheduling$(EXEEXT)
> 
> @@ -22,15 +23,19 @@ bin_PROGRAMS = $(EXECUTABLES) $(COMPILE_ONLY)
> 
> odp_crypto_LDFLAGS = $(AM_LDFLAGS) -static
> odp_crypto_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> +odp_pktio_ordered_LDFLAGS = $(AM_LDFLAGS) -static
> +odp_pktio_ordered_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> odp_sched_latency_LDFLAGS = $(AM_LDFLAGS) -static
> odp_sched_latency_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> odp_scheduling_LDFLAGS = $(AM_LDFLAGS) -static
> odp_scheduling_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> 
> noinst_HEADERS = \
> -   $(top_srcdir)/test/test_debug.h
> +   $(top_srcdir)/test/test_debug.h \
> +   dummy_crc.h
> 
> dist_odp_crypto_SOURCES = odp_crypto.c
> +dist_odp_pktio_ordered_SOURCES = odp_pktio_ordered.c
> dist_odp_sched_latency_SOURCES = odp_sched_latency.c
> dist_odp_scheduling_SOURCES = odp_scheduling.c
> dist_odp_pktio_perf_SOURCES = odp_pktio_perf.c
> diff --git a/test/common_plat/performance/dummy_crc.h 
> b/test/common_plat/performance/dummy_crc.h
> new file mode 100644
> index 000..38da444
> --- /dev/null
> +++ b/test/common_plat/performance/dummy_crc.h
> @@ -0,0 +1,493 @@
> +/* Copyright (c) 2016, Linaro Limited
> + * All rights reserved.
> + *
> + * SPDX-License-Identifier: BSD-3-Clause
> + */
> +
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2010-2013 Intel Corporation. All rights reserved.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + *   notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + *   notice, this list of conditions and the following disclaimer in
> + *   the documentation and/or other materials provided with the
> + *   distribution.
> + * * Neither the name of Intel Corporation nor the names of its
> + *   contributors may be used to endorse or promote products derived
> + *   from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL 

[lng-odp] [API-NEXT PATCHv7 03/13] linux-gen: use ishm as north API mem allocator

[Christophe Milard]

The odp shared_memory API is changed to use the ODP internal memory
allocator: _ishm.
_ishm supports memory sharing between processes, regardless of fork time.
The test testing the ODP_SHM_PROC flag is also changed to cope with the
new OS sharing interface used by _ishm (link in /tmp).

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/include/odp_internal.h  |   5 -
 platform/linux-generic/odp_init.c  |  19 -
 platform/linux-generic/odp_shared_memory.c | 406 ++---
 .../validation/api/shmem/shmem_linux.c | 100 -
 4 files changed, 111 insertions(+), 419 deletions(-)

diff --git a/platform/linux-generic/include/odp_internal.h 
b/platform/linux-generic/include/odp_internal.h
index 2f29d3d..6a80d9d 100644
--- a/platform/linux-generic/include/odp_internal.h
+++ b/platform/linux-generic/include/odp_internal.h
@@ -60,7 +60,6 @@ enum init_stage {
SYSINFO_INIT,
FDSERVER_INIT,
ISHM_INIT,
-   SHM_INIT,
THREAD_INIT,
POOL_INIT,
QUEUE_INIT,
@@ -90,10 +89,6 @@ int odp_thread_init_local(odp_thread_type_t type);
 int odp_thread_term_local(void);
 int odp_thread_term_global(void);
 
-int odp_shm_init_global(void);
-int odp_shm_term_global(void);
-int odp_shm_init_local(void);
-
 int odp_pool_init_global(void);
 int odp_pool_init_local(void);
 int odp_pool_term_global(void);
diff --git a/platform/linux-generic/odp_init.c 
b/platform/linux-generic/odp_init.c
index d4a8e09..169a83e 100644
--- a/platform/linux-generic/odp_init.c
+++ b/platform/linux-generic/odp_init.c
@@ -63,12 +63,6 @@ int odp_init_global(odp_instance_t *instance,
}
stage = ISHM_INIT;
 
-   if (odp_shm_init_global()) {
-   ODP_ERR("ODP shm init failed.\n");
-   goto init_failed;
-   }
-   stage = SHM_INIT;
-
if (odp_thread_init_global()) {
ODP_ERR("ODP thread init failed.\n");
goto init_failed;
@@ -222,13 +216,6 @@ int _odp_term_global(enum init_stage stage)
}
/* Fall through */
 
-   case SHM_INIT:
-   if (odp_shm_term_global()) {
-   ODP_ERR("ODP shm term failed.\n");
-   rc = -1;
-   }
-   /* Fall through */
-
case ISHM_INIT:
if (_odp_ishm_term_global()) {
ODP_ERR("ODP ishm term failed.\n");
@@ -286,12 +273,6 @@ int odp_init_local(odp_instance_t instance, 
odp_thread_type_t thr_type)
}
stage = ISHM_INIT;
 
-   if (odp_shm_init_local()) {
-   ODP_ERR("ODP shm local init failed.\n");
-   goto init_fail;
-   }
-   stage = SHM_INIT;
-
if (odp_thread_init_local(thr_type)) {
ODP_ERR("ODP thread local init failed.\n");
goto init_fail;
diff --git a/platform/linux-generic/odp_shared_memory.c 
b/platform/linux-generic/odp_shared_memory.c
index 550af27..9e916e9 100644
--- a/platform/linux-generic/odp_shared_memory.c
+++ b/platform/linux-generic/odp_shared_memory.c
@@ -4,434 +4,88 @@
  * SPDX-License-Identifier: BSD-3-Clause
  */
 
-#include 
-
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include 
 #include 
-
-#include 
-#include 
-#include 
-#include 
-#include 
-
-#include 
+#include 
+#include 
+#include 
+#include <_ishm_internal.h>
 #include 
-#include 
-#include 
 
 ODP_STATIC_ASSERT(ODP_CONFIG_SHM_BLOCKS >= ODP_CONFIG_POOLS,
  "ODP_CONFIG_SHM_BLOCKS < ODP_CONFIG_POOLS");
 
-typedef struct {
-   char  name[ODP_SHM_NAME_LEN];
-   uint64_t  size;
-   uint64_t  align;
-   uint64_t  alloc_size;
-   void  *addr_orig;
-   void  *addr;
-   int   huge;
-   odp_shm_t hdl;
-   uint32_t  flags;
-   uint64_t  page_sz;
-   int   fd;
-
-} odp_shm_block_t;
-
-
-typedef struct {
-   odp_shm_block_t block[ODP_CONFIG_SHM_BLOCKS];
-   odp_spinlock_t  lock;
-
-} odp_shm_table_t;
-
-
-#ifndef MAP_ANONYMOUS
-#define MAP_ANONYMOUS MAP_ANON
-#endif
-
-
-/* Global shared memory table */
-static odp_shm_table_t *odp_shm_tbl;
-
-
 static inline uint32_t from_handle(odp_shm_t shm)
 {
return _odp_typeval(shm) - 1;
 }
 
-
 static inline odp_shm_t to_handle(uint32_t index)
 {
return _odp_cast_scalar(odp_shm_t, index + 1);
 }
 
-
-int odp_shm_init_global(void)
-{
-   void *addr;
-
-#ifndef MAP_HUGETLB
-   ODP_DBG("NOTE: mmap does not support huge pages\n");
-#endif
-
-   addr = mmap(NULL, sizeof(odp_shm_table_t),
-   PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
-
-   if (addr == MAP_FAILED)
-   return -1;
-
-   odp_shm_tbl = addr;
-
-   memset(odp_shm_tbl, 0, sizeof(odp_shm_table_t));
-   odp_spinlock_init(_shm_tbl->lock);
-
-   return 0;
-}
-
-int odp_shm_term_global(void)
-{
-   

[lng-odp] [API-NEXT PATCHv7 02/13] linux-gen: _ishm: allow memory alloc/free at global init/term

[Christophe Milard]

_ishm.c assumed that both _ishm_init_global() and _ishm_init_local()
had been run to work properly. This assumption turns out the be a problem
if _ishm is to be used as main memory allocator, as many modules
init_global functions assume the availability of the odp_reserve()
function before any init_local function is called.
Likewise, many term_global() functions assume the availability
of the odp_shm_free() function after all odp_term_local() have run.
This patch runs _ishm_init_local() in advance for the main ODP thread
and postpones the execution of_ishm_term_local() to init_global()
time for the main process, hence making the ishm_reserve()
and ishm_free() functions available at init_global/term_global time.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/_ishm.c | 189 +
 1 file changed, 118 insertions(+), 71 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index 9018fb9..a988ab9 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -156,7 +156,6 @@ typedef struct ishm_block {
char name[ISHM_NAME_MAXLEN];/* name for the ishm block (if any) */
char filename[ISHM_FILENAME_MAXLEN]; /* name of the .../odp-* file  */
char exptname[ISHM_FILENAME_MAXLEN]; /* name of the export file */
-   int  main_odpthread; /* The thread which did the initial reserve*/
uint32_t user_flags; /* any flags the user want to remember.*/
uint32_t flags;  /* block creation flags.   */
uint64_t user_len;   /* length, as requested at reserve time.   */
@@ -178,6 +177,7 @@ typedef struct ishm_block {
 typedef struct {
odp_spinlock_t  lock;
uint64_t dev_seq;   /* used when creating device names */
+   uint32_t odpthread_cnt; /* number of running ODP threads   */
ishm_block_t  block[ISHM_MAX_NB_BLOCKS];
 } ishm_table_t;
 static ishm_table_t *ishm_tbl;
@@ -864,7 +864,6 @@ int _odp_ishm_reserve(const char *name, uint64_t size, int 
fd,
new_block->user_flags = user_flags;
new_block->seq++;
new_block->refcnt = 1;
-   new_block->main_odpthread = odp_thread_id();
new_block->start = addr; /* only for SINGLE_VA*/
 
/* the allocation succeeded: update the process local view */
@@ -907,10 +906,8 @@ static int block_free(int block_index)
 
proc_index = procfind_block(block_index);
if (proc_index >= 0) {
-   /* close the fd, unless if it was externaly provided */
-   if ((block->filename[0] != 0) ||
-   (odp_thread_id() != block->main_odpthread))
-   close(ishm_proctable->entry[proc_index].fd);
+   /* close the related fd */
+   close(ishm_proctable->entry[proc_index].fd);
 
/* remove the mapping and possible fragment */
do_unmap(ishm_proctable->entry[proc_index].start,
@@ -1201,12 +1198,62 @@ int _odp_ishm_info(int block_index, _odp_ishm_info_t 
*info)
return 0;
 }
 
+static int do_odp_ishm_init_local(void)
+{
+   int i;
+   int block_index;
+
+   /*
+* the ishm_process table is local to each linux process
+* Check that no other linux threads (of same or ancestor processes)
+* have already created the table, and create it if needed.
+* We protect this with the general ishm lock to avoid
+* init race condition of different running threads.
+*/
+   odp_spinlock_lock(_tbl->lock);
+   ishm_tbl->odpthread_cnt++; /* count ODPthread (pthread or process) */
+   if (!ishm_proctable) {
+   ishm_proctable = malloc(sizeof(ishm_proctable_t));
+   if (!ishm_proctable) {
+   odp_spinlock_unlock(_tbl->lock);
+   return -1;
+   }
+   memset(ishm_proctable, 0, sizeof(ishm_proctable_t));
+   }
+   if (syscall(SYS_gettid) != getpid())
+   ishm_proctable->thrd_refcnt++;  /* new linux thread  */
+   else
+   ishm_proctable->thrd_refcnt = 1;/* new linux process */
+
+   /*
+* if this ODP thread is actually a new linux process, (as opposed
+* to a pthread), i.e, we just forked, then all shmem blocks
+* of the parent process are mapped into this child by inheritance.
+* (The process local table is inherited as well). We hence have to
+* increase the process refcount for each of the inherited mappings:
+*/
+   if (syscall(SYS_gettid) == getpid()) {
+   for (i = 0; i < ishm_proctable->nb_entries; i++) {
+   block_index = ishm_proctable->entry[i].block_index;
+   ishm_tbl->block[block_index].refcnt++;
+   }
+   }
+
+   odp_spinlock_unlock(_tbl->lock);
+   return 0;
+}
+
 int 

[lng-odp] [API-NEXT PATCHv7 05/13] api: shm: add flags to shm_reserve and function to find external mem

[Christophe Milard]

The ODP_SHM_SINGLE_VA flag is created: when set (at odp_shm_reserve()),
this flag guarantees that all ODP threads sharing this memory
block will see the block at the same address (regadless of ODP
thread type -pthread vs process- or fork time)

The flag ODP_SHM_EXPORT is added: when passed at odp_shm_reserve() time
the memory block becomes visible to other ODP instances.
The function odp_shm_import() is added: this function enables to
reserve block of memories exported by other ODP instances (using the
ODP_SHM_EXPORT flag).

Signed-off-by: Christophe Milard 
---
 include/odp/api/spec/shared_memory.h | 46 
 1 file changed, 41 insertions(+), 5 deletions(-)

diff --git a/include/odp/api/spec/shared_memory.h 
b/include/odp/api/spec/shared_memory.h
index 8c76807..885751d 100644
--- a/include/odp/api/spec/shared_memory.h
+++ b/include/odp/api/spec/shared_memory.h
@@ -14,6 +14,7 @@
 #ifndef ODP_API_SHARED_MEMORY_H_
 #define ODP_API_SHARED_MEMORY_H_
 #include 
+#include 
 
 #ifdef __cplusplus
 extern "C" {
@@ -43,12 +44,25 @@ extern "C" {
 #define ODP_SHM_NAME_LEN 32
 
 /*
- * Shared memory flags
+ * Shared memory flags:
  */
-
-/* Share level */
-#define ODP_SHM_SW_ONLY 0x1 /**< Application SW only, no HW access */
-#define ODP_SHM_PROC0x2 /**< Share with external processes */
+#define ODP_SHM_SW_ONLY0x1 /**< Application SW only, no HW 
access   */
+#define ODP_SHM_PROC   0x2 /**< Share with external processes   */
+/**
+ * Single virtual address
+ *
+ * When set, this flag guarantees that all ODP threads sharing this
+ * memory block will see the block at the same address - regardless
+ * of ODP thread type (e.g. pthread vs. process (or fork process time)).
+ */
+#define ODP_SHM_SINGLE_VA  0x4
+/**
+ * Export memory
+ *
+ * When set, the memory block becomes visible to other ODP instances
+ * through odp_shm_import().
+ */
+#define ODP_SHM_EXPORT 0x08
 
 /**
  * Shared memory block info
@@ -135,6 +149,28 @@ int odp_shm_free(odp_shm_t shm);
  */
 odp_shm_t odp_shm_lookup(const char *name);
 
+/**
+ * Import a block of shared memory, exported by another ODP instance
+ *
+ * This call creates a new handle for accessing a shared memory block created
+ * (with ODP_SHM_EXPORT flag) by another ODP instance. An instance may have
+ * only a single handle to the same block. Application must not access the
+ * block after freeing the handle. When an imported handle is freed, only
+ * the calling instance is affected. The exported block may be freed only
+ * after all other instances have stopped accessing the block.
+ *
+ * @param remote_name  Name of the block, in the remote ODP instance
+ * @param odp_inst Remote ODP instance, as returned by odp_init_global()
+ * @param local_name   Name given to the block, in the local ODP instance
+ *May be NULL, if the application doesn't need a name
+ *(for a lookup).
+ *
+ * @return A handle to access a block exported by another ODP instance.
+ * @retval ODP_SHM_INVALID on failure
+ */
+odp_shm_t odp_shm_import(const char *remote_name,
+odp_instance_t odp_inst,
+const char *local_name);
 
 /**
  * Shared memory block address
-- 
2.7.4

[lng-odp] [API-NEXT PATCHv7 06/13] linux-gen: shm: new ODP_SHM_SINGLE_VA flag implementation

[Christophe Milard]

This flag guarentess the unicity the the block address on all ODP threads.
The patch just exposes the _ODP_ISHM_SINGLE_VA flag of the internal memory
allocator, ishm.

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/odp_shared_memory.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/platform/linux-generic/odp_shared_memory.c 
b/platform/linux-generic/odp_shared_memory.c
index 9e916e9..2377f16 100644
--- a/platform/linux-generic/odp_shared_memory.c
+++ b/platform/linux-generic/odp_shared_memory.c
@@ -43,6 +43,7 @@ odp_shm_t odp_shm_reserve(const char *name, uint64_t size, 
uint64_t align,
 
/* set internal ishm flags according to API flags: */
flgs |= (flags & ODP_SHM_PROC) ? _ODP_ISHM_EXPORT : 0;
+   flgs |= (flags & ODP_SHM_SINGLE_VA) ? _ODP_ISHM_SINGLE_VA : 0;
 
/* all mem reserved through this interface is requested to be locked: */
flgs |= (flags & _ODP_ISHM_LOCK);
-- 
2.7.4

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Savolainen, Petri (Nokia - FI/Espoo)]

From: Bill Fischofer [mailto:bill.fischo...@linaro.org] 
Sent: Tuesday, November 15, 2016 3:03 PM
To: Savolainen, Petri (Nokia - FI/Espoo) 
Cc: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added 
support for segmented packets



On Tue, Nov 15, 2016 at 6:20 AM, Savolainen, Petri (Nokia - FI/Espoo) 
 wrote:


From: Bill Fischofer [mailto:bill.fischo...@linaro.org]
Sent: Monday, November 14, 2016 3:37 AM
To: Savolainen, Petri (Nokia - FI/Espoo) 
Cc: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added 
support for segmented packets



On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen  
wrote:
Added support for multi-segmented packets. The first segments
is the packet descriptor, which contains all metadata and
pointers to other segments.

Signed-off-by: Petri Savolainen 
---
 .../include/odp/api/plat/packet_types.h            |   6 +-
 .../linux-generic/include/odp_buffer_inlines.h     |  11 -
 .../linux-generic/include/odp_buffer_internal.h    |  23 +-
 .../linux-generic/include/odp_config_internal.h    |  39 +-
 .../linux-generic/include/odp_packet_internal.h    |  80 +--
 platform/linux-generic/include/odp_pool_internal.h |   3 -
 platform/linux-generic/odp_buffer.c                |   8 +-
 platform/linux-generic/odp_crypto.c                |   8 +-
 platform/linux-generic/odp_packet.c                | 712 +
 platform/linux-generic/odp_pool.c                  | 123 ++--
 platform/linux-generic/pktio/netmap.c              |   4 +-
 platform/linux-generic/pktio/socket.c              |   3 +-
 12 files changed, 692 insertions(+), 328 deletions(-)




diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index 2eee775..a5c6ff4 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -20,12 +20,155 @@
 #include 
 #include 

-/*
- *
- * Alloc and free
- * 
- *
- */
+static inline odp_packet_t packet_handle(odp_packet_hdr_t *pkt_hdr)
+{
+       return (odp_packet_t)pkt_hdr->buf_hdr.handle.handle;
+}
+
+static inline odp_buffer_t buffer_handle(odp_packet_hdr_t *pkt_hdr)
+{
+       return pkt_hdr->buf_hdr.handle.handle;
+}
+
+static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr,
+                                     uint32_t seg_idx)
+{
+       return pkt_hdr->buf_hdr.seg[seg_idx].len;
+}
+
+static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t 
seg_idx)

This is more usefully typed as returning uint8_t * rather than void * to permit 
easy arithmetic on the result. The uint8_t * converts automatically to void * 
returns for the external API calls that use this.



This is actually the only place packet_seg_data() is called. The API function 
returns (void *), so the inline function fits that.

void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg)
{
        odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);

        if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount))
                return NULL;

        return packet_seg_data(pkt_hdr, seg);
}

Also (void *) can be assigned to any pointer type without cast. For example:

uint8_t *data;
data = packet_seg_data(pkt_hdr, seg);


So, I think there's no benefit to change.



I do use this function in the packet reference extensions, and need to perform 
arithmetic on the return, which is why having it return uint8_t * rather than 
void * is more useful. I can include this change as part of my patch series if 
you don't want to make it part of this base patch.
 
<2nd reply to HTML mail>

It's better to change it on a future patch that actually needs the change.






+{
+       return pkt_hdr->buf_hdr.seg[seg_idx].data;
+}
+
+static inline int packet_last_seg(odp_packet_hdr_t *pkt_hdr)
+{
+       if (CONFIG_PACKET_MAX_SEGS == 1)
+               return 0;
+       else
+               return pkt_hdr->buf_hdr.segcount - 1;
+}
+
+static inline uint32_t packet_first_seg_len(odp_packet_hdr_t *pkt_hdr)
+{
+       return packet_seg_len(pkt_hdr, 0);
+}
+
+static inline uint32_t packet_last_seg_len(odp_packet_hdr_t *pkt_hdr)
+{
+       int last = packet_last_seg(pkt_hdr);
+
+       return packet_seg_len(pkt_hdr, last);
+}
+
+static inline void *packet_data(odp_packet_hdr_t *pkt_hdr)
+{
+       return pkt_hdr->buf_hdr.seg[0].data;


Given the earlier definition of the packet_seg_data helper this would more 
consistently be:

        return packet_seg_data(pkt_hdr, 0);
 



There's no need to change the return type of packet_seg_data() to be able to 
use it here.

Anyway, I decided not to hide buf_hdr.seg[0].data reads behind a function call, 
since this way it's easy to find all reads and *writes* of it 

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Bill Fischofer]

On Tue, Nov 15, 2016 at 8:13 AM, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia-bell-labs.com> wrote:

>
>
> From: Bill Fischofer [mailto:bill.fischo...@linaro.org]
> Sent: Tuesday, November 15, 2016 3:03 PM
> To: Savolainen, Petri (Nokia - FI/Espoo)  labs.com>
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added
> support for segmented packets
>
>
>
> On Tue, Nov 15, 2016 at 6:20 AM, Savolainen, Petri (Nokia - FI/Espoo) <
> petri.savolai...@nokia-bell-labs.com> wrote:
>
>
> From: Bill Fischofer [mailto:bill.fischo...@linaro.org]
> Sent: Monday, November 14, 2016 3:37 AM
> To: Savolainen, Petri (Nokia - FI/Espoo)  labs.com>
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added
> support for segmented packets
>
>
>
> On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
> petri.savolai...@nokia.com> wrote:
> Added support for multi-segmented packets. The first segments
> is the packet descriptor, which contains all metadata and
> pointers to other segments.
>
> Signed-off-by: Petri Savolainen 
> ---
>  .../include/odp/api/plat/packet_types.h|   6 +-
>  .../linux-generic/include/odp_buffer_inlines.h |  11 -
>  .../linux-generic/include/odp_buffer_internal.h|  23 +-
>  .../linux-generic/include/odp_config_internal.h|  39 +-
>  .../linux-generic/include/odp_packet_internal.h|  80 +--
>  platform/linux-generic/include/odp_pool_internal.h |   3 -
>  platform/linux-generic/odp_buffer.c|   8 +-
>  platform/linux-generic/odp_crypto.c|   8 +-
>  platform/linux-generic/odp_packet.c| 712
> +
>  platform/linux-generic/odp_pool.c  | 123 ++--
>  platform/linux-generic/pktio/netmap.c  |   4 +-
>  platform/linux-generic/pktio/socket.c  |   3 +-
>  12 files changed, 692 insertions(+), 328 deletions(-)
>
>
>
>
> diff --git a/platform/linux-generic/odp_packet.c
> b/platform/linux-generic/odp_packet.c
> index 2eee775..a5c6ff4 100644
> --- a/platform/linux-generic/odp_packet.c
> +++ b/platform/linux-generic/odp_packet.c
> @@ -20,12 +20,155 @@
>  #include 
>  #include 
>
> -/*
> - *
> - * Alloc and free
> - * 
> - *
> - */
> +static inline odp_packet_t packet_handle(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return (odp_packet_t)pkt_hdr->buf_hdr.handle.handle;
> +}
> +
> +static inline odp_buffer_t buffer_handle(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return pkt_hdr->buf_hdr.handle.handle;
> +}
> +
> +static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr,
> + uint32_t seg_idx)
> +{
> +   return pkt_hdr->buf_hdr.seg[seg_idx].len;
> +}
> +
> +static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t
> seg_idx)
>
> This is more usefully typed as returning uint8_t * rather than void * to
> permit easy arithmetic on the result. The uint8_t * converts automatically
> to void * returns for the external API calls that use this.
>
> 
>
> This is actually the only place packet_seg_data() is called. The API
> function returns (void *), so the inline function fits that.
>
> void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg)
> {
> odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);
>
> if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount))
> return NULL;
>
> return packet_seg_data(pkt_hdr, seg);
> }
>
> Also (void *) can be assigned to any pointer type without cast. For
> example:
>
> uint8_t *data;
> data = packet_seg_data(pkt_hdr, seg);
>
>
> So, I think there's no benefit to change.
>
> 
>
> I do use this function in the packet reference extensions, and need to
> perform arithmetic on the return, which is why having it return uint8_t *
> rather than void * is more useful. I can include this change as part of my
> patch series if you don't want to make it part of this base patch.
>
> <2nd reply to HTML mail>
>
> It's better to change it on a future patch that actually needs the change.
>
> 
>
>
>
>
> +{
> +   return pkt_hdr->buf_hdr.seg[seg_idx].data;
> +}
> +
> +static inline int packet_last_seg(odp_packet_hdr_t *pkt_hdr)
> +{
> +   if (CONFIG_PACKET_MAX_SEGS == 1)
> +   return 0;
> +   else
> +   return pkt_hdr->buf_hdr.segcount - 1;
> +}
> +
> +static inline uint32_t packet_first_seg_len(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return packet_seg_len(pkt_hdr, 0);
> +}
> +
> +static inline uint32_t packet_last_seg_len(odp_packet_hdr_t *pkt_hdr)
> +{
> +   int last = packet_last_seg(pkt_hdr);
> +
> +   return packet_seg_len(pkt_hdr, last);
> +}
> +
> +static inline void *packet_data(odp_packet_hdr_t *pkt_hdr)
> +{
> +   return 

[lng-odp] [API-NEXT PATCHv2 2/5] linux-generic: packet: implement reference apis

[Bill Fischofer]

Implement the APIs:
- odp_packet_ref_static()
- odp_packet_ref()
- odp_packet_ref_pkt()
- odp_packet_is_ref()
- odp_packet_has_ref()
- odp_packet_unshared_len()

This also involves functional upgrades to the existing packet manipulation
APIs to work with packet references as input.

Signed-off-by: Bill Fischofer 
---
 .../linux-generic/include/odp_packet_internal.h|  49 ++-
 platform/linux-generic/odp_packet.c| 331 -
 2 files changed, 308 insertions(+), 72 deletions(-)

diff --git a/platform/linux-generic/include/odp_packet_internal.h 
b/platform/linux-generic/include/odp_packet_internal.h
index d09231e..35d1ca2 100644
--- a/platform/linux-generic/include/odp_packet_internal.h
+++ b/platform/linux-generic/include/odp_packet_internal.h
@@ -167,7 +167,7 @@ typedef struct {
  * packet_init(). Because of this any new fields added must be reviewed for
  * initialization requirements.
  */
-typedef struct {
+typedef struct odp_packet_hdr_t {
/* common buffer header */
odp_buffer_hdr_t buf_hdr;
 
@@ -178,6 +178,10 @@ typedef struct {
uint32_t headroom;
uint32_t tailroom;
 
+   uint32_t ref_offset;
+   struct odp_packet_hdr_t *ref_hdr;
+   odp_atomic_u32_t ref_count;
+
odp_pktio_t input;
 
/* Members below are not initialized by packet_init() */
@@ -200,6 +204,38 @@ static inline odp_packet_hdr_t 
*odp_packet_hdr(odp_packet_t pkt)
return (odp_packet_hdr_t *)buf_hdl_to_hdr((odp_buffer_t)pkt);
 }
 
+static inline odp_packet_hdr_t *odp_packet_last_hdr(odp_packet_t pkt,
+   uint32_t *offset)
+{
+   odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);
+   uint32_t ref_offset = 0;
+
+   while (pkt_hdr->ref_hdr) {
+   ref_offset = pkt_hdr->ref_offset;
+   pkt_hdr= pkt_hdr->ref_hdr;
+   }
+
+   if (offset)
+   *offset = ref_offset;
+   return pkt_hdr;
+}
+
+static inline odp_packet_hdr_t *odp_packet_prev_hdr(odp_packet_hdr_t *pkt_hdr,
+   odp_packet_hdr_t *cur_hdr,
+   uint32_t *offset)
+{
+   uint32_t ref_offset = 0;
+
+   while (pkt_hdr->ref_hdr != cur_hdr) {
+   ref_offset = pkt_hdr->ref_offset;
+   pkt_hdr= pkt_hdr->ref_hdr;
+   }
+
+   if (offset)
+   *offset = ref_offset;
+   return pkt_hdr;
+}
+
 static inline void copy_packet_parser_metadata(odp_packet_hdr_t *src_hdr,
   odp_packet_hdr_t *dst_hdr)
 {
@@ -227,7 +263,16 @@ static inline void pull_tail(odp_packet_hdr_t *pkt_hdr, 
uint32_t len)
 
 static inline uint32_t packet_len(odp_packet_hdr_t *pkt_hdr)
 {
-   return pkt_hdr->frame_len;
+   uint32_t pkt_len = 0;
+   uint32_t offset  = 0;
+
+   do {
+   pkt_len += pkt_hdr->frame_len - offset;
+   offset   = pkt_hdr->ref_offset;
+   pkt_hdr  = pkt_hdr->ref_hdr;
+   } while (pkt_hdr);
+
+   return pkt_len;
 }
 
 static inline void packet_set_len(odp_packet_hdr_t *pkt_hdr, uint32_t len)
diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index 0d3fd05..0e01c0d 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -36,7 +36,8 @@ static inline uint32_t packet_seg_len(odp_packet_hdr_t 
*pkt_hdr,
return pkt_hdr->buf_hdr.seg[seg_idx].len;
 }
 
-static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t 
seg_idx)
+static inline uint8_t *packet_seg_data(odp_packet_hdr_t *pkt_hdr,
+  uint32_t seg_idx)
 {
return pkt_hdr->buf_hdr.seg[seg_idx].data;
 }
@@ -137,7 +138,14 @@ static inline void *packet_map(odp_packet_hdr_t *pkt_hdr,
int seg = 0;
int seg_count = pkt_hdr->buf_hdr.segcount;
 
-   if (odp_unlikely(offset >= pkt_hdr->frame_len))
+   /* Special processing for references */
+   while (offset >= pkt_hdr->frame_len && pkt_hdr->ref_hdr) {
+   offset   -= (pkt_hdr->frame_len - pkt_hdr->ref_offset);
+   pkt_hdr   = pkt_hdr->ref_hdr;
+   seg_count = pkt_hdr->buf_hdr.segcount;
+   }
+
+   if (odp_unlikely(offset > pkt_hdr->frame_len))
return NULL;
 
if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || seg_count == 1)) {
@@ -231,6 +239,13 @@ static inline void packet_init(odp_packet_hdr_t *pkt_hdr, 
uint32_t len,
 CONFIG_PACKET_TAILROOM;
 
pkt_hdr->input = ODP_PKTIO_INVALID;
+
+   /* By default packet has no references */
+   pkt_hdr->ref_hdr = NULL;
+   pkt_hdr->ref_offset = 0;
+
+   /* Start with a ref count of 1 since packet is allocated */
+   odp_atomic_init_u32(_hdr->ref_count, 1);
 }
 
 static inline odp_packet_hdr_t *init_segments(odp_buffer_t 

Re: [lng-odp] [API-NEXT PATCH 0/5] name argument definitions in *_create() functions

[Elo, Matias (Nokia - FI/Espoo)]

Ping. This patch set has been reviewed and tested.

Reviewed-and-tested-by: Bill Fischofer 
>
Reviewed-by: Petri Savolainen 
>


On 24 Oct 2016, at 13:44, Elo, Matias (Nokia - FI/Espoo) 
> wrote:


Ping. This patch set has been reviewed and tested.

-Matias

On 20 Oct 2016, at 1.08, Bill Fischofer 
>
 wrote:

For this series:

Reviewed-and-tested-by: Bill Fischofer 
>

On Fri, Oct 14, 2016 at 3:49 AM, Matias Elo 
>
 wrote:
This patch set improves name argument definitions in *_create() functions to
clearly state when NULL is a valid value and when the name string doesn't have
to be unique.

The related function implementions are fixed to handle NULL name values
correctly. NULL values are also tested in the validation suite.

Note: NULL name values are not yet working (and tested) for odp_pool_create()
and odp_timer_pool_create(). Fixing these functions would require editing odp
shared memory code which is currently being updated in a separate patch set.

Matias Elo (5):
 api: improve name argument definitions in *_create() functions
 linux-gen: schedule: fix creating event group with no name
 linux-gen: queue: fix creating queue with no name
 linux-gen: classification: fix creating cos with no name
 linux-gen: timer: fix creating timer pool with no name

include/odp/api/spec/classification.h  | 10 +
include/odp/api/spec/pool.h| 17 +-
include/odp/api/spec/queue.h   |  9 +---
include/odp/api/spec/schedule.h| 12 +-
include/odp/api/spec/timer.h   |  5 -
platform/linux-generic/odp_classification.c| 11 ++---
platform/linux-generic/odp_queue.c |  8 +--
platform/linux-generic/odp_schedule.c  | 26 +++---
platform/linux-generic/odp_schedule_sp.c   | 11 +++--
platform/linux-generic/odp_timer.c | 14 
.../api/classification/odp_classification_basic.c  |  4 +---
test/common_plat/validation/api/queue/queue.c  | 10 -
.../validation/api/scheduler/scheduler.c   |  9 +++-
13 files changed, 97 insertions(+), 49 deletions(-)

--
2.7.4

[lng-odp] [API-NEXT PATCHv2 5/5] doc: userguide: add user documentation for packet references

[Bill Fischofer]

Signed-off-by: Bill Fischofer 
---
 doc/users-guide/users-guide-packet.adoc | 261 +++-
 1 file changed, 260 insertions(+), 1 deletion(-)

diff --git a/doc/users-guide/users-guide-packet.adoc 
b/doc/users-guide/users-guide-packet.adoc
index e3be23c..e34bbd2 100644
--- a/doc/users-guide/users-guide-packet.adoc
+++ b/doc/users-guide/users-guide-packet.adoc
@@ -246,7 +246,7 @@ packet pool as the original packet.
 The opposite operation is performed by the `odp_packet_concat()` API. This API
 takes a destination and source packet as arguments and the result is that
 the source packet is concatenated to the destination packet and ceases to
-have any separete identity. Note that it is legal to concatenate a packet to
+have any separate identity. Note that it is legal to concatenate a packet to
 itself, in which case the result is a packet with double the length of the
 original packet.
 
@@ -282,3 +282,262 @@ larger than the underlying segment size. The call may 
also fail if the
 requested alignment is too high. Alignment limits will vary among different ODP
 implementations, however ODP requires that all implementations support
 requested alignments of at least 32 bytes.
+
+ Packet References
+To support efficient multicast, retransmit, and related processing, ODP
+supports two additional types of packet manipulation: static and dynamic
+_references_.
+
+= Static References
+The simplest type of reference is the _static reference_. A static reference is
+created by the call:
+
+[source,c]
+-
+ref_pkt = odp_packet_ref_static(base_pkt);
+-
+
+If the reference fails, `ODP_PACKET_INVALID` is returned and `base_pkt`
+remains unchanged.
+
+The effect of this call is shown below:
+
+.Static Packet Reference
+image::refstatic.svg[align="center"]
+
+A static reference provides a simple and efficient means of creating an alias
+for a packet handle that prevents the packet itself from being freed until all
+references to it have been released via `odp_packet_free()` calls. This is
+useful, for example, to support retransmission processing, since as part of
+packet TX processing, `odp_pktout_send()` or `odp_tm_enq()` will free
+the packet after it has been transmitted.
+
+`odp_packet_ref_static()` might be used in a transmit routine wrapper
+function like:
+
+[source,c]
+-
+int xmit_pkt(odp_pktout_queue_t queue, odp_packet_t pkt)
+{
+   odp_packet_t ref = odp_packet_ref_static(pkt);
+   return ref == ODP_PACKET_INVALID ? -1 : odp_pktout_send(queue, ref, 1);
+}
+-
+
+This transmits a reference to `pkt` so that `pkt` is retained by the caller,
+which means that the caller is free to retransmit it if needed at a later
+time. When a higher level protocol (_e.g.,_ receipt of a TCP ACK packet)
+confirms that the transmission was successful, `pkt` can then be discarded via
+an `odp_packet_free()` call.
+
+The key characteristic of a static alias is that because
+there are multiple independent handles that refer to the same packet, the
+caller should treat the packet as read only following the creation of a
+reference until all other references are freed. This is because all static
+references are simply aliases of the same packet, so if multiple threads were
+independently manipulating the packet this would lead to unpredictable race
+conditions.
+
+To assist in determining whether there are other references to a packet, ODP
+provides the API:
+
+[source,c]
+-
+int odp_packet_has_ref(odp_packet_t pkt);
+-
+
+that indicates how many other references to `pkt` exist. If this routine
+returns 0 then `pkt` has no references and hence the caller can be assured
+that it is safe to modify it. Otherwise for best safety and portability, the
+caller should treat `pkt` as read-only.
+
+= Dynamic References
+While static references are convenient and efficient, they are limited by the
+need to be treated as read-only. For example, consider an application that
+needs to _multicast_ a packet. Here the same packet needs to be sent to two or
+more different destinations. While the packet payload may be the same, each
+sent copy of the packet requires its own unique header to specify the
+destination that is to receive the packet.
+
+To address this need, ODP provides _dynamic references_. These are created
+by the call:
+
+[source,c]
+-
+ref_pkt = odp_packet_ref(base_pkt, offset);
+-
+
+The `offset` parameter specifies the byte offset into `base_pkt` at which the
+reference is to begin. This must be in the range
+0..`odp_packet_len(base_pkt)`-1. As before, if the reference is unable to be
+created `ODP_PACKET_INVALID` is returned and `base_pkt` is unchanged,
+otherwise the result is as shown below:
+
+.Dynamic Packet Reference
+image::ref.svg[align="center"]
+
+Once again, following a successful reference the `base_pkt` should be treated
+as read-only since multiple references may point into it, however each
+reference has its own 

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Savolainen, Petri (Nokia - FI/Espoo)]

Something like this would not need application to load any drivers, just open 
pktio with correct name
1) Place all driver libs into a well know directory e.g. odp/drv/libs (user can 
which drivers to install into this dir)
2) Only drivers in the directory are considered available
3) Names and drivers are assigned to interfaces, if not done already by the 
system. Today this is done outside of ODP, with driver interface it could be 
done also during global_init()
4) odp_global_init() finds out which libs (== drivers) are in the dir and may 
be uses also config file (which driver is preferred for an interface)
5) The drv interface includes function calls for odp to request additional 
information about a driver: which NICs are supported, which drivers are already 
connected to which interface, ..., etc. Anyway, after this point ODP knows 
mapping between name ("eth0") -> interface (PCIe slot 5, 10GE port #2) -> 
driver ("odpdrv_niantic.so")
6) Application opens pktios with the interface names it got from the user 
("eth0"). Driver may be "loaded" as part of pktio_open() if not done already, 
at least the drv interface functions are bound to the correct lib.


-Petri


From: Christophe Milard [mailto:christophe.mil...@linaro.org] 
Sent: Wednesday, November 16, 2016 1:49 PM
To: Francois Ozog 
Cc: Savolainen, Petri (Nokia - FI/Espoo) 
; yi...@linaro.org; 
mike.hol...@linaro.org; bill.fischo...@linaro.org; lng-odp@lists.linaro.org
Subject: Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function 
added

Command line parameters will go to the application. As far as I know, we have 
no ODP parameters at this time, and they would require some command line 
processing cooperation with the application (main(argv, argc)) is in the app, 
we are just a lib). That put apps using getopts in a bad position. argparse is 
really the command line parser which thought about subparsing, but is not 
always used.
Command line parameters also assume a command :-). If ODP comes with a 
shell-less system this becomes less obvious.
I agree that in some cases (command line from linux) it would be clearer, but...

Scanning a directory makes assumption on the directory structure, unless we can 
specify that directory from the north interface (and then, aren't we back more 
or less to the initial proposal?), or command line.

I have hard to see the problem with enabling driver loading from the north 
interface: The apps which needs a specific driver (because they run on a 
specific HW) have a chance to load what they need. 
I agree a set of default driver would make sense, and possibly command line 
args if they come into picture. But dynamic loading make sense to me: both 
command line args and directory scan implies init-only driver loading, right? 
(of course on linux one could receive an event on directory write...)

In dpdk, many drivers come with dpdk. Possibly we could do that too, and that 
could reduce the usage on the driver_load function...

Christophe



On 16 November 2016 at 12:20, Francois Ozog  wrote:
Why not just scanning a directory or give a command line parameter?

On 16 November 2016 at 12:05, Christophe Milard  
wrote:
So what?
If we don't allow to load a driver from the north interface, where should they 
be loaded from??
Or are we saying that we go for a predefined list of driver only?

So drivers have to be .so files (because of autotools), but we don't give the 
possibility to load a free driver??

Christophe 

On 16 November 2016 at 11:45, Francois Ozog  wrote:
Hello,

If the north API is the one visible by ODP applications then I don't think it 
is a good idea to expose that.
DPDK exposed it at the beginning and is now internal.

That said there must be a standard way to manage drivers fir the benefit of the 
device framework.

I don't think the idea of integrating it with packetio open because packetio is 
not really a device. It is an abstract way of dealing with ports which may be 
hosted on a device (NIC) or on the platform (SoC). 
It can be done as a PoC (that's what I do with virtio-net exploratiry project) 
but that is not a long term solution.

FF


Le mercredi 16 novembre 2016, Christophe Milard  
a écrit :
On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
 wrote:
>
>>  /**
>> + * Driver loading
>> + *
>> + * This function is used by the application to load NIC drivers into ODP.
>> + * Calls to this function are optional, but should be performed (if any)
>> + * after odp_init_global
>> + *
>> + *
>> + * @param filename        Driver shared lib (dynamic library)
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on failure
>> + *
>> + */
>> +int odp_load_driver(const char *filename);
>
> Why application should explicitly need to load a driver ? Why it cannot 

Re: [lng-odp] [API-NEXT PATCH 0/5] name argument definitions in *_create() functions

[Maxim Uvarov]

Merged,
Maxim.

On 11/16/16 13:43, Elo, Matias (Nokia - FI/Espoo) wrote:


Ping. This patch set has been reviewed and tested.

Reviewed-and-tested-by: Bill Fischofer >
Reviewed-by: Petri Savolainen >



On 24 Oct 2016, at 13:44, Elo, Matias (Nokia - FI/Espoo) 
> wrote:



Ping. This patch set has been reviewed and tested.

-Matias

On 20 Oct 2016, at 1.08, Bill Fischofer > 
wrote:


For this series:

Reviewed-and-tested-by: Bill Fischofer >


On Fri, Oct 14, 2016 at 3:49 AM, Matias Elo > wrote:
This patch set improves name argument definitions in *_create() 
functions to
clearly state when NULL is a valid value and when the name string 
doesn't have

to be unique.

The related function implementions are fixed to handle NULL name values
correctly. NULL values are also tested in the validation suite.

Note: NULL name values are not yet working (and tested) for 
odp_pool_create()
and odp_timer_pool_create(). Fixing these functions would require 
editing odp
shared memory code which is currently being updated in a separate 
patch set.


Matias Elo (5):
 api: improve name argument definitions in *_create() functions
 linux-gen: schedule: fix creating event group with no name
 linux-gen: queue: fix creating queue with no name
 linux-gen: classification: fix creating cos with no name
 linux-gen: timer: fix creating timer pool with no name

include/odp/api/spec/classification.h  | 10 +
include/odp/api/spec/pool.h| 17 +-
include/odp/api/spec/queue.h   |  9 +---
include/odp/api/spec/schedule.h| 12 +-
include/odp/api/spec/timer.h   |  5 -
platform/linux-generic/odp_classification.c| 11 ++---
platform/linux-generic/odp_queue.c |  8 +--
platform/linux-generic/odp_schedule.c  | 26 
+++---

platform/linux-generic/odp_schedule_sp.c   | 11 +++--
platform/linux-generic/odp_timer.c | 14 
.../api/classification/odp_classification_basic.c  |  4 +---
test/common_plat/validation/api/queue/queue.c  | 10 -
.../validation/api/scheduler/scheduler.c   |  9 +++-
13 files changed, 97 insertions(+), 49 deletions(-)

--
2.7.4

Re: [lng-odp] concat and split semantics

[Joe Savage]

> ODP specifies functional behavior of it's APIs. Implementations are free to
> choose whatever techniques or other internals they wish to realize this
> behavior. So things like lazy/deferred/batched operations are entirely
> possible as long as the external semantics are matched.

Sure, and that's really what I was asking: is there sufficient flexibility
in the defined behaviour of concat and split to allow for a zero-copy
implementation to exist in the context of fragmentation/reassembly?

>From your response I assume the answer is yes, but that wasn't necessarily
clear to me as an ODP newbie. Particularly, I'm unsure if implementations
are able to create packets with segment of differing lengths, as may be
useful in an efficient zero-copy software implementation.


> odp-linux is designed to provide a simple and clear reference
> implementation of the ODP APIs. These are not necessarily optimized for
> performance.

Sure, I totally appreciate that, but I'm more interested in the API in the
abstract here. What I'm trying to get at is whether split and concat were
designed with a particular use case in mind. If so, its not unreasonable to
think that implementations might optimise for this common use case, the
requirements of which may not match my desired behaviour for fragmentation
and reassembly. After all, it's not always the case that one size fits all,
and perhaps a disparity in requirements between seemingly similar packet
splitting or concatenation tasks could indicate that a new API would be
useful.

Not to say that I necessarily think doing this would be necessary here, but
just that I'm trying to better understand what you might expect a decidedly
"good" general implementation to do, and whether that would fit with my
desired behaviour for this task.


> The key difference between split/concat and references is that the latter
> imposes usage restrictions on the end-product while the former do not. So
> the "shared" portion of references should be treated as read/only by the 
> application.

Reading the proposal I completely agree that this is a key difference, but as
per my point above regarding how the expected usage of APIs will guide the
behaviour of implementations, I think it's entirely possible that there will
be other deep-rooted disparities between how implementations handle these
two seemingly similar concepts. As such, I'm just trying to figure out
whether — on the API level — you have an idea about what those differences
might be. And, indeed, how those differences might lead me to choosing to use
concat over references (or visa versa) when writing my reassembly code.

Re: [lng-odp] [API-NEXT PATCHv2 0/3] physical address query on DRV interface

[Yi He]

For this series of patch:

Reviewed-by: Yi He 


On 11 November 2016 at 23:05, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> NOTE: Must be applied on top of "getting physical addresses from
> _ishmphy", V2
>
> since V1:
> -name changes (Maxim, Francois)
> -comments errors (Maxim)
>
> Brings the physical address query functions on the driver interface.
>
> Christophe Milard (3):
>   drv: shm: function to query for physical addresses
>   linux-gen: drv: functions to query for physical addresses
>   test: drv: shm: adding test for physical address queries
>
>  include/odp/drv/spec/shm.h | 35
> 
>  platform/linux-generic/drv_shm.c   | 12 +++
>  platform/linux-generic/include/_ishmphy_internal.h |  3 +-
>  .../linux-generic/include/odp/drv/plat/shm_types.h |  3 ++
>  .../common_plat/validation/drv/drvshmem/drvshmem.c | 37
> ++
>  .../common_plat/validation/drv/drvshmem/drvshmem.h |  1 +
>  6 files changed, 90 insertions(+), 1 deletion(-)
>
> --
> 2.7.4
>
>

Re: [lng-odp] [API-NEXT PATCHv2 0/2] getting physical addresses from _ishmphy

[Yi He]

Sorry, strange to reply into a blank mail thread, my two comments are
solved and for this series of patch:

Reviewed-by: Yi He 

On 11 November 2016 at 22:50, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> since V1:
> -changes to match names suggested by Maxim. (Christophe)
>
> This patch series brings a set of functions to retrieve physical addresses
> (from virtual addresses). These functions will be needed to populate the
> physical addresses of packet segments. In the absence of iommu, drivers
> need to know physical addresses.
> The first patch brings the function to retrieve physical addresses.
> The second patch implements a debug function for printing out the mapping
> of some area.
> Here follows a dump for a 8Mb memry area:
> With Huge pages
> _ishmphy.c:290:_odp_ishmphy_dumpphy():Phy Dump:
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7ffb4fc0 <-->  Phy:
> 2a2c0   512 PFNs, 2097152 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7ffb4fe0 <-->  Phy:
> 2a2a0   512 PFNs, 2097152 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7ffb5000 <-->  Phy:
> 2a280   512 PFNs, 2097152 bytes
> _ishmphy.c:323:_odp_ishmphy_dumpphy():Virtual: 7ffb5020 <-->  Phy:
> 2a260   512 PFNs, 2097152 bytes
>
> Without Huge pages
> _ishmphy.c:290:_odp_ishmphy_dumpphy():Phy Dump:
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb812598000 <-->  Phy:
> 2a6c31000   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb812599000 <-->  Phy:
> 2a6c3   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb81259a000 <-->  Phy:
> 287bfd000   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb81259b000 <-->  Phy:
> 287bfc000   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb81259c000 <-->  Phy:
> 285a79000   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb81259d000 <-->  Phy:
> 285a78000   1 PFNs, 4096 bytes
> _ishmphy.c:309:_odp_ishmphy_dumpphy():Virtual: 7fb81259e000 <-->  Phy:
> 3a8a11000   1 PFNs, 4096 bytes
> ...
>
>
> Christophe Milard (2):
>   linux-gen: _ishmphy: adding function for physical address query
>   linux-gen: _ishmphy: adding debug function for pysical address mapping
>
>  platform/linux-generic/_ishmphy.c  | 140
> +
>  platform/linux-generic/include/_ishmphy_internal.h |  19 +++
>  2 files changed, 159 insertions(+)
>
> --
> 2.7.4
>
>

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Savolainen, Petri (Nokia - FI/Espoo)]

From: Bill Fischofer [mailto:bill.fischo...@linaro.org] 
Sent: Monday, November 14, 2016 3:37 AM
To: Savolainen, Petri (Nokia - FI/Espoo) 
Cc: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added 
support for segmented packets



On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen  
wrote:
Added support for multi-segmented packets. The first segments
is the packet descriptor, which contains all metadata and
pointers to other segments.

Signed-off-by: Petri Savolainen 
---
 .../include/odp/api/plat/packet_types.h            |   6 +-
 .../linux-generic/include/odp_buffer_inlines.h     |  11 -
 .../linux-generic/include/odp_buffer_internal.h    |  23 +-
 .../linux-generic/include/odp_config_internal.h    |  39 +-
 .../linux-generic/include/odp_packet_internal.h    |  80 +--
 platform/linux-generic/include/odp_pool_internal.h |   3 -
 platform/linux-generic/odp_buffer.c                |   8 +-
 platform/linux-generic/odp_crypto.c                |   8 +-
 platform/linux-generic/odp_packet.c                | 712 +
 platform/linux-generic/odp_pool.c                  | 123 ++--
 platform/linux-generic/pktio/netmap.c              |   4 +-
 platform/linux-generic/pktio/socket.c              |   3 +-
 12 files changed, 692 insertions(+), 328 deletions(-)





diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index 2eee775..a5c6ff4 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -20,12 +20,155 @@
 #include 
 #include 

-/*
- *
- * Alloc and free
- * 
- *
- */
+static inline odp_packet_t packet_handle(odp_packet_hdr_t *pkt_hdr)
+{
+       return (odp_packet_t)pkt_hdr->buf_hdr.handle.handle;
+}
+
+static inline odp_buffer_t buffer_handle(odp_packet_hdr_t *pkt_hdr)
+{
+       return pkt_hdr->buf_hdr.handle.handle;
+}
+
+static inline uint32_t packet_seg_len(odp_packet_hdr_t *pkt_hdr,
+                                     uint32_t seg_idx)
+{
+       return pkt_hdr->buf_hdr.seg[seg_idx].len;
+}
+
+static inline void *packet_seg_data(odp_packet_hdr_t *pkt_hdr, uint32_t 
seg_idx)

This is more usefully typed as returning uint8_t * rather than void * to permit 
easy arithmetic on the result. The uint8_t * converts automatically to void * 
returns for the external API calls that use this.




This is actually the only place packet_seg_data() is called. The API function 
returns (void *), so the inline function fits that. 

void *odp_packet_seg_data(odp_packet_t pkt, odp_packet_seg_t seg)
{
odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);

if (odp_unlikely(seg >= pkt_hdr->buf_hdr.segcount))
return NULL;

return packet_seg_data(pkt_hdr, seg);
}

Also (void *) can be assigned to any pointer type without cast. For example:

uint8_t *data;
data = packet_seg_data(pkt_hdr, seg);


So, I think there's no benefit to change.




+{
+       return pkt_hdr->buf_hdr.seg[seg_idx].data;
+}
+
+static inline int packet_last_seg(odp_packet_hdr_t *pkt_hdr)
+{
+       if (CONFIG_PACKET_MAX_SEGS == 1)
+               return 0;
+       else
+               return pkt_hdr->buf_hdr.segcount - 1;
+}
+
+static inline uint32_t packet_first_seg_len(odp_packet_hdr_t *pkt_hdr)
+{
+       return packet_seg_len(pkt_hdr, 0);
+}
+
+static inline uint32_t packet_last_seg_len(odp_packet_hdr_t *pkt_hdr)
+{
+       int last = packet_last_seg(pkt_hdr);
+
+       return packet_seg_len(pkt_hdr, last);
+}
+
+static inline void *packet_data(odp_packet_hdr_t *pkt_hdr)
+{
+       return pkt_hdr->buf_hdr.seg[0].data;


Given the earlier definition of the packet_seg_data helper this would more 
consistently be:

        return packet_seg_data(pkt_hdr, 0);
 





There's no need to change the return type of packet_seg_data() to be able to 
use it here.

Anyway, I decided not to hide buf_hdr.seg[0].data reads behind a function call, 
since this way it's easy to find all reads and *writes* of it throughout the 
file.





+}
+
+static inline void *packet_tail(odp_packet_hdr_t *pkt_hdr)
+{
+       int last = packet_last_seg(pkt_hdr);
+       uint32_t seg_len = pkt_hdr->buf_hdr.seg[last].len;
+
+       return pkt_hdr->buf_hdr.seg[last].data + seg_len;
+}
+
+static inline void push_head(odp_packet_hdr_t *pkt_hdr, uint32_t len)
+{
+   pkt_hdr->headroom  -= len;
+   pkt_hdr->frame_len += len;
+   pkt_hdr->buf_hdr.seg[0].data -= len;
+   pkt_hdr->buf_hdr.seg[0].len  += len;
+}




 int odp_packet_num_segs(odp_packet_t pkt)
 {
-       return odp_packet_hdr(pkt)->buf_hdr.segcount;
+       odp_packet_hdr_t *pkt_hdr = odp_packet_hdr(pkt);
+
+       return pkt_hdr->buf_hdr.segcount;
 }

 odp_packet_seg_t odp_packet_first_seg(odp_packet_t pkt)
 {
-       return (odp_packet_seg_t)pkt;
+       (void)pkt;
+
+       return 

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Francois Ozog]

Hello,

If the north API is the one visible by ODP applications then I don't think
it is a good idea to expose that.
DPDK exposed it at the beginning and is now internal.

That said there must be a standard way to manage drivers fir the benefit of
the device framework.

I don't think the idea of integrating it with packetio open because
packetio is not really a device. It is an abstract way of dealing with
ports which may be hosted on a device (NIC) or on the platform (SoC).
It can be done as a PoC (that's what I do with virtio-net exploratiry
project) but that is not a long term solution.

FF

Le mercredi 16 novembre 2016, Christophe Milard <
christophe.mil...@linaro.org> a écrit :

> On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
> > wrote:
> >
> >>  /**
> >> + * Driver loading
> >> + *
> >> + * This function is used by the application to load NIC drivers into
> ODP.
> >> + * Calls to this function are optional, but should be performed (if
> any)
> >> + * after odp_init_global
> >> + *
> >> + *
> >> + * @param filenameDriver shared lib (dynamic library)
> >> + *
> >> + * @retval 0 on success
> >> + * @retval <0 on failure
> >> + *
> >> + */
> >> +int odp_load_driver(const char *filename);
> >
> > Why application should explicitly need to load a driver ? Why it cannot
> happen as part of odp_pktio_open() ?
> >
> > To me a better approach would be:
> > * user reads implementation documentation or uses system commands to
> find a list of available interfaces
> > * odp_global_init() finds out available pktio interfaces and drivers for
> those
> > * application opens the interface the user commands (just like today)
> > * pktio_open call loads the driver
> >
> > Or with more ODP support:
> > * odp_global_init() finds out available pktio interfaces and drivers for
> those
> > * a new API call lists all available interfaces
> > * application chooses an interface from the list and calls open
> > * pktio_open call loads the driver
> >
> >
> > -Petri
>
>
> Having ODP finding the driver by itself means "hard-coding" the list
> of available drivers (in the sense that ODP must know in advance what
> drivers matches what interface and where to find these drivers).
>
> The approach above let people do their own driver and attach it to ODP
> whithout any pre-requirements on ODP.
> When we have a set of "drivers we like", nothing prevent us from
> loading a default set of drivers at init.
> Unless the driver name (and path) is given as part as the pktio device
> name, which would just be awfull, we need such a load function to
> enable private driver loading.
>
> Moreover, the driver must be loaded to be probed: The drivers must
> tell what they can do, not ODP.
>
> I think the application should as a little aware as possible of
> drivers (and what HW they match) but should be given a way to load
> private drivers. The approach here enable that. Load your own driver
> (or none if it is in the default set), then open pktios without
> specifying any driver (note that 1 driver may be handling many
> interfaces)
>
> I think having an API call to list available interfaces makes sense,
> and does stress my proposal: Drivers must be loaded and probed to
> build that list.
>
> Christophe.
>


-- 
[image: Linaro] 
François-Frédéric Ozog | *Director Linaro Networking Group*
T: +33.67221.6485
francois.o...@linaro.org | Skype: ffozog

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Christophe Milard]

Command line parameters will go to the application. As far as I know, we
have no ODP parameters at this time, and they would require some command
line processing cooperation with the application (main(argv, argc)) is in
the app, we are just a lib). That put apps using getopts in a bad position.
argparse is really the command line parser which thought about subparsing,
but is not always used.
Command line parameters also assume a command :-). If ODP comes with a
shell-less system this becomes less obvious.
I agree that in some cases (command line from linux) it would be clearer,
but...

Scanning a directory makes assumption on the directory structure, unless we
can specify that directory from the north interface (and then, aren't we
back more or less to the initial proposal?), or command line.

I have hard to see the problem with enabling driver loading from the north
interface: The apps which needs a specific driver (because they run on a
specific HW) have a chance to load what they need.
I agree a set of default driver would make sense, and possibly command line
args if they come into picture. But dynamic loading make sense to me: both
command line args and directory scan implies init-only driver loading,
right? (of course on linux one could receive an event on directory write...)

In dpdk, many drivers come with dpdk. Possibly we could do that too, and
that could reduce the usage on the driver_load function...

Christophe



On 16 November 2016 at 12:20, Francois Ozog 
wrote:

> Why not just scanning a directory or give a command line parameter?
>
> On 16 November 2016 at 12:05, Christophe Milard <
> christophe.mil...@linaro.org> wrote:
>
>> So what?
>> If we don't allow to load a driver from the north interface, where should
>> they be loaded from??
>> Or are we saying that we go for a predefined list of driver only?
>>
>> So drivers have to be .so files (because of autotools), but we don't give
>> the possibility to load a free driver??
>>
>> Christophe
>>
>> On 16 November 2016 at 11:45, Francois Ozog 
>> wrote:
>>
>>> Hello,
>>>
>>> If the north API is the one visible by ODP applications then I don't
>>> think it is a good idea to expose that.
>>> DPDK exposed it at the beginning and is now internal.
>>>
>>> That said there must be a standard way to manage drivers fir the benefit
>>> of the device framework.
>>>
>>> I don't think the idea of integrating it with packetio open because
>>> packetio is not really a device. It is an abstract way of dealing with
>>> ports which may be hosted on a device (NIC) or on the platform (SoC).
>>> It can be done as a PoC (that's what I do with virtio-net exploratiry
>>> project) but that is not a long term solution.
>>>
>>> FF
>>>
>>>
>>> Le mercredi 16 novembre 2016, Christophe Milard <
>>> christophe.mil...@linaro.org> a écrit :
>>>
 On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
  wrote:
 >
 >>  /**
 >> + * Driver loading
 >> + *
 >> + * This function is used by the application to load NIC drivers
 into ODP.
 >> + * Calls to this function are optional, but should be performed (if
 any)
 >> + * after odp_init_global
 >> + *
 >> + *
 >> + * @param filenameDriver shared lib (dynamic library)
 >> + *
 >> + * @retval 0 on success
 >> + * @retval <0 on failure
 >> + *
 >> + */
 >> +int odp_load_driver(const char *filename);
 >
 > Why application should explicitly need to load a driver ? Why it
 cannot happen as part of odp_pktio_open() ?
 >
 > To me a better approach would be:
 > * user reads implementation documentation or uses system commands to
 find a list of available interfaces
 > * odp_global_init() finds out available pktio interfaces and drivers
 for those
 > * application opens the interface the user commands (just like today)
 > * pktio_open call loads the driver
 >
 > Or with more ODP support:
 > * odp_global_init() finds out available pktio interfaces and drivers
 for those
 > * a new API call lists all available interfaces
 > * application chooses an interface from the list and calls open
 > * pktio_open call loads the driver
 >
 >
 > -Petri


 Having ODP finding the driver by itself means "hard-coding" the list
 of available drivers (in the sense that ODP must know in advance what
 drivers matches what interface and where to find these drivers).

 The approach above let people do their own driver and attach it to ODP
 whithout any pre-requirements on ODP.
 When we have a set of "drivers we like", nothing prevent us from
 loading a default set of drivers at init.
 Unless the driver name (and path) is given as part as the pktio device
 name, which would just be awfull, we need such a load function to
 enable private driver 

Re: [lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Bill Fischofer]

Since we did make some minor changes to the Traffic Manager API in this
release, need to discuss whether this should be v1.10.2.0 or v1.12.0.0.
I'll send a v2 if we want to pick a different release number.

It would be good to bump this to v1.12 since Monarch-LTS is v1.11 and this
is a higher level so continuing with v1.10 looks odd here even apart from
the API difference.

On Wed, Nov 16, 2016 at 9:06 PM, Bill Fischofer 
wrote:

> Signed-off-by: Bill Fischofer 
> ---
>  CHANGELOG | 211 ++
> 
>  1 file changed, 211 insertions(+)
>
> diff --git a/CHANGELOG b/CHANGELOG
> index d8230cd..3ab7354 100644
> --- a/CHANGELOG
> +++ b/CHANGELOG
> @@ -1,3 +1,214 @@
> +== OpenDataPlane (1.10.2.0)
> +
> +=== New Features
> +
> + APIs
> +ODP v1.10.2.0 is a minor API revision level and introduces a small change
> to
> +the Traffic Manager API as well as some documentation clarification
> surrounding
> +other APIs.
> +
> + Traffic Manager Egress Function
> +The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
> +(`egress_fcn_supported`) that indicates whether the TM system supports
> +a user-provided egress function. When specified this function is called to
> +"output" a packet rather than having TM transmit it directly to a PktIO
> +interface.
> +
> + Traffic Manager Coupling Change
> +The `odp_tm_egress_t` struct has been changed to associate a TM system
> with an
> +output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`. This
> makes
> +an explicit 1-to-1 map between a TM system and a PktIO interface.
> +
> + Default huge page size clarification
> +The documentation for the `odp_sys_huge_page_size()` API has been
> reworded to
> +clarify that this API refers to default huge page size.
> +
> + Strict Priority (SP) Scheduler
> +Building on the modular scheduler framework added in v1.10.1.0, an
> alternate
> +Strict Priority (SP) Scheduler is now available for latency-sensitive
> +workloads. Applications wishing to use the SP scheduler should specify
> +the `./configure` option `--enable-schedule-sp`. This scheduler
> emphasizes low
> +latency processing of high priority events at the expense of throughput.
> This
> +alternate scheduler is considered experimental and should not be used for
> +production at this time.
> +
> + Application Binary Interface (ABI) Support
> +Support is added to enable ODP applications to be binary compatible across
> +different implementations of ODP sharing the same Instruction Set
> Architecture
> +(ISA). This support introduces a new `configure` option:
> +
> +`--enable-abi-compat=yes`::
> +This is the default and specifies that the ODP library is to be built to
> +support ABI compatibility mode. In this mode ODP APIs are never inlined.
> ABI
> +compatibility ensures maximum application portability in cloud
> environments.
> +
> +`--enable-abi-compat=no`::
> +Specify this option to enable the inlining of ODP APIs. This may result in
> +improved performance at the cost of ABI compatibility and is suitable for
> +applications running in embedded environments.
> +
> +Note that ODP applications retain source code portability between ODP
> +implementations regardless of the ABI mode chosen. To move to a different
> ODP
> +application running on a different ISA, code need simply be recompiled
> against
> +that target ODP implementation.
> +
> + SCTP Parsing Support
> +The ODP classifier adds support for recognizing Stream Control
> Transmission
> +Protocol (SCTP) packets. The APIs for this were previously not
> implemented.
> +
> +=== Packaging and Implementation Refinements
> +
> + Remove dependency on Linux headers
> +ODP no longer has a dependency on Linux headers. This will help make the
> +odp-linux reference implementation more easily portable to non-Linux
> +environments.
> +
> + Remove dependency on helpers
> +The odp-linux implementation has been made independent of the helper
> library
> +to avoid circular dependency issues with packaging. Helper functions may
> use
> +ODP APIs, however ODP implementations should not use helper functions.
> +
> + Reorganization of `test` directory
> +The `test` directory has been reorganized to better support a unified
> approach
> +to ODP component testing. API tests now live in
> +`test/common_plat/validation/api` instead of the former
> +`test/validation`. With this change performance and validation tests, as
> well
> +as common and platform-specific tests can all be part of a unified test
> +hierarchy.
> +
> +The resulting test tree now looks like:
> +
> +.New `test` directory hierarchy
> +-
> +test
> +├── common_plat
> +│   ├── common
> +│   ├── m4
> +│   ├── miscellaneous
> +│   ├── performance
> +│   └── validation
> +│   └── api
> +│   ├── atomic
> +│   ├── barrier
> +│   ├── buffer
> +│   ├── classification
> +│   

Re: [lng-odp] [PATCH v2 1/2] test: perf: add new ordered pktio application

[Elo, Matias (Nokia - FI/Espoo)]

Ping.

> On 1 Nov 2016, at 10:53, Matias Elo  wrote:
> 
> Add new test application for ordered queue functionality and performance
> validation. The application sets sequence numbers to incoming packets using
> ordered pktin queues and ordered context locks. After being tagged packets
> are enqueued to atomic queues based on flow hash (IPv4 5-tuple). In atomic
> flow processing the sequence number is validated and packet is sent to
> selected output interface.
> 
> Main options:
> -m: Input queue type can be changed to atomic or parallel to enable easy
>performance comparison. With parallel input queues the packet order is
>not maintained.
> -r: Number of input queues per interface
> -f: Number of atomic flow queues per interface
> -e: Number of extra input processing rounds. This can be used to simulate
>"fat pipe" traffic processing.
> 
> Signed-off-by: Matias Elo 
> ---
> 
> V2:
> - Fixed missing/invalid documentation (Maxim)
> - Fixed resource leak (Maxim)
> - Check odp_shm_reserve() return value (Maxim)
> - Check that CPU count command line argument is valid
> 
> test/common_plat/performance/.gitignore  |1 +
> test/common_plat/performance/Makefile.am |7 +-
> test/common_plat/performance/dummy_crc.h |  493 
> test/common_plat/performance/odp_pktio_ordered.c | 1337 ++
> 4 files changed, 1837 insertions(+), 1 deletion(-)
> create mode 100644 test/common_plat/performance/dummy_crc.h
> create mode 100644 test/common_plat/performance/odp_pktio_ordered.c
> 
> diff --git a/test/common_plat/performance/.gitignore 
> b/test/common_plat/performance/.gitignore
> index 1527d25..8bb18f5 100644
> --- a/test/common_plat/performance/.gitignore
> +++ b/test/common_plat/performance/.gitignore
> @@ -3,6 +3,7 @@
> odp_atomic
> odp_crypto
> odp_l2fwd
> +odp_pktio_ordered
> odp_pktio_perf
> odp_sched_latency
> odp_scheduling
> diff --git a/test/common_plat/performance/Makefile.am 
> b/test/common_plat/performance/Makefile.am
> index f184609..790ddae 100644
> --- a/test/common_plat/performance/Makefile.am
> +++ b/test/common_plat/performance/Makefile.am
> @@ -5,6 +5,7 @@ TESTS_ENVIRONMENT += TEST_DIR=${builddir}
> EXECUTABLES = odp_crypto$(EXEEXT) odp_pktio_perf$(EXEEXT)
> 
> COMPILE_ONLY = odp_l2fwd$(EXEEXT) \
> +odp_pktio_ordered$(EXEEXT) \
>  odp_sched_latency$(EXEEXT) \
>  odp_scheduling$(EXEEXT)
> 
> @@ -22,15 +23,19 @@ bin_PROGRAMS = $(EXECUTABLES) $(COMPILE_ONLY)
> 
> odp_crypto_LDFLAGS = $(AM_LDFLAGS) -static
> odp_crypto_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> +odp_pktio_ordered_LDFLAGS = $(AM_LDFLAGS) -static
> +odp_pktio_ordered_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> odp_sched_latency_LDFLAGS = $(AM_LDFLAGS) -static
> odp_sched_latency_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> odp_scheduling_LDFLAGS = $(AM_LDFLAGS) -static
> odp_scheduling_CFLAGS = $(AM_CFLAGS) -I${top_srcdir}/test
> 
> noinst_HEADERS = \
> -   $(top_srcdir)/test/test_debug.h
> +   $(top_srcdir)/test/test_debug.h \
> +   dummy_crc.h
> 
> dist_odp_crypto_SOURCES = odp_crypto.c
> +dist_odp_pktio_ordered_SOURCES = odp_pktio_ordered.c
> dist_odp_sched_latency_SOURCES = odp_sched_latency.c
> dist_odp_scheduling_SOURCES = odp_scheduling.c
> dist_odp_pktio_perf_SOURCES = odp_pktio_perf.c
> diff --git a/test/common_plat/performance/dummy_crc.h 
> b/test/common_plat/performance/dummy_crc.h
> new file mode 100644
> index 000..38da444
> --- /dev/null
> +++ b/test/common_plat/performance/dummy_crc.h
> @@ -0,0 +1,493 @@
> +/* Copyright (c) 2016, Linaro Limited
> + * All rights reserved.
> + *
> + * SPDX-License-Identifier: BSD-3-Clause
> + */
> +
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2010-2013 Intel Corporation. All rights reserved.
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + *   notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + *   notice, this list of conditions and the following disclaimer in
> + *   the documentation and/or other materials provided with the
> + *   distribution.
> + * * Neither the name of Intel Corporation nor the names of its
> + *   contributors may be used to endorse or promote products derived
> + *   from this software without specific prior written permission.
> + *
> + *   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
> + *   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> + *   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
> + *   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL 

[lng-odp] [API-NEXT PATCHv7 12/13] linux_gen: _ishm: decreasing the number of error messages when no huge pages

[Christophe Milard]

Signed-off-by: Christophe Milard 
---
 platform/linux-generic/_ishm.c | 22 --
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/platform/linux-generic/_ishm.c b/platform/linux-generic/_ishm.c
index 0586a96..782c32f 100644
--- a/platform/linux-generic/_ishm.c
+++ b/platform/linux-generic/_ishm.c
@@ -439,8 +439,12 @@ static int create_file(int block_index, huge_flag_t huge, 
uint64_t len,
 
fd = open(filename, oflag, S_IRUSR | S_IWUSR | S_IRGRP | S_IROTH);
if (fd < 0) {
-   ODP_ERR("open failed for %s: %s.\n",
-   filename, strerror(errno));
+   if (huge == HUGE)
+   ODP_DBG("open failed for %s: %s.\n",
+   filename, strerror(errno));
+   else
+   ODP_ERR("open failed for %s: %s.\n",
+   filename, strerror(errno));
return -1;
}
 
@@ -762,6 +766,7 @@ int _odp_ishm_reserve(const char *name, uint64_t size, int 
fd,
void *addr = NULL;/* mapping address */
int new_proc_entry;
struct stat statbuf;
+   static int  huge_error_printed;   /* to avoid millions of error...*/
 
odp_spinlock_lock(_tbl->lock);
 
@@ -836,11 +841,16 @@ int _odp_ishm_reserve(const char *name, uint64_t size, 
int fd,
len = (size + (page_hp_size - 1)) & (-page_hp_size);
addr = do_map(new_index, len, hp_align, flags, HUGE, );
 
-   if (addr == NULL)
-   ODP_DBG("No huge pages, fall back to normal pages, "
-   "check: /proc/sys/vm/nr_hugepages.\n");
-   else
+   if (addr == NULL) {
+   if (!huge_error_printed) {
+   ODP_ERR("No huge pages, fall back to normal "
+   "pages. "
+   "check: /proc/sys/vm/nr_hugepages.\n");
+   huge_error_printed = 1;
+   }
+   } else {
new_block->huge = HUGE;
+   }
}
 
/* Try normal pages if huge pages failed */
-- 
2.7.4

Re: [lng-odp] Classification Queue Group

[Bala Manoharan]

Regards,
Bala


On 11 November 2016 at 13:26, Brian Brooks  wrote:
> On 11/10 15:17:15, Bala Manoharan wrote:
>> On 10 November 2016 at 13:26, Brian Brooks  wrote:
>> > On 11/07 16:46:12, Bala Manoharan wrote:
>> >> Hi,
>> >
>> > Hiya
>> >
>> >> This mail thread discusses the design of classification queue group
>> >> RFC. The same can be found in the google doc whose link is given
>> >> below.
>> >> Users can provide their comments either in this mail thread or in the
>> >> google doc as per their convenience.
>> >>
>> >> https://docs.google.com/document/d/1fOoG9WDR0lMpVjgMAsx8QsMr0YFK9slR93LZ8VXqM2o/edit?usp=sharing
>> >>
>> >> The basic issues with queues as being a single target for a CoS are two 
>> >> fold:
>> >>
>> >> Queues must be created and deleted individually. This imposes a
>> >> significant burden when queues are used to represent individual flows
>> >> since the application may need to process thousands (or millions) of
>> >> flows.
>> >
>> > Wondering why there is an issue with creating and deleting queues 
>> > individually
>> > if queue objects represent millions of flows..
>>
>> The queue groups are mainly required for hashing the incoming packets
>> to multiple flows based on the hash configuration.
>> So from application point of view it just needs a queue to have
>> packets belonging to same flow and that packets belonging to different
>> flows are placed in different queues respectively.It does not matter
>> who creates the flow/queue.
>
> When the application receives an event from odp_schedule() call, how does it
> know whether the odp_queue_t was previously created by the application from
> odp_queue_create() or whether it was created by the implementation?

odp_schedule() call returns the queue from which the event was part of.
The type of the queue can be got from odp_queue_type() API.
But the question is there an use-case where the application need to know?
The application has the information of the queue it has created and
the queues created by implementation are destroyed by implementation.

>
>> It is actually simpler if implementation creates a flow since in that
>> case implementation need not accumulate meta-data for all possible
>> hash values in a queue group and it can be created when traffic
>> arrives in that particular flow.
>>
>> >
>> > Could an application ever call odp_schedule() and receive an event (e.g. 
>> > packet)
>> > from a queue (of opaque type odp_queue_t) and that queue has never been 
>> > created
>> > by the application (via odp_queue_create())? Could that ever happen from 
>> > the
>> > hardware, and could the application ever handle that?
>>
>> No. All the queues in the system are created by the application either
>> directly or in-directly.
>> In-case of queue groups the queues are in-directly created by the
>> application by configuring a queue group.
>>
>> > Or, is it related to memory usage? The reference implementation
>> > struct queue_entry_s is 320 bytes on a 64-bit machine.
>> >
>> >   2^28 ~= 268,435,456 queues -> 81.920 GB
>> >   2^26 ~=  67,108,864 queues -> 20.480 GB
>> >   2^22 ~=   4,194,304 queues ->  1.280 GB
>> >
>> > Forget about 320 bytes per queue, if each queue was represented by a 32-bit
>> > integer (4 bytes!) the usage would be:
>> >
>> >   2^28 ~= 268,435,456 queues ->  1.024 GB
>> >   2^26 ~=  67,108,864 queues ->256 MB
>> >   2^22 ~=   4,194,304 queues -> 16 MB
>> >
>> > That still might be a lot of usage if the application must explicitly 
>> > create
>> > every queue (before it is used) and require an ODP implementation to map
>> > between every ODP queue object (opaque type) and the internal queue.
>> >
>> > Lets say ODP API has two classes of handles: 1) pointers, 2) integers. An 
>> > opaque
>> > pointer is used to point to some other software object. This object should 
>> > be
>> > larger than 64 bits (or 32 bits on a chip in 32-bit pointer mode) 
>> > otherwise it
>> > could just be represented in a 64-bit (or 32-bit) integer type value!
>> >
>> > To support millions of queues (flows) should odp_queue_t be an integer 
>> > type in
>> > the API? A software-only implementation may still use 320 bytes per queue 
>> > and
>> > use that integer as an index into an array or as a key for lookup 
>> > operation on a
>> > data structure containing queues. An implementation with hardware assist 
>> > may
>> > use this integer value directly when interfacing with hardware!
>>
>> I believe I have answered this question based on explanation above.
>> Pls feel free to point out if something is not clear.
>>
>> >
>> > Would it still be necessary to assign a "name" to each queue (flow)?
>>
>> "name" per queue might not be required since it would mean a character
>> based lookup across millions of items.
>>
>> >
>> > Would a queue (flow) also require an "op type" to explicitly specify 
>> > whether
>> > access to the queue (flow) is threadsafe? Atomic queues are 

[lng-odp] [API-NEXT PATCHv2 1/5] api: packet: add support for packet references

[Bill Fischofer]

Introduce three new APIs that support efficient sharing of portions of
packets.

odp_packet_ref_static() creates an alias for a base packet

odp_packet_ref() creates a reference to a base packet

odp_packet_ref_pkt() creates a reference to a base packet from a supplied
header packet

In addition, three other APIs simplify working with references

odp_packet_is_ref() says whether a packet is a reference

odp_packet_has_ref() says whether a packet has had a reference made to it

odp_packet_unshared_len() gives the length of the prefix bytes that are
unique to this reference. These are the only bytes of the packet that may
be modified safely.

Signed-off-by: Bill Fischofer 
---
 include/odp/api/spec/packet.h | 168 ++
 1 file changed, 168 insertions(+)

diff --git a/include/odp/api/spec/packet.h b/include/odp/api/spec/packet.h
index faf62e2..137024f 100644
--- a/include/odp/api/spec/packet.h
+++ b/include/odp/api/spec/packet.h
@@ -256,6 +256,23 @@ uint32_t odp_packet_seg_len(odp_packet_t pkt);
 uint32_t odp_packet_len(odp_packet_t pkt);
 
 /**
+ * Packet unshared data len
+ *
+ * Returns the sum of data lengths over all unshared packet segments. These
+ * are the only bytes that should be modified by the caller. The rest of the
+ * packet should be treated as read-only.
+ *
+ * This routine will return 0 if odp_packet_has_ref() != 0 and will be the
+ * same as odp_packet_len() if odp_packet_has_ref() == 0 and
+ * odp_packet_is_ref() == 0.
+ *
+ * @param pkt Packet handle
+ *
+ * @return Packet unshared data length
+ */
+uint32_t odp_packet_unshared_len(odp_packet_t pkt);
+
+/**
  * Packet headroom length
  *
  * Returns the current packet level headroom length.
@@ -847,6 +864,157 @@ int odp_packet_split(odp_packet_t *pkt, uint32_t len, 
odp_packet_t *tail);
 
 /*
  *
+ * References
+ * 
+ *
+ */
+
+/**
+ * Create a static reference to a packet
+ *
+ * A static reference is used to obtain an additional handle for referring to
+ * a packet so that the storage behind it is not freed until all references to
+ * the packet are freed. This can be used, for example, to support efficient
+ * retransmission processing.
+ *
+ * The intent of a static reference is that both the base packet and the
+ * returned reference will be treated as read-only after this call. Results
+ * are unpredictable if this restriction is not observed.
+ *
+ * Static references have restrictions but may have performance advantages on
+ * some platforms if the caller does not intend to modify the reference
+ * packet. If modification is needed (e.g., to prefix a unique header onto the
+ * packet) then odp_packet_ref() or odp_packet_ref_pkt() should be used.
+ *
+ * @param pktHandle of the base packet for which a static reference is
+ *   to be created.
+ *
+ * @returnHandle of the static reference packet
+ * @retval ODP_PACKET_INVALID Operation failed. Base packet remains unchanged.
+ */
+odp_packet_t odp_packet_ref_static(odp_packet_t pkt);
+
+/**
+ * Create a reference to a packet
+ *
+ * Create a dynamic reference to a base packet starting at a specified byte
+ * offset. This reference may be used as an argument to header manipulation
+ * APIs to prefix a unique header onto the shared base. The storage associated
+ * with the base packet is not freed until all references to it are freed,
+ * which permits easy multicasting or retransmission processing to be
+ * performed. Following a successful call, the base packet should be treated
+ * as read-only. Results are unpredictable if this restriction is not
+ * observed.
+ *
+ * This operation prepends a zero-length header onto the base packet beginning
+ * at the specified offset. This header is always drawn from the same pool as
+ * the base packet.
+ *
+ * Because references are unique packets, any bytes pushed onto the head of a
+ * reference via odp_packet_push_head() or odp_packet_extend_head() are unique
+ * to this reference and are not seen by the base packet or by any other
+ * reference to the same base packet.
+ *
+ * The base packet used as input to this routine may itself by a reference to
+ * some other base packet. Implementations MAY restrict the ability to create
+ * such compound references. Attempts to exceed any implementation limits in
+ * this regard will result in this call failing and returning
+ * ODP_PACKET_INVALID.
+ *
+ * If the caller does not indend to push a header onto the returned reference,
+ * the odp_packet_ref_static() API may be used. This may be a more efficient
+ * means of obtaining another reference to a base packet that will be treated
+ * as read-only.
+ *
+ * @param pktHandle of the base packet for which a reference is to be
+ *   created.
+ *
+ * @param offset Byte offset in the base packet at which the shared reference
+ *   is to begin. Must be in the range 

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Bill Fischofer]

Trying again as the repost doesn't seem to show up on the list either.

For this series:

Reviewed-and-tested-by: Bill Fischofer 

On Tue, Nov 15, 2016 at 5:55 PM, Bill Fischofer 
wrote:

> Reposting this since it doesn't seem to have made it to the mailing list.
>
> For this series:
>
> Reviewed-and-tested-by: Bill Fischofer 
>
> On Tue, Nov 15, 2016 at 8:41 AM, Bill Fischofer  > wrote:
>
>> For this series:
>>
>> Reviewed-and-tested-by: Bill Fischofer 
>>
>> On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
>> petri.savolai...@nokia.com> wrote:
>>
>>> Pool performance is optimized by using a ring as the global buffer
>>> storage.
>>> IPC build is disabled, since it needs large modifications due to
>>> dependency to
>>> pool internals. Old pool implementation was based on locks and linked
>>> list of
>>> buffer headers. New implementation maintain a ring of buffer handles,
>>> which
>>> enable fast, burst based allocs and frees. Also ring scales better with
>>> number
>>> of cpus than a list (enq and deq operations update opposite ends of the
>>> pool).
>>>
>>> L2fwd link rate (%), 2 x 40GE, 64 byte packets
>>>
>>> direct- parallel-   atomic-
>>> cpusorigdirect  difforigparall  difforigatomic
>>> diff
>>> 1   7 % 8 % 1 % 6 % 6 % 2 % 5.4 %   5.6 %
>>>  4 %
>>> 2   14 %15 %7 % 9 % 9 % 5 % 8 % 9 %
>>>  8 %
>>> 4   28 %30 %6 % 13 %14 %13 %12 %15 %
>>> 19 %
>>> 6   42 %44 %6 % 16 %19 %19 %8 % 20 %
>>> 150 %
>>> 8   46 %59 %28 %19 %23 %26 %18 %24 %
>>> 34 %
>>> 10  55 %57 %3 % 20 %27 %37 %8 % 28 %
>>> 264 %
>>> 12  56 %56 %-1 %22 %31 %43 %7 % 32 %
>>> 357 %
>>>
>>> Max packet rate of NICs are reached with 10-12 cpu on direct mode.
>>> Otherwise,
>>> all cases were improved. Especially, scheduler driven cases suffered on
>>> bad
>>> pool scalability.
>>>
>>> changed in v3:
>>> * rebased
>>> * ipc disabled with #ifdef
>>> * added support for multi-segment packets
>>> * API: added explicit limits for packet length in alloc calls
>>> * Corrected validation test and example application bugs found during
>>>   segmentation implementation
>>>
>>> changed in v2:
>>> * rebased to api-next branch
>>> * added a comment that ring size must be larger than number of items in
>>> it
>>> * fixed clang build issue
>>> * added parens in align macro
>>>
>>> v1 reviews:
>>> Reviewed-by: Brian Brooks 
>>>
>>>
>>>
>>> Petri Savolainen (19):
>>>   linux-gen: ipc: disable build of ipc pktio
>>>   linux-gen: pktio: do not free zero packets
>>>   linux-gen: ring: created common ring implementation
>>>   linux-gen: align: added round up power of two
>>>   linux-gen: pool: reimplement pool with ring
>>>   linux-gen: ring: added multi enq and deq
>>>   linux-gen: pool: use ring multi enq and deq operations
>>>   linux-gen: pool: optimize buffer alloc
>>>   linux-gen: pool: clean up pool inlines functions
>>>   linux-gen: pool: ptr instead of hdl in buffer_alloc_multi
>>>   test: validation: buf: test alignment
>>>   test: performance: crypto: use capability to select max packet
>>>   test: correctly initialize pool parameters
>>>   test: validation: packet: fix bugs in tailroom and concat tests
>>>   linux-gen: packet: added support for segmented packets
>>>   test: validation: packet: improved multi-segment alloc test
>>>   api: packet: added limits for packet len on alloc
>>>   linux-gen: packet: remove zero len support from alloc
>>>   linux-gen: packet: enable multi-segment packets
>>>
>>>  example/generator/odp_generator.c  |2 +-
>>>  include/odp/api/spec/packet.h  |9 +-
>>>  include/odp/api/spec/pool.h|6 +
>>>  platform/linux-generic/Makefile.am |1 +
>>>  .../include/odp/api/plat/packet_types.h|6 +-
>>>  .../include/odp/api/plat/pool_types.h  |6 -
>>>  .../linux-generic/include/odp_align_internal.h |   34 +-
>>>  .../linux-generic/include/odp_buffer_inlines.h |  167 +--
>>>  .../linux-generic/include/odp_buffer_internal.h|  120 +-
>>>  .../include/odp_classification_datamodel.h |2 +-
>>>  .../linux-generic/include/odp_config_internal.h|   55 +-
>>>  .../linux-generic/include/odp_packet_internal.h|   87 +-
>>>  platform/linux-generic/include/odp_pool_internal.h |  289 +---
>>>  platform/linux-generic/include/odp_ring_internal.h |  176 +++
>>>  .../linux-generic/include/odp_timer_internal.h |4 -
>>>  platform/linux-generic/odp_buffer.c|   22 +-
>>>  platform/linux-generic/odp_classification.c|   25 +-
>>>  

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Bill Fischofer]

git bisect shows this problem starts surfacing with this commit:

eea929d61e25106adc2598448c865f40e4a6f13b is the first bad commit
commit eea929d61e25106adc2598448c865f40e4a6f13b
Author: Petri Savolainen 
Date:   Thu Nov 10 13:07:39 2016 +0200

linux-gen: pool: reimplement pool with ring

Used the ring data structure to implement pool. Also
buffer structure was simplified to enable future driver
interface. Every buffer includes a packet header, so each
buffer can be used as a packet head or segment. Segmentation
was disabled and segment size was fixed to a large number
(64kB) to limit the number of modification in the commit.

Signed-off-by: Petri Savolainen 

I don't think the issue is necessarily with this patch but rather that the
efficiency improvements are probably exposing a latent race condition
elsewhere in ordered queue handling. This needs further investigation.

On Wed, Nov 16, 2016 at 3:01 PM, Bill Fischofer 
wrote:

> Trying to reproduce this I'm seeing sporadic failures in the scheduler
> validation test that don't seem to appear in the base api-next branch.
> Issue seems to be failures in the ordered queue tests:
>
>   Test: scheduler_test_multi_mq_mt_prio_n 
> ...linux.c:273:odpthread_run_start_routine():helper:
> ODP worker thread started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> passed
>   Test: scheduler_test_multi_mq_mt_prio_a 
> ...linux.c:273:odpthread_run_start_routine():helper:
> ODP worker thread started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> passed
>   Test: scheduler_test_multi_mq_mt_prio_o 
> ...linux.c:273:odpthread_run_start_routine():helper:
> ODP worker thread started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> linux.c:273:odpthread_run_start_routine():helper: ODP worker thread
> started as linux pthread. (pid=6274)
> FAILED
> 1. scheduler.c:871  - bctx->sequence == seq
> 2. scheduler.c:871  - bctx->sequence == seq
>   Test: scheduler_test_multi_1q_mt_a_excl 
> ...linux.c:273:odpthread_run_start_routine():helper:
> ODP worker thread started as linux pthread. (pid=6274)
>
> We had seen these earlier but they were never consistently reproducible.
> Petri: are you able to recreate this on your local systems?
>
>
> On Wed, Nov 16, 2016 at 2:03 PM, Maxim Uvarov 
> wrote:
>
>> I can not test patch by patch this series because it fails (one time it
>> was TM, one time kernel died, other time OOM killer killed tests then hang
>> kernel).
>>
>> And for all patches test/common_plat/validation/api/pktio/pktio_main
>> hangs forever:
>>
>>
>> Program received signal SIGINT, Interrupt.
>> 0x2afbe69ffb80 in __nanosleep_nocancel () at
>> ../sysdeps/unix/syscall-template.S:81
>> 81in ../sysdeps/unix/syscall-template.S
>> (gdb) bt
>> #0  0x2afbe69ffb80 in __nanosleep_nocancel () at
>> ../sysdeps/unix/syscall-template.S:81
>> #1  0x00415ced in odp_pktin_recv_tmo (queue=...,
>> packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
>> wait=wait@entry=18446744073709551615) at
>> ../../../platform/linux-generic/odp_packet_io.c:1584
>> #2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
>> pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
>> seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1,
>> mode=mode@entry=RECV_TMO, tmo=tmo@entry=18446744073709551615, ns=ns@entry
>> =0)
>> at ../../../../../../test/common_plat/validation/api/pktio/pkti
>> o.c:515
>> #3  0x004075f8 in test_recv_tmo (mode=RECV_TMO) at
>> ../../../../../../test/common_plat/validation/api/pktio/pktio.c:940
>> #4  0x2afbe61cc482 in run_single_test () from
>> /usr/local/lib/libcunit.so.1
>> #5  0x2afbe61cc0b2 in run_single_suite () from
>> /usr/local/lib/libcunit.so.1
>> #6  0x2afbe61c9d55 in CU_run_all_tests () from
>> /usr/local/lib/libcunit.so.1
>> #7  0x2afbe61ce245 in basic_run_all_tests () from
>> /usr/local/lib/libcunit.so.1
>> #8  0x2afbe61cdfe7 in CU_basic_run_tests () from
>> /usr/local/lib/libcunit.so.1
>> #9  0x00409361 in odp_cunit_run () at
>> 

Re: [lng-odp] [API-NEXT PATCHv2 1/2] linux-gen: _ishmphy: adding function for physical address query

[Yi He]

One comment inline:

On 11 November 2016 at 22:50, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> The function _odp_ishmphy_virt_to_phys() is added to query for physical
> addresses (given a virtual address)
> This function is meant to be used to populate the physical address
> of packet segments which are to be used by drivers (without iommu).
> The function _odp_ishmphy_can_virt_to_phys(), also added, return true if
> _odp_ishmphy_virt_to_phys() is able to works (as it requires specific
> permission)
>
> Signed-off-by: Christophe Milard 
> ---
>  platform/linux-generic/_ishmphy.c  | 82
> ++
>  platform/linux-generic/include/_ishmphy_internal.h | 14 
>  2 files changed, 96 insertions(+)
>
> diff --git a/platform/linux-generic/_ishmphy.c b/platform/linux-generic/_
> ishmphy.c
> index 2b2d100..8c0f46e 100644
> --- a/platform/linux-generic/_ishmphy.c
> +++ b/platform/linux-generic/_ishmphy.c
> @@ -29,6 +29,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include <_ishmphy_internal.h>
>
>  static void *common_va_address;
> @@ -38,6 +39,8 @@ static uint64_t common_va_len;
>  #define MAP_ANONYMOUS MAP_ANON
>  #endif
>
> +#define PAGEMAP_FILE "/proc/self/pagemap"
> +
>  /* Book some virtual address space
>   * This function is called at odp_init_global() time to pre-book some
>   * virtual address space inherited by all odpthreads (i.e. descendant
> @@ -183,3 +186,82 @@ int _odp_ishmphy_unmap(void *start, uint64_t len, int
> flags)
> ODP_ERR("_ishmphy_free failure: %s\n", strerror(errno));
> return ret;
>  }
> +
> +/*
> + * Get physical address from virtual address addr.
> + */
>

I saw DPDK API commented that "The page must be locked", and provides an
API rte_mem_lock_page to do so, in this API is the locking guarenteed by
like _odp_ishm_reserve calls already?

+phys_addr_t _odp_ishmphy_virt_to_phys(const void *addr)
> +{
> +   int page_sz;
> +   int fd;
> +   off_t offset;
> +   int  read_bytes;
> +   uint64_t page;
> +   phys_addr_t phys_addr;
> +
> +   /* get normal page sizes: */
> +   page_sz = odp_sys_page_size();
> +
> +   /* read 8 bytes (uint64_t) at position N*8, where N is
> addr/page_sz */
> +   fd = open(PAGEMAP_FILE, O_RDONLY);
> +   if (fd < 0) {
> +   ODP_ERR("cannot open " PAGEMAP_FILE ": %s\n",
> +   strerror(errno));
> +   return PHYS_ADDR_INVALID;
> +   }
> +
> +   offset = ((unsigned long)addr / page_sz) * sizeof(uint64_t);
> +   if (lseek(fd, offset, SEEK_SET) == (off_t)-1) {
> +   ODP_ERR("cannot seek " PAGEMAP_FILE ": %s\n",
> +   strerror(errno));
> +   close(fd);
> +   return PHYS_ADDR_INVALID;
> +   }
> +
> +   read_bytes = read(fd, , sizeof(uint64_t));
> +   close(fd);
> +   if (read_bytes < 0) {
> +   ODP_ERR("cannot read " PAGEMAP_FILE ": %s\n",
> +   strerror(errno));
> +   return PHYS_ADDR_INVALID;
> +   } else if (read_bytes != sizeof(uint64_t)) {
> +   ODP_ERR("read %d bytes from " PAGEMAP_FILE " "
> +   "but expected %d:\n",
> +   read_bytes, sizeof(uint64_t));
> +   return PHYS_ADDR_INVALID;
> +   }
> +
> +   /* some kernel return PFN zero when permission is denied: */
> +   if (!(page & 0x7fULL))
> +   return PHYS_ADDR_INVALID;
> +
> +   /*
> +* the pfn (page frame number) are bits 0-54 (see
> +* pagemap.txt in linux Documentation)
> +*/
> +   phys_addr = ((page & 0x7fULL) * page_sz)
> +   + ((unsigned long)addr % page_sz);
> +
> +   return phys_addr;
> +}
> +
> +/*
> + * check if physical address are readable
> + * return true if physical addresses can be retrieved.
> + * Just do a test to see if it works
> + */
> +int _odp_ishmphy_can_virt_to_phys(void)
> +{
> +   int block_index;
> +   phys_addr_t phy;
> +
> +   /* allocate a block, locked in memory and try to grab its phy
> address */
> +   block_index = _odp_ishm_reserve(NULL, 10, -1, 0, _ODP_ISHM_LOCK,
> 0);
> +   phy = _odp_ishmphy_virt_to_phys((void *)_odp_ishm_address(block_
> index));
> +   _odp_ishm_free_by_index(block_index);
> +
> +   if (phy == PHYS_ADDR_INVALID)
> +   return 0;
> +
> +   return 1;
> +}
> diff --git a/platform/linux-generic/include/_ishmphy_internal.h
> b/platform/linux-generic/include/_ishmphy_internal.h
> index 4fe560f..2022590 100644
> --- a/platform/linux-generic/include/_ishmphy_internal.h
> +++ b/platform/linux-generic/include/_ishmphy_internal.h
> @@ -13,11 +13,25 @@ extern "C" {
>
>  #include 
>
> +typedef uint64_t phys_addr_t; /* Physical address definition. */
> +#define PHYS_ADDR_INVALID ((phys_addr_t)-1)
> +
>  void *_odp_ishmphy_book_va(uintptr_t 

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Savolainen, Petri (Nokia - FI/Espoo)]

 
>  /**
> + * Driver loading
> + *
> + * This function is used by the application to load NIC drivers into ODP.
> + * Calls to this function are optional, but should be performed (if any)
> + * after odp_init_global
> + *
> + *
> + * @param filenameDriver shared lib (dynamic library)
> + *
> + * @retval 0 on success
> + * @retval <0 on failure
> + *
> + */
> +int odp_load_driver(const char *filename);

Why application should explicitly need to load a driver ? Why it cannot happen 
as part of odp_pktio_open() ?

To me a better approach would be:
* user reads implementation documentation or uses system commands to find a 
list of available interfaces
* odp_global_init() finds out available pktio interfaces and drivers for those
* application opens the interface the user commands (just like today)
* pktio_open call loads the driver

Or with more ODP support:
* odp_global_init() finds out available pktio interfaces and drivers for those
* a new API call lists all available interfaces
* application chooses an interface from the list and calls open
* pktio_open call loads the driver


-Petri

Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched mode

[Savolainen, Petri (Nokia - FI/Espoo)]

Ping. This set has been on the list two months now.

We use l2fwd to measure raw packet IO performance with and without scheduler. 
As long as the scheduler mode of the application is un-optimized, scheduler 
mode results are worse than those should be. A perfect HW scheduler does not 
help, if the test itself loses cycles.

-Petri



> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> Savolainen, Petri (Nokia - FI/Espoo)
> Sent: Thursday, October 20, 2016 4:27 PM
> To: lng-odp@lists.linaro.org
> Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> mode
> 
> Ping. Scheduler vs. direct mode performance comparison is more fair with
> these optimizations. Also corrects bugs.
> 
> 
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > Savolainen, Petri (Nokia - FI/Espoo)
> > Sent: Wednesday, October 05, 2016 9:38 AM
> > To: lng-odp@lists.linaro.org
> > Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > mode
> >
> > Ping. Review needed. Optimizes sched mode and corrects bugs.
> >
> >
> > > -Original Message-
> > > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > > Savolainen, Petri (Nokia - FI/Espoo)
> > > Sent: Thursday, September 22, 2016 10:44 AM
> > > To: lng-odp@lists.linaro.org
> > > Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > mode
> > >
> > > Ping. This series optimizes l2fwd schedule mode performance by
> enabling
> > > lockless packet output (when possible) and corrects a bug in
> queue/plain
> > > mode forwarding.
> > >
> > >
> > >
> > > > -Original Message-
> > > > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > > Petri
> > > > Savolainen
> > > > Sent: Thursday, September 15, 2016 12:51 PM
> > > > To: lng-odp@lists.linaro.org
> > > > Subject: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > mode
> > > >
> > > > Replaced linear destination port search in scheduler mode
> > > > with a lookup table. Table index is API provided pktio_index
> > > > and data is application port index.
> > > >
> > > > Signed-off-by: Petri Savolainen 
> > > > ---
> > > >  test/common_plat/performance/odp_l2fwd.c | 68 +++--
> --
> > --
> > > --
> > > > -
> > > >  1 file changed, 41 insertions(+), 27 deletions(-)

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Christophe Milard]

So what?
If we don't allow to load a driver from the north interface, where should
they be loaded from??
Or are we saying that we go for a predefined list of driver only?

So drivers have to be .so files (because of autotools), but we don't give
the possibility to load a free driver??

Christophe

On 16 November 2016 at 11:45, Francois Ozog 
wrote:

> Hello,
>
> If the north API is the one visible by ODP applications then I don't think
> it is a good idea to expose that.
> DPDK exposed it at the beginning and is now internal.
>
> That said there must be a standard way to manage drivers fir the benefit
> of the device framework.
>
> I don't think the idea of integrating it with packetio open because
> packetio is not really a device. It is an abstract way of dealing with
> ports which may be hosted on a device (NIC) or on the platform (SoC).
> It can be done as a PoC (that's what I do with virtio-net exploratiry
> project) but that is not a long term solution.
>
> FF
>
>
> Le mercredi 16 novembre 2016, Christophe Milard <
> christophe.mil...@linaro.org> a écrit :
>
>> On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
>>  wrote:
>> >
>> >>  /**
>> >> + * Driver loading
>> >> + *
>> >> + * This function is used by the application to load NIC drivers into
>> ODP.
>> >> + * Calls to this function are optional, but should be performed (if
>> any)
>> >> + * after odp_init_global
>> >> + *
>> >> + *
>> >> + * @param filenameDriver shared lib (dynamic library)
>> >> + *
>> >> + * @retval 0 on success
>> >> + * @retval <0 on failure
>> >> + *
>> >> + */
>> >> +int odp_load_driver(const char *filename);
>> >
>> > Why application should explicitly need to load a driver ? Why it cannot
>> happen as part of odp_pktio_open() ?
>> >
>> > To me a better approach would be:
>> > * user reads implementation documentation or uses system commands to
>> find a list of available interfaces
>> > * odp_global_init() finds out available pktio interfaces and drivers
>> for those
>> > * application opens the interface the user commands (just like today)
>> > * pktio_open call loads the driver
>> >
>> > Or with more ODP support:
>> > * odp_global_init() finds out available pktio interfaces and drivers
>> for those
>> > * a new API call lists all available interfaces
>> > * application chooses an interface from the list and calls open
>> > * pktio_open call loads the driver
>> >
>> >
>> > -Petri
>>
>>
>> Having ODP finding the driver by itself means "hard-coding" the list
>> of available drivers (in the sense that ODP must know in advance what
>> drivers matches what interface and where to find these drivers).
>>
>> The approach above let people do their own driver and attach it to ODP
>> whithout any pre-requirements on ODP.
>> When we have a set of "drivers we like", nothing prevent us from
>> loading a default set of drivers at init.
>> Unless the driver name (and path) is given as part as the pktio device
>> name, which would just be awfull, we need such a load function to
>> enable private driver loading.
>>
>> Moreover, the driver must be loaded to be probed: The drivers must
>> tell what they can do, not ODP.
>>
>> I think the application should as a little aware as possible of
>> drivers (and what HW they match) but should be given a way to load
>> private drivers. The approach here enable that. Load your own driver
>> (or none if it is in the default set), then open pktios without
>> specifying any driver (note that 1 driver may be handling many
>> interfaces)
>>
>> I think having an API call to list available interfaces makes sense,
>> and does stress my proposal: Drivers must be loaded and probed to
>> build that list.
>>
>> Christophe.
>>
>
>
> --
> [image: Linaro] 
> François-Frédéric Ozog | *Director Linaro Networking Group*
> T: +33.67221.6485
> francois.o...@linaro.org | Skype: ffozog
>
>
>

Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched mode

[Savolainen, Petri (Nokia - FI/Espoo)]

Didn't make it to the list. Sending again.

> -Original Message-
> From: Savolainen, Petri (Nokia - FI/Espoo)
> Sent: Wednesday, November 16, 2016 11:21 AM
> To: lng-odp@lists.linaro.org
> Subject: RE: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> mode
> 
> Ping. This set has been on the list two months now.
> 
> We use l2fwd to measure raw packet IO performance with and without
> scheduler. As long as the scheduler mode of the application is un-
> optimized, scheduler mode results are worse than those should be. A
> perfect HW scheduler does not help, if the test itself loses cycles.
> 
> -Petri
> 
> 
> 
> > -Original Message-
> > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > Savolainen, Petri (Nokia - FI/Espoo)
> > Sent: Thursday, October 20, 2016 4:27 PM
> > To: lng-odp@lists.linaro.org
> > Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > mode
> >
> > Ping. Scheduler vs. direct mode performance comparison is more fair with
> > these optimizations. Also corrects bugs.
> >
> >
> > > -Original Message-
> > > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > > Savolainen, Petri (Nokia - FI/Espoo)
> > > Sent: Wednesday, October 05, 2016 9:38 AM
> > > To: lng-odp@lists.linaro.org
> > > Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > > mode
> > >
> > > Ping. Review needed. Optimizes sched mode and corrects bugs.
> > >
> > >
> > > > -Original Message-
> > > > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> > > > Savolainen, Petri (Nokia - FI/Espoo)
> > > > Sent: Thursday, September 22, 2016 10:44 AM
> > > > To: lng-odp@lists.linaro.org
> > > > Subject: Re: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for
> sched
> > > mode
> > > >
> > > > Ping. This series optimizes l2fwd schedule mode performance by
> > enabling
> > > > lockless packet output (when possible) and corrects a bug in
> > queue/plain
> > > > mode forwarding.
> > > >
> > > >
> > > >
> > > > > -Original Message-
> > > > > From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf
> Of
> > > > Petri
> > > > > Savolainen
> > > > > Sent: Thursday, September 15, 2016 12:51 PM
> > > > > To: lng-odp@lists.linaro.org
> > > > > Subject: [lng-odp] [PATCH 1/3] test: l2fwd: lookup table for sched
> > > mode
> > > > >
> > > > > Replaced linear destination port search in scheduler mode
> > > > > with a lookup table. Table index is API provided pktio_index
> > > > > and data is application port index.
> > > > >
> > > > > Signed-off-by: Petri Savolainen 
> > > > > ---
> > > > >  test/common_plat/performance/odp_l2fwd.c | 68
> +++--
> > --
> > > --
> > > > --
> > > > > -
> > > > >  1 file changed, 41 insertions(+), 27 deletions(-)

Re: [lng-odp] Classification Queue Group

[Bala Manoharan]

Regards,
Bala


On 15 November 2016 at 22:43, Brian Brooks  wrote:
> On Mon, Nov 14, 2016 at 2:12 AM, Bala Manoharan
>  wrote:
>> Regards,
>> Bala
>>
>>
>> On 11 November 2016 at 13:26, Brian Brooks  wrote:
>>> On 11/10 15:17:15, Bala Manoharan wrote:
 On 10 November 2016 at 13:26, Brian Brooks  wrote:
 > On 11/07 16:46:12, Bala Manoharan wrote:
 >> Hi,
 >
 > Hiya
 >
 >> This mail thread discusses the design of classification queue group
 >> RFC. The same can be found in the google doc whose link is given
 >> below.
 >> Users can provide their comments either in this mail thread or in the
 >> google doc as per their convenience.
 >>
 >> https://docs.google.com/document/d/1fOoG9WDR0lMpVjgMAsx8QsMr0YFK9slR93LZ8VXqM2o/edit?usp=sharing
 >>
 >> The basic issues with queues as being a single target for a CoS are two 
 >> fold:
 >>
 >> Queues must be created and deleted individually. This imposes a
 >> significant burden when queues are used to represent individual flows
 >> since the application may need to process thousands (or millions) of
 >> flows.
 >
 > Wondering why there is an issue with creating and deleting queues 
 > individually
 > if queue objects represent millions of flows..

 The queue groups are mainly required for hashing the incoming packets
 to multiple flows based on the hash configuration.
 So from application point of view it just needs a queue to have
 packets belonging to same flow and that packets belonging to different
 flows are placed in different queues respectively.It does not matter
 who creates the flow/queue.
>>>
>>> When the application receives an event from odp_schedule() call, how does it
>>> know whether the odp_queue_t was previously created by the application from
>>> odp_queue_create() or whether it was created by the implementation?
>>
>> odp_schedule() call returns the queue from which the event was part of.
>> The type of the queue can be got from odp_queue_type() API.
>> But the question is there an use-case where the application need to know?
>> The application has the information of the queue it has created and
>> the queues created by implementation are destroyed by implementation.
>
> If certain fields of the packet are hashed to a queue handle, and this queue
> handle has not previously been created via odp_queue_create(), there might
> be a use case where the application needs to be aware of a new "flow"..
> Maybe the application ages flows.

It is very difficult in network traffic to predict the exact flows
which will be coming in an interface.
The application can configure for all the possible flows in that case.

>
>>
>>>
 It is actually simpler if implementation creates a flow since in that
 case implementation need not accumulate meta-data for all possible
 hash values in a queue group and it can be created when traffic
 arrives in that particular flow.

 >
 > Could an application ever call odp_schedule() and receive an event (e.g. 
 > packet)
 > from a queue (of opaque type odp_queue_t) and that queue has never been 
 > created
 > by the application (via odp_queue_create())? Could that ever happen from 
 > the
 > hardware, and could the application ever handle that?

 No. All the queues in the system are created by the application either
 directly or in-directly.
 In-case of queue groups the queues are in-directly created by the
 application by configuring a queue group.

 > Or, is it related to memory usage? The reference implementation
 > struct queue_entry_s is 320 bytes on a 64-bit machine.
 >
 >   2^28 ~= 268,435,456 queues -> 81.920 GB
 >   2^26 ~=  67,108,864 queues -> 20.480 GB
 >   2^22 ~=   4,194,304 queues ->  1.280 GB
 >
 > Forget about 320 bytes per queue, if each queue was represented by a 
 > 32-bit
 > integer (4 bytes!) the usage would be:
 >
 >   2^28 ~= 268,435,456 queues ->  1.024 GB
 >   2^26 ~=  67,108,864 queues ->256 MB
 >   2^22 ~=   4,194,304 queues -> 16 MB
 >
 > That still might be a lot of usage if the application must explicitly 
 > create
 > every queue (before it is used) and require an ODP implementation to map
 > between every ODP queue object (opaque type) and the internal queue.
 >
 > Lets say ODP API has two classes of handles: 1) pointers, 2) integers. 
 > An opaque
 > pointer is used to point to some other software object. This object 
 > should be
 > larger than 64 bits (or 32 bits on a chip in 32-bit pointer mode) 
 > otherwise it
 > could just be represented in a 64-bit (or 32-bit) integer type value!
 >
 > To support millions of queues (flows) should odp_queue_t be an integer 
 > type in

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Savolainen, Petri (Nokia - FI/Espoo)]

I have reported a bug https://bugs.linaro.org/show_bug.cgi?id=2595 about the 
new VLAN test hang. Could this be the same issue? It hangs for me sometimes, 
most times not. It happens both at the tip of the master and api-next.

-Petri



> -Original Message-
> From: Maxim Uvarov [mailto:maxim.uva...@linaro.org]
> Sent: Wednesday, November 16, 2016 10:03 PM
> To: Bill Fischofer ; Savolainen, Petri (Nokia -
> FI/Espoo) ; Mike Holmes
> 
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization
> 
> I can not test patch by patch this series because it fails (one time it
> was TM, one time kernel died, other time OOM killer killed tests then
> hang kernel).
> 
> And for all patches test/common_plat/validation/api/pktio/pktio_main
> hangs forever:
> 
> 
> Program received signal SIGINT, Interrupt.
> 0x2afbe69ffb80 in __nanosleep_nocancel () at
> ../sysdeps/unix/syscall-template.S:81
> 81in ../sysdeps/unix/syscall-template.S
> (gdb) bt
> #0  0x2afbe69ffb80 in __nanosleep_nocancel () at
> ../sysdeps/unix/syscall-template.S:81
> #1  0x00415ced in odp_pktin_recv_tmo (queue=...,
> packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
>  wait=wait@entry=18446744073709551615) at
> ../../../platform/linux-generic/odp_packet_io.c:1584
> #2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
> pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
>  seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1,
> mode=mode@entry=RECV_TMO, tmo=tmo@entry=18446744073709551615,
> ns=ns@entry=0)
>  at
> ../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
> #3  0x004075f8 in test_recv_tmo (mode=RECV_TMO) at
> ../../../../../../test/common_plat/validation/api/pktio/pktio.c:940
> #4  0x2afbe61cc482 in run_single_test () from
> /usr/local/lib/libcunit.so.1
> #5  0x2afbe61cc0b2 in run_single_suite () from
> /usr/local/lib/libcunit.so.1
> #6  0x2afbe61c9d55 in CU_run_all_tests () from
> /usr/local/lib/libcunit.so.1
> #7  0x2afbe61ce245 in basic_run_all_tests () from
> /usr/local/lib/libcunit.so.1
> #8  0x2afbe61cdfe7 in CU_basic_run_tests () from
> /usr/local/lib/libcunit.so.1
> #9  0x00409361 in odp_cunit_run () at
> ../../../../test/common_plat/common/odp_cunit_common.c:298
> #10 0x2afbe6c2ff45 in __libc_start_main (main=0x403850 ,
> argc=1, argv=0x7ffed64d9878, init=,
>  fini=, rtld_fini=,
> stack_end=0x7ffed64d9868) at libc-start.c:287
> #11 0x0040387e in _start ()
> (gdb) up
> #1  0x00415ced in odp_pktin_recv_tmo (queue=...,
> packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
>  wait=wait@entry=18446744073709551615) at
> ../../../platform/linux-generic/odp_packet_io.c:1584
> 1584nanosleep(, NULL);
> (gdb) p ts
> $1 = {tv_sec = 0, tv_nsec = 1000}
> (gdb) l
> 1579}
> 1580
> 1581wait--;
> 1582}
> 1583
> 1584nanosleep(, NULL);
> 1585}
> 1586}
> 1587
> 1588int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[],
> unsigned num_q,
> (gdb) up
> #2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
> pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
>  seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1,
> mode=mode@entry=RECV_TMO, tmo=tmo@entry=18446744073709551615,
> ns=ns@entry=0)
>  at
> ../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
> 515n = odp_pktin_recv_tmo(pktin[0], pkt_tmp, num - num_rx,
> (gdb) p num - num_rx
> $2 = 1
> (gdb) l
> 510/** Multiple odp_pktin_recv_tmo()/odp_pktin_recv_mq_tmo()
> calls may be
> 511 *  required to discard possible non-test packets. */
> 512do {
> 513ts1 = odp_time_global();
> 514if (mode == RECV_TMO)
> 515n = odp_pktin_recv_tmo(pktin[0], pkt_tmp, num - num_rx,
> 516   tmo);
> 517else
> 518n = odp_pktin_recv_mq_tmo(pktin, (unsigned)num_q,
> 519  from, pkt_tmp,
> (gdb) p tmo
> $3 = 18446744073709551615
> 
> 
> I applied patches and following script under root:
> CLEANUP=0 GIT_URL=/opt/Linaro/odp3.git GIT_BRANCH=api-next ./build.sh
> 
> Need more investigation into this issue... Not applied yet.
> 
> Maxim.
> 
> On 11/16/16 02:58, Bill Fischofer wrote:
> > Trying again as the repost doesn't seem to show up on the list either.
> >
> > For this series:
> >
> > Reviewed-and-tested-by: Bill Fischofer  > >
> >
> > On Tue, Nov 15, 2016 at 5:55 PM, Bill Fischofer
> > > wrote:
> >
> > Reposting this since it doesn't seem to have made it to the
> > mailing list.
> >
> > For this series:
> >
> > Reviewed-and-tested-by: Bill Fischofer 

[lng-odp] [API-NEXT PATCHv7 00/13] using _ishm as north API mem allocator

[Christophe Milard]

since v6:
- All points according to Petri's request i.e.:
Odp_shm_find_external() changed again: now odp_shm_import().
Function description updated.

since v5:
-fixed doxygen issue (Bill)

since v4:
- All points of v3 according to Petri's request i.e.:
-All API patches merges as one single patch.
-lock flag removed: memory is now always locked.
-alignement and flag inherited from remote ODP instance when sharing
 memory.
-function name changed to: odp_shm_find_external().

since v3:
-Comments from Petri addressed, with exceptions for:
-Patches related to the API not merged as they are part of different
 functions.
-Lock flag not removed: Why wouldn't any ODP application being allowed
 to reserve memory for slower (control or other things) businnes.
 Should really ALL shared memory bee locked?
-Alignment and flag parameter of odp_shm_reserve_external() kept:
 The alignment (and some flags) affect virtual memory, which is different
 between for each ODP instance.
-function name odp_shm_reserve_external() kept, as this function does
 return a new handle (which requires free() as any handle would)
These 4 items are sheduled for arch call today

since v2:
-some minor changes on the doc (Bill)
-doc for the ODP_SHM_EXPORT flag (Christophe)
-reduction of memory reguirement for shm tests (Mattias)

since v1:
-flag _ODP_SHM_PROC_NOCREAT and _ODP_SHM_O_EXCL get new values
(but remain useless: Should be removed when IPC is updated)  (Maxim)

-In get_ishm_flags(), odp_shm_capability() local variable flgs renamed
(for be better distinction from other "flags" variable. (Maxim)

-Added doc updates with shm api extensions.

This Patch series aims at using _ishm as north API memory allocator.
odp_shared_memory.c just becomes a wrapper around _ishm.
_ishm supports "process mode", i.e. memory allocated  with _ishm
is sharable by all ODP threads of a given ODP instance, regardless of
thread type (e.g. process) or thread creation time (for time).

NOTE: This patch series will break IPC: This is due to the fact that
IPC relied on a "special case" in the former memory allocator that broke
ODP instance scoping. I don't think this should be kept.
I have included in this patch series a function to share memory between
designated ODP instances. If we want to have IPC, it should use that.

Christophe Milard (13):
  linux-gen: _ishm: create description file for external memory sharing
  linux-gen: _ishm: allow memory alloc/free at global init/term
  linux-gen: use ishm as north API mem allocator
  linux-gen: Push internal flag definition
  api: shm: add flags to shm_reserve and function to find external mem
  linux-gen: shm: new ODP_SHM_SINGLE_VA flag implementation
  test: api: shmem: new proper tests for shm API
  linux-gen: _ishm: adding function to map memory from other ODP
  linux-gen: shm: add flag and function to share memory between ODP
instances
  test: linux-gen: api: shmem: test sharing memory between ODP instances
  linux-gen: _ishm: cleaning remaining block at odp_term_global
  linux_gen: _ishm: decreasing the number of error messages when no huge
pages
  doc: updating docs for the shm interface extension

 doc/users-guide/users-guide.adoc   |  68 +-
 include/odp/api/spec/shared_memory.h   |  46 +-
 platform/linux-generic/_ishm.c | 456 ++
 platform/linux-generic/include/_ishm_internal.h|   6 +
 platform/linux-generic/include/odp_internal.h  |   5 -
 platform/linux-generic/include/odp_shm_internal.h  |   4 +-
 platform/linux-generic/odp_init.c  |  19 -
 platform/linux-generic/odp_shared_memory.c | 412 ++--
 test/common_plat/validation/api/shmem/shmem.c  | 687 -
 test/common_plat/validation/api/shmem/shmem.h  |   5 +-
 test/linux-generic/validation/api/shmem/.gitignore |   3 +-
 .../linux-generic/validation/api/shmem/Makefile.am |  22 +-
 .../validation/api/shmem/shmem_linux.c | 220 +--
 .../api/shmem/{shmem_odp.c => shmem_odp1.c}|  10 +-
 .../api/shmem/{shmem_odp.h => shmem_odp1.h}|   0
 .../validation/api/shmem/shmem_odp2.c  |  95 +++
 .../validation/api/shmem/shmem_odp2.h  |   7 +
 17 files changed, 1464 insertions(+), 601 deletions(-)
 rename test/linux-generic/validation/api/shmem/{shmem_odp.c => shmem_odp1.c} 
(81%)
 rename test/linux-generic/validation/api/shmem/{shmem_odp.h => shmem_odp1.h} 
(100%)
 create mode 100644 test/linux-generic/validation/api/shmem/shmem_odp2.c
 create mode 100644 test/linux-generic/validation/api/shmem/shmem_odp2.h

-- 
2.7.4

Re: [lng-odp] [API-NEXT PATCH v3 15/19] linux-gen: packet: added support for segmented packets

[Bill Fischofer]

On Thu, Nov 10, 2016 at 5:07 AM, Petri Savolainen <
petri.savolai...@nokia.com> wrote:

> Added support for multi-segmented packets. The first segments
> is the packet descriptor, which contains all metadata and
> pointers to other segments.
>
> Signed-off-by: Petri Savolainen 
> ---
>  .../include/odp/api/plat/packet_types.h|   6 +-
>  .../linux-generic/include/odp_buffer_inlines.h |  11 -
>  .../linux-generic/include/odp_buffer_internal.h|  23 +-
>  .../linux-generic/include/odp_config_internal.h|  39 +-
>  .../linux-generic/include/odp_packet_internal.h|  80 +--
>  platform/linux-generic/include/odp_pool_internal.h |   3 -
>  platform/linux-generic/odp_buffer.c|   8 +-
>  platform/linux-generic/odp_crypto.c|   8 +-
>  platform/linux-generic/odp_packet.c| 712
> +
>  platform/linux-generic/odp_pool.c  | 123 ++--
>  platform/linux-generic/pktio/netmap.c  |   4 +-
>  platform/linux-generic/pktio/socket.c  |   3 +-
>  12 files changed, 692 insertions(+), 328 deletions(-)
>
> diff --git a/platform/linux-generic/include/odp/api/plat/packet_types.h
> b/platform/linux-generic/include/odp/api/plat/packet_types.h
> index b5345ed..864494d 100644
> --- a/platform/linux-generic/include/odp/api/plat/packet_types.h
> +++ b/platform/linux-generic/include/odp/api/plat/packet_types.h
> @@ -32,9 +32,11 @@ typedef ODP_HANDLE_T(odp_packet_t);
>
>  #define ODP_PACKET_OFFSET_INVALID (0x0fff)
>
> -typedef ODP_HANDLE_T(odp_packet_seg_t);
> +/* A packet segment handle stores a small index. Strong type handles are
> + * pointers, which would be wasteful in this case. */
> +typedef uint8_t odp_packet_seg_t;
>
> -#define ODP_PACKET_SEG_INVALID _odp_cast_scalar(odp_packet_seg_t,
> 0x)
> +#define ODP_PACKET_SEG_INVALID ((odp_packet_seg_t)-1)
>
>  /** odp_packet_color_t assigns names to the various pkt "colors" */
>  typedef enum {
> diff --git a/platform/linux-generic/include/odp_buffer_inlines.h
> b/platform/linux-generic/include/odp_buffer_inlines.h
> index f8688f6..cf817d9 100644
> --- a/platform/linux-generic/include/odp_buffer_inlines.h
> +++ b/platform/linux-generic/include/odp_buffer_inlines.h
> @@ -23,22 +23,11 @@ odp_event_type_t _odp_buffer_event_type(odp_buffer_t
> buf);
>  void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
>  int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
>
> -void *buffer_map(odp_buffer_hdr_t *buf, uint32_t offset, uint32_t *seglen,
> -uint32_t limit);
> -
>  static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
>  {
> return hdr->handle.handle;
>  }
>
> -static inline uint32_t pool_id_from_buf(odp_buffer_t buf)
> -{
> -   odp_buffer_bits_t handle;
> -
> -   handle.handle = buf;
> -   return handle.pool_id;
> -}
> -
>  #ifdef __cplusplus
>  }
>  #endif
> diff --git a/platform/linux-generic/include/odp_buffer_internal.h
> b/platform/linux-generic/include/odp_buffer_internal.h
> index 0ca13f8..4e75908 100644
> --- a/platform/linux-generic/include/odp_buffer_internal.h
> +++ b/platform/linux-generic/include/odp_buffer_internal.h
> @@ -33,10 +33,6 @@ extern "C" {
>  #include 
>  #include 
>
> -ODP_STATIC_ASSERT(ODP_CONFIG_PACKET_SEG_LEN_MIN >= 256,
> - "ODP Segment size must be a minimum of 256 bytes");
> -
> -
>  typedef union odp_buffer_bits_t {
> odp_buffer_t handle;
>
> @@ -65,6 +61,20 @@ struct odp_buffer_hdr_t {
> int burst_first;
> struct odp_buffer_hdr_t *burst[BUFFER_BURST_SIZE];
>
> +   struct {
> +   void *hdr;
> +   uint8_t  *data;
> +   uint32_t  len;
> +   } seg[CONFIG_PACKET_MAX_SEGS];
> +
> +   /* max data size */
> +   uint32_t size;
> +
> +   /* Initial buffer data pointer and length */
> +   void *base_data;
> +   uint32_t  base_len;
> +   uint8_t  *buf_end;
> +
> union {
> uint32_t all;
> struct {
> @@ -75,7 +85,6 @@ struct odp_buffer_hdr_t {
>
> int8_t   type;   /* buffer type */
> odp_event_type_t event_type; /* for reuse as event */
> -   uint32_t size;   /* max data size */
> odp_pool_t   pool_hdl;   /* buffer pool handle */
> union {
> uint64_t buf_u64;/* user u64 */
> @@ -86,8 +95,6 @@ struct odp_buffer_hdr_t {
> uint32_t uarea_size; /* size of user area */
> uint32_t segcount;   /* segment count */
> uint32_t segsize;/* segment size */
> -   /* block addrs */
> -   void*addr[ODP_CONFIG_PACKET_MAX_SEGS];
> uint64_t order;  /* sequence for ordered
> queues */
> queue_entry_t   *origin_qe;  /* ordered queue 

[lng-odp] [API-NEXT PATCHv2 0/5] Packet References

[Bill Fischofer]

Add support for packet reference APIs. Note that this patch series applies
on top of Petri's v3 patch series for pool enhancements:
http://patches.opendataplane.org/patch/7299/

Changes for v2:
- Rebased on top of Petri's pool restructure patch series

Bill Fischofer (5):
  api: packet: add support for packet references
  linux-generic: packet: implement reference apis
  validation: packet: add packet reference tests
  doc: images: add images for packet reference documentation
  doc: userguide: add user documentation for packet references

 doc/images/ref.svg |  58 
 doc/images/reflen.svg  |  45 +++
 doc/images/refpkt1.svg |  43 +++
 doc/images/refpkt2.svg |  43 +++
 doc/images/refpktmulti.svg |  75 +
 doc/images/refpktsingle.svg|  76 +
 doc/images/refstatic.svg   |  39 +++
 doc/users-guide/users-guide-packet.adoc| 261 +++-
 include/odp/api/spec/packet.h  | 168 +++
 .../linux-generic/include/odp_packet_internal.h|  49 ++-
 platform/linux-generic/odp_packet.c| 331 -
 test/common_plat/validation/api/packet/packet.c| 241 +++
 test/common_plat/validation/api/packet/packet.h|   1 +
 13 files changed, 1357 insertions(+), 73 deletions(-)
 create mode 100644 doc/images/ref.svg
 create mode 100644 doc/images/reflen.svg
 create mode 100644 doc/images/refpkt1.svg
 create mode 100644 doc/images/refpkt2.svg
 create mode 100644 doc/images/refpktmulti.svg
 create mode 100644 doc/images/refpktsingle.svg
 create mode 100644 doc/images/refstatic.svg

-- 
2.7.4

[lng-odp] [API-NEXT PATCHv2 3/5] validation: packet: add packet reference tests

[Bill Fischofer]

Add validation tests for the new packet reference APIs:
- odp_packet_ref_static()
- odp_packet_ref()
- odp_packet_ref_pkt()
- odp_packet_is_ref()
- odp_packet_has_ref()
- odp_packet_unshared_len()

Signed-off-by: Bill Fischofer 
---
 test/common_plat/validation/api/packet/packet.c | 241 
 test/common_plat/validation/api/packet/packet.h |   1 +
 2 files changed, 242 insertions(+)

diff --git a/test/common_plat/validation/api/packet/packet.c 
b/test/common_plat/validation/api/packet/packet.c
index bf7eab2..f84560b 100644
--- a/test/common_plat/validation/api/packet/packet.c
+++ b/test/common_plat/validation/api/packet/packet.c
@@ -1393,6 +1393,246 @@ void packet_test_offset(void)
CU_ASSERT_PTR_NOT_NULL(ptr);
 }
 
+void packet_test_ref(void)
+{
+   odp_packet_t base_pkt, segmented_base_pkt, hdr_pkt[4],
+   ref_pkt[4], refhdr_pkt[4], hdr_cpy;
+   uint32_t pkt_len, segmented_pkt_len, hdr_len[4], offset[4], hr[4];
+   int i;
+
+   base_pkt = odp_packet_copy(test_packet, odp_packet_pool(test_packet));
+   pkt_len  = odp_packet_len(test_packet);
+   CU_ASSERT_FATAL(base_pkt != ODP_PACKET_INVALID);
+
+   segmented_base_pkt =
+   odp_packet_copy(segmented_test_packet,
+   odp_packet_pool(segmented_test_packet));
+   segmented_pkt_len = odp_packet_len(segmented_test_packet);
+   CU_ASSERT_FATAL(segmented_base_pkt != ODP_PACKET_INVALID);
+
+   CU_ASSERT(odp_packet_is_ref(base_pkt) == 0);
+   CU_ASSERT(odp_packet_has_ref(base_pkt) == 0);
+
+   hdr_pkt[0] =
+   odp_packet_copy_part(segmented_test_packet, 0,
+odp_packet_len(segmented_test_packet) / 4,
+odp_packet_pool(segmented_test_packet));
+   CU_ASSERT_FATAL(hdr_pkt[0] != ODP_PACKET_INVALID);
+   hdr_len[0] = odp_packet_len(hdr_pkt[0]);
+   offset[0]  = 0;
+
+   hdr_pkt[1] =
+   odp_packet_copy_part(segmented_test_packet, 10,
+odp_packet_len(segmented_test_packet) / 8,
+odp_packet_pool(segmented_test_packet));
+   CU_ASSERT_FATAL(hdr_pkt[1] != ODP_PACKET_INVALID);
+   hdr_len[1] = odp_packet_len(hdr_pkt[1]);
+   offset[1]  = 5;
+
+   hdr_pkt[2] = odp_packet_copy_part(test_packet, 0,
+ odp_packet_len(test_packet) / 4,
+ odp_packet_pool(test_packet));
+   CU_ASSERT_FATAL(hdr_pkt[2] != ODP_PACKET_INVALID);
+   hdr_len[2] = odp_packet_len(hdr_pkt[2]);
+   offset[2]  = 64;
+
+   hdr_pkt[3] = odp_packet_copy_part(test_packet, 0,
+ odp_packet_len(test_packet) / 4,
+ odp_packet_pool(test_packet));
+   CU_ASSERT_FATAL(hdr_pkt[3] != ODP_PACKET_INVALID);
+   hdr_len[3] = odp_packet_len(hdr_pkt[3]);
+   offset[3]  = 64;
+
+   /* Nothing is a ref or has a ref before we start */
+   for (i = 0; i < 4; i++) {
+   CU_ASSERT(odp_packet_is_ref(hdr_pkt[i]) == 0);
+   CU_ASSERT(odp_packet_has_ref(hdr_pkt[i]) == 0);
+   CU_ASSERT(odp_packet_len(hdr_pkt[i]) ==
+ odp_packet_unshared_len(hdr_pkt[i]));
+   }
+
+   /* Attempt an invalid ref */
+   refhdr_pkt[0] = odp_packet_ref_pkt(base_pkt, pkt_len, hdr_pkt[0]);
+   CU_ASSERT(refhdr_pkt[0] == ODP_PACKET_INVALID);
+   CU_ASSERT(odp_packet_is_ref(hdr_pkt[0]) == 0);
+   CU_ASSERT(odp_packet_is_ref(base_pkt) == 0);
+   CU_ASSERT(odp_packet_has_ref(hdr_pkt[0]) == 0);
+   CU_ASSERT(odp_packet_has_ref(base_pkt) == 0);
+
+   /* We can't ref to ourselves */
+   refhdr_pkt[0] = odp_packet_ref_pkt(hdr_pkt[0], 0, hdr_pkt[0]);
+   CU_ASSERT(refhdr_pkt[0] == ODP_PACKET_INVALID);
+   CU_ASSERT(odp_packet_is_ref(hdr_pkt[0]) == 0);
+   CU_ASSERT(odp_packet_has_ref(hdr_pkt[0]) == 0);
+
+   /* Now create a couple of valid refs */
+   refhdr_pkt[0] = odp_packet_ref_pkt(base_pkt, offset[0], hdr_pkt[0]);
+   refhdr_pkt[1] = odp_packet_ref_pkt(base_pkt, offset[1], hdr_pkt[1]);
+
+   CU_ASSERT(refhdr_pkt[0] != ODP_PACKET_INVALID);
+   CU_ASSERT(refhdr_pkt[1] != ODP_PACKET_INVALID);
+
+   CU_ASSERT(odp_packet_is_ref(refhdr_pkt[0]) == 1);
+   CU_ASSERT(odp_packet_is_ref(refhdr_pkt[1]) == 1);
+   CU_ASSERT(odp_packet_is_ref(base_pkt) == 0);
+
+   CU_ASSERT(odp_packet_has_ref(base_pkt) == 2);
+   CU_ASSERT(odp_packet_has_ref(refhdr_pkt[0]) == 0);
+   CU_ASSERT(odp_packet_has_ref(refhdr_pkt[1]) == 0);
+
+   /* Verify ref lengths and contents */
+   CU_ASSERT(odp_packet_unshared_len(base_pkt) == 0);
+
+   CU_ASSERT(odp_packet_len(refhdr_pkt[0]) ==
+ hdr_len[0] + pkt_len - offset[0]);
+   CU_ASSERT(odp_packet_unshared_len(refhdr_pkt[0]) == 

Re: [lng-odp] [API-NEXT PATCHv2 3/5] api: init: driver load function added

[Christophe Milard]

On 16 November 2016 at 10:07, Savolainen, Petri (Nokia - FI/Espoo)
 wrote:
>
>>  /**
>> + * Driver loading
>> + *
>> + * This function is used by the application to load NIC drivers into ODP.
>> + * Calls to this function are optional, but should be performed (if any)
>> + * after odp_init_global
>> + *
>> + *
>> + * @param filenameDriver shared lib (dynamic library)
>> + *
>> + * @retval 0 on success
>> + * @retval <0 on failure
>> + *
>> + */
>> +int odp_load_driver(const char *filename);
>
> Why application should explicitly need to load a driver ? Why it cannot 
> happen as part of odp_pktio_open() ?
>
> To me a better approach would be:
> * user reads implementation documentation or uses system commands to find a 
> list of available interfaces
> * odp_global_init() finds out available pktio interfaces and drivers for those
> * application opens the interface the user commands (just like today)
> * pktio_open call loads the driver
>
> Or with more ODP support:
> * odp_global_init() finds out available pktio interfaces and drivers for those
> * a new API call lists all available interfaces
> * application chooses an interface from the list and calls open
> * pktio_open call loads the driver
>
>
> -Petri


Having ODP finding the driver by itself means "hard-coding" the list
of available drivers (in the sense that ODP must know in advance what
drivers matches what interface and where to find these drivers).

The approach above let people do their own driver and attach it to ODP
whithout any pre-requirements on ODP.
When we have a set of "drivers we like", nothing prevent us from
loading a default set of drivers at init.
Unless the driver name (and path) is given as part as the pktio device
name, which would just be awfull, we need such a load function to
enable private driver loading.

Moreover, the driver must be loaded to be probed: The drivers must
tell what they can do, not ODP.

I think the application should as a little aware as possible of
drivers (and what HW they match) but should be given a way to load
private drivers. The approach here enable that. Load your own driver
(or none if it is in the default set), then open pktios without
specifying any driver (note that 1 driver may be handling many
interfaces)

I think having an API call to list available interfaces makes sense,
and does stress my proposal: Drivers must be loaded and probed to
build that list.

Christophe.

Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

[Maxim Uvarov]

On 11/17/16 11:36, Savolainen, Petri (Nokia - FI/Espoo) wrote:

I have reported a bug https://bugs.linaro.org/show_bug.cgi?id=2595 about the 
new VLAN test hang. Could this be the same issue? It hangs for me sometimes, 
most times not. It happens both at the tip of the master and api-next.

-Petri


Vlan testing hang runs:
test/linux-generic/mmap_vlan_ins/mmap_vlan_ins.sh

where it can hang only if your #!/bin/sh does not support this syntax 
for KILL command.


# Listen on veth pipe and write to pcap Send pcap
plat_mmap_vlan_ins${EXEEXT} pktiop0p1 pcap:out=${PCAP_OUT} \
00:02:03:04:05:06 00:08:09:0a:0b:0c &
P1=$!
# Send pcap file to veth interface
plat_mmap_vlan_ins${EXEEXT} pcap:in=${PCAP_IN} pktiop0p1 \
01:02:03:04:05:06 01:08:09:0a:0b:0c &
P2=$!

sleep 1
kill -s INT ${P1}
kill -s INT ${P2}


But for me it works well with both /bin/sh and /bin/bash


In your case 'pktio_main' binary rolls in infinite loop. So I would say 
it's different issues.


Maxim.




-Original Message-
From: Maxim Uvarov [mailto:maxim.uva...@linaro.org]
Sent: Wednesday, November 16, 2016 10:03 PM
To: Bill Fischofer ; Savolainen, Petri (Nokia -
FI/Espoo) ; Mike Holmes

Cc: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCH v3 00/19] pool optimization

I can not test patch by patch this series because it fails (one time it
was TM, one time kernel died, other time OOM killer killed tests then
hang kernel).

And for all patches test/common_plat/validation/api/pktio/pktio_main
hangs forever:


Program received signal SIGINT, Interrupt.
0x2afbe69ffb80 in __nanosleep_nocancel () at
../sysdeps/unix/syscall-template.S:81
81in ../sysdeps/unix/syscall-template.S
(gdb) bt
#0  0x2afbe69ffb80 in __nanosleep_nocancel () at
../sysdeps/unix/syscall-template.S:81
#1  0x00415ced in odp_pktin_recv_tmo (queue=...,
packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
  wait=wait@entry=18446744073709551615) at
../../../platform/linux-generic/odp_packet_io.c:1584
#2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
  seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1,
mode=mode@entry=RECV_TMO, tmo=tmo@entry=18446744073709551615,
ns=ns@entry=0)
  at
../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
#3  0x004075f8 in test_recv_tmo (mode=RECV_TMO) at
../../../../../../test/common_plat/validation/api/pktio/pktio.c:940
#4  0x2afbe61cc482 in run_single_test () from
/usr/local/lib/libcunit.so.1
#5  0x2afbe61cc0b2 in run_single_suite () from
/usr/local/lib/libcunit.so.1
#6  0x2afbe61c9d55 in CU_run_all_tests () from
/usr/local/lib/libcunit.so.1
#7  0x2afbe61ce245 in basic_run_all_tests () from
/usr/local/lib/libcunit.so.1
#8  0x2afbe61cdfe7 in CU_basic_run_tests () from
/usr/local/lib/libcunit.so.1
#9  0x00409361 in odp_cunit_run () at
../../../../test/common_plat/common/odp_cunit_common.c:298
#10 0x2afbe6c2ff45 in __libc_start_main (main=0x403850 ,
argc=1, argv=0x7ffed64d9878, init=,
  fini=, rtld_fini=,
stack_end=0x7ffed64d9868) at libc-start.c:287
#11 0x0040387e in _start ()
(gdb) up
#1  0x00415ced in odp_pktin_recv_tmo (queue=...,
packets=packets@entry=0x7ffed64d8bd0, num=num@entry=1,
  wait=wait@entry=18446744073709551615) at
../../../platform/linux-generic/odp_packet_io.c:1584
1584nanosleep(, NULL);
(gdb) p ts
$1 = {tv_sec = 0, tv_nsec = 1000}
(gdb) l
1579}
1580
1581wait--;
1582}
1583
1584nanosleep(, NULL);
1585}
1586}
1587
1588int odp_pktin_recv_mq_tmo(const odp_pktin_queue_t queues[],
unsigned num_q,
(gdb) up
#2  0x004047fa in recv_packets_tmo (pktio=pktio@entry=0x2,
pkt_tbl=pkt_tbl@entry=0x7ffed64d9500,
  seq_tbl=seq_tbl@entry=0x7ffed64d94b0, num=num@entry=1,
mode=mode@entry=RECV_TMO, tmo=tmo@entry=18446744073709551615,
ns=ns@entry=0)
  at
../../../../../../test/common_plat/validation/api/pktio/pktio.c:515
515n = odp_pktin_recv_tmo(pktin[0], pkt_tmp, num - num_rx,
(gdb) p num - num_rx
$2 = 1
(gdb) l
510/** Multiple odp_pktin_recv_tmo()/odp_pktin_recv_mq_tmo()
calls may be
511 *  required to discard possible non-test packets. */
512do {
513ts1 = odp_time_global();
514if (mode == RECV_TMO)
515n = odp_pktin_recv_tmo(pktin[0], pkt_tmp, num - num_rx,
516   tmo);
517else
518n = odp_pktin_recv_mq_tmo(pktin, (unsigned)num_q,
519  from, pkt_tmp,
(gdb) p tmo
$3 = 18446744073709551615


I applied patches and following script under root:
CLEANUP=0 GIT_URL=/opt/Linaro/odp3.git GIT_BRANCH=api-next ./build.sh

Need more investigation into this issue... Not applied yet.

Maxim.

On 11/16/16 02:58, Bill 

[lng-odp] [API-NEXT PATCHv7 10/13] test: linux-gen: api: shmem: test sharing memory between ODP instances

[Christophe Milard]

The platform tests odp/test/linux-generic/validation/api/shmem
are updated to both test ODP<->linux process memory sharing, but also test
ODP to ODP (different instances) memory sharing.
shmem_linux is the main test process, and shmem_linux.c contains (at
file top) a chart flow of the test procedure.

Signed-off-by: Christophe Milard 
---
 test/linux-generic/validation/api/shmem/.gitignore |   3 +-
 .../linux-generic/validation/api/shmem/Makefile.am |  22 ++--
 .../validation/api/shmem/shmem_linux.c | 140 +++--
 .../api/shmem/{shmem_odp.c => shmem_odp1.c}|  10 +-
 .../api/shmem/{shmem_odp.h => shmem_odp1.h}|   0
 .../validation/api/shmem/shmem_odp2.c  |  95 ++
 .../validation/api/shmem/shmem_odp2.h  |   7 ++
 7 files changed, 230 insertions(+), 47 deletions(-)
 rename test/linux-generic/validation/api/shmem/{shmem_odp.c => shmem_odp1.c} 
(81%)
 rename test/linux-generic/validation/api/shmem/{shmem_odp.h => shmem_odp1.h} 
(100%)
 create mode 100644 test/linux-generic/validation/api/shmem/shmem_odp2.c
 create mode 100644 test/linux-generic/validation/api/shmem/shmem_odp2.h

diff --git a/test/linux-generic/validation/api/shmem/.gitignore 
b/test/linux-generic/validation/api/shmem/.gitignore
index 7627079..74195f5 100644
--- a/test/linux-generic/validation/api/shmem/.gitignore
+++ b/test/linux-generic/validation/api/shmem/.gitignore
@@ -1,2 +1,3 @@
 shmem_linux
-shmem_odp
+shmem_odp1
+shmem_odp2
diff --git a/test/linux-generic/validation/api/shmem/Makefile.am 
b/test/linux-generic/validation/api/shmem/Makefile.am
index 341747f..b0ae627 100644
--- a/test/linux-generic/validation/api/shmem/Makefile.am
+++ b/test/linux-generic/validation/api/shmem/Makefile.am
@@ -2,19 +2,27 @@ include ../Makefile.inc
 
 #the main test program is shmem_linux, which, in turn, starts a shmem_odp:
 test_PROGRAMS = shmem_linux$(EXEEXT)
-test_extra_PROGRAMS = shmem_odp$(EXEEXT)
+test_extra_PROGRAMS = shmem_odp1$(EXEEXT) shmem_odp2$(EXEEXT)
 test_extradir = $(testdir)
 
 #shmem_linux is stand alone, pure linux (no ODP):
 dist_shmem_linux_SOURCES = shmem_linux.c
 shmem_linux_LDFLAGS = $(AM_LDFLAGS) -lrt
 
-#shmem_odp is the odp part:
-dist_shmem_odp_SOURCES = shmem_odp.c
-shmem_odp_CFLAGS = $(AM_CFLAGS) \
+#shmem_odp1 and shmem_odp2 are the 2 ODP processes:
+dist_shmem_odp1_SOURCES = shmem_odp1.c
+shmem_odp1_CFLAGS = $(AM_CFLAGS) \
   $(INCCUNIT_COMMON) \
   $(INCODP)
-shmem_odp_LDFLAGS = $(AM_LDFLAGS)
-shmem_odp_LDADD = $(LIBCUNIT_COMMON) $(LIBODP)
+shmem_odp1_LDFLAGS = $(AM_LDFLAGS)
+shmem_odp1_LDADD = $(LIBCUNIT_COMMON) $(LIBODP)
 
-noinst_HEADERS = shmem_common.h shmem_linux.h shmem_odp.h
+dist_shmem_odp2_SOURCES = shmem_odp2.c
+shmem_odp2_CFLAGS = $(AM_CFLAGS) \
+  $(INCCUNIT_COMMON) \
+  $(INCODP)
+shmem_odp2_LDFLAGS = $(AM_LDFLAGS)
+shmem_odp2_LDADD = $(LIBCUNIT_COMMON) $(LIBODP)
+
+
+noinst_HEADERS = shmem_common.h shmem_linux.h shmem_odp1.h shmem_odp2.h
diff --git a/test/linux-generic/validation/api/shmem/shmem_linux.c 
b/test/linux-generic/validation/api/shmem/shmem_linux.c
index 9ab0e2b..39473f3 100644
--- a/test/linux-generic/validation/api/shmem/shmem_linux.c
+++ b/test/linux-generic/validation/api/shmem/shmem_linux.c
@@ -5,8 +5,10 @@
  */
 
 /* this test makes sure that odp shared memory created with the ODP_SHM_PROC
- * flag is visible under linux. It therefore checks both that the device
- * name under /dev/shm is correct, and also checks that the memory contents
+ * flag is visible under linux, and checks that memory created with the
+ * ODP_SHM_EXPORT flag is visible by other ODP instances.
+ * It therefore checks both that the link
+ * name under /tmp is correct, and also checks that the memory contents
  * is indeed shared.
  * we want:
  * -the odp test to run using C UNIT
@@ -15,18 +17,47 @@
  *
  * To achieve this, the flow of operations is as follows:
  *
- *   linux process (main, non odp) |   ODP process
- *   (shmem_linux.c)   |   (shmem_odp.c)
+ *   linux process (main, non odp) |
+ *   (shmem_linux.c)   |
+ * |
+ * |
  * |
  *   main()|
- *   forks odp process |  allocate shmem
- *   wait for named pipe creation  |  populate shmem
+ *   forks odp_app1 process|
+ *   wait for named pipe creation  |
+ * |
+ * |   ODP_APP1 process
+ * |   (shmem_odp1.c)
+ * |
+ * |  allocate shmem
+ * |  populate shmem
  * |  create named pipe
- *   read shared memory

[lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Bill Fischofer]

Signed-off-by: Bill Fischofer 
---
 CHANGELOG | 211 ++
 1 file changed, 211 insertions(+)

diff --git a/CHANGELOG b/CHANGELOG
index d8230cd..3ab7354 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,214 @@
+== OpenDataPlane (1.10.2.0)
+
+=== New Features
+
+ APIs
+ODP v1.10.2.0 is a minor API revision level and introduces a small change to
+the Traffic Manager API as well as some documentation clarification surrounding
+other APIs.
+
+ Traffic Manager Egress Function
+The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
+(`egress_fcn_supported`) that indicates whether the TM system supports
+a user-provided egress function. When specified this function is called to
+"output" a packet rather than having TM transmit it directly to a PktIO
+interface.
+
+ Traffic Manager Coupling Change
+The `odp_tm_egress_t` struct has been changed to associate a TM system with an
+output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`. This makes
+an explicit 1-to-1 map between a TM system and a PktIO interface.
+
+ Default huge page size clarification
+The documentation for the `odp_sys_huge_page_size()` API has been reworded to
+clarify that this API refers to default huge page size.
+
+ Strict Priority (SP) Scheduler
+Building on the modular scheduler framework added in v1.10.1.0, an alternate
+Strict Priority (SP) Scheduler is now available for latency-sensitive
+workloads. Applications wishing to use the SP scheduler should specify
+the `./configure` option `--enable-schedule-sp`. This scheduler emphasizes low
+latency processing of high priority events at the expense of throughput. This
+alternate scheduler is considered experimental and should not be used for
+production at this time.
+
+ Application Binary Interface (ABI) Support
+Support is added to enable ODP applications to be binary compatible across
+different implementations of ODP sharing the same Instruction Set Architecture
+(ISA). This support introduces a new `configure` option:
+
+`--enable-abi-compat=yes`::
+This is the default and specifies that the ODP library is to be built to
+support ABI compatibility mode. In this mode ODP APIs are never inlined. ABI
+compatibility ensures maximum application portability in cloud environments.
+
+`--enable-abi-compat=no`::
+Specify this option to enable the inlining of ODP APIs. This may result in
+improved performance at the cost of ABI compatibility and is suitable for
+applications running in embedded environments.
+
+Note that ODP applications retain source code portability between ODP
+implementations regardless of the ABI mode chosen. To move to a different ODP
+application running on a different ISA, code need simply be recompiled against
+that target ODP implementation.
+
+ SCTP Parsing Support
+The ODP classifier adds support for recognizing Stream Control Transmission
+Protocol (SCTP) packets. The APIs for this were previously not implemented.
+
+=== Packaging and Implementation Refinements
+
+ Remove dependency on Linux headers
+ODP no longer has a dependency on Linux headers. This will help make the
+odp-linux reference implementation more easily portable to non-Linux
+environments.
+
+ Remove dependency on helpers
+The odp-linux implementation has been made independent of the helper library
+to avoid circular dependency issues with packaging. Helper functions may use
+ODP APIs, however ODP implementations should not use helper functions.
+
+ Reorganization of `test` directory
+The `test` directory has been reorganized to better support a unified approach
+to ODP component testing. API tests now live in
+`test/common_plat/validation/api` instead of the former
+`test/validation`. With this change performance and validation tests, as well
+as common and platform-specific tests can all be part of a unified test
+hierarchy.
+
+The resulting test tree now looks like:
+
+.New `test` directory hierarchy
+-
+test
+├── common_plat
+│   ├── common
+│   ├── m4
+│   ├── miscellaneous
+│   ├── performance
+│   └── validation
+│   └── api
+│   ├── atomic
+│   ├── barrier
+│   ├── buffer
+│   ├── classification
+│   ├── cpumask
+│   ├── crypto
+│   ├── errno
+│   ├── hash
+│   ├── init
+│   ├── lock
+│   ├── packet
+│   ├── pktio
+│   ├── pool
+│   ├── queue
+│   ├── random
+│   ├── scheduler
+│   ├── shmem
+│   ├── std_clib
+│   ├── system
+│   ├── thread
+│   ├── time
+│   ├── timer
+│   └── traffic_mngr
+├── linux-generic
+│   ├── m4
+│   ├── mmap_vlan_ins
+│   ├── performance
+│   ├── pktio_ipc
+│   ├── ring
+│   └── validation
+│   └── api
+│   ├── pktio
+│   └── shmem
+└── m4
+-
+
+ Pools
+The maximum number of pools that may be created 

Re: [lng-odp] [API-NEXT PATCHv2 2/2] linux-gen: _ishmphy: adding debug function for pysical address mapping

[Yi He]

One comment inline:

On 11 November 2016 at 22:50, Christophe Milard <
christophe.mil...@linaro.org> wrote:

> _odp_ishmphy_memmap_print() prints the virtual to physical address
> mapping of some memory region (defined by virtuall address + length).
>
> Signed-off-by: Christophe Milard 
> ---
>  platform/linux-generic/_ishmphy.c  | 58
> ++
>  platform/linux-generic/include/_ishmphy_internal.h |  5 ++
>  2 files changed, 63 insertions(+)
>
> diff --git a/platform/linux-generic/_ishmphy.c b/platform/linux-generic/_
> ishmphy.c
> index 8c0f46e..21f6bd1 100644
> --- a/platform/linux-generic/_ishmphy.c
> +++ b/platform/linux-generic/_ishmphy.c
> @@ -265,3 +265,61 @@ int _odp_ishmphy_can_virt_to_phys(void)
>
> return 1;
>  }
> +
> +/*
> + * dump the physical mapping from virtual address addr, length len.
> + */
> +void _odp_ishmphy_memmap_print(void *curr_addr, uint64_t len)
> +{
> +   int page_sz;
> +   phys_addr_t curr_phy;
> +   phys_addr_t start_phy;
> +   void *start_address = 0;
> +   uint32_t curr_len = 0;
> +   uint32_t pfn_count = 0;
> +
> +   /* get normal page sizes: */
> +   page_sz = odp_sys_page_size();
> +
> +   curr_phy = _odp_ishmphy_virt_to_phys(curr_addr);
> +   if (curr_phy == PHYS_ADDR_INVALID) {
> +   ODP_DBG("Phy Dump failed (permission?).\n");
> +   return;
> +   }
> +
> +   ODP_DBG("Phy Dump:\n");
> +   start_address = curr_addr;
> +   start_phy = curr_phy;
> +
> +   curr_len += page_sz;
> +   curr_addr = (void *)((char *)curr_addr + page_sz);
> +   pfn_count++;
> +   while (curr_len < len) {
> +   if (_odp_ishmphy_virt_to_phys(curr_addr) ==
> +   curr_phy + page_sz) {
> +   curr_len += page_sz;
> +   curr_addr = (void *)((char *)curr_addr + page_sz);
> +   curr_phy += page_sz;
> +   pfn_count++;
> +   continue;
> +   }
> +
> +   ODP_DBG("Virtual: %08" PRIx64 " <-->  Phy: %08" PRIx64
> +   "   %" PRIu32 " PFNs, %" PRIu32 " bytes\n",
> +   (uint64_t)start_address, start_phy,
> +   pfn_count, pfn_count * page_sz);
> +   curr_phy =  _odp_ishmphy_virt_to_phys(curr_addr);
>

Should here check the success or failure of the call? And break in case
failed.


> +   start_address = curr_addr;
> +   start_phy = curr_phy;
> +   pfn_count = 0;
> +   curr_len += page_sz;
> +   curr_addr = (void *)((char *)curr_addr + page_sz);
> +   pfn_count++;
> +   }
> +
> +   if (pfn_count)
> +   ODP_DBG("Virtual: %08" PRIx64 " <-->  Phy: %08" PRIx64
> +   "   %" PRIu32 " PFNs, %" PRIu32 " bytes\n",
> +   (uint64_t)start_address, start_phy,
> +   pfn_count, pfn_count * page_sz);
> +}
> diff --git a/platform/linux-generic/include/_ishmphy_internal.h
> b/platform/linux-generic/include/_ishmphy_internal.h
> index 2022590..c8752c0 100644
> --- a/platform/linux-generic/include/_ishmphy_internal.h
> +++ b/platform/linux-generic/include/_ishmphy_internal.h
> @@ -32,6 +32,11 @@ int _odp_ishmphy_can_virt_to_phys(void);
>   */
>  phys_addr_t _odp_ishmphy_virt_to_phys(const void *addr);
>
> +/*
> + * dump the physical mapping from virtual address addr, length len.
> + */
> +void _odp_ishmphy_memmap_print(void *addr, uint64_t len);
> +
>  #ifdef __cplusplus
>  }
>  #endif
> --
> 2.7.4
>
>

Re: [lng-odp] [API-NEXT PATCHv2 1/2] linux-gen: _ishmphy: adding function for physical address query

[Christophe Milard]

Th page must be locked, indeed: if the page is swapped out its
physical address does not exist...

_ishm (the internal ODP memory allocator has a flag to lock memory:
_ODP_ISHM_LOCK

On the north API (odp_*), any memory reserved is locked anyway. This
was a requirement from Petri: that is any  odp_shm_reserve() will get
locked page. Assuming we are using the _ishm memory allocator on the
north interface (this patch serie is still waiting for Petri's
blessing)

On the south interface (odpdrv_*) both the driver still has the
choice, through a flag: ODPDRV_SHM_LOCK
ODPDRV_SHM_LOCK maps to the _ishm _ODP_ISHM_LOCK flag.

Christophe

On 14 November 2016 at 06:00, Yi He  wrote:
> One comment inline:
>
> On 11 November 2016 at 22:50, Christophe Milard
>  wrote:
>>
>> The function _odp_ishmphy_virt_to_phys() is added to query for physical
>> addresses (given a virtual address)
>> This function is meant to be used to populate the physical address
>> of packet segments which are to be used by drivers (without iommu).
>> The function _odp_ishmphy_can_virt_to_phys(), also added, return true if
>> _odp_ishmphy_virt_to_phys() is able to works (as it requires specific
>> permission)
>>
>> Signed-off-by: Christophe Milard 
>> ---
>>  platform/linux-generic/_ishmphy.c  | 82
>> ++
>>  platform/linux-generic/include/_ishmphy_internal.h | 14 
>>  2 files changed, 96 insertions(+)
>>
>> diff --git a/platform/linux-generic/_ishmphy.c
>> b/platform/linux-generic/_ishmphy.c
>> index 2b2d100..8c0f46e 100644
>> --- a/platform/linux-generic/_ishmphy.c
>> +++ b/platform/linux-generic/_ishmphy.c
>> @@ -29,6 +29,7 @@
>>  #include 
>>  #include 
>>  #include 
>> +#include 
>>  #include <_ishmphy_internal.h>
>>
>>  static void *common_va_address;
>> @@ -38,6 +39,8 @@ static uint64_t common_va_len;
>>  #define MAP_ANONYMOUS MAP_ANON
>>  #endif
>>
>> +#define PAGEMAP_FILE "/proc/self/pagemap"
>> +
>>  /* Book some virtual address space
>>   * This function is called at odp_init_global() time to pre-book some
>>   * virtual address space inherited by all odpthreads (i.e. descendant
>> @@ -183,3 +186,82 @@ int _odp_ishmphy_unmap(void *start, uint64_t len, int
>> flags)
>> ODP_ERR("_ishmphy_free failure: %s\n", strerror(errno));
>> return ret;
>>  }
>> +
>> +/*
>> + * Get physical address from virtual address addr.
>> + */
>
>
> I saw DPDK API commented that "The page must be locked", and provides an API
> rte_mem_lock_page to do so, in this API is the locking guarenteed by like
> _odp_ishm_reserve calls already?
>
>> +phys_addr_t _odp_ishmphy_virt_to_phys(const void *addr)
>> +{
>> +   int page_sz;
>> +   int fd;
>> +   off_t offset;
>> +   int  read_bytes;
>> +   uint64_t page;
>> +   phys_addr_t phys_addr;
>> +
>> +   /* get normal page sizes: */
>> +   page_sz = odp_sys_page_size();
>> +
>> +   /* read 8 bytes (uint64_t) at position N*8, where N is
>> addr/page_sz */
>> +   fd = open(PAGEMAP_FILE, O_RDONLY);
>> +   if (fd < 0) {
>> +   ODP_ERR("cannot open " PAGEMAP_FILE ": %s\n",
>> +   strerror(errno));
>> +   return PHYS_ADDR_INVALID;
>> +   }
>> +
>> +   offset = ((unsigned long)addr / page_sz) * sizeof(uint64_t);
>> +   if (lseek(fd, offset, SEEK_SET) == (off_t)-1) {
>> +   ODP_ERR("cannot seek " PAGEMAP_FILE ": %s\n",
>> +   strerror(errno));
>> +   close(fd);
>> +   return PHYS_ADDR_INVALID;
>> +   }
>> +
>> +   read_bytes = read(fd, , sizeof(uint64_t));
>> +   close(fd);
>> +   if (read_bytes < 0) {
>> +   ODP_ERR("cannot read " PAGEMAP_FILE ": %s\n",
>> +   strerror(errno));
>> +   return PHYS_ADDR_INVALID;
>> +   } else if (read_bytes != sizeof(uint64_t)) {
>> +   ODP_ERR("read %d bytes from " PAGEMAP_FILE " "
>> +   "but expected %d:\n",
>> +   read_bytes, sizeof(uint64_t));
>> +   return PHYS_ADDR_INVALID;
>> +   }
>> +
>> +   /* some kernel return PFN zero when permission is denied: */
>> +   if (!(page & 0x7fULL))
>> +   return PHYS_ADDR_INVALID;
>> +
>> +   /*
>> +* the pfn (page frame number) are bits 0-54 (see
>> +* pagemap.txt in linux Documentation)
>> +*/
>> +   phys_addr = ((page & 0x7fULL) * page_sz)
>> +   + ((unsigned long)addr % page_sz);
>> +
>> +   return phys_addr;
>> +}
>> +
>> +/*
>> + * check if physical address are readable
>> + * return true if physical addresses can be retrieved.
>> + * Just do a test to see if it works
>> + */
>> +int _odp_ishmphy_can_virt_to_phys(void)
>> +{
>> +   int block_index;
>> +   phys_addr_t phy;
>> +
>> +   /* allocate a block, locked in memory and try 

Re: [lng-odp] Classification Queue Group

[Brian Brooks]

On Mon, Nov 14, 2016 at 2:12 AM, Bala Manoharan
 wrote:
> Regards,
> Bala
>
>
> On 11 November 2016 at 13:26, Brian Brooks  wrote:
>> On 11/10 15:17:15, Bala Manoharan wrote:
>>> On 10 November 2016 at 13:26, Brian Brooks  wrote:
>>> > On 11/07 16:46:12, Bala Manoharan wrote:
>>> >> Hi,
>>> >
>>> > Hiya
>>> >
>>> >> This mail thread discusses the design of classification queue group
>>> >> RFC. The same can be found in the google doc whose link is given
>>> >> below.
>>> >> Users can provide their comments either in this mail thread or in the
>>> >> google doc as per their convenience.
>>> >>
>>> >> https://docs.google.com/document/d/1fOoG9WDR0lMpVjgMAsx8QsMr0YFK9slR93LZ8VXqM2o/edit?usp=sharing
>>> >>
>>> >> The basic issues with queues as being a single target for a CoS are two 
>>> >> fold:
>>> >>
>>> >> Queues must be created and deleted individually. This imposes a
>>> >> significant burden when queues are used to represent individual flows
>>> >> since the application may need to process thousands (or millions) of
>>> >> flows.
>>> >
>>> > Wondering why there is an issue with creating and deleting queues 
>>> > individually
>>> > if queue objects represent millions of flows..
>>>
>>> The queue groups are mainly required for hashing the incoming packets
>>> to multiple flows based on the hash configuration.
>>> So from application point of view it just needs a queue to have
>>> packets belonging to same flow and that packets belonging to different
>>> flows are placed in different queues respectively.It does not matter
>>> who creates the flow/queue.
>>
>> When the application receives an event from odp_schedule() call, how does it
>> know whether the odp_queue_t was previously created by the application from
>> odp_queue_create() or whether it was created by the implementation?
>
> odp_schedule() call returns the queue from which the event was part of.
> The type of the queue can be got from odp_queue_type() API.
> But the question is there an use-case where the application need to know?
> The application has the information of the queue it has created and
> the queues created by implementation are destroyed by implementation.

If certain fields of the packet are hashed to a queue handle, and this queue
handle has not previously been created via odp_queue_create(), there might
be a use case where the application needs to be aware of a new "flow"..
Maybe the application ages flows.

>
>>
>>> It is actually simpler if implementation creates a flow since in that
>>> case implementation need not accumulate meta-data for all possible
>>> hash values in a queue group and it can be created when traffic
>>> arrives in that particular flow.
>>>
>>> >
>>> > Could an application ever call odp_schedule() and receive an event (e.g. 
>>> > packet)
>>> > from a queue (of opaque type odp_queue_t) and that queue has never been 
>>> > created
>>> > by the application (via odp_queue_create())? Could that ever happen from 
>>> > the
>>> > hardware, and could the application ever handle that?
>>>
>>> No. All the queues in the system are created by the application either
>>> directly or in-directly.
>>> In-case of queue groups the queues are in-directly created by the
>>> application by configuring a queue group.
>>>
>>> > Or, is it related to memory usage? The reference implementation
>>> > struct queue_entry_s is 320 bytes on a 64-bit machine.
>>> >
>>> >   2^28 ~= 268,435,456 queues -> 81.920 GB
>>> >   2^26 ~=  67,108,864 queues -> 20.480 GB
>>> >   2^22 ~=   4,194,304 queues ->  1.280 GB
>>> >
>>> > Forget about 320 bytes per queue, if each queue was represented by a 
>>> > 32-bit
>>> > integer (4 bytes!) the usage would be:
>>> >
>>> >   2^28 ~= 268,435,456 queues ->  1.024 GB
>>> >   2^26 ~=  67,108,864 queues ->256 MB
>>> >   2^22 ~=   4,194,304 queues -> 16 MB
>>> >
>>> > That still might be a lot of usage if the application must explicitly 
>>> > create
>>> > every queue (before it is used) and require an ODP implementation to map
>>> > between every ODP queue object (opaque type) and the internal queue.
>>> >
>>> > Lets say ODP API has two classes of handles: 1) pointers, 2) integers. An 
>>> > opaque
>>> > pointer is used to point to some other software object. This object 
>>> > should be
>>> > larger than 64 bits (or 32 bits on a chip in 32-bit pointer mode) 
>>> > otherwise it
>>> > could just be represented in a 64-bit (or 32-bit) integer type value!
>>> >
>>> > To support millions of queues (flows) should odp_queue_t be an integer 
>>> > type in
>>> > the API? A software-only implementation may still use 320 bytes per queue 
>>> > and
>>> > use that integer as an index into an array or as a key for lookup 
>>> > operation on a
>>> > data structure containing queues. An implementation with hardware assist 
>>> > may
>>> > use this integer value directly when interfacing with hardware!
>>>
>>> I believe I have answered this 

[lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Bill Fischofer]

Signed-off-by: Bill Fischofer 
---
 CHANGELOG | 211 ++
 1 file changed, 211 insertions(+)

diff --git a/CHANGELOG b/CHANGELOG
index d8230cd..3ab7354 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,214 @@
+== OpenDataPlane (1.10.2.0)
+
+=== New Features
+
+ APIs
+ODP v1.10.2.0 is a minor API revision level and introduces a small change to
+the Traffic Manager API as well as some documentation clarification surrounding
+other APIs.
+
+ Traffic Manager Egress Function
+The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
+(`egress_fcn_supported`) that indicates whether the TM system supports
+a user-provided egress function. When specified this function is called to
+"output" a packet rather than having TM transmit it directly to a PktIO
+interface.
+
+ Traffic Manager Coupling Change
+The `odp_tm_egress_t` struct has been changed to associate a TM system with an
+output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`. This makes
+an explicit 1-to-1 map between a TM system and a PktIO interface.
+
+ Default huge page size clarification
+The documentation for the `odp_sys_huge_page_size()` API has been reworded to
+clarify that this API refers to default huge page size.
+
+ Strict Priority (SP) Scheduler
+Building on the modular scheduler framework added in v1.10.1.0, an alternate
+Strict Priority (SP) Scheduler is now available for latency-sensitive
+workloads. Applications wishing to use the SP scheduler should specify
+the `./configure` option `--enable-schedule-sp`. This scheduler emphasizes low
+latency processing of high priority events at the expense of throughput. This
+alternate scheduler is considered experimental and should not be used for
+production at this time.
+
+ Application Binary Interface (ABI) Support
+Support is added to enable ODP applications to be binary compatible across
+different implementations of ODP sharing the same Instruction Set Architecture
+(ISA). This support introduces a new `configure` option:
+
+`--enable-abi-compat=yes`::
+This is the default and specifies that the ODP library is to be built to
+support ABI compatibility mode. In this mode ODP APIs are never inlined. ABI
+compatibility ensures maximum application portability in cloud environments.
+
+`--enable-abi-compat=no`::
+Specify this option to enable the inlining of ODP APIs. This may result in
+improved performance at the cost of ABI compatibility and is suitable for
+applications running in embedded environments.
+
+Note that ODP applications retain source code portability between ODP
+implementations regardless of the ABI mode chosen. To move to a different ODP
+application running on a different ISA, code need simply be recompiled against
+that target ODP implementation.
+
+ SCTP Parsing Support
+The ODP classifier adds support for recognizing Stream Control Transmission
+Protocol (SCTP) packets. The APIs for this were previously not implemented.
+
+=== Packaging and Implementation Refinements
+
+ Remove dependency on Linux headers
+ODP no longer has a dependency on Linux headers. This will help make the
+odp-linux reference implementation more easily portable to non-Linux
+environments.
+
+ Remove dependency on helpers
+The odp-linux implementation has been made independent of the helper library
+to avoid circular dependency issues with packaging. Helper functions may use
+ODP APIs, however ODP implementations should not use helper functions.
+
+ Reorganization of `test` directory
+The `test` directory has been reorganized to better support a unified approach
+to ODP component testing. API tests now live in
+`test/common_plat/validation/api` instead of the former
+`test/validation`. With this change performance and validation tests, as well
+as common and platform-specific tests can all be part of a unified test
+hierarchy.
+
+The resulting test tree now looks like:
+
+.New `test` directory hierarchy
+-
+test
+├── common_plat
+│   ├── common
+│   ├── m4
+│   ├── miscellaneous
+│   ├── performance
+│   └── validation
+│   └── api
+│   ├── atomic
+│   ├── barrier
+│   ├── buffer
+│   ├── classification
+│   ├── cpumask
+│   ├── crypto
+│   ├── errno
+│   ├── hash
+│   ├── init
+│   ├── lock
+│   ├── packet
+│   ├── pktio
+│   ├── pool
+│   ├── queue
+│   ├── random
+│   ├── scheduler
+│   ├── shmem
+│   ├── std_clib
+│   ├── system
+│   ├── thread
+│   ├── time
+│   ├── timer
+│   └── traffic_mngr
+├── linux-generic
+│   ├── m4
+│   ├── mmap_vlan_ins
+│   ├── performance
+│   ├── pktio_ipc
+│   ├── ring
+│   └── validation
+│   └── api
+│   ├── pktio
+│   └── shmem
+└── m4
+-
+
+ Pools
+The maximum number of pools that may be created 

[lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Bill Fischofer]

Signed-off-by: Bill Fischofer 
---
 CHANGELOG | 211 ++
 1 file changed, 211 insertions(+)

diff --git a/CHANGELOG b/CHANGELOG
index d8230cd..3ab7354 100644
--- a/CHANGELOG
+++ b/CHANGELOG
@@ -1,3 +1,214 @@
+== OpenDataPlane (1.10.2.0)
+
+=== New Features
+
+ APIs
+ODP v1.10.2.0 is a minor API revision level and introduces a small change to
+the Traffic Manager API as well as some documentation clarification surrounding
+other APIs.
+
+ Traffic Manager Egress Function
+The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
+(`egress_fcn_supported`) that indicates whether the TM system supports
+a user-provided egress function. When specified this function is called to
+"output" a packet rather than having TM transmit it directly to a PktIO
+interface.
+
+ Traffic Manager Coupling Change
+The `odp_tm_egress_t` struct has been changed to associate a TM system with an
+output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`. This makes
+an explicit 1-to-1 map between a TM system and a PktIO interface.
+
+ Default huge page size clarification
+The documentation for the `odp_sys_huge_page_size()` API has been reworded to
+clarify that this API refers to default huge page size.
+
+ Strict Priority (SP) Scheduler
+Building on the modular scheduler framework added in v1.10.1.0, an alternate
+Strict Priority (SP) Scheduler is now available for latency-sensitive
+workloads. Applications wishing to use the SP scheduler should specify
+the `./configure` option `--enable-schedule-sp`. This scheduler emphasizes low
+latency processing of high priority events at the expense of throughput. This
+alternate scheduler is considered experimental and should not be used for
+production at this time.
+
+ Application Binary Interface (ABI) Support
+Support is added to enable ODP applications to be binary compatible across
+different implementations of ODP sharing the same Instruction Set Architecture
+(ISA). This support introduces a new `configure` option:
+
+`--enable-abi-compat=yes`::
+This is the default and specifies that the ODP library is to be built to
+support ABI compatibility mode. In this mode ODP APIs are never inlined. ABI
+compatibility ensures maximum application portability in cloud environments.
+
+`--enable-abi-compat=no`::
+Specify this option to enable the inlining of ODP APIs. This may result in
+improved performance at the cost of ABI compatibility and is suitable for
+applications running in embedded environments.
+
+Note that ODP applications retain source code portability between ODP
+implementations regardless of the ABI mode chosen. To move to a different ODP
+application running on a different ISA, code need simply be recompiled against
+that target ODP implementation.
+
+ SCTP Parsing Support
+The ODP classifier adds support for recognizing Stream Control Transmission
+Protocol (SCTP) packets. The APIs for this were previously not implemented.
+
+=== Packaging and Implementation Refinements
+
+ Remove dependency on Linux headers
+ODP no longer has a dependency on Linux headers. This will help make the
+odp-linux reference implementation more easily portable to non-Linux
+environments.
+
+ Remove dependency on helpers
+The odp-linux implementation has been made independent of the helper library
+to avoid circular dependency issues with packaging. Helper functions may use
+ODP APIs, however ODP implementations should not use helper functions.
+
+ Reorganization of `test` directory
+The `test` directory has been reorganized to better support a unified approach
+to ODP component testing. API tests now live in
+`test/common_plat/validation/api` instead of the former
+`test/validation`. With this change performance and validation tests, as well
+as common and platform-specific tests can all be part of a unified test
+hierarchy.
+
+The resulting test tree now looks like:
+
+.New `test` directory hierarchy
+-
+test
+├── common_plat
+│   ├── common
+│   ├── m4
+│   ├── miscellaneous
+│   ├── performance
+│   └── validation
+│   └── api
+│   ├── atomic
+│   ├── barrier
+│   ├── buffer
+│   ├── classification
+│   ├── cpumask
+│   ├── crypto
+│   ├── errno
+│   ├── hash
+│   ├── init
+│   ├── lock
+│   ├── packet
+│   ├── pktio
+│   ├── pool
+│   ├── queue
+│   ├── random
+│   ├── scheduler
+│   ├── shmem
+│   ├── std_clib
+│   ├── system
+│   ├── thread
+│   ├── time
+│   ├── timer
+│   └── traffic_mngr
+├── linux-generic
+│   ├── m4
+│   ├── mmap_vlan_ins
+│   ├── performance
+│   ├── pktio_ipc
+│   ├── ring
+│   └── validation
+│   └── api
+│   ├── pktio
+│   └── shmem
+└── m4
+-
+
+ Pools
+The maximum number of pools that may be created 

Re: [lng-odp] Classification Queue Group

[Bill Fischofer]

On Thu, Nov 17, 2016 at 3:05 AM, Bala Manoharan 
wrote:

> Regards,
> Bala
>
>
> On 15 November 2016 at 22:43, Brian Brooks 
> wrote:
> > On Mon, Nov 14, 2016 at 2:12 AM, Bala Manoharan
> >  wrote:
> >> Regards,
> >> Bala
> >>
> >>
> >> On 11 November 2016 at 13:26, Brian Brooks 
> wrote:
> >>> On 11/10 15:17:15, Bala Manoharan wrote:
>  On 10 November 2016 at 13:26, Brian Brooks 
> wrote:
>  > On 11/07 16:46:12, Bala Manoharan wrote:
>  >> Hi,
>  >
>  > Hiya
>  >
>  >> This mail thread discusses the design of classification queue group
>  >> RFC. The same can be found in the google doc whose link is given
>  >> below.
>  >> Users can provide their comments either in this mail thread or in
> the
>  >> google doc as per their convenience.
>  >>
>  >> https://docs.google.com/document/d/1fOoG9WDR0lMpVjgMAsx8QsMr0YFK9
> slR93LZ8VXqM2o/edit?usp=sharing
>  >>
>  >> The basic issues with queues as being a single target for a CoS
> are two fold:
>  >>
>  >> Queues must be created and deleted individually. This imposes a
>  >> significant burden when queues are used to represent individual
> flows
>  >> since the application may need to process thousands (or millions)
> of
>  >> flows.
>  >
>  > Wondering why there is an issue with creating and deleting queues
> individually
>  > if queue objects represent millions of flows..
> 
>  The queue groups are mainly required for hashing the incoming packets
>  to multiple flows based on the hash configuration.
>  So from application point of view it just needs a queue to have
>  packets belonging to same flow and that packets belonging to different
>  flows are placed in different queues respectively.It does not matter
>  who creates the flow/queue.
> >>>
> >>> When the application receives an event from odp_schedule() call, how
> does it
> >>> know whether the odp_queue_t was previously created by the application
> from
> >>> odp_queue_create() or whether it was created by the implementation?
> >>
> >> odp_schedule() call returns the queue from which the event was part of.
> >> The type of the queue can be got from odp_queue_type() API.
> >> But the question is there an use-case where the application need to
> know?
> >> The application has the information of the queue it has created and
> >> the queues created by implementation are destroyed by implementation.
> >
> > If certain fields of the packet are hashed to a queue handle, and this
> queue
> > handle has not previously been created via odp_queue_create(), there
> might
> > be a use case where the application needs to be aware of a new "flow"..
> > Maybe the application ages flows.
>

I'm not sure I understand the concern being raised here. Packet fields are
matched against PMRs to get a matching CoS. That CoS, in turn, is
associated with with a queue or a queue group. If the latter then specified
subfields within the packet are hashed to generate an index into that queue
group to select the individual queue within the target queue group that is
to receive the packet. Whether these queues have been preallocated at
odp_queue_group_create() time, or allocated dynamically on first reference
is up to the implementation, however especially in the case of "large"
queue groups it can be expected that the number of actual queues in use
will be sparse so a deferred allocation strategy will most likely be used.

Applications are aware of flows because that's what an individual queue
coming out of the classifier represents. An interesting question arises is
if a higher-level protocol (e.g., a TCP FIN sequence) ends a given flow,
meaning that the context represented by an individual queue within a queue
group can be released. Especially in the case of sparse queue groups it
might be worthwhile to have an API that can communicate this flow release
back to the classifier to facilitate queue resource management.


>
> It is very difficult in network traffic to predict the exact flows
> which will be coming in an interface.
> The application can configure for all the possible flows in that case.
>

Not sure what you mean by the application configuring. If the hash is for a
UDP port, for example, then the queue group has 64K (logical) queues
associated with it.  Which of these are active (and hence require
instantiation) depends on the inbound traffic that is received, which may
be unpredictable. But the management of this is an ODP implementation
concern rather than an application concern, unless we extend the API with a
flow release hint as suggested above.


>
> >
> >>
> >>>
>  It is actually simpler if implementation creates a flow since in that
>  case implementation need not accumulate meta-data for all possible
>  hash values in a queue group and it can be created 

Re: [lng-odp] [PATCH] travis: linux-gen: add dpdk pktio

[Mike Holmes]

On 7 November 2016 at 10:49, Maxim Uvarov  wrote:
> Turn on linux-generic dpdk and pcap pktios with their validation
> tests.
>
> Signed-off-by: Maxim Uvarov 

Reviewed-by: Mike Holmes 

We need to make this complete and add netmap pktio to cover all the
possibilities
We could also do with a matrix to cover the different schedulers possibly.


> ---
>  .travis.yml | 22 ++
>  1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/.travis.yml b/.travis.yml
> index 0d1add3..1092cd0 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -11,6 +11,7 @@ sudo: required
>  before_install:
>  - sudo apt-get -qq update
>  - sudo apt-get install automake autoconf libtool libssl-dev graphviz 
> mscgen doxygen
> +- sudo apt-get install libpcap-dev linux-headers-`uname -r`
>  - gem install asciidoctor
>
>  #   Install cunit for the validation tests because distro version is too 
> old and fails C99 compile
> @@ -24,12 +25,25 @@ before_install:
>  - sudo make install
>  - cd ..
>  - export LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH"
> -
> -install:
> -- ./bootstrap
> +#  DPDK pktio
> +- TARGET=${TARGET:-"x86_64-native-linuxapp-gcc"}
> +- git clone http://dpdk.org/git/dpdk dpdk
> +- pushd dpdk
> +- git checkout -b bv16.07 v16.07
> +- make config T=${TARGET} O=${TARGET}
> +- pushd ${TARGET}
> +- sed -ri 's,(CONFIG_RTE_LIBRTE_PMD_PCAP=).*,\1y,' .config
> +- popd
> +- make install T=${TARGET} EXTRA_CFLAGS="-fPIC"
> +- popd
>
>  script:
> -- ./configure --enable-test-cpp --enable-test-vald 
> --enable-test-helper --enable-test-perf --enable-user-guides 
> --enable-test-perf-proc --enable-test-example
> +
> +- ./bootstrap
> +- ./configure --enable-test-cpp --enable-test-vald 
> --enable-test-helper --enable-test-perf --enable-user-guides 
> --enable-test-perf-proc --enable-test-example 
> --with-dpdk-path=`pwd`/dpdk/${TARGET}
>  - make check
> +- git clean -f -d -x && rm -rf dpdk
> +- ./bootstrap
> +- ./configure
>  - make doxygen-doc
>  - make distcheck
> --
> 2.7.1.250.gff4ea60
>



-- 
Mike Holmes
Program Manager - Linaro Networking Group
Linaro.org │ Open source software for ARM SoCs
"Work should be fun and collaborative, the rest follows"

Re: [lng-odp] concat and split semantics

[Bill Fischofer]

These are good questions. Answers inline.

On Thu, Nov 17, 2016 at 10:52 AM, Joe Savage  wrote:

> > ODP specifies functional behavior of it's APIs. Implementations are free
> to
> > choose whatever techniques or other internals they wish to realize this
> > behavior. So things like lazy/deferred/batched operations are entirely
> > possible as long as the external semantics are matched.
>
> Sure, and that's really what I was asking: is there sufficient flexibility
> in the defined behaviour of concat and split to allow for a zero-copy
> implementation to exist in the context of fragmentation/reassembly?
>
> From your response I assume the answer is yes, but that wasn't necessarily
> clear to me as an ODP newbie. Particularly, I'm unsure if implementations
> are able to create packets with segment of differing lengths, as may be
> useful in an efficient zero-copy software implementation.
>

ODP packets are opaque objects of type odp_packet_t that have observable
behavior as defined by the ODP API specification. Packet structure is only
observable on the part of applications only as a byproduct of requesting
addressability to a packet. For this, ODP recognizes that implementations
may store packets in one or more discontiguous segments of
implementation-defined length. Applications may request minimum segment
lengths at the time packet pools are created, but implementations are free
to use larger lengths for internal reasons.

Whenever a packet is made addressable (by returning a void * pointer to the
offset in the packet that is requested) one of the return parameter is an
integer length that tells the caller how many contiguously addressable
bytes are available to it at the returned address. If the application
wishes to address beyond that point, another API call is required to get
addressability to the next segment. There is no requirement that segments
be of any particular length, though zero-length segments, if they exist at
all, are not visible via the ODP APIs. So the returned segment length will
always be a minimum of 1 byte in length.

This is discussed in the ODP User's Guide, which I recommend reading.


>
>
> > odp-linux is designed to provide a simple and clear reference
> > implementation of the ODP APIs. These are not necessarily optimized for
> > performance.
>
> Sure, I totally appreciate that, but I'm more interested in the API in the
> abstract here. What I'm trying to get at is whether split and concat were
> designed with a particular use case in mind. If so, its not unreasonable to
> think that implementations might optimise for this common use case, the
> requirements of which may not match my desired behaviour for fragmentation
> and reassembly. After all, it's not always the case that one size fits all,
> and perhaps a disparity in requirements between seemingly similar packet
> splitting or concatenation tasks could indicate that a new API would be
> useful.
>

IP Fragment reassembly can be realized with these APIs. Packet references
can also be used for this as individual fragments can be assembled as a
series of compound references (at least in the current proposed definition
of these APIs). However since these APIs have not yet been finalized
pending confirmation that the major platforms that are implementing ODP are
comfortable with these definitions. Part of the process of defining ODP
APIs is we try to satisfy two parties: application writers, who want a
robust and useful set of semantics, and platform implementers, who want to
be sure that defined ODP APIs can be mapped to their platforms efficiently
and can usefully exploit HW features found on those platforms.


>
> Not to say that I necessarily think doing this would be necessary here, but
> just that I'm trying to better understand what you might expect a decidedly
> "good" general implementation to do, and whether that would fit with my
> desired behaviour for this task.
>
>
> > The key difference between split/concat and references is that the latter
> > imposes usage restrictions on the end-product while the former do not. So
> > the "shared" portion of references should be treated as read/only by the
> > application.
>
> Reading the proposal I completely agree that this is a key difference, but
> as
> per my point above regarding how the expected usage of APIs will guide the
> behaviour of implementations, I think it's entirely possible that there
> will
> be other deep-rooted disparities between how implementations handle these
> two seemingly similar concepts. As such, I'm just trying to figure out
> whether — on the API level — you have an idea about what those differences
> might be. And, indeed, how those differences might lead me to choosing to
> use
> concat over references (or visa versa) when writing my reassembly code.
>

As with most APIs, there is usually more than one combination of calls that
can be used to accomplish the same objective. It's also possible that while
all ODP 

[lng-odp] concat and split semantics

[Joe Savage]

Hey,

I have recently been looking into the suitability of the ODP APIs for the
fragmentation and reassembly of IP packets. Particularly, I've been focused
on the split and concat APIs — using the former to break larger packets into
smaller fragments, and the latter to glue such fragments back together.

I'm not entirely sure to what the degree these APIs were designed to support
this kind of use case, though. Does the specification allow, for example, for
"zero-copy" implementations of split and concat, in which the operations act
purely on packet metadata (segments etc.) rather than the packet data itself?

And more broadly, if it is possible, is this the kind of behaviour that one
should expect from a good implementation? I'm not sure of the exact purposes
for which concat and split were designed, but it may be the case that this
kind of zero-copy ideal is outside the primary use case for these APIs, and
hence goes against the grain of what many implementations will do.

It's also interesting to contrast the semantics of concat and split to the
splice/reference API proposals, which more explicitly indicate that the
operations need not be implemented via costly copies. Do people have a good
idea of when it might, on a purely semantic level, be a better idea to use
concat rather than, say, nested references?

Thanks,

Joe

Re: [lng-odp] Classification Queue Group

[Honnappa Nagarahalli]

Hi Bala,
   I am trying to catch up on this conversation. I have few questions here,
not sure if they are discussed already.

1) Why is there a need for such large number of queues (Million) in a
system? If the system meets the line rate, the packets (and the flows)
waiting to be processed is small. So, a small number of queues should
suffice, if we want to have to queue per flow. I agree that the system will
receive packets belonging to million flows over a period of time.

2) Are there any thoughts on implementing a software scheduler for
linux-generic which supports large number of queues?

3) What will be the arbitration method (round robin, strict priority etc)
among the queues in the queue group? If it is weighted round robin or
strict priority, how are the priorities assigned to these queues?

4) Is the intent, to propagate the flow ID from classification down the
packet processing pipeline?

Thank you,
Honnappa


On 17 November 2016 at 13:59, Bill Fischofer 
wrote:

> On Thu, Nov 17, 2016 at 3:05 AM, Bala Manoharan  >
> wrote:
>
> > Regards,
> > Bala
> >
> >
> > On 15 November 2016 at 22:43, Brian Brooks 
> > wrote:
> > > On Mon, Nov 14, 2016 at 2:12 AM, Bala Manoharan
> > >  wrote:
> > >> Regards,
> > >> Bala
> > >>
> > >>
> > >> On 11 November 2016 at 13:26, Brian Brooks 
> > wrote:
> > >>> On 11/10 15:17:15, Bala Manoharan wrote:
> >  On 10 November 2016 at 13:26, Brian Brooks  >
> > wrote:
> >  > On 11/07 16:46:12, Bala Manoharan wrote:
> >  >> Hi,
> >  >
> >  > Hiya
> >  >
> >  >> This mail thread discusses the design of classification queue
> group
> >  >> RFC. The same can be found in the google doc whose link is given
> >  >> below.
> >  >> Users can provide their comments either in this mail thread or in
> > the
> >  >> google doc as per their convenience.
> >  >>
> >  >> https://docs.google.com/document/d/
> 1fOoG9WDR0lMpVjgMAsx8QsMr0YFK9
> > slR93LZ8VXqM2o/edit?usp=sharing
> >  >>
> >  >> The basic issues with queues as being a single target for a CoS
> > are two fold:
> >  >>
> >  >> Queues must be created and deleted individually. This imposes a
> >  >> significant burden when queues are used to represent individual
> > flows
> >  >> since the application may need to process thousands (or millions)
> > of
> >  >> flows.
> >  >
> >  > Wondering why there is an issue with creating and deleting queues
> > individually
> >  > if queue objects represent millions of flows..
> > 
> >  The queue groups are mainly required for hashing the incoming
> packets
> >  to multiple flows based on the hash configuration.
> >  So from application point of view it just needs a queue to have
> >  packets belonging to same flow and that packets belonging to
> different
> >  flows are placed in different queues respectively.It does not matter
> >  who creates the flow/queue.
> > >>>
> > >>> When the application receives an event from odp_schedule() call, how
> > does it
> > >>> know whether the odp_queue_t was previously created by the
> application
> > from
> > >>> odp_queue_create() or whether it was created by the implementation?
> > >>
> > >> odp_schedule() call returns the queue from which the event was part
> of.
> > >> The type of the queue can be got from odp_queue_type() API.
> > >> But the question is there an use-case where the application need to
> > know?
> > >> The application has the information of the queue it has created and
> > >> the queues created by implementation are destroyed by implementation.
> > >
> > > If certain fields of the packet are hashed to a queue handle, and this
> > queue
> > > handle has not previously been created via odp_queue_create(), there
> > might
> > > be a use case where the application needs to be aware of a new "flow"..
> > > Maybe the application ages flows.
> >
>
> I'm not sure I understand the concern being raised here. Packet fields are
> matched against PMRs to get a matching CoS. That CoS, in turn, is
> associated with with a queue or a queue group. If the latter then specified
> subfields within the packet are hashed to generate an index into that queue
> group to select the individual queue within the target queue group that is
> to receive the packet. Whether these queues have been preallocated at
> odp_queue_group_create() time, or allocated dynamically on first reference
> is up to the implementation, however especially in the case of "large"
> queue groups it can be expected that the number of actual queues in use
> will be sparse so a deferred allocation strategy will most likely be used.
>
> Applications are aware of flows because that's what an individual queue
> coming out of the classifier represents. An interesting question arises is
> if a higher-level protocol (e.g., 

Re: [lng-odp] [PATCH] changelog: summary of changes for odp v1.10.2.0

[Bill Fischofer]

OK, sent v2 with level changed to v1.11.1.0

On Thu, Nov 17, 2016 at 3:17 PM, Mike Holmes  wrote:

> On 16 November 2016 at 22:06, Bill Fischofer 
> wrote:
> > Signed-off-by: Bill Fischofer 
> > ---
> >  CHANGELOG | 211 ++
> 
> >  1 file changed, 211 insertions(+)
> >
> > diff --git a/CHANGELOG b/CHANGELOG
> > index d8230cd..3ab7354 100644
> > --- a/CHANGELOG
> > +++ b/CHANGELOG
> > @@ -1,3 +1,214 @@
> > +== OpenDataPlane (1.10.2.0)
>
> https://git.linaro.org/lng/odp.git
>
> latest tag is v1.11.0.0_monarch  should the mainline be 1.11.1.0
> since mainline is monarch + latest stuff
>
> > +
> > +=== New Features
> > +
> > + APIs
> > +ODP v1.10.2.0 is a minor API revision level and introduces a small
> change to
> > +the Traffic Manager API as well as some documentation clarification
> surrounding
> > +other APIs.
> > +
> > + Traffic Manager Egress Function
> > +The `odp_tm_egress_t` struct has been changed to add an explicit Boolean
> > +(`egress_fcn_supported`) that indicates whether the TM system supports
> > +a user-provided egress function. When specified this function is called
> to
> > +"output" a packet rather than having TM transmit it directly to a PktIO
> > +interface.
> > +
> > + Traffic Manager Coupling Change
> > +The `odp_tm_egress_t` struct has been changed to associate a TM system
> with an
> > +output `odp_pktio_t` rather than the previous `odp_pktout_queue_t`.
> This makes
> > +an explicit 1-to-1 map between a TM system and a PktIO interface.
> > +
> > + Default huge page size clarification
> > +The documentation for the `odp_sys_huge_page_size()` API has been
> reworded to
> > +clarify that this API refers to default huge page size.
> > +
> > + Strict Priority (SP) Scheduler
> > +Building on the modular scheduler framework added in v1.10.1.0, an
> alternate
> > +Strict Priority (SP) Scheduler is now available for latency-sensitive
> > +workloads. Applications wishing to use the SP scheduler should specify
> > +the `./configure` option `--enable-schedule-sp`. This scheduler
> emphasizes low
> > +latency processing of high priority events at the expense of
> throughput. This
> > +alternate scheduler is considered experimental and should not be used
> for
> > +production at this time.
> > +
> > + Application Binary Interface (ABI) Support
> > +Support is added to enable ODP applications to be binary compatible
> across
> > +different implementations of ODP sharing the same Instruction Set
> Architecture
> > +(ISA). This support introduces a new `configure` option:
> > +
> > +`--enable-abi-compat=yes`::
> > +This is the default and specifies that the ODP library is to be built to
> > +support ABI compatibility mode. In this mode ODP APIs are never
> inlined. ABI
> > +compatibility ensures maximum application portability in cloud
> environments.
> > +
> > +`--enable-abi-compat=no`::
> > +Specify this option to enable the inlining of ODP APIs. This may result
> in
> > +improved performance at the cost of ABI compatibility and is suitable
> for
> > +applications running in embedded environments.
> > +
> > +Note that ODP applications retain source code portability between ODP
> > +implementations regardless of the ABI mode chosen. To move to a
> different ODP
> > +application running on a different ISA, code need simply be recompiled
> against
> > +that target ODP implementation.
> > +
> > + SCTP Parsing Support
> > +The ODP classifier adds support for recognizing Stream Control
> Transmission
> > +Protocol (SCTP) packets. The APIs for this were previously not
> implemented.
> > +
> > +=== Packaging and Implementation Refinements
> > +
> > + Remove dependency on Linux headers
> > +ODP no longer has a dependency on Linux headers. This will help make the
> > +odp-linux reference implementation more easily portable to non-Linux
> > +environments.
> > +
> > + Remove dependency on helpers
> > +The odp-linux implementation has been made independent of the helper
> library
> > +to avoid circular dependency issues with packaging. Helper functions
> may use
> > +ODP APIs, however ODP implementations should not use helper functions.
> > +
> > + Reorganization of `test` directory
> > +The `test` directory has been reorganized to better support a unified
> approach
> > +to ODP component testing. API tests now live in
> > +`test/common_plat/validation/api` instead of the former
> > +`test/validation`. With this change performance and validation tests,
> as well
> > +as common and platform-specific tests can all be part of a unified test
> > +hierarchy.
> > +
> > +The resulting test tree now looks like:
> > +
> > +.New `test` directory hierarchy
> > +-
> > +test
> > +├── common_plat
> > +│   ├── common
> > +│   ├── m4
> > +│   ├── miscellaneous
> > +│   ├── performance
> > +│   └── validation
> > +│   └── api
> > +│