Re: [lng-odp] questions about buffer allocation in linux-generic

[Bill Fischofer]

On Fri, Jul 7, 2017 at 4:55 PM, Brian Brooks  wrote:
> Why is a buffer's "user area" not adjacent (virtually) to the buffer
> header itself?
> If by design, then a simpler way is to let the user manage that memory
> and pass a pointer that gets associated with the buffer (context,
> usr_ptr, cookie, ...).

ODP does not mandate where any metadata is with respect to the rest of
a packet since that would impose undesirable restrictions on
implementation flexibility, especially for platforms that make use of
HW accelerators. In such implementations the main packet data may be
managed by HW while the metadata is handled by SW, so questions of
adjacency are left up to the implementation.

In odp-linux, since this area is optional and variable sized, there's
no way to make it adjacent and still have compile-time access to
various internal packet fields that can be held at fixed offsets.

>
> Why are the sizes here round up to nearest cache line?
> Alignment bytes are allocated, which is enough to place the object at
> the requested alignment.

Keeping this area in its own cache line(s) means that this optional
area doesn't impinge on the cache footprint consumed by the rest of
the header. This can be fine-tuned if measurements show that this
decision is sub-optimal.

>
> sizeof(odp_packet_hdr_t) bytes are allocated per buffer regardless of
> the pool type.

Currently we don't support optional metadata that's not in the main
odp_packet_hdr_t. As we refine the odp_mbuf definition we may want to
split out optional metadata areas (e.g., in support of event sub-types
like ODP_PACKET_IPSEC, etc.). That's a tuning detail.

>
> odp_pool_t opaque type in linux-generic can be used as a pointer to
> pool_t instead of the u32 pool id.  This can save a level of
> indirection by avoiding looking up the pool in the array of pools.
> Opaque pointers are sizeof(void *), so it is not saving any space to
> store a u32 there.

This is one of the changes Petri is introducing in his latest buffer
restructure patch. Again, these are tuning optimizations since the API
remains opaque.

[lng-odp] questions about buffer allocation in linux-generic

[Brian Brooks]

Why is a buffer's "user area" not adjacent (virtually) to the buffer
header itself?
If by design, then a simpler way is to let the user manage that memory
and pass a pointer that gets associated with the buffer (context,
usr_ptr, cookie, ...).

Why are the sizes here round up to nearest cache line?
Alignment bytes are allocated, which is enough to place the object at
the requested alignment.

sizeof(odp_packet_hdr_t) bytes are allocated per buffer regardless of
the pool type.

odp_pool_t opaque type in linux-generic can be used as a pointer to
pool_t instead of the u32 pool id.  This can save a level of
indirection by avoiding looking up the pool in the array of pools.
Opaque pointers are sizeof(void *), so it is not saving any space to
store a u32 there.

Re: [lng-odp] [API-NEXT PATCH v2 0/6] Queue interface clean up

[Bill Fischofer]

For this series:

Reviewed-and-tested-by: Bill Fischofer 

On Tue, Jul 4, 2017 at 4:53 AM, Petri Savolainen
 wrote:
> APPLIES ON TOP OF: [API-NEXT PATCH 0/4] Clean up scheduler interface
>
> Clean up queue interface definitions and usage. Remove unneccesary 
> conversions,
> functions and file. Don't use qentry as a variable name when type is queue_t.
> Don't let TM to change queue type, since current API specifies only plain and
> sched queues as queue types.
>
> v2:
>   * Rebased on top of latest api-next and "Clean up scheduler interface"
> patch set
>
> Petri Savolainen (6):
>   linux-gen: cls: avoid queue conversions
>   linux-gen: pktio: dont use qentry for queue_t
>   linux-gen: pktio: convert queue handle only once
>   linux-gen: pktio: remove unused header file
>   linux-gen: queue: combine queue interface set functions
>   linux-gen: queue: remove type set from interface
>
>  platform/linux-generic/Makefile.am |   1 -
>  .../linux-generic/include/odp_packet_internal.h|   3 +-
>  .../linux-generic/include/odp_packet_io_internal.h |   2 +
>  .../linux-generic/include/odp_packet_io_queue.h|  48 
>  platform/linux-generic/include/odp_queue_if.h  |  20 ++-
>  platform/linux-generic/odp_classification.c|  10 +-
>  platform/linux-generic/odp_packet_io.c | 137 
> +++--
>  platform/linux-generic/odp_queue.c |  39 +++---
>  platform/linux-generic/odp_queue_scalable.c|  39 +++---
>  platform/linux-generic/odp_traffic_mngr.c  |  10 +-
>  10 files changed, 118 insertions(+), 191 deletions(-)
>  delete mode 100644 platform/linux-generic/include/odp_packet_io_queue.h
>
> --
> 2.13.0
>

Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove schedule interface depedency to qentry

[Honnappa Nagarahalli]

On 7 July 2017 at 01:46, Savolainen, Petri (Nokia - FI/Espoo)
 wrote:
>> >>  typedef struct schedule_fn_t {
>> >> +   int status_sync;
>> >
>> > this structure should contain functions that are provided by scheduler
>> > to other components of ODP. 'status_sync' seems to be an internal
>> > mechanism between the default scheduler and default queue. Hence it
>> > should not be here.
>> >
>> Any update on this comment?
>
> I did answer it already.

Ok, found your answer. Should this variable be moved to queue internal
structure which is set only for iQuery scheduler?

This structure should contain only the functions exposed by the
scheduler to other components of ODP. It should not contain anything
related to the interface between queue and scheduler (they are being
considered as a single module).

Re: [lng-odp] [PATCH 7/7] linux-gen: dpdk: enable zero-copy operation

[Maxim Uvarov]

On 07/07/17 16:46, Elo, Matias (Nokia - FI/Espoo) wrote:
> 
>> On 7 Jul 2017, at 0:21, Maxim Uvarov  wrote:
>>
>> On 07/03/17 15:01, Matias Elo wrote:
>>> +zero_copy=0
>>> +AC_ARG_ENABLE([dpdk-zero-copy],
>>> +[  --enable-dpdk-zero-copy  enable experimental zero-copy DPDK pktio 
>>> mode],
>>> +[if test x$enableval = xyes; then
>>> +zero_copy=1
>>> +fi])
>>> +
>>
>> please add corresponding check to his to .travis.yaml
>>
>> Maxim.
> 
> This seems to require some major changes to the travis configuration file. 
> The zero-copy dpdk pktio is enabled by adding '--enable-dpdk-zero-copy' 
> configure flag. So, to test both modes ODP would have to be configured, 
> built, and tested twice.
> 
> I've pretty much zero experience working with Travis, but to me it looks like 
> the minimum change would be to repeat the lines 118-122:
> 
> - ./bootstrap
> - ./configure --prefix=$HOME/odp-install  --enable-test-cpp 
> --enable-test-vald --enable-test-helper --enable-test-perf 
> --enable-user-guides --enable-test-perf-proc --enable-test-example 
> --with-dpdk-path=`pwd`/dpdk/${TARGET} --with-netmap-path=`pwd`/netmap $CONF
> - make -j $(nproc)
> - sudo LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" make check
> - make install
> 
> Then do 'make clean' and repeat the process adding the 
> '--enable-dpdk-zero-copy' flag.
> 
> Then there is the code coverage test which is another problem. I've no way of 
> testing this script, so I don't feel that comfortable doing any major 
> changes. As you are much more experienced working with Travis would it be 
> possible for you to do the necessary modifications?  
> 
> -Matias
> 

For first question - yes. Lets merge Dmitris patch before.

For travis, please first see how result matrix looks like:

https://travis-ci.org/Linaro/odp/builds/251051276?utm_source=github_status_medium=notification


So you need separate:
- stage: test
  env: TEST=distcheck

entry.
Like clone entry "TEST=coverage" and put your options there.


If you only need to provide one option, then you need just add one line
here:

 env:
- CONF=""
- CONF="--disable-abi-compat"
- CONF="--enable-schedule-sp"
- CONF="--enable-schedule-iquery"
- CONF="--enable-schedule-scalable"
- CONF="--enable-dpdk-zero-copy"


and it will be tested with clang/gcc and what we have. You can test how
it works on your private github repo.


Maxim.

Re: [lng-odp] [PATCH 7/7] linux-gen: dpdk: enable zero-copy operation

[Elo, Matias (Nokia - FI/Espoo)]

> On 7 Jul 2017, at 0:21, Maxim Uvarov  wrote:
> 
> On 07/03/17 15:01, Matias Elo wrote:
>> +zero_copy=0
>> +AC_ARG_ENABLE([dpdk-zero-copy],
>> +[  --enable-dpdk-zero-copy  enable experimental zero-copy DPDK pktio 
>> mode],
>> +[if test x$enableval = xyes; then
>> +zero_copy=1
>> +fi])
>> +
> 
> please add corresponding check to his to .travis.yaml
> 
> Maxim.

This seems to require some major changes to the travis configuration file. The 
zero-copy dpdk pktio is enabled by adding '--enable-dpdk-zero-copy' configure 
flag. So, to test both modes ODP would have to be configured, built, and tested 
twice.

I've pretty much zero experience working with Travis, but to me it looks like 
the minimum change would be to repeat the lines 118-122:

- ./bootstrap
- ./configure --prefix=$HOME/odp-install  --enable-test-cpp 
--enable-test-vald --enable-test-helper --enable-test-perf --enable-user-guides 
--enable-test-perf-proc --enable-test-example 
--with-dpdk-path=`pwd`/dpdk/${TARGET} --with-netmap-path=`pwd`/netmap $CONF
- make -j $(nproc)
- sudo LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" make check
- make install

Then do 'make clean' and repeat the process adding the 
'--enable-dpdk-zero-copy' flag.

Then there is the code coverage test which is another problem. I've no way of 
testing this script, so I don't feel that comfortable doing any major changes. 
As you are much more experienced working with Travis would it be possible for 
you to do the necessary modifications?  

-Matias

[lng-odp] [PATCH 1/3] linux-gen: buffer: change buffer handle to pointer

[Petri Savolainen]

Changed buffer, event and timeout handles to be pointers.
Packet handles are already pointers. This enabled code
optimization as some conversions may be removed and remaining
once are just type casts.

Signed-off-by: Petri Savolainen 
---
 include/odp/arch/default/api/abi/buffer.h  |   7 +-
 include/odp/arch/default/api/abi/event.h   |   2 +-
 .../include/odp/api/plat/buffer_types.h|   6 +-
 .../include/odp/api/plat/event_types.h |   2 +-
 .../include/odp/api/plat/timer_types.h |   2 +-
 .../linux-generic/include/odp_buffer_inlines.h |   9 +-
 .../linux-generic/include/odp_buffer_internal.h|  23 ++--
 .../linux-generic/include/odp_packet_internal.h|   5 +-
 platform/linux-generic/include/odp_pool_internal.h |  30 +-
 platform/linux-generic/odp_buffer.c|   4 +-
 platform/linux-generic/odp_packet.c|  56 +-
 platform/linux-generic/odp_pool.c  | 120 +++--
 platform/linux-generic/odp_queue.c |   6 +-
 platform/linux-generic/odp_traffic_mngr.c  |   2 +-
 platform/linux-generic/pktio/ipc.c |  11 +-
 15 files changed, 126 insertions(+), 159 deletions(-)

diff --git a/include/odp/arch/default/api/abi/buffer.h 
b/include/odp/arch/default/api/abi/buffer.h
index eec6f01f..d8bfc913 100644
--- a/include/odp/arch/default/api/abi/buffer.h
+++ b/include/odp/arch/default/api/abi/buffer.h
@@ -14,18 +14,13 @@ extern "C" {
 /** @internal Dummy type for strong typing */
 typedef struct { char dummy; /**< @internal Dummy */ } _odp_abi_buffer_t;
 
-/** @internal Dummy type for strong typing */
-typedef struct { char dummy; /**< @internal Dummy */ } _odp_abi_buffer_seg_t;
-
 /** @ingroup odp_buffer
  *  @{
  */
 
 typedef _odp_abi_buffer_t *odp_buffer_t;
-typedef _odp_abi_buffer_seg_t *odp_buffer_seg_t;
 
-#define ODP_BUFFER_INVALID   ((odp_buffer_t)0x)
-#define ODP_SEGMENT_INVALID  ((odp_buffer_seg_t)0x)
+#define ODP_BUFFER_INVALID   ((odp_buffer_t)NULL)
 
 /**
  * @}
diff --git a/include/odp/arch/default/api/abi/event.h 
b/include/odp/arch/default/api/abi/event.h
index 4f6596b1..fd86f25c 100644
--- a/include/odp/arch/default/api/abi/event.h
+++ b/include/odp/arch/default/api/abi/event.h
@@ -22,7 +22,7 @@ typedef struct { char dummy; /**< @internal Dummy */ } 
_odp_abi_event_t;
 
 typedef _odp_abi_event_t *odp_event_t;
 
-#define ODP_EVENT_INVALID  ((odp_event_t)0x)
+#define ODP_EVENT_INVALID  ((odp_event_t)NULL)
 
 typedef enum odp_event_type_t {
ODP_EVENT_BUFFER   = 1,
diff --git a/platform/linux-generic/include/odp/api/plat/buffer_types.h 
b/platform/linux-generic/include/odp/api/plat/buffer_types.h
index 809768f3..8b79bb52 100644
--- a/platform/linux-generic/include/odp/api/plat/buffer_types.h
+++ b/platform/linux-generic/include/odp/api/plat/buffer_types.h
@@ -31,11 +31,7 @@ extern "C" {
 
 typedef ODP_HANDLE_T(odp_buffer_t);
 
-#define ODP_BUFFER_INVALID _odp_cast_scalar(odp_buffer_t, 0x)
-
-typedef ODP_HANDLE_T(odp_buffer_seg_t);
-
-#define ODP_SEGMENT_INVALID ((odp_buffer_seg_t)ODP_BUFFER_INVALID)
+#define ODP_BUFFER_INVALID _odp_cast_scalar(odp_buffer_t, NULL)
 
 /**
  * @}
diff --git a/platform/linux-generic/include/odp/api/plat/event_types.h 
b/platform/linux-generic/include/odp/api/plat/event_types.h
index a1aa0e45..559a2fa0 100644
--- a/platform/linux-generic/include/odp/api/plat/event_types.h
+++ b/platform/linux-generic/include/odp/api/plat/event_types.h
@@ -32,7 +32,7 @@ extern "C" {
 
 typedef ODP_HANDLE_T(odp_event_t);
 
-#define ODP_EVENT_INVALID _odp_cast_scalar(odp_event_t, 0x)
+#define ODP_EVENT_INVALID _odp_cast_scalar(odp_event_t, NULL)
 
 typedef enum odp_event_type_t {
ODP_EVENT_BUFFER   = 1,
diff --git a/platform/linux-generic/include/odp/api/plat/timer_types.h 
b/platform/linux-generic/include/odp/api/plat/timer_types.h
index 8821bed6..a8891f11 100644
--- a/platform/linux-generic/include/odp/api/plat/timer_types.h
+++ b/platform/linux-generic/include/odp/api/plat/timer_types.h
@@ -38,7 +38,7 @@ typedef ODP_HANDLE_T(odp_timer_t);
 
 typedef ODP_HANDLE_T(odp_timeout_t);
 
-#define ODP_TIMEOUT_INVALID  _odp_cast_scalar(odp_timeout_t, 0x)
+#define ODP_TIMEOUT_INVALID  _odp_cast_scalar(odp_timeout_t, NULL)
 
 /**
  * @}
diff --git a/platform/linux-generic/include/odp_buffer_inlines.h 
b/platform/linux-generic/include/odp_buffer_inlines.h
index cf817d90..1dbc725b 100644
--- a/platform/linux-generic/include/odp_buffer_inlines.h
+++ b/platform/linux-generic/include/odp_buffer_inlines.h
@@ -23,9 +23,14 @@ odp_event_type_t _odp_buffer_event_type(odp_buffer_t buf);
 void _odp_buffer_event_type_set(odp_buffer_t buf, int ev);
 int odp_buffer_snprint(char *str, uint32_t n, odp_buffer_t buf);
 
-static inline odp_buffer_t odp_hdr_to_buf(odp_buffer_hdr_t *hdr)
+static inline odp_buffer_t buf_from_buf_hdr(odp_buffer_hdr_t *hdr)
 {
-   

[lng-odp] [PATCH 3/3] linux-gen: pool: avoid extra conversions

[Petri Savolainen]

Avoid conversions between buffer/packet handles and
buffer/packet header pointers.

Signed-off-by: Petri Savolainen 
---
 platform/linux-generic/include/odp_pool_internal.h |  5 +--
 platform/linux-generic/odp_packet.c| 49 +++---
 platform/linux-generic/odp_pool.c  | 49 +++---
 3 files changed, 42 insertions(+), 61 deletions(-)

diff --git a/platform/linux-generic/include/odp_pool_internal.h 
b/platform/linux-generic/include/odp_pool_internal.h
index 1fb55d7d..a4c0e532 100644
--- a/platform/linux-generic/include/odp_pool_internal.h
+++ b/platform/linux-generic/include/odp_pool_internal.h
@@ -96,9 +96,8 @@ static inline odp_buffer_hdr_t *buf_hdl_to_hdr(odp_buffer_t 
buf)
return (odp_buffer_hdr_t *)(uintptr_t)buf;
 }
 
-int buffer_alloc_multi(pool_t *pool, odp_buffer_t buf[],
-  odp_buffer_hdr_t *buf_hdr[], int num);
-void buffer_free_multi(const odp_buffer_t buf[], int num_free);
+int buffer_alloc_multi(pool_t *pool, odp_buffer_hdr_t *buf_hdr[], int num);
+void buffer_free_multi(odp_buffer_hdr_t *buf_hdr[], int num_free);
 
 #ifdef __cplusplus
 }
diff --git a/platform/linux-generic/odp_packet.c 
b/platform/linux-generic/odp_packet.c
index e6de2558..3cde2086 100644
--- a/platform/linux-generic/odp_packet.c
+++ b/platform/linux-generic/odp_packet.c
@@ -361,14 +361,14 @@ static inline void copy_num_segs(odp_packet_hdr_t *to, 
odp_packet_hdr_t *from,
 
 static inline odp_packet_hdr_t *alloc_segments(pool_t *pool, int num)
 {
-   odp_buffer_t buf[num];
odp_packet_hdr_t *pkt_hdr[num];
int ret;
 
-   ret = buffer_alloc_multi(pool, buf, (odp_buffer_hdr_t **)pkt_hdr, num);
+   ret = buffer_alloc_multi(pool, (odp_buffer_hdr_t **)pkt_hdr, num);
+
if (odp_unlikely(ret != num)) {
if (ret > 0)
-   buffer_free_multi(buf, ret);
+   buffer_free_multi((odp_buffer_hdr_t **)pkt_hdr, ret);
 
return NULL;
}
@@ -427,12 +427,12 @@ static inline odp_packet_hdr_t 
*add_segments(odp_packet_hdr_t *pkt_hdr,
 static inline void free_bufs(odp_packet_hdr_t *pkt_hdr, int first, int num)
 {
int i;
-   odp_buffer_t buf[num];
+   odp_buffer_hdr_t *buf_hdr[num];
 
for (i = 0; i < num; i++)
-   buf[i] = buffer_handle(pkt_hdr->buf_hdr.seg[first + i].hdr);
+   buf_hdr[i] = pkt_hdr->buf_hdr.seg[first + i].hdr;
 
-   buffer_free_multi(buf, num);
+   buffer_free_multi(buf_hdr, num);
 }
 
 static inline odp_packet_hdr_t *free_segments(odp_packet_hdr_t *pkt_hdr,
@@ -444,10 +444,10 @@ static inline odp_packet_hdr_t 
*free_segments(odp_packet_hdr_t *pkt_hdr,
if (head) {
odp_packet_hdr_t *new_hdr;
int i;
-   odp_buffer_t buf[num];
+   odp_buffer_hdr_t *buf_hdr[num];
 
for (i = 0; i < num; i++)
-   buf[i] = buffer_handle(pkt_hdr->buf_hdr.seg[i].hdr);
+   buf_hdr[i] = pkt_hdr->buf_hdr.seg[i].hdr;
 
/* First remaining segment is the new packet descriptor */
new_hdr = pkt_hdr->buf_hdr.seg[num].hdr;
@@ -464,7 +464,7 @@ static inline odp_packet_hdr_t 
*free_segments(odp_packet_hdr_t *pkt_hdr,
 
pkt_hdr = new_hdr;
 
-   buffer_free_multi(buf, num);
+   buffer_free_multi(buf_hdr, num);
} else {
/* Free last 'num' bufs */
free_bufs(pkt_hdr, num_remain, num);
@@ -487,10 +487,9 @@ static inline int packet_alloc(pool_t *pool, uint32_t len, 
int max_pkt,
int num_buf, i;
int num = max_pkt;
int max_buf = max_pkt * num_seg;
-   odp_buffer_t buf[max_buf];
odp_packet_hdr_t *pkt_hdr[max_buf];
 
-   num_buf = buffer_alloc_multi(pool, buf, (odp_buffer_hdr_t **)pkt_hdr,
+   num_buf = buffer_alloc_multi(pool, (odp_buffer_hdr_t **)pkt_hdr,
 max_buf);
 
/* Failed to allocate all segments */
@@ -500,8 +499,12 @@ static inline int packet_alloc(pool_t *pool, uint32_t len, 
int max_pkt,
num  = num_buf / num_seg;
num_free = num_buf - (num * num_seg);
 
-   if (num_free > 0)
-   buffer_free_multi([num_buf - num_free], num_free);
+   if (num_free > 0) {
+   odp_buffer_hdr_t **p;
+
+   p = (odp_buffer_hdr_t **)_hdr[num_buf - num_free];
+   buffer_free_multi(p, num_free);
+   }
 
if (num == 0)
return 0;
@@ -584,43 +587,39 @@ void odp_packet_free(odp_packet_t pkt)
int num_seg = pkt_hdr->buf_hdr.segcount;
 
if (odp_likely(CONFIG_PACKET_MAX_SEGS == 1 || num_seg == 1))
-   buffer_free_multi(, 1);
+   buffer_free_multi((odp_buffer_hdr_t **), 1);
else
 

[lng-odp] [PATCH 2/3] linux-gen: queue: avoid extra conversions

[Petri Savolainen]

Conversions are not needed as event handles are buffer header
pointers.

Signed-off-by: Petri Savolainen 
---
 .../linux-generic/include/odp_buffer_inlines.h |  5 --
 platform/linux-generic/odp_queue.c | 61 +-
 2 files changed, 14 insertions(+), 52 deletions(-)

diff --git a/platform/linux-generic/include/odp_buffer_inlines.h 
b/platform/linux-generic/include/odp_buffer_inlines.h
index 1dbc725b..a5658e81 100644
--- a/platform/linux-generic/include/odp_buffer_inlines.h
+++ b/platform/linux-generic/include/odp_buffer_inlines.h
@@ -28,11 +28,6 @@ static inline odp_buffer_t buf_from_buf_hdr(odp_buffer_hdr_t 
*hdr)
return (odp_buffer_t)hdr;
 }
 
-static inline odp_event_t event_from_buf_hdr(odp_buffer_hdr_t *hdr)
-{
-   return (odp_event_t)hdr;
-}
-
 #ifdef __cplusplus
 }
 #endif
diff --git a/platform/linux-generic/odp_queue.c 
b/platform/linux-generic/odp_queue.c
index 3f6e7365..e246d1a3 100644
--- a/platform/linux-generic/odp_queue.c
+++ b/platform/linux-generic/odp_queue.c
@@ -479,31 +479,23 @@ int queue_enq(queue_entry_t *queue, odp_buffer_hdr_t 
*buf_hdr)
 
 int odp_queue_enq_multi(odp_queue_t handle, const odp_event_t ev[], int num)
 {
-   odp_buffer_hdr_t *buf_hdr[QUEUE_MULTI_MAX];
-   queue_entry_t *queue;
-   int i;
+   queue_entry_t *queue = queue_to_qentry(handle);
+
+   if (odp_unlikely(num == 0))
+   return 0;
 
if (num > QUEUE_MULTI_MAX)
num = QUEUE_MULTI_MAX;
 
-   queue = queue_to_qentry(handle);
-
-   for (i = 0; i < num; i++)
-   buf_hdr[i] = buf_hdl_to_hdr(odp_buffer_from_event(ev[i]));
-
-   return num == 0 ? 0 : queue->s.enqueue_multi(queue, buf_hdr,
-num);
+   return queue->s.enqueue_multi(queue, (odp_buffer_hdr_t **)(uintptr_t)ev,
+ num);
 }
 
 int odp_queue_enq(odp_queue_t handle, odp_event_t ev)
 {
-   odp_buffer_hdr_t *buf_hdr;
-   queue_entry_t *queue;
-
-   queue   = queue_to_qentry(handle);
-   buf_hdr = buf_hdl_to_hdr(odp_buffer_from_event(ev));
+   queue_entry_t *queue = queue_to_qentry(handle);
 
-   return queue->s.enqueue(queue, buf_hdr);
+   return queue->s.enqueue(queue, (odp_buffer_hdr_t *)(uintptr_t)ev);
 }
 
 static inline int deq_multi(queue_entry_t *queue, odp_buffer_hdr_t *buf_hdr[],
@@ -596,38 +588,21 @@ odp_buffer_hdr_t *queue_deq(queue_entry_t *queue)
return NULL;
 }
 
-int odp_queue_deq_multi(odp_queue_t handle, odp_event_t events[], int num)
+int odp_queue_deq_multi(odp_queue_t handle, odp_event_t ev[], int num)
 {
-   queue_entry_t *queue;
-   odp_buffer_hdr_t *buf_hdr[QUEUE_MULTI_MAX];
-   int i, ret;
+   queue_entry_t *queue = queue_to_qentry(handle);
 
if (num > QUEUE_MULTI_MAX)
num = QUEUE_MULTI_MAX;
 
-   queue = queue_to_qentry(handle);
-
-   ret = queue->s.dequeue_multi(queue, buf_hdr, num);
-
-   for (i = 0; i < ret; i++)
-   events[i] = event_from_buf_hdr(buf_hdr[i]);
-
-   return ret;
+   return queue->s.dequeue_multi(queue, (odp_buffer_hdr_t **)ev, num);
 }
 
-
 odp_event_t odp_queue_deq(odp_queue_t handle)
 {
-   queue_entry_t *queue;
-   odp_buffer_hdr_t *buf_hdr;
-
-   queue   = queue_to_qentry(handle);
-   buf_hdr = queue->s.dequeue(queue);
-
-   if (buf_hdr)
-   return event_from_buf_hdr(buf_hdr);
+   queue_entry_t *queue = queue_to_qentry(handle);
 
-   return ODP_EVENT_INVALID;
+   return (odp_event_t)queue->s.dequeue(queue);
 }
 
 void queue_lock(queue_entry_t *queue)
@@ -726,17 +701,9 @@ odp_queue_t sched_cb_queue_handle(uint32_t queue_index)
 
 int sched_cb_queue_deq_multi(uint32_t queue_index, odp_event_t ev[], int num)
 {
-   int i, ret;
queue_entry_t *qe = get_qentry(queue_index);
-   odp_buffer_hdr_t *buf_hdr[num];
 
-   ret = deq_multi(qe, buf_hdr, num);
-
-   if (ret > 0)
-   for (i = 0; i < ret; i++)
-   ev[i] = event_from_buf_hdr(buf_hdr[i]);
-
-   return ret;
+   return deq_multi(qe, (odp_buffer_hdr_t **)ev, num);
 }
 
 int sched_cb_queue_empty(uint32_t queue_index)
-- 
2.13.0

[lng-odp] [PATCH 0/3] Change buffer handle to pointer

[Petri Savolainen]

Change buffer (event, timeout) handle to odp_buffer_hdr_t pointer. Packet 
handle is already a pointer. This removes need for many conversions as handle 
arrays may used directly.

All 'make check' tests pass except pktio_ipc. This can be verified by disabling 
the IPC test.

test/linux-generic/Makefile.am
# TESTS += pktio_ipc/pktio_ipc_run.sh
# SUBDIRS += pktio_ipc

L2fwd packet rate (odp-linux / pktio:dpdk) improves about 3%.

Petri Savolainen (3):
  linux-gen: buffer: change buffer handle to pointer
  linux-gen: queue: avoid extra conversions
  linux-gen: pool: avoid extra conversions

 include/odp/arch/default/api/abi/buffer.h  |   7 +-
 include/odp/arch/default/api/abi/event.h   |   2 +-
 .../include/odp/api/plat/buffer_types.h|   6 +-
 .../include/odp/api/plat/event_types.h |   2 +-
 .../include/odp/api/plat/timer_types.h |   2 +-
 .../linux-generic/include/odp_buffer_inlines.h |   4 +-
 .../linux-generic/include/odp_buffer_internal.h|  23 ++--
 .../linux-generic/include/odp_packet_internal.h|   5 +-
 platform/linux-generic/include/odp_pool_internal.h |  35 +-
 platform/linux-generic/odp_buffer.c|   4 +-
 platform/linux-generic/odp_packet.c| 105 +-
 platform/linux-generic/odp_pool.c  | 123 ++---
 platform/linux-generic/odp_queue.c |  61 +++---
 platform/linux-generic/odp_traffic_mngr.c  |   2 +-
 platform/linux-generic/pktio/ipc.c |  11 +-
 15 files changed, 151 insertions(+), 241 deletions(-)

-- 
2.13.0

Re: [lng-odp] [PATCH 7/7] linux-gen: dpdk: enable zero-copy operation

[Elo, Matias (Nokia - FI/Espoo)]

> On 7 Jul 2017, at 0:21, Maxim Uvarov  wrote:
> 
> On 07/03/17 15:01, Matias Elo wrote:
>> +zero_copy=0
>> +AC_ARG_ENABLE([dpdk-zero-copy],
>> +[  --enable-dpdk-zero-copy  enable experimental zero-copy DPDK pktio 
>> mode],
>> +[if test x$enableval = xyes; then
>> +zero_copy=1
>> +fi])
>> +
> 
> please add corresponding check to his to .travis.yaml
> 
> Maxim.


OK, will do.

Btw. is Dmitry's "Rework the way ODP links with other libraries" patch set 
going to be merged soon? My patch set has a small conflict with it, so it's 
probably better for me to wait until that one is merged until I send V2.

-Matias

Re: [lng-odp] [PATCH 7/7] linux-gen: dpdk: enable zero-copy operation

[Krishna Garapati]

On 3 July 2017 at 14:01, Matias Elo  wrote:

> Implements experimental zero-copy mode for DPDK pktio. This can be enabled
> with additional '--enable-dpdk-zero-copy' configure flag.
>
> This feature has been put behind an extra configure flag as it doesn't
> entirely adhere to the DPDK API and may behave unexpectedly with untested
> DPDK NIC drivers. Zero-copy operation has been tested with pcap, ixgbe, and
> i40e drivers.
>
> Signed-off-by: Matias Elo 
> ---
>  .../linux-generic/include/odp_buffer_internal.h|   2 +-
>  .../linux-generic/include/odp_packet_internal.h|  13 +
>  platform/linux-generic/include/odp_pool_internal.h |   4 +
>  platform/linux-generic/m4/odp_dpdk.m4  |  14 +-
>  platform/linux-generic/odp_pool.c  |   2 +
>  platform/linux-generic/pktio/dpdk.c| 676
> -
>  6 files changed, 562 insertions(+), 149 deletions(-)
>
> diff --git a/platform/linux-generic/include/odp_buffer_internal.h
> b/platform/linux-generic/include/odp_buffer_internal.h
> index 076abe9..78ea527 100644
> --- a/platform/linux-generic/include/odp_buffer_internal.h
> +++ b/platform/linux-generic/include/odp_buffer_internal.h
> @@ -109,7 +109,7 @@ struct odp_buffer_hdr_t {
>
> /* Data or next header */
> uint8_t data[0];
> -};
> +} ODP_ALIGNED_CACHE;
>
>  ODP_STATIC_ASSERT(CONFIG_PACKET_MAX_SEGS < 256,
>   "CONFIG_PACKET_MAX_SEGS_TOO_LARGE");
> diff --git a/platform/linux-generic/include/odp_packet_internal.h
> b/platform/linux-generic/include/odp_packet_internal.h
> index 11f2fdc..78569b6 100644
> --- a/platform/linux-generic/include/odp_packet_internal.h
> +++ b/platform/linux-generic/include/odp_packet_internal.h
> @@ -92,6 +92,12 @@ typedef struct {
> uint32_t l4_offset; /**< offset to L4 hdr (TCP, UDP, SCTP, also
> ICMP) */
>  } packet_parser_t;
>
> +/* Packet extra data length */
> +#define PKT_EXTRA_LEN 128
> +
> +/* Packet extra data types */
> +#define PKT_EXTRA_TYPE_DPDK 1
> +
>  /**
>   * Internal Packet header
>   *
> @@ -131,6 +137,13 @@ typedef struct {
> /* Result for crypto */
> odp_crypto_generic_op_result_t op_result;
>
> +#ifdef ODP_PKTIO_DPDK
> +   /* Type of extra data */
> +   uint8_t extra_type;
> +   /* Extra space for packet descriptors. E.g. DPDK mbuf  */
> +   uint8_t extra[PKT_EXTRA_LEN] ODP_ALIGNED_CACHE;
> +#endif
> +
> /* Packet data storage */
> uint8_t data[0];
>  } odp_packet_hdr_t;
> diff --git a/platform/linux-generic/include/odp_pool_internal.h
> b/platform/linux-generic/include/odp_pool_internal.h
> index ebb779d..acea079 100644
> --- a/platform/linux-generic/include/odp_pool_internal.h
> +++ b/platform/linux-generic/include/odp_pool_internal.h
> @@ -68,6 +68,10 @@ typedef struct pool_t {
> uint8_t *base_addr;
> uint8_t *uarea_base_addr;
>
> +   /* Used by DPDK zero-copy pktio */
> +   void*ext_desc;
> +   uint16_t ext_ref_count;
> +
> pool_cache_t local_cache[ODP_THREAD_COUNT_MAX];
>
> odp_shm_tring_shm;
> diff --git a/platform/linux-generic/m4/odp_dpdk.m4
> b/platform/linux-generic/m4/odp_dpdk.m4
> index 58d1472..edcc4c8 100644
> --- a/platform/linux-generic/m4/odp_dpdk.m4
> +++ b/platform/linux-generic/m4/odp_dpdk.m4
> @@ -9,6 +9,16 @@ AC_HELP_STRING([--with-dpdk-path=DIR   path to dpdk
> build directory]),
>  pktio_dpdk_support=yes],[])
>
>  
> ##
> +# Enable zero-copy DPDK pktio
> +###
> ###
> +zero_copy=0
> +AC_ARG_ENABLE([dpdk-zero-copy],
> +[  --enable-dpdk-zero-copy  enable experimental zero-copy DPDK pktio
> mode],
> +[if test x$enableval = xyes; then
> +zero_copy=1
> +fi])
> +
> +###
> ###
>  # Save and set temporary compilation flags
>  
> ##
>  OLD_CPPFLAGS=$CPPFLAGS
> @@ -38,9 +48,9 @@ then
>  done
>  DPDK_PMD+=--no-whole-archive
>
> -ODP_CFLAGS="$ODP_CFLAGS -DODP_PKTIO_DPDK"
> +ODP_CFLAGS="$ODP_CFLAGS -DODP_PKTIO_DPDK -DODP_DPDK_ZERO_COPY=$zero_
> copy"
>  AM_LDFLAGS="$AM_LDFLAGS -L$DPDK_PATH/lib -Wl,$DPDK_PMD"
> -LIBS="$LIBS -ldpdk -ldl -lpcap"
> +LIBS="$LIBS -ldpdk -ldl -lpcap -lm"
>  else
>  pktio_dpdk_support=no
>  fi
> diff --git a/platform/linux-generic/odp_pool.c
> b/platform/linux-generic/odp_pool.c
> index 5360b94..8a27c8a 100644
> --- a/platform/linux-generic/odp_pool.c
> +++ b/platform/linux-generic/odp_pool.c
> @@ -395,6 +395,8 @@ static odp_pool_t pool_create(const char *name,
> odp_pool_param_t *params,
> pool->uarea_size = uarea_size;
> pool->shm_size   = num * block_size;
> pool->uarea_shm_size = num * 

Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove schedule interface depedency to qentry

[Maxim Uvarov]

Hello Honnappa, it looks like there is no blocker issue to merge these
patches. Improvements can be done later and clean up can go in now. Do you
agree?

Maxim.

On 7 July 2017 at 09:46, Savolainen, Petri (Nokia - FI/Espoo) <
petri.savolai...@nokia.com> wrote:

> > >>  typedef struct schedule_fn_t {
> > >> +   int status_sync;
> > >
> > > this structure should contain functions that are provided by scheduler
> > > to other components of ODP. 'status_sync' seems to be an internal
> > > mechanism between the default scheduler and default queue. Hence it
> > > should not be here.
> > >
> > Any update on this comment?
>
> I did answer it already.
>

[lng-odp] [Linaro/odp] 5a2ed3: test:queue:update order queue test for locks.

[GitHub]

  Branch: refs/heads/master
  Home:   https://github.com/Linaro/odp
  Commit: 5a2ed3eefb1a263139282c376741adb2c355bd5a
  
https://github.com/Linaro/odp/commit/5a2ed3eefb1a263139282c376741adb2c355bd5a
  Author: Nikhil Agarwal 
  Date:   2017-07-07 (Fri, 07 Jul 2017)

  Changed paths:
M test/common_plat/validation/api/queue/queue.c

  Log Message:
  ---
  test:queue:update order queue test for locks.

Order queue test should reserve ordered locks only if
ordered locks count is more than 1.

Signed-off-by: Nikhil Agarwal 
Reviewed-by: Bill Fischofer 
Signed-off-by: Maxim Uvarov 

[lng-odp] [Linaro/odp] 8fea9f: api: system_info: add function for fetching all su...

[GitHub]

  Branch: refs/heads/api-next
  Home:   https://github.com/Linaro/odp
  Commit: 8fea9ff7d8fde984981e681f49133dbda8429b7f
  
https://github.com/Linaro/odp/commit/8fea9ff7d8fde984981e681f49133dbda8429b7f
  Author: Matias Elo 
  Date:   2017-07-07 (Fri, 07 Jul 2017)

  Changed paths:
M include/odp/api/spec/system_info.h

  Log Message:
  ---
  api: system_info: add function for fetching all supported huge page sizes

A system may simultaneously support multiple huge page sizes. Add a new API
function odp_sys_huge_page_size_all() which returns all supported page
sizes. odp_sys_huge_page_size() stays unmodified to maintain backward
compatibility.

Signed-off-by: Matias Elo 
Reviewed-and-tested-by: Bill Fischofer 
Signed-off-by: Maxim Uvarov 


  Commit: eb564bfb6f813ec3f2fe8a8d4ce0da25220b4215
  
https://github.com/Linaro/odp/commit/eb564bfb6f813ec3f2fe8a8d4ce0da25220b4215
  Author: Matias Elo 
  Date:   2017-07-07 (Fri, 07 Jul 2017)

  Changed paths:
M platform/linux-generic/odp_system_info.c

  Log Message:
  ---
  linux-gen: system_info: implement odp_sys_huge_page_size_all()

Directory /sys/kernel/mm/hugepages contains subdirectories for all huge
page sizes supported by the running kernel. Loop through the contents of
this directory to find the supported huge page sizes.

Signed-off-by: Matias Elo 
Reviewed-and-tested-by: Bill Fischofer 
Signed-off-by: Maxim Uvarov 


  Commit: b668182d6ea0cb942c2cf43771c618c9457bf146
  
https://github.com/Linaro/odp/commit/b668182d6ea0cb942c2cf43771c618c9457bf146
  Author: Matias Elo 
  Date:   2017-07-07 (Fri, 07 Jul 2017)

  Changed paths:
M test/common_plat/validation/api/system/system.c
M test/common_plat/validation/api/system/system.h

  Log Message:
  ---
  validation: system_info: add test for odp_sys_huge_page_size_all()

Signed-off-by: Matias Elo 
Reviewed-and-tested-by: Bill Fischofer 
Signed-off-by: Maxim Uvarov 


Compare: https://github.com/Linaro/odp/compare/15c97427d01c...b668182d6ea0

Re: [lng-odp] [API-NEXT PATCH 0/4] Clean up scheduler interface

[Savolainen, Petri (Nokia - FI/Espoo)]

This is the first step to clean up queue / scheduler dependencies. Could we 
merge this now and continue towards the next steps. Master and api-next should 
be synced soon otherwise the code base delta just increases for no real reason 
(API is the same).

-Petri

> -Original Message-
> From: Bill Fischofer [mailto:bill.fischo...@linaro.org]
> Sent: Friday, June 30, 2017 6:51 PM
> To: Petri Savolainen 
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH 0/4] Clean up scheduler interface
> 
> For this series:
> 
> Reviewed-and-tested-by: Bill Fischofer 
> 
> On Fri, Jun 30, 2017 at 9:10 AM, Petri Savolainen
>  wrote:
> > First removed dependency to queue internals from scheduler interface.
> Then
> > removed almost all references to queue internals from default and iquery
> > schedulers. This required move of ordered queue data structure from
> queue to
> > scheduler files. The last references require queue / pktio interface
> change and
> > thus is left as the next step. Scheduler performance increased couple of
> > percents.
> >
> >
> > Petri Savolainen (4):
> >   linux-gen: sched: remove schedule interface depedency to qentry
> >   linux-gen: sched: use config max ordered locks
> >   linux-gen: sched: remove most dependecies to qentry
> >   linux-gen: sched: remove unused sched interface functions
> >
> >  .../linux-generic/include/odp_config_internal.h|   2 +-
> >  .../linux-generic/include/odp_queue_internal.h |   7 -
> >  platform/linux-generic/include/odp_schedule_if.h   |  13 +-
> >  platform/linux-generic/odp_queue.c |  68 +
> >  platform/linux-generic/odp_schedule.c  | 161 +-
> ---
> >  platform/linux-generic/odp_schedule_iquery.c   | 158 --
> --
> >  platform/linux-generic/odp_schedule_sp.c   |  18 +--
> >  7 files changed, 211 insertions(+), 216 deletions(-)
> >
> > --
> > 2.13.0
> >

Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove schedule interface depedency to qentry

[Savolainen, Petri (Nokia - FI/Espoo)]

> >>  typedef struct schedule_fn_t {
> >> +   int status_sync;
> >
> > this structure should contain functions that are provided by scheduler
> > to other components of ODP. 'status_sync' seems to be an internal
> > mechanism between the default scheduler and default queue. Hence it
> > should not be here.
> >
> Any update on this comment?

I did answer it already.

Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove schedule interface depedency to qentry

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: Honnappa Nagarahalli [mailto:honnappa.nagaraha...@linaro.org]
> Sent: Thursday, July 06, 2017 11:07 PM
> To: Savolainen, Petri (Nokia - FI/Espoo) 
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove
> schedule interface depedency to qentry
> 
> On 5 July 2017 at 01:31, Savolainen, Petri (Nokia - FI/Espoo)
>  wrote:
> >
> >
> >> -Original Message-
> >> From: Honnappa Nagarahalli [mailto:honnappa.nagaraha...@linaro.org]
> >> Sent: Wednesday, July 05, 2017 7:04 AM
> >> To: Petri Savolainen 
> >> Cc: lng-odp-forward 
> >> Subject: Re: [lng-odp] [API-NEXT PATCH 1/4] linux-gen: sched: remove
> >> schedule interface depedency to qentry
> >>
> >> On 30 June 2017 at 09:10, Petri Savolainen
> 
> >> wrote:
> >> > Do not use queue internal type in schedule interface.
> >> >
> >> > Signed-off-by: Petri Savolainen 
> >> > ---
> >> >  platform/linux-generic/include/odp_schedule_if.h |  8 +++--
> >> >  platform/linux-generic/odp_queue.c   |  9 --
> >> >  platform/linux-generic/odp_schedule.c| 18 ---
> >> >  platform/linux-generic/odp_schedule_iquery.c | 41 +-
> ---
> >> ---
> >> >  platform/linux-generic/odp_schedule_sp.c | 18 +++
> >> >  5 files changed, 45 insertions(+), 49 deletions(-)
> >> >
> >> > diff --git a/platform/linux-generic/include/odp_schedule_if.h
> >> b/platform/linux-generic/include/odp_schedule_if.h
> >> > index 5877a1cd..5abbb732 100644
> >> > --- a/platform/linux-generic/include/odp_schedule_if.h
> >> > +++ b/platform/linux-generic/include/odp_schedule_if.h
> >> > @@ -35,9 +35,10 @@ typedef int (*schedule_term_local_fn_t)(void);
> >> >  typedef void (*schedule_order_lock_fn_t)(void);
> >> >  typedef void (*schedule_order_unlock_fn_t)(void);
> >> >  typedef unsigned (*schedule_max_ordered_locks_fn_t)(void);
> >> > -typedef void (*schedule_save_context_fn_t)(queue_entry_t *queue);
> >> > +typedef void (*schedule_save_context_fn_t)(uint32_t queue_index,
> void
> >> *ptr);
> >> >
> >> >  typedef struct schedule_fn_t {
> >> > +   int status_sync;
> >>
> >> this structure should contain functions that are provided by scheduler
> >> to other components of ODP. 'status_sync' seems to be an internal
> >> mechanism between the default scheduler and default queue. Hence it
> >> should not be here.
> >
> > This flags if unsched_queue() and save_context() needs to be called.
> Those calls are only needed by iquery scheduler. With this flag, queue
> needs to check only single variable if those calls are needed or not.
> Today, these calls are made always, which hurt performance.
> >
> >>
> >> > schedule_pktio_start_fn_t   pktio_start;
> >> > schedule_thr_add_fn_t   thr_add;
> >> > schedule_thr_rem_fn_t   thr_rem;
> >> > @@ -45,7 +46,6 @@ typedef struct schedule_fn_t {
> >> > schedule_init_queue_fn_tinit_queue;
> >> > schedule_destroy_queue_fn_t destroy_queue;
> >> > schedule_sched_queue_fn_t   sched_queue;
> >> > -   schedule_unsched_queue_fn_t unsched_queue;
> >> these queue related functions are not used by other components within
> >> ODP. These are specific to default queue and default schedulers. These
> >> should not be part of this file.
> >
> > Didn't add or remove those functions in this patch set. This discussion
> can be done in context of another patch set.
> >
> Are you just cleaning up the scheduler and queue interface in this
> patch? If not this change can be taken up in this patch.

This patch set cleans odp_schedule_if.h interface, which currently defines 
input and output interfaces for the scheduler. This set does not redesign the 
interface but cleans and optimizes the usage. The main point is to remove type 
dependencies, functions and function calls that are not needed.

These functions above are needed only by iquery. This change removes empty 
function defines and extra calls when iquery is not used. One option to get 
rid-off these functions is to remove iquery if it's not developed anymore.

-Petri

Re: [lng-odp] [API-NEXT PATCH 2/4] linux-gen: sched: use config max ordered locks

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: Honnappa Nagarahalli [mailto:honnappa.nagaraha...@linaro.org]
> Sent: Thursday, July 06, 2017 11:00 PM
> To: Savolainen, Petri (Nokia - FI/Espoo) 
> Cc: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCH 2/4] linux-gen: sched: use config
> max ordered locks
> 
> On 5 July 2017 at 01:35, Savolainen, Petri (Nokia - FI/Espoo)
>  wrote:
> >
> >> > diff --git a/platform/linux-generic/include/odp_config_internal.h
> >> b/platform/linux-generic/include/odp_config_internal.h
> >> > index 3cff0045..469396df 100644
> >> > --- a/platform/linux-generic/include/odp_config_internal.h
> >> > +++ b/platform/linux-generic/include/odp_config_internal.h
> >> > @@ -27,7 +27,7 @@
> >> >  /*
> >> >   * Maximum number of ordered locks per queue
> >> >   */
> >> > -#define CONFIG_QUEUE_MAX_ORD_LOCKS 4
> >> > +#define CONFIG_QUEUE_MAX_ORD_LOCKS 2
> >>
> >> This is unnecessary change for this patch. This patch does not need
> this
> >> change.
> >
> > With this value (2), the current situation does not change. Internal
> defines limited the number into 2, so this keeps it 2.
> 
> This change affects other implementations of the scheduler. If the
> default scheduler is implemented independent of what the value of
> CONFIG_QUEUE_MAX_ORD_LOCKS is, this change is not required.
> This change should not be necessary for this patch.

I try to keep properties as they were before the patch. I have not heard 
anybody complaining that 2 locks is not sufficient. Have you? We can later 
expand the value if needed. Your scheduler is not widely used yet (it's been in 
api-next a week or so), so people using this feature have not been exposed to 
number of locks being 4.

It was 2, I keep it 2 and we can increase it later if someone needs a larger 
value.

-Petri