Re: [lng-odp] [PATCH v2 1/2] test: perf: add new scheduling latency test

[Brian Brooks]

On 09/14 11:53:06, Matias Elo wrote:
> +
> + /* Clear possible locally stored buffers */
> + odp_schedule_pause();
> +
> + while (1) {
> + ev = odp_schedule(_queue, ODP_SCHED_NO_WAIT);
> +
> + if (ev == ODP_EVENT_INVALID)
> + break;
> +
> + if (odp_queue_enq(src_queue, ev)) {
> + LOG_ERR("[%i] Queue enqueue failed.\n", thr);
> + odp_event_free(ev);
> + return -1;
> + }
> + }
> +
> + odp_schedule_resume();

Is it possible to skip this and go straight to draining the queues?

Locally pre-scheduled work is an implementation detail that should be hidden
by the scheduling APIs.

A hardware scheduler may not pre-schedule work to cores the way the current
software implementation does. The ODP implementation for that environment
would have to turn the scheduling call into a nop for that core if it is
paused by use of these APIs. Another way to implement it would be to remove
this core from all queue scheduling groups and leave the schedule call as-is.
If implemented by the first method, the application writer could simply just
not call the API to schedule work. If implemented by the second method, there
are already scheduling group APIs to do this.

Are odp_schedule_pause() and odp_schedule_resume() deprecated?

> + odp_barrier_wait(>barrier);
> +
> + clear_sched_queues();

Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

[Maxim Uvarov]

On 09/16/16 12:51, Savolainen, Petri (Nokia - FI/Espoo) wrote:



-Original Message-
From: Maxim Uvarov [mailto:maxim.uva...@linaro.org]
Sent: Friday, September 16, 2016 10:05 AM
To: Savolainen, Petri (Nokia - FI/Espoo) ; lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

On 09/16/16 09:41, Savolainen, Petri (Nokia - FI/Espoo) wrote:

-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of

Maxim

Uvarov
Sent: Thursday, September 15, 2016 7:07 PM
To: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

On 09/15/16 15:18, Maxim Uvarov wrote:

there is odp_time_diff(), forget about this patch.


Sorry, odp_time_diff() is for difference. odp_time_sub() still needed
for thing like calculate interval for execution.

Please review api patch.

Maxim.


Could you give an example. E.g. this kind of interval calculation does

not need substract:

t1 = time();
t2 = t1 + wait_time;

while (time() < t2)
  spin();

// now it's time to continue
foo();


-Petri

Looks like I lost again. "diff" word is confusing for me:
/**
   * Time difference
   *
   * @param t2Second time stamp
   * @param t1First time stamp
   *
   * @return Difference of time stamps
   */
odp_time_t odp_time_diff(odp_time_t t2, odp_time_t t1);


In fact this function (at least in linux-generic) subtracts t1 from t2.
Assuming that t2 > t1.

I'm going to send proposal api patch to implement function:
int odp_schedule_stats(odp_schedule_stats_t *stats);
which will return times spend in and out scheduler. Later extend it
with more counters like context switches and etc.

t1 = odp_time_local();
ev = sched_api->schedule(from, wait);
t2 = odp_time_local();

Then time in schedule will be:
odp_time_t sched_time =  odp_time_diff(t2, t1)

So I think everything ok but I would refine doxygen to be more clear
what is odp_time_diff() for
and probably rename it to odp_time_sub() to be more symmetrical to
odp_time_sum().

Maxim.



Diff is commonly used for time difference where only t2 - t1 is allowed (not t1 
- t2). Sub is more generic term and would indicate that also t1 - t2 is 
allowed, which would produce a *negative* result. We'd not want to support 
negative time. Also odp_time_diff is similar to e.g. POSIX difftime( time_t 
time_end, time_t time_beg )

ok.

About scheduler statistics. Features that are commonly HW accelerated fit into 
the API, while those that are not commonly supported should not be added. When 
e.g. scheduler execution time statistics is not commonly calculated by the HW, 
it's better to leave it to the (interested) application. Additional statistics 
add implementation complexity and decrease performance. Time read is bad for 
performance since it requires synchronization (system call, read a SoC global 
register, etc) and cannot be reordered.

Application can easily do:

t1 = odp_time_local();
ev = odp_schedule(from, wait);
t2 = odp_time_local();

there's no room for implementation to do it any faster.


-Petri

Yes, anyway there will be slow down. I'm looking to card ODP-332 
. Time spend for app is 
odp_time_local() - sched_time. And sched time calculated as code above.
Time spend can be calculated with cpu cycles or performance counters on 
cpu. But any accounting will take time for such critical function. And 
probably there
will be no common values which we can get from hardware or from OS. 
Maybe it's better to close that card.


Maxim.

Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: Maxim Uvarov [mailto:maxim.uva...@linaro.org]
> Sent: Friday, September 16, 2016 10:05 AM
> To: Savolainen, Petri (Nokia - FI/Espoo)  labs.com>; lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub
> 
> On 09/16/16 09:41, Savolainen, Petri (Nokia - FI/Espoo) wrote:
> >
> >> -Original Message-
> >> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of
> Maxim
> >> Uvarov
> >> Sent: Thursday, September 15, 2016 7:07 PM
> >> To: lng-odp-forward 
> >> Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub
> >>
> >> On 09/15/16 15:18, Maxim Uvarov wrote:
> >>> there is odp_time_diff(), forget about this patch.
> >>>
> >> Sorry, odp_time_diff() is for difference. odp_time_sub() still needed
> >> for thing like calculate interval for execution.
> >>
> >> Please review api patch.
> >>
> >> Maxim.
> >>
> > Could you give an example. E.g. this kind of interval calculation does
> not need substract:
> >
> > t1 = time();
> > t2 = t1 + wait_time;
> >
> > while (time() < t2)
> >  spin();
> >
> > // now it's time to continue
> > foo();
> >
> >
> > -Petri
> Looks like I lost again. "diff" word is confusing for me:
> /**
>   * Time difference
>   *
>   * @param t2Second time stamp
>   * @param t1First time stamp
>   *
>   * @return Difference of time stamps
>   */
> odp_time_t odp_time_diff(odp_time_t t2, odp_time_t t1);
> 
> 
> In fact this function (at least in linux-generic) subtracts t1 from t2.
> Assuming that t2 > t1.
> 
> I'm going to send proposal api patch to implement function:
> int odp_schedule_stats(odp_schedule_stats_t *stats);
> which will return times spend in and out scheduler. Later extend it
> with more counters like context switches and etc.
> 
> t1 = odp_time_local();
> ev = sched_api->schedule(from, wait);
> t2 = odp_time_local();
> 
> Then time in schedule will be:
> odp_time_t sched_time =  odp_time_diff(t2, t1)
> 
> So I think everything ok but I would refine doxygen to be more clear
> what is odp_time_diff() for
> and probably rename it to odp_time_sub() to be more symmetrical to
> odp_time_sum().
> 
> Maxim.
> 
> 

Diff is commonly used for time difference where only t2 - t1 is allowed (not t1 
- t2). Sub is more generic term and would indicate that also t1 - t2 is 
allowed, which would produce a *negative* result. We'd not want to support 
negative time. Also odp_time_diff is similar to e.g. POSIX difftime( time_t 
time_end, time_t time_beg )

About scheduler statistics. Features that are commonly HW accelerated fit into 
the API, while those that are not commonly supported should not be added. When 
e.g. scheduler execution time statistics is not commonly calculated by the HW, 
it's better to leave it to the (interested) application. Additional statistics 
add implementation complexity and decrease performance. Time read is bad for 
performance since it requires synchronization (system call, read a SoC global 
register, etc) and cannot be reordered.

Application can easily do:

t1 = odp_time_local();
ev = odp_schedule(from, wait);
t2 = odp_time_local();

there's no room for implementation to do it any faster.


-Petri

[lng-odp] [PATCH v2 5/5] example: l3fwd: simplify worker main loop

[Matias Elo]

Reduce extra code in the fast path.

Signed-off-by: Matias Elo 
---
 example/l3fwd/odp_l3fwd.c | 154 +++---
 1 file changed, 89 insertions(+), 65 deletions(-)

diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c
index 9e29e1b..5539711 100644
--- a/example/l3fwd/odp_l3fwd.c
+++ b/example/l3fwd/odp_l3fwd.c
@@ -62,6 +62,7 @@ struct thread_arg_s {
uint64_t rx_drops;
uint64_t tx_drops;
struct {
+   int if_idx; /* interface index */
int nb_rxq; /* number of rxq this thread will access */
int rxq[MAX_NB_QUEUE];  /* rxq[i] is index in pktio.ifin[] */
int txq_idx;/* index in pktio.ifout[] */
@@ -175,7 +176,7 @@ static inline void ipv4_dec_ttl_csum_update(odph_ipv4hdr_t 
*ip)
ip->chksum += odp_cpu_to_be_16(1 << 8);
 }
 
-static int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
+static inline int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
 {
fwd_db_entry_t *entry;
ipv4_tuple5_t key;
@@ -217,7 +218,7 @@ static int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
return dif;
 }
 
-static int l3fwd_pkt_lpm(odp_packet_t pkt, int sif)
+static inline int l3fwd_pkt_lpm(odp_packet_t pkt, int sif)
 {
odph_ipv4hdr_t *ip;
odph_ethhdr_t *eth;
@@ -275,75 +276,83 @@ static inline int drop_err_pkts(odp_packet_t pkt_tbl[], 
unsigned num)
return dropped;
 }
 
-static void l3fwd_one_queue(uint32_t sif, int rxq_idx, void *thr_arg)
+static int run_worker(void *arg)
 {
-   odp_packet_t *tbl;
-   odp_pktout_queue_t outq;
+   int if_idx;
+   struct thread_arg_s *thr_arg = arg;
odp_pktin_queue_t inq;
+   int input_ifs[thr_arg->nb_pktio];
+   odp_pktin_queue_t input_queues[thr_arg->nb_pktio];
+   odp_pktout_queue_t output_queues[global.cmd_args.if_count];
odp_packet_t pkt_tbl[MAX_PKT_BURST];
-   struct thread_arg_s *arg;
+   odp_packet_t *tbl;
int pkts, drop, sent;
int dst_port, dif;
-   int i;
+   int i, j;
+   int pktio = 0;
+   int num_pktio = 0;
 
-   arg = thr_arg;
-   inq = global.l3fwd_pktios[sif].ifin[rxq_idx];
-   pkts = odp_pktin_recv(inq, pkt_tbl, MAX_PKT_BURST);
-   if (pkts <= 0)
-   return;
+   /* Copy all required handles to local memory */
+   for (i = 0; i < global.cmd_args.if_count; i++) {
+   int txq_idx = thr_arg->pktio[i].txq_idx;
 
-   arg->packets += pkts;
-   drop = drop_err_pkts(pkt_tbl, pkts);
-   pkts -= drop;
-   arg->rx_drops += drop;
+   output_queues[i] =  global.l3fwd_pktios[i].ifout[txq_idx];
 
-   dif = global.fwd_func(pkt_tbl[0], sif);
-   tbl = _tbl[0];
-   while (pkts) {
-   int txq_idx;
+   if_idx = thr_arg->pktio[i].if_idx;
+   for (j = 0; j < thr_arg->pktio[i].nb_rxq; j++) {
+   int rxq_idx = thr_arg->pktio[i].rxq[j];
 
-   dst_port = dif;
-   for (i = 1; i < pkts; i++) {
-   dif = global.fwd_func(tbl[i], sif);
-   if (dif != dst_port)
-   break;
+   inq = global.l3fwd_pktios[if_idx].ifin[rxq_idx];
+   input_ifs[num_pktio] = if_idx;
+   input_queues[num_pktio] = inq;
+   num_pktio++;
}
-
-   txq_idx = arg->pktio[dst_port].txq_idx;
-   outq = global.l3fwd_pktios[dst_port].ifout[txq_idx];
-   sent = odp_pktout_send(outq, tbl, i);
-   if (odp_unlikely(sent < i)) {
-   sent = sent < 0 ? 0 : sent;
-   odp_packet_free_multi([sent], i - sent);
-   arg->tx_drops += i - sent;
-   }
-
-   if (i < pkts)
-   tbl += i;
-
-   pkts -= i;
}
-}
 
-static int run_worker(void *arg)
-{
-   int if_idx, rxq, nb_rxq;
-   struct thread_arg_s *thr_arg = arg;
+   num_pktio = thr_arg->nb_pktio;
+   if_idx = input_ifs[pktio];
+   inq = input_queues[pktio];
 
odp_barrier_wait();
 
while (!exit_threads) {
-   for (if_idx = 0; if_idx < thr_arg->nb_pktio; if_idx++) {
-   nb_rxq = thr_arg->pktio[if_idx].nb_rxq;
-   if (!nb_rxq || thr_arg->thr_idx == INVALID_ID)
-   continue;
+   if (num_pktio > 1) {
+   if_idx = input_ifs[pktio];
+   inq = input_queues[pktio];
+   pktio++;
+   if (pktio == num_pktio)
+   pktio = 0;
+   }
 
-   for (rxq = 0; rxq < nb_rxq; rxq++) {
-   int rxq_idx;
+   pkts = odp_pktin_recv(inq, pkt_tbl, MAX_PKT_BURST);

[lng-odp] [PATCH v2 2/5] example: l3fwd: optimize ip checksum update

[Matias Elo]

When decrementing TTL the IPv4 checksum can be incrementally updated.

Signed-off-by: Matias Elo 
---
 example/l3fwd/odp_l3fwd.c | 22 ++
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c
index 44778b0..95c3d85 100644
--- a/example/l3fwd/odp_l3fwd.c
+++ b/example/l3fwd/odp_l3fwd.c
@@ -158,6 +158,22 @@ static void setup_fwd_db(void)
}
 }
 
+/**
+ * Decrement TTL and incrementally update checksum
+ *
+ * @param ip  IPv4 header
+ */
+static inline void ipv4_dec_ttl_csum_update(odph_ipv4hdr_t *ip)
+{
+   uint16_t a = ~odp_cpu_to_be_16(1 << 8);
+
+   ip->ttl--;
+   if (ip->chksum >= a)
+   ip->chksum -= a;
+   else
+   ip->chksum += odp_cpu_to_be_16(1 << 8);
+}
+
 static int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
 {
fwd_db_entry_t *entry;
@@ -186,8 +202,7 @@ static int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
key.dst_port = 0;
}
entry = find_fwd_db_entry();
-   ip->ttl--;
-   ip->chksum = odph_ipv4_csum_update(pkt);
+   ipv4_dec_ttl_csum_update(ip);
eth = odp_packet_l2_ptr(pkt, NULL);
if (entry) {
eth->src = entry->src_mac;
@@ -211,8 +226,7 @@ static int l3fwd_pkt_lpm(odp_packet_t pkt, int sif)
int ret;
 
ip = odp_packet_l3_ptr(pkt, );
-   ip->ttl--;
-   ip->chksum = odph_ipv4_csum_update(pkt);
+   ipv4_dec_ttl_csum_update(ip);
eth = odp_packet_l2_ptr(pkt, NULL);
 
/* network byte order maybe different from host */
-- 
2.7.4

[lng-odp] [PATCH v2 3/5] example: l3fwd: make packet error check optional

[Matias Elo]

Make packet error check optional as it forces full packet parse.

Signed-off-by: Matias Elo 
---
 example/l3fwd/odp_l3fwd.c | 24 ++--
 1 file changed, 18 insertions(+), 6 deletions(-)

diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c
index 95c3d85..f767fb4 100644
--- a/example/l3fwd/odp_l3fwd.c
+++ b/example/l3fwd/odp_l3fwd.c
@@ -80,6 +80,7 @@ typedef struct {
uint32_t duration; /* seconds to run */
uint8_t hash_mode; /* 1:hash, 0:lpm */
uint8_t dest_mac_changed[MAX_NB_PKTIO]; /* 1: dest mac from cmdline */
+   int error_check; /* Check packets for errors */
 } app_args_t;
 
 struct {
@@ -241,10 +242,10 @@ static int l3fwd_pkt_lpm(odp_packet_t pkt, int sif)
 }
 
 /**
- * Drop packets which input parsing marked as containing errors.
+ * Drop unsupported packets and packets containing errors.
  *
- * Frees packets with error and modifies pkt_tbl[] to only contain packets with
- * no detected errors.
+ * Frees packets with errors or unsupported protocol and modifies pkt_tbl[] to
+ * only contain valid packets.
  *
  * @param pkt_tbl  Array of packets
  * @param num  Number of packets in pkt_tbl[]
@@ -256,12 +257,16 @@ static inline int drop_err_pkts(odp_packet_t pkt_tbl[], 
unsigned num)
odp_packet_t pkt;
unsigned dropped = 0;
unsigned i, j;
+   int err;
 
for (i = 0, j = 0; i < num; ++i) {
pkt = pkt_tbl[i];
+   err = 0;
 
-   if (odp_unlikely(odp_packet_has_error(pkt) ||
-!odp_packet_has_ipv4(pkt))) {
+   if (global.cmd_args.error_check)
+   err = odp_packet_has_error(pkt);
+
+   if (odp_unlikely(err || !odp_packet_has_ipv4(pkt))) {
odp_packet_free(pkt);
dropped++;
} else if (odp_unlikely(i != j++)) {
@@ -475,6 +480,8 @@ static void print_usage(char *progname)
   "  -q, --queue  Configure rx queue(s) for port\n"
   "optional, format: [(port, queue, thread),...]\n"
   "for example: -q '(0, 0, 1),(1,0,2)'\n"
+  "  -e, --error_check 0: Don't check packet errors (default)\n"
+  "1: Check packet errors\n"
   "  -h, --help   Display help and exit.\n\n"
   "\n", NO_PATH(progname), NO_PATH(progname)
);
@@ -495,12 +502,13 @@ static void parse_cmdline_args(int argc, char *argv[], 
app_args_t *args)
{"duration", optional_argument, NULL, 'd'}, /* return 'd' */
{"thread", optional_argument, NULL, 't'},   /* return 't' */
{"queue", optional_argument, NULL, 'q'},/* return 'q' */
+   {"error_check", required_argument, NULL, 'e'},
{"help", no_argument, NULL, 'h'},   /* return 'h' */
{NULL, 0, NULL, 0}
};
 
while (1) {
-   opt = getopt_long(argc, argv, "+s:t:d:i:r:q:h",
+   opt = getopt_long(argc, argv, "+s:t:d:i:r:q:e:h",
  longopts, _index);
 
if (opt == -1)
@@ -585,6 +593,10 @@ static void parse_cmdline_args(int argc, char *argv[], 
app_args_t *args)
args->route_str[route_index++] = local;
break;
 
+   case 'e':
+   args->error_check = atoi(optarg);
+   break;
+
case 'h':
print_usage(argv[0]);
exit(EXIT_SUCCESS);
-- 
2.7.4

[lng-odp] [PATCH v2 4/5] example: l3fwd: remove unnecessary output arguments

[Matias Elo]

There is no need to save the number of data bytes remaining in the segment
when calling odp_packet_l3_ptr().

Signed-off-by: Matias Elo 
---
 example/l3fwd/odp_l3fwd.c | 6 ++
 1 file changed, 2 insertions(+), 4 deletions(-)

diff --git a/example/l3fwd/odp_l3fwd.c b/example/l3fwd/odp_l3fwd.c
index f767fb4..9e29e1b 100644
--- a/example/l3fwd/odp_l3fwd.c
+++ b/example/l3fwd/odp_l3fwd.c
@@ -182,10 +182,9 @@ static int l3fwd_pkt_hash(odp_packet_t pkt, int sif)
odph_ethhdr_t *eth;
odph_udphdr_t  *udp;
odph_ipv4hdr_t *ip;
-   uint32_t len;
int dif;
 
-   ip = odp_packet_l3_ptr(pkt, );
+   ip = odp_packet_l3_ptr(pkt, NULL);
key.dst_ip = odp_be_to_cpu_32(ip->dst_addr);
key.src_ip = odp_be_to_cpu_32(ip->src_addr);
key.proto = ip->proto;
@@ -222,11 +221,10 @@ static int l3fwd_pkt_lpm(odp_packet_t pkt, int sif)
 {
odph_ipv4hdr_t *ip;
odph_ethhdr_t *eth;
-   uint32_t len;
int dif;
int ret;
 
-   ip = odp_packet_l3_ptr(pkt, );
+   ip = odp_packet_l3_ptr(pkt, NULL);
ipv4_dec_ttl_csum_update(ip);
eth = odp_packet_l2_ptr(pkt, NULL);
 
-- 
2.7.4

[lng-odp] [PATCH v2 1/5] example: l3fwd: add missing gitignores

[Matias Elo]

Signed-off-by: Matias Elo 
---
 example/l3fwd/.gitignore | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/example/l3fwd/.gitignore b/example/l3fwd/.gitignore
index 74e501f..3411830 100644
--- a/example/l3fwd/.gitignore
+++ b/example/l3fwd/.gitignore
@@ -1 +1,3 @@
 odp_l3fwd
+*.log
+*.trs
-- 
2.7.4

Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

[Maxim Uvarov]

On 09/16/16 09:41, Savolainen, Petri (Nokia - FI/Espoo) wrote:



-Original Message-
From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Maxim
Uvarov
Sent: Thursday, September 15, 2016 7:07 PM
To: lng-odp-forward 
Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

On 09/15/16 15:18, Maxim Uvarov wrote:

there is odp_time_diff(), forget about this patch.


Sorry, odp_time_diff() is for difference. odp_time_sub() still needed
for thing like calculate interval for execution.

Please review api patch.

Maxim.


Could you give an example. E.g. this kind of interval calculation does not need 
substract:

t1 = time();
t2 = t1 + wait_time;

while (time() < t2)
 spin();

// now it's time to continue
foo();


-Petri

Looks like I lost again. "diff" word is confusing for me:
/**
 * Time difference
 *
 * @param t2Second time stamp
 * @param t1First time stamp
 *
 * @return Difference of time stamps
 */
odp_time_t odp_time_diff(odp_time_t t2, odp_time_t t1);


In fact this function (at least in linux-generic) subtracts t1 from t2.
Assuming that t2 > t1.

I'm going to send proposal api patch to implement function:
int odp_schedule_stats(odp_schedule_stats_t *stats);
which will return times spend in and out scheduler. Later extend it
with more counters like context switches and etc.

t1 = odp_time_local();
ev = sched_api->schedule(from, wait);
t2 = odp_time_local();

Then time in schedule will be:
odp_time_t sched_time =  odp_time_diff(t2, t1)

So I think everything ok but I would refine doxygen to be more clear 
what is odp_time_diff() for
and probably rename it to odp_time_sub() to be more symmetrical to 
odp_time_sum().


Maxim.

Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub

[Savolainen, Petri (Nokia - FI/Espoo)]

> -Original Message-
> From: lng-odp [mailto:lng-odp-boun...@lists.linaro.org] On Behalf Of Maxim
> Uvarov
> Sent: Thursday, September 15, 2016 7:07 PM
> To: lng-odp-forward 
> Subject: Re: [lng-odp] [API-NEXT PATCHv2] api: odp_time_sub
> 
> On 09/15/16 15:18, Maxim Uvarov wrote:
> > there is odp_time_diff(), forget about this patch.
> >
> Sorry, odp_time_diff() is for difference. odp_time_sub() still needed
> for thing like calculate interval for execution.
> 
> Please review api patch.
> 
> Maxim.
> 

Could you give an example. E.g. this kind of interval calculation does not need 
substract:

t1 = time();
t2 = t1 + wait_time;

while (time() < t2)
spin();

// now it's time to continue
foo();


-Petri