Just as an FYI, when I add a thread-local buffer cache to the new patch the numbers for odp_example are reduced to this:
Thread 1 starts on core 1 Thread 2 starts on core 0 [1] alloc_sng alloc+free 67 cycles, 19 ns [2] alloc_sng alloc+free 72 cycles, 21 ns [1] alloc_multi alloc+free 87 cycles, 25 ns [2] alloc_multi alloc+free 92 cycles, 27 ns which is equivalent to what we see with the current code. I'll post that patch as an optimization to the restructured code once it's reviewed and merged. There does, however, appear to be a serious problem with the current ODP thread model. If we implement thread-local caches like this, we need to ensure that these caches are flushed upon thread termination or else we'll "leak" cached buffers. The current buffer chunking code has the same problem. I've added a flush_cache() routine to do the flushing but there isn't a thread termination hook to call it from. The current ODP examples all have long-lived threads, however an application that created and destroyed threads with some frequency would quickly run into problems due to this. Something that needs to be addressed for the final v1.0 I'd think. Bill On Mon, Dec 1, 2014 at 9:36 AM, Ola Liljedahl <[email protected]> wrote: > On 1 December 2014 at 14:27, Savolainen, Petri (NSN - FI/Espoo) > <[email protected]> wrote: > > Lock free algorithms do re-tries and in addition there seems to be atomic > Taking locks (e.g. spin locks) also involve retries (on x86, looping > on the return value of a locked xchg instruction). It is just a > question if the loop is in the application code or in the underlying > lock implementation. > > > counter upgrades. It seems that the patch runs about 10x slower than the > Presumably those counters are there for debugging and can be disabled. > What performance do we get without them? On x86, updates on 64-bit > atomic counters will always imply full barriers so will come with some > unnecessary overhead. > > Bill, if the counters are not needed for the proper function, perhaps > they should be compile-time conditional and not enabled by default. > Only those who want that information will need to suffer the overhead. > > -- Ola > > > > > > > current alloc/free code in repo … if these numbers are correct … > > > > > > > > Thread 1 starts on core 1 > > > > Thread 2 starts on core 0 > > > > [2] alloc_sng alloc+free 766 cycles, 225 ns > > > > [1] alloc_sng alloc+free 775 cycles, 228 ns > > > > [2] alloc_multi alloc+free 859 cycles, 245 ns > > > > [1] alloc_multi alloc+free 874 cycles, 245 ns > > > > > > > > > > > > I have used to see numbers around 60-70 cycles in that test (on similar > > x86). > > > > > > > > -Petri > > > > > > > > > > > > > > > > From: [email protected] > > [mailto:[email protected]] On Behalf Of ext Bill > Fischofer > > Sent: Monday, December 01, 2014 2:54 PM > > To: Ola Liljedahl > > Cc: lng-odp-forward > > Subject: Re: [lng-odp] odp_example.c race condition > > > > > > > > The real answer would seem to be that there should be a wait option on > > odp_queue_deq() that doesn't return until a buffer is available for > dequeue. > > That's a fairly natural means of using queues as a synchronization > > mechanism. Busy waiting on a queue seems wasteful whether or not we > call it > > a "polled" queue. > > > > > > > > What seems to exacerbate this issue with the new buffer pool code is that > > odp_bufffer_alloc() and odp_buffer_free() are now lockless (thanks to the > > new odp_atomic_internal.h) and so there's less lock-stepping as threads > > fight over the pool lock for buffer allocates and frees. I suspect the > > latter tended to serialize things a bit more which smoothed these race > > conditions out a bit. > > > > > > > > Bill > > > > > > > > On Mon, Dec 1, 2014 at 6:01 AM, Ola Liljedahl <[email protected]> > > wrote: > > > > Wasn't there some discussion a while ago that the ODP test program > > expects buffer enqueue operations to be instantaneous (the buffer can > > immediately be popped or scheduled) while HW queue implementations > > likely will have some latency before the buffer becomes visible again. > > > > -- Ola > > > > > > > > On 30 November 2014 at 17:09, Bill Fischofer <[email protected]> > > wrote: > >> I'm testing my new buffer pool code and one thing I'm seeing is that the > >> odp_example.c program seems to have a race condition that causes it to > >> fail > >> intermittently. Has anyone else seen this issue? > >> > >> The failing call is on line 297: > >> > >> buf = odp_queue_deq(queue); > >> > >> if (!odp_buffer_is_valid(buf)) { > >> EXAMPLE_ERR(" [%i] Queue empty.\n", thr); > >> return -1; > >> } > >> > >> I modified odp_queue_deq() to retry dequeues up to 1 million times > before > >> reporting empty to allow for some race conditions and the results are > >> illuminating. There's a large variance in how many retries are needed > to > >> find a non-empty queue and this suggests that this is a design issue in > >> the > >> test program. The odp_buffer_pool_print() output shows that the pools > >> themselves seem normal with very few buffers actually in use, again > >> suggesting that this is a race condition in the example. > >> > >> I'll dig into the example code to try to determine how best to address > >> this > >> but just wondering if anyone else has seen this. > >> > >> Thanks. > >> > >> Bill > >> > >> bill@Ubuntu13:~/linaro/v10bufpool/example/odp_example$ ./odp_example > >> > >> ODP example starts > >> > >> Thread mode > >> odp_buffer_pool.c:84:odp_buffer_pool_init_global(): > >> Buffer pool init global > >> odp_buffer_pool.c:85:odp_buffer_pool_init_global(): pool_entry_s size > >> 256 > >> odp_buffer_pool.c:86:odp_buffer_pool_init_global(): pool_entry_t size > >> 256 > >> odp_buffer_pool.c:87:odp_buffer_pool_init_global(): odp_buffer_hdr_t > size > >> 120 > >> odp_buffer_pool.c:88:odp_buffer_pool_init_global(): > >> odp_queue.c:100:odp_queue_init_global():Queue init ... > >> odp_queue.c:120:odp_queue_init_global():done > >> odp_queue.c:121:odp_queue_init_global():Queue init global > >> odp_queue.c:123:odp_queue_init_global(): struct queue_entry_s size 192 > >> odp_queue.c:125:odp_queue_init_global(): queue_entry_t size 192 > >> odp_queue.c:126:odp_queue_init_global(): > >> odp_schedule.c:89:odp_schedule_init_global():Schedule init ... > >> odp_schedule.c:141:odp_schedule_init_global():done > >> odp_timer.c:220:odp_timer_init_global():Timer init > >> ...odp_timer.c:226:odp_timer_init_global():done > >> > >> ODP system info > >> --------------- > >> ODP API version: 0.3.0 > >> CPU model: Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz > >> CPU freq (hz): 3400000000 > >> Cache line size: 64 > >> Max core count: 2 > >> > >> num worker threads: 2 > >> first core: 1 > >> > >> Time accuracy test (2 sec) > >> clock_gettime 2000000024 ns > >> odp_time_cycles 6800000036 cycles > >> odp_time_cycles_to_ns 2000000010 ns > >> odp get cycle error -0.000001% > >> > >> > >> Shared memory > >> -------------- > >> page size: 4 kB > >> huge page size: 2048 kB > >> > >> id name kB align huge addr > >> 0 odp_thread_globals 1 64 0x7fd35a926000 > >> 1 odp_buffer_pools 4 256 0x7fd35a924000 > >> 2 odp_queues 192 192 0x7fd35a8d7000 > >> 3 odp_scheduler 0 64 0x7fd35a923000 > >> 4 odp_sched_pool 8192 4096 0x7fd358f24000 > >> 5 odp_pktio_entries 28 448 0x7fd35a91b080 > >> 6 crypto_pool 14 64 0x7fd35a917000 > >> 7 test_globals 0 64 0x7fd35a916000 > >> 8 msg_pool 65536 4096 0x7fd354f23000 > >> > >> Thread 1 starts on core 1 > >> Thread 2 starts on core 0 > >> [2] alloc_sng alloc+free 766 cycles, 225 ns > >> [1] alloc_sng alloc+free 775 cycles, 228 ns > >> [2] alloc_multi alloc+free 859 cycles, 245 ns > >> [1] alloc_multi alloc+free 874 cycles, 245 ns > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 1 > >> retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 1846 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 5220 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 640 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4235 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 484 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4186 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 479 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4172 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 474 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4159 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 472 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4161 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 471 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4155 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 474 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4185 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 477 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4174 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 481 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4195 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 520 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4224 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 572 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4225 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 569 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4201 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 481 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4187 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 470 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4206 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 569 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4217 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 572 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4157 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 474 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4174 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 472 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4183 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 472 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4159 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 473 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4154 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 476 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 4148 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 471 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 2483 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 567 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 430 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 370 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 363 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 359 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 357 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 350 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 355 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 357 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 357 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 352 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 358 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 357 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 357 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 344 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 353 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 430 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 427 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 356 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 356 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 355 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 353 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 356 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 345 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 361 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 358 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 342 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 362 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 365 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 345 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 342 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 334 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 333 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 333 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 318 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 322 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 320 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 322 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 322 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 318 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 332 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 336 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 320 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 318 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 320 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 332 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 314 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 806221 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 695 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 614 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 368 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 383 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 365 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 363 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 360 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 362 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 353 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 350 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 345 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 353 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 345 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 347 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 342 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 352 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 344 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 352 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 351 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 350 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 348 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 354 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 350 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 349 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 353 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 352 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 346 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 333 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 336 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 336 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 322 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 337 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 336 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 332 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 331 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 329 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 336 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 332 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 334 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 317 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 323 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 333 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 333 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 324 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 328 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 322 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 327 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 325 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 326 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 321 retries > >> odp_queue.c:448:odp_queue_deq():Got buf 0x7fd354f23500 from queue 61 > after > >> 330 retries > >> odp_example.c:297:test_poll_queue(): [2] Queue empty. > >> odp_example.c:298:test_poll_queue(): buf = ffffffff > >> odp_buffer_pool.c:477:odp_buffer_pool_print():Pool info > >> odp_buffer_pool.c:478:odp_buffer_pool_print():--------- > >> odp_buffer_pool.c:479:odp_buffer_pool_print(): pool 1 > >> odp_buffer_pool.c:481:odp_buffer_pool_print(): name > >> odp_sched_pool > >> odp_buffer_pool.c:487:odp_buffer_pool_print(): pool type raw > >> odp_buffer_pool.c:490:odp_buffer_pool_print(): pool storage ODP > managed > >> odp_buffer_pool.c:492:odp_buffer_pool_print(): pool status active > >> odp_buffer_pool.c:496:odp_buffer_pool_print(): pool opts > >> unsegmented, > >> non-zeroized, created > >> odp_buffer_pool.c:497:odp_buffer_pool_print(): pool base > >> 0x7fd358f24000 > >> odp_buffer_pool.c:499:odp_buffer_pool_print(): pool size 8388608 > >> (2048 > >> pages) > >> odp_buffer_pool.c:500:odp_buffer_pool_print(): udata size 0 > >> odp_buffer_pool.c:501:odp_buffer_pool_print(): buf size 4 > >> odp_buffer_pool.c:502:odp_buffer_pool_print(): num bufs 65536 > >> odp_buffer_pool.c:503:odp_buffer_pool_print(): bufs in use 128 > >> odp_buffer_pool.c:504:odp_buffer_pool_print(): buf allocs 128 > >> odp_buffer_pool.c:505:odp_buffer_pool_print(): buf frees 0 > >> odp_buffer_pool.c:506:odp_buffer_pool_print(): buf empty 0 > >> odp_buffer_pool.c:508:odp_buffer_pool_print(): blk size 0 > >> odp_buffer_pool.c:509:odp_buffer_pool_print(): blks available 0 > >> odp_buffer_pool.c:510:odp_buffer_pool_print(): blk allocs 0 > >> odp_buffer_pool.c:511:odp_buffer_pool_print(): blk frees 0 > >> odp_buffer_pool.c:512:odp_buffer_pool_print(): blk empty 0 > >> odp_buffer_pool.c:513:odp_buffer_pool_print(): high wm count 0 > >> odp_buffer_pool.c:514:odp_buffer_pool_print(): low wm count 0 > >> odp_buffer_pool.c:477:odp_buffer_pool_print():Pool info > >> odp_buffer_pool.c:478:odp_buffer_pool_print():--------- > >> odp_buffer_pool.c:479:odp_buffer_pool_print(): pool 2 > >> odp_buffer_pool.c:481:odp_buffer_pool_print(): name msg_pool > >> odp_buffer_pool.c:487:odp_buffer_pool_print(): pool type raw > >> odp_buffer_pool.c:490:odp_buffer_pool_print(): pool storage ODP > managed > >> odp_buffer_pool.c:492:odp_buffer_pool_print(): pool status active > >> odp_buffer_pool.c:496:odp_buffer_pool_print(): pool opts > >> unsegmented, > >> non-zeroized, created > >> odp_buffer_pool.c:497:odp_buffer_pool_print(): pool base > >> 0x7fd354f23000 > >> odp_buffer_pool.c:499:odp_buffer_pool_print(): pool size 67108864 > >> (16384 pages) > >> odp_buffer_pool.c:500:odp_buffer_pool_print(): udata size 0 > >> odp_buffer_pool.c:501:odp_buffer_pool_print(): buf size 8 > >> odp_buffer_pool.c:502:odp_buffer_pool_print(): num bufs 524288 > >> odp_buffer_pool.c:503:odp_buffer_pool_print(): bufs in use 2 > >> odp_buffer_pool.c:504:odp_buffer_pool_print(): buf allocs 75497474 > >> odp_buffer_pool.c:505:odp_buffer_pool_print(): buf frees 75497472 > >> odp_buffer_pool.c:506:odp_buffer_pool_print(): buf empty 0 > >> odp_buffer_pool.c:508:odp_buffer_pool_print(): blk size 0 > >> odp_buffer_pool.c:509:odp_buffer_pool_print(): blks available 0 > >> odp_buffer_pool.c:510:odp_buffer_pool_print(): blk allocs 0 > >> odp_buffer_pool.c:511:odp_buffer_pool_print(): blk frees 0 > >> odp_buffer_pool.c:512:odp_buffer_pool_print(): blk empty 0 > >> odp_buffer_pool.c:513:odp_buffer_pool_print(): high wm count 0 > >> odp_buffer_pool.c:514:odp_buffer_pool_print(): low wm count 0 > >> [1] poll_queue enq+deq 328 cycles, 96 ns > >> > >> > > > >> _______________________________________________ > >> lng-odp mailing list > >> [email protected] > >> http://lists.linaro.org/mailman/listinfo/lng-odp > >> > > > > >
_______________________________________________ lng-odp mailing list [email protected] http://lists.linaro.org/mailman/listinfo/lng-odp
