[Xenomai-git] New commits on branch wip/heapmem

[git repository hosting]

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=a651f0ccce8eb10a36edceb60d69552cd08d1a0e
Author: Philippe Gerum 
Date:   Sun May 13 19:00:50 2018 +0200

cobalt/heap: rebase on HEAPMEM algorithm

Address the issue mentioned in [1] regarding the core (xnheap)
allocator, using a variant of the McKusick scheme significantly
improving the performance figures.

As a by-product of this overhaul, the core allocator can now manage
heaps up to (4GB - PAGE_SIZE).

The performance report log obtained by testing on imx6qp
is as follows:

== memcheck started
 seq_heap_size=2048k
 random_alloc_rounds=1024
 pattern_heap_size=128k
 pattern_check_rounds=128

[SEQUENTIAL ALLOC->FREE, ^2 BLOCK SIZES] ON 'xnheap'

HEAPSZ  test heap size
BLOCKSZ tested block size
NRBLKS  number of blocks allocatable in heap
AVG-A   average time to allocate block (us)
AVG-F   average time to free block (us)
MAX-A   max time to allocate block (us)
MAX-F   max time to free block (us)
FLAGS   +shuffle: randomized free
+realloc: measure after initial alloc/free pass (hot heap)

sorted by: max alloc time
  HEAPSZ  BLOCKSZ   NRBLKS  AVG-A  AVG-F  MAX-A  MAX-F   FLAGS
   1024k   3232768  0  0  8  6
   1024k   3232768  0  0  7  2   +shuffle +realloc
   1024k   1665536  0  0  7  2   +realloc
   1024k   1665536  0  0  6  7   +shuffle +realloc
  ... (364 results following) ...

sorted by: max free time
  HEAPSZ  BLOCKSZ   NRBLKS  AVG-A  AVG-F  MAX-A  MAX-F   FLAGS
   1024k  128 8192  0  1  2  8
   1024k   1665536  0  0  6  7   +shuffle +realloc
   1024k   3232768  0  0  8  6
512k   3216384  0  0  5  6   +realloc
  ... (364 results following) ...

overall:
  worst alloc time: 8 (us)
  worst free time: 8 (us)
  average of max. alloc times: 1 (us)
  average of max. free times: 2 (us)
  average alloc time: 1 (us)
  average free time: 1 (us)

[SEQUENTIAL ALLOC->FREE, RANDOM BLOCK SIZES] ON 'xnheap'

HEAPSZ  test heap size
BLOCKSZ tested block size
NRBLKS  number of blocks allocatable in heap
AVG-A   average time to allocate block (us)
AVG-F   average time to free block (us)
MAX-A   max time to allocate block (us)
MAX-F   max time to free block (us)
FLAGS   +shuffle: randomized free
+realloc: measure after initial alloc/free pass (hot heap)

sorted by: max alloc time
  HEAPSZ  BLOCKSZ   NRBLKS  AVG-A  AVG-F  MAX-A  MAX-F   FLAGS
512k   17k  28  1  1  8  2   +shuffle
512k   45k  11  1  1  7  2
   1024k   2432768  0  0  7  6   +shuffle
128k  820  128  1  1  6  2   +shuffle
  ... (32764 results following) ...

sorted by: max free time
  HEAPSZ  BLOCKSZ   NRBLKS  AVG-A  AVG-F  MAX-A  MAX-F   FLAGS
   1024k3k 292  1  1  1  8   +shuffle
256k  174 1024  1  1  1  6   +shuffle
   1024k   2432768  0  0  7  6   +shuffle
 32k   12k   2  2  3  1  5
  ... (32764 results following) ...

overall:
  worst alloc time: 8 (us)
  worst free time: 8 (us)
  average of max. alloc times: 1 (us)
  average of max. free times: 1 (us)
  average alloc time: 1 (us)
  average free time: 1 (us)

[1] http://www.xenomai.org/pipermail/xenomai/2018-April/038883.html

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=77d4fa2c9f93394bd935e93f8e29ea6d9a7420b5
Author: Philippe Gerum 
Date:   Tue May 15 18:24:02 2018 +0200

boilerplate/heapmem: align on HEAPMEM_PAGE_SIZE for non-bucketed blocks

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=b0afd6dcd67e4038f25ad7f2c9e3ea4fd3a268a9
Author: Philippe Gerum 
Date:   Tue May 15 15:29:44 2018 +0200

testsuite/smokey: memcheck: fix test heap sizes

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=434d42036a852ea2171a89a55b197dfc98d4b0f5
Author: Philippe Gerum 
Date:   Sun May 13 19:00:36 2018 +0200

drivers/ipc: align pool memory on native page size

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=b63991a72eb3fd1cf7bea203a4dda4daa10c80c0
Author: Philippe Gerum 
Date:   Sun May 13 17:17:06 2018 +0200

boilerplate/heapmem: move heap limit to ~4GB

URL:

[Xenomai-git] New commits on branch wip/heapmem

[git repository hosting]

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=e1f60e82c09119b63bab8674356ae744aa460cdd
Author: Philippe Gerum 
Date:   Sun Apr 22 19:50:57 2018 +0200

copperplate/heapobj: enable heapmem for private memory

Make HEAPMEM the default private memory allocator for real-time
configurations (cobalt || (mercury && non-debug)).

This setting can be reverted by passing --with-localmem=tlsf.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=f20c742f6585fd53e5ff12be6b04d5081ec4a158
Author: Philippe Gerum 
Date:   Sun Apr 22 18:20:04 2018 +0200

testsuite/smokey: add test suite for memory allocators

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=ebc1dbb901e4db2a3175ecf4fda668201d47227b
Author: Philippe Gerum 
Date:   Mon Apr 23 10:47:28 2018 +0200

copperplate/heapobj-pshared: add helper to return the total heap size

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=99cad85489858b0557f4461bf85bcb7d66a93559
Author: Philippe Gerum 
Date:   Fri Apr 13 12:15:15 2018 +0200

boilerplate: add heapmem allocator

This is the umpteenth incarnation of the McKusick allocator, aimed at
replacing other allocators which have serious issues:

- TLSF is fast but not that good when it comes to memory overhead with
  small sizes (i.e. < 512 bytes) on 64bit.

- heapobj-pshared has decent overhead figures but may be insanely slow
  at deallocating blocks from large, hot heaps.

- the Cobalt core allocator is even worse than heapobj-pshared in
  deallocating blocks, although the system heap should be generally
  small enough not to trigger the most pathological cases in practice,
  hopefully. Nevertheless, the performances are unacceptable.

The original McKusick algorithm implements a quick fit allocation
scheme, based on bucket management of ^2-sized chunks, which
automatically switches to a page-based allocation method for blocks
larger than twice the base page size.

This variant maintains the free page list in AVL trees for fast
lookups of multi-page memory ranges, and pages holding bucketed memory
have a quick allocation bitmap to manage their blocks internally.

The current implementation can replace TLSF for delivering
process-local memory with similar performances but lesser overhead
with small sizes. Most importantly, a kernel variant of HEAPMEM should
replace the Cobalt core allocator. Likewise, heapobj-pshared which is
beyond repair should be replaced with a process-shareable variant as
well, assuming the average size and allocation patterns of real-time
objects are similar in all contexts.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=43347d7fee5dd752eb0c93ac3d5568e5ac80b810
Author: Philippe Gerum 
Date:   Mon Apr 23 10:28:49 2018 +0200

boilerplate: compiler: add more helpers and attribute tags

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=1c556dc7b8f8bd5fe7bd981a0e8ecb715f44b8aa
Author: Philippe Gerum 
Date:   Mon Apr 23 18:29:46 2018 +0200

smokey: add helper to retrieve size expr

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=94a220039395ef6d89d911ce3aec86744b5607d6
Author: Philippe Gerum 
Date:   Mon Apr 23 18:03:07 2018 +0200

smokey: argument accessors using variables

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=138588cba6f58af2b98c011833e5678c4365001c
Author: Gilles Chanteperdrix 
Date:   Tue Jul 12 20:29:22 2016 +0200

boilerplate/avl: merge pshared support for AVL trees

Make the AVL tree usable in shared memory when AVL_SHARED is defined
at build time, switching to offset-based memory references.

Gilles published this code in July 2016 as part of his personal
toolkit for hobby projects aka 'libchutils'.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=171beb05487c7e34ea04db82fda6782d6ad89ffa
Author: Norbert Lange 
Date:   Wed Apr 25 15:05:00 2018 +0200

demo/posix, testsuite/smokey: silences some compiler warnings

Signed-off-by: Norbert Lange 

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=bc646282da46d32ce4c203aa477b72ea294ae535
Author: Jan Kiszka 
Date:   Wed Apr 18 14:51:00 2018 +0200

cobalt/synch: Clear PP boost prior to handing over the ownership

Otherwise we first add the synch object to the new owner's boost list
before we remove it from the old one's, corrupting the list.

Signed-off-by: Jan Kiszka 

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=95193cce02aae6d075662d749370c8bff962ab33
Author: Jan Kiszka 
Date:   Wed Apr 18 14:51:00 2018 +0200

[Xenomai-git] New commits on branch wip/heapmem

[git repository hosting]

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=984ce322de865a3ba7896ff9f5272bc07f0c4075
Author: Philippe Gerum 
Date:   Sun Apr 22 19:50:57 2018 +0200

copperplate/heapobj: enable heapmem for private memory

Make HEAPMEM the default private memory allocator for real-time
configurations (cobalt || (mercury && non-debug)).

This setting can be reverted by passing --with-localmem=tlsf.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=871d062a1ca1e38413fd924fc05490079ad86e3c
Author: Philippe Gerum 
Date:   Sun Apr 22 18:20:04 2018 +0200

testsuite/smokey: add test suite for memory allocators

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=d7c0f165bbe49e29dd50c3c74597cb5cf8e59c37
Author: Philippe Gerum 
Date:   Mon Apr 23 10:47:28 2018 +0200

copperplate/heapobj-pshared: add helper to return the total heap size

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=0c6d84b42ef5873476a32b729bb520fc0bb072ad
Author: Philippe Gerum 
Date:   Fri Apr 13 12:15:15 2018 +0200

boilerplate: add heapmem allocator

This is the umpteenth incarnation of the McKusick allocator, aimed at
replacing other allocators which have serious issues:

- TLSF is fast but not that good when it comes to memory overhead with
  small sizes (i.e. < 512 bytes) on 64bit.

- heapobj-pshared has decent overhead figures but may be insanely slow
  at deallocating blocks from large, hot heaps.

- the Cobalt core allocator is even worse than heapobj-pshared in
  deallocating blocks, although the system heap should be generally
  small enough not to trigger the most pathological cases in practice,
  hopefully. Nevertheless, the performances are unacceptable.

The original McKusick algorithm implements a quick fit allocation
scheme, based on bucket management of ^2-sized chunks, which
automatically switches to a page-based allocation method for blocks
larger than twice the base page size.

This variant maintains the free page list in AVL trees for fast
lookups of multi-page memory ranges, and pages holding bucketed memory
have a quick allocation bitmap to manage their blocks internally.

The current implementation can replace TLSF for delivering
process-local memory with similar performances but lesser overhead
with small sizes. Most importantly, a kernel variant of HEAPMEM should
replace the Cobalt core allocator. Likewise, heapobj-pshared which is
beyond repair should be replaced with a process-shareable variant as
well, assuming the average size and allocation patterns of real-time
objects are similar in all contexts.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=2af5432db53d1c73a0efa35cdb534269b8b9bb94
Author: Philippe Gerum 
Date:   Mon Apr 23 10:28:49 2018 +0200

boilerplate: compiler: add more helpers and attribute tags

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=9499058e27f0191f09b5fb4b191b967f62affe75
Author: Philippe Gerum 
Date:   Mon Apr 23 18:29:46 2018 +0200

smokey: add helper to retrieve size expr

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=cb30ed6e5fcb1e9c4c822fc174460d008e438243
Author: Philippe Gerum 
Date:   Mon Apr 23 18:03:07 2018 +0200

smokey: argument accessors using variables

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=3ed3f8587cca9a823c128c96fb669d060059bd49
Author: Gilles Chanteperdrix 
Date:   Tue Jul 12 20:29:22 2016 +0200

boilerplate/avl: merge pshared support for AVL trees

Make the AVL tree usable in shared memory when AVL_SHARED is defined
at build time, switching to offset-based memory references.

Gilles published this code in July 2016 as part of his personal
toolkit for hobby projects aka 'libchutils'.

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=d044252681c6740fcd83095a001f07f11366e51f
Author: Norbert Lange 
Date:   Wed Apr 25 15:05:00 2018 +0200

demo/posix, testsuite/smokey: silences some compiler warnings

Signed-off-by: Norbert Lange 

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=fb754c807de950132bc6c07e2eaf6628fb22e355
Author: Jan Kiszka 
Date:   Wed Apr 18 14:51:00 2018 +0200

cobalt/synch: Clear PP boost prior to handing over the ownership

Otherwise we first add the synch object to the new owner's boost list
before we remove it from the old one's, corrupting the list.

Signed-off-by: Jan Kiszka 

URL:
http://git.xenomai.org/?p=xenomai-3.git;a=commit;h=06744569820cc09ac1efb8270ccc4c37579a7e42
Author: Jan Kiszka 
Date:   Wed Apr 18 14:51:00 2018 +0200