On Wed, Dec 4, 2013 at 3:02 PM, Glenn Fowler <glenn.s.fow...@gmail.com> wrote:
> On Sun, Dec 1, 2013 at 4:58 PM, Lionel Cons <lionelcons1...@gmail.com>
> wrote:
>>
>> On 1 December 2013 17:26, Glenn Fowler <glenn.s.fow...@gmail.com> wrote:
>> > I believe this is related to vmalloc changes between 2013-05-31 and
>> > 2013-06-09
>> > re-run the tests with
>> > export VMALLOC_OPTIONS=getmem=safe
>> > if that's the problem then it gives a clue on a general solution
>> > details after confirmation
>> >
>>
>> timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0;
>> while read i ; do key="$i$((k++))" ; a["$key"]="$i" ; done ; printf
>> "%s\n" "${a[@]}" ; } ; print "${.sh.version}" ; nanosort <xxx >yyy'
>> Version AIJMP 93v- 2013-10-08
>>
>> real 34.60
>> user 33.27
>> sys 1.19
>>
>> VMALLOC_OPTIONS=getmem=safe timex ~/bin/ksh -c 'function nanosort {
>> typeset -A a ; integer k=0; while read i ; do key="$i$((k++))" ;
>> a["$key"]="$i" ; done ; printf "%s\n" "${a[@]}" ; } ; print
>> "${.sh.version}" ; nanosort <xxx >yyy'
>> Version AIJMP 93v- 2013-10-08
>> real 15.34
>> user 14.67
>> sys 0.52
>>
>> So your hunch that VMALLOC_OPTIONS=getmem=safe fixes the problem is
>> correct.
>>
>> What does VMALLOC_OPTIONS=getmem=safe do?
>
>
> vmalloc has an internal discipline/method for getting memory from the system
> several methods are available with varying degrees of thread safety etc.
> see src/lib/libast/vmalloc/vmdcsystem.c for the code
> and src/lib/libast/vmalloc/malloc.c for the latest VMALLOC_OPTIONS
> description (vmalloc.3 update shortly)
>
> ** getmemory=f enable f[,g] getmemory() functions if supported, all
> by default
> ** anon: mmap(MAP_ANON)
> ** break|sbrk: sbrk()
> ** native: native malloc()
> ** safe: safe sbrk() emulation via mmap(MAP_ANON)
> ** zero: mmap(/dev/zero)
>
> i believe the performance regression with "anon" is that on linux
> mmap(0....MAP_ANON|MAP_PRIVATE...),
> which lets the system decide the address, returns adjacent (when possible)
> region addresses from highest to lowest order
> and the reverse order at minimum tends to fragment more memory
> "zero" has the same hi=>lo characteristic
> i suspect it adversely affects the vmalloc coalescing algorithm but have not
> dug deeper
> for now the probe order in vmalloc/vmdcsystem.c was simply changed to favor
> "safe"
MAP_FIXED should be avoided because its only there for special
purposes like the runtime linker ld.so.1 or debuggers.
Using this for a general-purpose memory allocator causes serious problems:
1. On some systems this is a privileged operation and only available
for users with root privileges
2. SPARC T4 with 256GB and Solaris 11.1 the use of 'safe' degraded the
performance from 9 seconds to almost 15 minutes because it utterly
destroys the systems concept of large pages. If two MAP_FIXED mappings
follow directly each other the system downgrades the page size to the
smallest possible size, even trying to break up larger pages, which in
turn must be done by a special deamon (vmtasks)
3. MAP_PRIVATE|MAP_FIXED|MAP_ANON may no longer be available in future
versions of Solaris
4. Using the 'safe' allocator on SmartOS (solaris 11 clone) triggers a SEGV:
map(0xFFFFCD800B482000, 1048576, PROT_READ|PROT_WRITE,
MAP_PRIVATE|MAP_FIXED|MAP_ANON, 4294967295, 0) = 0xFFFFCD800B482000
sigaction(SIGSEGV, 0xFFFFFD7FFFDFDE50, 0xFFFFFD7FFFDFDED0) = 0
Incurred fault #6, FLTBOUNDS %pc = 0x0052FE06
siginfo: SIGSEGV SEGV_MAPERR addr=0xFFFFCD800B582000
Received signal #11, SIGSEGV [caught]
siginfo: SIGSEGV SEGV_MAPERR addr=0xFFFFCD800B582000
lwp_sigmask(SIG_SETMASK, 0x00000400, 0x00000000, 0x00000000,
0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
Irek
_______________________________________________
ast-developers mailing list
ast-developers@lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
