On Sun, Dec 1, 2013 at 4:58 PM, Lionel Cons <[email protected]>wrote:
> On 1 December 2013 17:26, Glenn Fowler <[email protected]> wrote: > > I believe this is related to vmalloc changes between 2013-05-31 and > > 2013-06-09 > > re-run the tests with > > export VMALLOC_OPTIONS=getmem=safe > > if that's the problem then it gives a clue on a general solution > > details after confirmation > > > > timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; > while read i ; do key="$i$((k++))" ; a["$key"]="$i" ; done ; printf > "%s\n" "${a[@]}" ; } ; print "${.sh.version}" ; nanosort <xxx >yyy' > Version AIJMP 93v- 2013-10-08 > > real 34.60 > user 33.27 > sys 1.19 > > VMALLOC_OPTIONS=getmem=safe timex ~/bin/ksh -c 'function nanosort { > typeset -A a ; integer k=0; while read i ; do key="$i$((k++))" ; > a["$key"]="$i" ; done ; printf "%s\n" "${a[@]}" ; } ; print > "${.sh.version}" ; nanosort <xxx >yyy' > Version AIJMP 93v- 2013-10-08 > real 15.34 > user 14.67 > sys 0.52 > > So your hunch that VMALLOC_OPTIONS=getmem=safe fixes the problem is > correct. > > What does VMALLOC_OPTIONS=getmem=safe do? vmalloc has an internal discipline/method for getting memory from the system several methods are available with varying degrees of thread safety etc. see src/lib/libast/vmalloc/vmdcsystem.c for the code and src/lib/libast/vmalloc/malloc.c for the latest VMALLOC_OPTIONS description (vmalloc.3 update shortly) ** getmemory=f enable f[,g] getmemory() functions if supported, all by default ** anon: mmap(MAP_ANON) ** break|sbrk: sbrk() ** native: native malloc() ** safe: safe sbrk() emulation via mmap(MAP_ANON) ** zero: mmap(/dev/zero) i believe the performance regression with "anon" is that on linux mmap(0....MAP_ANON|MAP_PRIVATE...), which lets the system decide the address, returns adjacent (when possible) region addresses from highest to lowest order and the reverse order at minimum tends to fragment more memory "zero" has the same hi=>lo characteristic i suspect it adversely affects the vmalloc coalescing algorithm but have not dug deeper for now the probe order in vmalloc/vmdcsystem.c was simply changed to favor "safe"
_______________________________________________ ast-users mailing list [email protected] http://lists.research.att.com/mailman/listinfo/ast-users
