edit src/lib/libast/vmalloc/vmmaddress.c and change
#define VMCHKMEM 0
this affects vmalloc detecting overbooked memory but will disable the
MAP_FIXED codepath
On Thu, Dec 5, 2013 at 4:50 PM, Irek Szczesniak <iszczesn...@gmail.com>wrote:
> On Wed, Dec 4, 2013 at 3:02 PM, Glenn Fowler <glenn.s.fow...@gmail.com>
> wrote:
> > On Sun, Dec 1, 2013 at 4:58 PM, Lionel Cons <lionelcons1...@gmail.com>
> > wrote:
> >>
> >> On 1 December 2013 17:26, Glenn Fowler <glenn.s.fow...@gmail.com>
> wrote:
> >> > I believe this is related to vmalloc changes between 2013-05-31 and
> >> > 2013-06-09
> >> > re-run the tests with
> >> > export VMALLOC_OPTIONS=getmem=safe
> >> > if that's the problem then it gives a clue on a general solution
> >> > details after confirmation
> >> >
> >>
> >> timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0;
> >> while read i ; do key="$i$((k++))" ; a["$key"]="$i" ; done ; printf
> >> "%s\n" "${a[@]}" ; } ; print "${.sh.version}" ; nanosort <xxx >yyy'
> >> Version AIJMP 93v- 2013-10-08
> >>
> >> real 34.60
> >> user 33.27
> >> sys 1.19
> >>
> >> VMALLOC_OPTIONS=getmem=safe timex ~/bin/ksh -c 'function nanosort {
> >> typeset -A a ; integer k=0; while read i ; do key="$i$((k++))" ;
> >> a["$key"]="$i" ; done ; printf "%s\n" "${a[@]}" ; } ; print
> >> "${.sh.version}" ; nanosort <xxx >yyy'
> >> Version AIJMP 93v- 2013-10-08
> >> real 15.34
> >> user 14.67
> >> sys 0.52
> >>
> >> So your hunch that VMALLOC_OPTIONS=getmem=safe fixes the problem is
> >> correct.
> >>
> >> What does VMALLOC_OPTIONS=getmem=safe do?
> >
> >
> > vmalloc has an internal discipline/method for getting memory from the
> system
> > several methods are available with varying degrees of thread safety etc.
> > see src/lib/libast/vmalloc/vmdcsystem.c for the code
> > and src/lib/libast/vmalloc/malloc.c for the latest VMALLOC_OPTIONS
> > description (vmalloc.3 update shortly)
> >
> > ** getmemory=f enable f[,g] getmemory() functions if supported,
> all
> > by default
> > ** anon: mmap(MAP_ANON)
> > ** break|sbrk: sbrk()
> > ** native: native malloc()
> > ** safe: safe sbrk() emulation via
> mmap(MAP_ANON)
> > ** zero: mmap(/dev/zero)
> >
> > i believe the performance regression with "anon" is that on linux
> > mmap(0....MAP_ANON|MAP_PRIVATE...),
> > which lets the system decide the address, returns adjacent (when
> possible)
> > region addresses from highest to lowest order
> > and the reverse order at minimum tends to fragment more memory
> > "zero" has the same hi=>lo characteristic
> > i suspect it adversely affects the vmalloc coalescing algorithm but have
> not
> > dug deeper
> > for now the probe order in vmalloc/vmdcsystem.c was simply changed to
> favor
> > "safe"
>
> MAP_FIXED should be avoided because its only there for special
> purposes like the runtime linker ld.so.1 or debuggers.
>
> Using this for a general-purpose memory allocator causes serious problems:
> 1. On some systems this is a privileged operation and only available
> for users with root privileges
>
> 2. SPARC T4 with 256GB and Solaris 11.1 the use of 'safe' degraded the
> performance from 9 seconds to almost 15 minutes because it utterly
> destroys the systems concept of large pages. If two MAP_FIXED mappings
> follow directly each other the system downgrades the page size to the
> smallest possible size, even trying to break up larger pages, which in
> turn must be done by a special deamon (vmtasks)
>
> 3. MAP_PRIVATE|MAP_FIXED|MAP_ANON may no longer be available in future
> versions of Solaris
>
> 4. Using the 'safe' allocator on SmartOS (solaris 11 clone) triggers a
> SEGV:
> map(0xFFFFCD800B482000, 1048576, PROT_READ|PROT_WRITE,
> MAP_PRIVATE|MAP_FIXED|MAP_ANON, 4294967295, 0) = 0xFFFFCD800B482000
> sigaction(SIGSEGV, 0xFFFFFD7FFFDFDE50, 0xFFFFFD7FFFDFDED0) = 0
> Incurred fault #6, FLTBOUNDS %pc = 0x0052FE06
> siginfo: SIGSEGV SEGV_MAPERR addr=0xFFFFCD800B582000
> Received signal #11, SIGSEGV [caught]
> siginfo: SIGSEGV SEGV_MAPERR addr=0xFFFFCD800B582000
> lwp_sigmask(SIG_SETMASK, 0x00000400, 0x00000000, 0x00000000,
> 0x00000000) = 0xFFBFFEFF [0xFFFFFFFF]
>
> Irek
>
_______________________________________________
ast-developers mailing list
ast-developers@lists.research.att.com
http://lists.research.att.com/mailman/listinfo/ast-developers
