Re: Vim 7 performance notes

[Yakov Lerner]

On 2/4/07, Bram Moolenaar <[EMAIL PROTECTED]> wrote:

Alexei Alexandrov wrote:

> I'm doing some performance investigations of Vim code trying to
> understand whether there are any possibilities to improve it.
> Currently I've made the following observations (all investigations are
> done on Windows):
>
> Redundant work is done during regexp operations in syntax
> highlighting. On some files it is very noticable. The stack of the
> hotspot is ... > syn_current_attr > syn_regexec > vim_regexec_multi >
> vim_regexec_both > regtry > regmatch > ga_grow > alloc_clear > memset.
> So alloc_clear spends quite a few clockticks in lalloc() and memset().
> The reason for this is pessimistically big grow size for regset
> growing array:
>
>     ga_init2(&regstack, 1, 10000);
>
> This is not very good: many regexp operations don't go deep -
> non-match is detected very quickly. But even one element on the stack
> will lead to allocating at least 10000 bytes (which should be fast
> with good CRT memory allocator) and (worse) initializing these 10000
> bytes with zeros (won't be that fast).
>
> One possible solution would be to keep regstack alive across calls to
> vim_regexec_both, but I'm not sure if it's can be done safely. What I
> did was replacing the grow size with smaller number and making the
> grow size for growing arrays dynamic with increase of 25%:
>
> --- regexp.c (revision 136)
> +++ regexp.c (working copy)
> @@ -3350,7 +3350,7 @@
>      /* Init the regstack empty.  Use an item size of 1 byte, since we push
>       * different things onto it.  Use a large grow size to avoid reallocating
>       * it too often. */
> -    ga_init2(&regstack, 1, 10000);
> +    ga_init2(&regstack, 1, 64);
>
>      /* Init the backpos table empty. */
>      ga_init2(&backpos, sizeof(backpos_T), 10);
>
> --- misc2.c (revision 136)
> +++ misc2.c (working copy)
> @@ -1905,6 +1905,7 @@
>
>      {
>   if (n < gap->ga_growsize)
>       n = gap->ga_growsize;
> +        gap->ga_growsize += (gap->ga_growsize >> 2);
>   len = gap->ga_itemsize * (gap->ga_len + n);
>   pp = alloc_clear((unsigned)len);
>   if (pp == NULL)
>
> With this change I can see serious performance improvements, but I'm
> not sure if they are safe.
> Bram, does it look making any sense?

This is a tradeoff between allocating too much and allocating too often.
What works best completely depends on the implementation of malloc() and
free().  I know that there are many implementations where they are quite
slow.  Then allocating larger chunks less often is better.  On some
systems they are fast and the chunks can be smaller.

Gnu malloc (glibc) is exceptionally fast, iirc. It is possible
to benchmark the malloc speed during  the ./configure  time.
And auto-select the initital size depending on the results.

The procmail this similar technique in configure: It automatically
benchmarks  it's own builtin strstr() vs system's strstr() and selects
the one which is faster.

Yakov