Bram, if you're going to be a doing a fair amount of work on the vimscript
side of things, one thing I would like to request is to have a mind towards
being able to support a full-on vimscript debugger. I realise there is a
built-in debugging mode, but it would be exceptionally cool to be able to
implement a Debug Adapter Protocol
<https://microsoft.github.io/debug-adapter-protocol/> server within vim, so
that something like Vimspector <https://github.com/puremourning/vimspector>
could
be used to debug plugin code externally.
Obviously doesn't have to be done right away, but having it as a medium
term goal would be great. As always, happy to contribute.
On Tuesday, December 17, 2019 at 4:49:16 PM UTC, Bram Moolenaar wrote:
>
>
> Vim 8.2 has been released, so what is next?
>
>
> 1. FASTER VIM SCRIPT
>
> The third item on the poll results of 2018, after popup windows and text
> properties, is faster Vim script. So how do we do that?
>
> I have been throwing some ideas around, and soon came to the conclusion
> that the current way functions are called and executed, with
> dictionaries for the arguments and local variables, is never going to be
> very fast. We're lucky if we can make it twice as fast. The overhead
> of a function call and executing every line is just too high.
>
> So what then? We can only make something fast by having a new way of
> defining a function, with similar but different properties of the old
> way:
> - Arguments are only available by name, not through the a: dictionary or
> the a:000 list.
> - Local variables are not available in an l: dictionary.
> - A few more things that slow us down, such as exception handling details.
>
> I Implemented a "proof of concept" and measured the time to run a simple
> for loop with an addition (Justin used this example in his presentation,
> code is below):
>
> Vim old function: 5.018541
> Python: 0.369598
> Lua: 0.078817
> Vim new function: 0.073595
>
> That looks very hopeful! It's just one example, but it shows how much
> we can gain, and also that Vim script can be faster than builtin
> interfaces.
>
> How does this work? The function is first compiled into a sequence of
> instructions. Each instruction has one or two parameters and a stack is
> used to store intermediate results. Local variables are also on the
> stack, space is reserved during compilation. This is a fairly normal
> way of compilation into an intermediate format, specialized for Vim,
> e.g. each stack item is a typeval_T. And one of the instructions is
> "execute Ex command", for commands that are not compiled.
>
>
> 2. PHASING OUT INTERFACES
>
> Attempts have been made to implement functionality with built-in script
> languages such as Python, Perl, Lua, Tcl and Ruby. This never gained much
> foothold, for various reasons.
>
> Instead of using script language support in Vim:
> - Encourage implementing external tools in any language and communicate
> with them. The job and channel support already makes this possible.
> Really any language can be used, also Java and Go, which are not
> available built-in.
> - Phase out the built-in language interfaces, make maintenance a bit
> easier
> and executables easier to build. They will be kept for backwards
> compatibility, no new features.
> - Improve the Vim script language, so that it can be used when an
> external tool is undesired.
>
> All together this creates a clear situation: Vim with the +eval feature
> will be sufficient for most plugins, while some plugins require
> installing a tool that can be written in any language. No confusion
> about having Vim but the plugin not working because some specific
> language is missing. This is a good long term goal.
>
> Rationale: Why is it easier to run a tool separately from Vim than using a
> built-in interface and interpreter? Take for example something that is
> written in Python:
> - The built-in interface uses the embedded python interpreter. This is
> less
> well maintained than the python command. Building Vim with it requires
> installing developer packages. If loaded dynamically there can be a
> version
> mismatch.
> - When running the tool externally the standard python command can be
> used,
> which is quite often available by default or can be easily installed.
> - A .py file can be compiled into a .pyc file and execute much faster.
> - Inside Vim multi-threading can cause problems, since the Vim core is
> single
> threaded. In an external tool there are no such problems.
> - The Vim part is written in .vim files, the Python part is in .py files,
> this
> is nicely separated.
> - Disadvantage: An interface needs to be made between Vim and Python.
> JSON is available for this, and it's fairly easy to use. But it still
> requires implementing asynchronous communication.
>
>
> 3. BETTER VIM SCRIPT
>
> To make Vim faster a new way of defining a function needs to be added.
> While we are doing that, since the lines in this function won't be fully
> backwards compatible anyway, we can also make Vim script easier to use.
> In other words: "less weird". Making it work more like modern
> programming languages will help. No surprises.
>
> A good example is how in a function the arguments are prefixed with
> "a:". No other language I know does that, so let's drop it.
>
> It should be possible to convert code from other languages to Vim
> script. We can add functionality to make this easier. This still needs
> to be discussed, but we can consider adding type checking and a simple
> form of classes. If you look at JavaScript for example, it has gone
> through these stages over time, adding real class support and now
> Typescript adds type checking. But we'll have to see how much of that
> we actually want to include in Vim script. Ideally a conversion tool
> can take Python, Javascript or Typescript code and convert it to Vim
> script, with only some things that cannot be converted.
>
> Vim script won't work the same as any specific language, but we can use
> mechanisms that are commonly known, ideally with the same syntax. One
> thing I have been thinking of is assignments without ":let". I often
> make that mistake (after writing JavaScript especially). I think it is
> possible, if we make local variables shadow commands. That should be OK,
> if you shadow a command you want to use, just rename the variable.
> Using "let" and "const" to declare a variable, like in JavaScript and
> Typescript, can work:
>
> def MyFunction(arg)
> let local = 1
> const ADD = 88
> while arg > 0
> local += ADD
> --arg
> endwhile
> return local
> enddef
>
> Just some ideas, this will take time to design, discuss and implement.
> Eventually this will lead to Vim 9!
>
>
> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
>
> Code for the timing measurements (build with -O2):
>
> func VimOld()
> let sum = 0
> for i in range(1, 2999999)
> let sum += i
> endfor
> return sum
> endfunc
>
> func Python()
> py3 << END
> sum = 0
> for i in range(1, 3000000):
> sum += i
> END
> return py3eval('sum')
> endfunc
>
> func Lua()
> lua << END
> sum = 0
> for i = 1, 2999999 do
> sum = sum + i
> end
> END
> return luaeval('sum')
> endfunc
>
> def VimNew()
> let sum = 0
> for i in range(1, 2999999)
> let sum += i
> endfor
> return sum
> enddef
>
> let start = reltime()
> echo VimOld()
> echo 'Vim old: ' .. reltimestr(reltime(start))
>
> let start = reltime()
> echo Python()
> echo 'Python: ' .. reltimestr(reltime(start))
>
> let start = reltime()
> echo Lua()
> echo 'Lua: ' .. reltimestr(reltime(start))
>
> let start = reltime()
> echo VimNew()
> echo 'Vim new: ' .. reltimestr(reltime(start))
>
> A couple of remarks:
> - Python is picky about the indent, has to be left aligned.
> - Lua doesn't support "sum += i".
>
> --
> Facepalm statement #6: "Estland is a fantasy place, just like Middle Earth
> and
> Madagaskar"
>
> /// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net
> \\\
> /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/
> \\\
> \\\ an exciting new programming language -- http://www.Zimbu.org
> ///
> \\\ help me help AIDS victims -- http://ICCF-Holland.org
> ///
>
--
--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
---
You received this message because you are subscribed to the Google Groups
"vim_dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/vim_dev/9aff18df-b3e0-4523-8ca6-a1c8d1e102e1%40googlegroups.com.
