Prabir Shrestha wrote:
> Glad to see you are already thinking of vim 9 and it is good to see the > rythm of having a new goal ever year. I was personally waiting to hear more > people comment on the ECMAScript > (https:/ta.github.io/WasmExplorer//github.com/vim/vim/pull/5198) support > before I share more details but here it goes. > > > Just one minor thought, have you thought about storing the intermediate Vim > > script format into a new binary file (similar to how python uses .py and > > .pyc)? > > This is exactly what WASM (WebAssembly) is. Why not make WASM a first class > citizen in Vim 9 instead? > > FASTER VIMSCRIPT wasm does a fabulous job here and was designed for this. > Here is an exact quote from their github repo. > https://github.com/WebAssembly/design I have considered using something like WASM. WASM is a very low level execution engine. It has no concept of higher level data structures, such as lists and dictionaries. It would require a very complex compiler to turn a Vim script into WASM instructions. Not something we would include in the Vim binary. It would require an external WASM compiler, probably with some intermediate format and a final step that produces the binary code. And then Vim needs to provide a library to interface WASM code with the rest of Vim. That's going to be very complicated. What I'm aiming for is using very high level instructions, such as "create a list". It is executed by the C code, thus as fast as it can be. The result is a variable of typval_T, which we already have. Thus it integrates very well with existing code and can be passed to Vim script lines that are not compiled. This is something I can implement in a few months, it is feasible (I implemented that "create a list" instruction in 10 minutes). > > "WebAssembly is efficient and fast: Wasm bytecode is designed to be > > encoded in a size- and load-time-efficient binary format. WebAssembly aims > > to execute at native speed by taking advantage of common hardware > > capabilities available on a wide range of platforms." > > Currently the new function prototype only makes function fast. There will > be lot more work to make rest of the features fast. While I would love > vimscript to be fast I'm not sure if it worth it. As I mentioned, it only makes sense to compile code that is executed more than once. That is functions, autocommands and user commands. If you do have a loop at the script level, you can put it in a function and then call that function. This nicely separates the old, backwards compatible (and necessarily slow) Vim script from the new fast Vim script. > I would rather have > vimscript compile to WASM and then execute. This way we only think about > making the runtime fast. There are also lot of WASM runtimes currently some > of which even supports JIT and native compilation and other optimizations. > You can try this website and see what native assembly code it generates. > https://mbebenita.github.io/WasmExplorer/ As mentioned above, no matter how nice WASM is, compiling Vim script into it appears to be infeasible. > PHASING OUT INTERFACES > Lot of languages now supports compiling to WASM and this would solve the > issue of having multiple language interfaces. Another option would be to > use jobs and channels but currently someone still needs to write a small > wrapper to take full advantage of this. Neovim has a remote RPC concept and > official libraries for their languages and hence there is more abundance of > the neovim plugins taking advantage of this and doing more complicated > stuff then vim8 plugins. I personally don't use any of these interfaces but > there are lot of people who use python plugins just so they can have the > speed or use libraries that come with languages. > WASM fixes this issues about supporting someones best language. Not sure how this relates to WASM, you can write a tool in any language. Neovim needs a complex RPC mechanism because they externalize things so you can run a separate UI. Plugins can do with a much simpler interface. You mention "I personally don't use any of these interfaces", which I think is typical for plugin writers. When you write a UI on top of Neovim then you would need that RPC mechanism. > BETTER VIMSCRIPT > This is the one I'm worried the most. I would personally be against > breaking the language compatibility as it would be pain for us to support > as a plugin author. There is no such thing, old Vim scripts will just keep working. > I don't think comparing javascript with vimscript on adding breaking > languages is the right comparison. While most devs might write in the ES6 > or newer they either compile it down to ES3 or ES5 so it works in older > browsers or they compile it to both versions but give hints to browsers to > only pick one of them so that it continues to work in older browsers. > > Here is an example: > <script src="es5.js" nomodule"></script> > <script src="es6.js" type="module"></script> > > > Old browsers will only use es5.js because if type is not specified it > automatically treats it as text/javascript and nomodule is not understood > by them and they just ignore it. es6.js has type module so while it may > download that file it doesn't know how to execute it so in terms of > executing it becomes a noop. For mordern browsers, they understand nomodule > so it doesn't download es5.js but understands type=module and treats it as > javascript file and executes it. How would vimscript do this? Same for > typescript, it compiles down to either ES5 or ES6 or some variant of > javascript with mostly typings removed. It doesn't matter what your ES6 is translated into, what matters is what you write. As it happens, the way variables are declared with "let" and "const" matches with what Vim script does, thus we can use that concept. Many other things from ES6 won't ever make it into Vim script, either because we prefer a mechanism from another language or we don't need it. We'll have to see what works best. > Typing is also something I miss a lot in vimscript I'm going to experiment with this to see how well it works. It's one of those things that some people consider essential while others are annoyed by it. If you want to discuss it, please join the Python discussions on this topic... > because as a vim plugin author I would most likely > continue to use func instead of def even if def is faster if I wanted to > support vimscript. If I really wanted it to be fast I would rather continue > to use Python or lua interfaces or WASM. I personally prefer > WASM>LUA>VimScript. Lua is a good alternative if it ships in vim primarily > because the language it self is very much set and is very powerful with > features such as co-routines allowing us to express async code in an easy > way (this is actually my personal pain compared to speed of vimscript). Lua is not a popular language. It doesn't even support "var += i", as I found out from writing the example. Python is a lot more popular, but the embedding doesn't work that great. And it's quite slow, as my measurements also show. The embedded Lua also isn't that fast either, you probably need to run Lua as a separate binary for that. We just have to come to the conclusion that plugin writers don't use the interfaces much, so let's phase them out. Multi-threading and coroutines are very complex mechanisms that do not fit well with the Vim core. Would be an awful lot of work to implement. Adding a language interface doesn't solve that. I do miss it sometimes for functionality that really is asynchronous. Maybe for Vim 10? So write a tool in any language you like and communicate with it from Vim. In Vim we'll just use Vim script. > To me I think vim9 should all be about WASM. This solves everything > mentioned eariler in the thread and in a better way without too much work. > > Why? > * It is fast, really fast. This will only continue to be better due to the > energy the current industry is betting on. Create a compiler to compile > vimscript to wasm How fast can WASM write a file? Or change a line in a buffer? It doesn't do any of that, it only provides a way to run code in a portable way. That compiler from Vim script to WASM would be very complex and I think it would take too long to implement. And then still it would have to run as a separate binary and require installing WASM support. It makes much more sense for a plugin to write code in any language, communicate with Vim using the existing communication mechanisms, and run it as a separate tool. It can perhaps be compiled to WASM to make the tool more portable, assuming one has a WASM runtime (is there a WRE like there is JRE for Java?). But when Vim script is much faster (think 100 times) it will be good enough for lots of plugins. Something that now takes 3 seconds becomes 30 msec, goes from slow to unnoticeable. > * I can write in whatever language I want solving the language interface > problem. WASM is most likely also officially supported by your favorite > language. > * WASI Interface > https://hacks.mozilla.org/2019/03/standardizing-wasi-a-webassembly-system-interface/ > > Think of this as standard c library which is portable. > * WASM threads https://github.com/WebAssembly/threads ex: using immutable > data structures to fuzzy search effeciently by sharing between threads. > * WASM webgl https://rustwasm.github.io/wasm-bindgen/examples/webgl.html > Similar to popup window but imagine getting a raw canvas to draw in gui vim > ex: rendering markdown files. Who knows might be one day we will get 3d > games running inside vim :) > * WASM c api https://github.com/WebAssembly/wasm-c-api Once this takes off > I'm hoping it will be easy to swap out runtimes into a faster runtime. All that doesn't seem useful to execute Vim script. You can use it for the tool you are writing, but you still need to communicate with Vim. Perhaps Vim can include a WASM runtime to execute that code in a separate thread, but it won't be different from running it as a separate process. -- GALAHAD: No. Look, I can tackle this lot single-handed! GIRLS: Yes, yes, let him Tackle us single-handed! "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD /// Bram Moolenaar -- [email protected] -- http://www.Moolenaar.net \\\ /// sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org /// \\\ help me help AIDS victims -- http://ICCF-Holland.org /// -- -- You received this message from the "vim_dev" maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php --- You received this message because you are subscribed to the Google Groups "vim_dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/vim_dev/201912181226.xBICQhsB002372%40masaka.moolenaar.net.
