Hi, I've been thinking some about native compilation, and how to do ahead-of-time and just-in-time compilation, and not totally explode Guile's mental footprint -- the number of cases and things that one has to keep in mind when debugging Guile code.
Currently, Guile has a compiler to a custom virtual machine, and the associated toolchain: assemblers and disassemblers, stack walkers, the debugger, etc. One can get the source location of a particular instruction pointer, for example. It's all mostly understandable, and mostly uniform across platforms (modulo word size and byte order, and perhaps we should make that uniform as well, especially if we can do native compilation.) So, my thought is to extend procedures with an additional pointer, a pointer to a "native" code structure. The native code could be written out ahead-of-time, or compiled at runtime. But procedures would still have bytecode, for various purposes for example to enable code coverage via the next-instruction hook, and in the JIT case, because only some procedures will be native-compiled. We keep the same stack representation, so stack walkers and the debugger still work. Some local variables can be allocated into registers, but procedure args are still passed and returned on the stack. Though the procedure's arity and other metadata would be the same, the local variable allocations and source locations would differ, so we would need some additional debugger support, but we can work on that when the time comes. All Scheme procedures, bytecode and native, will run inside the VM. If a bytecode procedure calls a native procedure, the machine registers are saved, and some machine-specific stub transfers control to the native code. Native code calling native code uses the same stack as the VM, though it has its own conventions over what registers to save; and native code calling bytecode prepares the Scheme stack, then restores the VM-saved machine registers. In this way we can do incremental experimentation with compiled procedures, compiling at runtime, or ahead-of-time. Now, what technology to choose for the compiler itself? Dunno. For a JIT, it would be useful to use something portable, and perhaps do the JIT compilation on the bytecode itself, without more source information. It would not produce the fastest code, but it would run fast. AIUI the hotspot compiler actually does an SSA transformation of Java bytecode, then works on that. I'm not particularly interested in something like that; I'm more interested in something direct and fast, and obviously correct and understandable by our debugging infrastructure. I think we can produce better native code ahead-of-time coming from the tree-il layer directly. I feel like eventually we'll need to replace GLIL with something else, but I don't really know; we'll find out in the future I guess. But I do want to do ahead-of-time compilation, because I want Guile programs to start up very quickly and not consume much memory. Anyway, just some thoughts here. I'm not going to focus on native compilation in the coming months, as there are other things to do, but this is how I think it should be done :-) Cheers, Andy -- http://wingolog.org/
