Hi Phil, On Wed, Dec 16, 2015 at 10:14 AM, [email protected] <[email protected]> wrote:
> Eliot, > > The C code generator has been externalized, so it is not dependent on > VMMaker. > > http://www.smalltalkhub.com/#!/~PavelKrivanek/CCodeGenerator > > As far as we can have an ASM DSL that would be working along with that > generator, there is no reason why we couldn't have assembly code part > emitted by that CCodeGenerator as inlined asm bits ( > https://gcc.gnu.org/onlinedocs/gcc/Using-Assembly-Language-with-C.html) > > e.g. https://gcc.gnu.org/onlinedocs/gcc/Extended-Asm.html#Extended-Asm > Yes, but you have to invoke gcc to translate the C and extended asm into assembler, and an assembler to translate that output into machine code, except that the assembler produces object files, not raw machine code. > > So, AsmJit syntax would extend CCodeGenerator and output C and inlined asm > that would then be compiled and invoked with FFI ( > https://clementbera.wordpress.com/2013/06/19/optimizing-pharo-to-c-speed-with-nativeboost-ffi/ > ) > > This may not be the use case for writing FFI but that is a use case for > writing and invoking ASMx86 in Smalltalk (or whatever else, based on the > AsmJIT syntax for a given architecture + backend for CCodeGenerator). > > What do you think? > I don't think it solves the problem. AsmJIT is a source to machine-code translator but Slang is a source to source translator. So I don't see how it solves the key problem, which is to generate the machine code for FFI call argument marshalling according to the rules of the underlying platform's ABI. Maybe I'm being dense, but it doesn't seem to me to be relevant. Right now, the Cog JIT contains an assembler and translates byte coded methods into sequences of assembler instructions and these into machine code embedded in CogMethod objects in the VM. AsmJIT contains an assembler which translates assembler instructions to machine code and embeds those machine code sequences in the trailer of compiled methods and then contains a hack to get the Cog JIT to call that machine code. None of these assembler to machine code steps are provided (at least not conveniently) by your proposed tool chain above. There is obvious duplication in having Cog contain an assembler and AsmJIT contain an assembler. But Cog also contains an escape sequence of byte codes we are using to generate efficient, unsafe code. This can be used to generate the marshalling code we need, and do it in a cross-platform manner, avoiding the duplication inherent in AsmJIT above Cog, and avoiding the hacks necessary to get Cog to be able to use the code embedded in method trailers (such as making the entire heap executable, and hacking Cog to call these escape sequences). I'm making myself parse here articulating what I think is an elegant, general and powerful architecture. But people find it boring. I don't understand. I think I'm going to shut up :-/ Phil > > On Wed, Dec 16, 2015 at 6:06 PM, Eliot Miranda <[email protected]> > wrote: > >> Hi Phillippe, >> >> On Wed, Dec 16, 2015 at 2:14 AM, [email protected] < >> [email protected]> wrote: >> >>> Couldn't we reuse the AsmJIT DSL and generate some C code with Asm >>> directives through Slang? >>> >> >> C is too high-level for the intended purposes (e.g. performing FFI calls) >> and Slang is not at all genial purpose or suitable for being used at >> run-time. Its tightly coupled to the VMaker. >> >> And call that C code through FFI? >>> >> >> Since the main use case for this system is to implement the marshalling >> code for FFI calls this doesn't work. >> >>> Seems cleaner and more debuggable. >>> And could be moved to x64. >>> >>> Phil >>> >>> On Dec 16, 2015 6:49 AM, "Clément Bera" <[email protected]> wrote: >>> > >>> > >>> > >>> > 2015-12-15 20:22 GMT+01:00 Jan Vrany <[email protected]>: >>> >> >>> >> Hi Esteban, Eliot, Clemont >>> >> >>> >> thanks very much for elaborate answer. I'm aware of all what you said >>> >> (well, most :-), but my actual needs are very low and primitive. >>> >> >>> >> All I need now is something that allows me to turn x86_64 assembly >>> >> into a machine code using a sane Smalltalk API, which I can later copy >>> >> to VM code space. Nothing more, nothing less :-) I do not fancy >>> >> spending time writing yet-another-x86-assembler hence my interest in >>> >> asmjit. >>> > >>> > >>> > As far as I know AsmJIT has stable support for x86 but not x86_64 >>> >> >>> >> >>> >> Thanks! Jan >>> >> >>> >> >>> >> >>> >> On Tue, 2015-12-15 at 18:34 +0100, Clément Bera wrote: >>> >> > We're getting away from Jan initial question here ... >>> >> > >>> >> > I think the point here is that Cog has: >>> >> > - 2 stable back ends: ARMv5 and x86 >>> >> > - 2 backends stable in the simulator with production planned for ~ >>> >> > April: x64 and MIPS >>> >> > and Tim is willing to do the ARMv8 backend. >>> >> > >>> >> > All the 4, and soon 5, cog back ends are maintained: >>> >> > - x86 and x64 maintained by Eliot >>> >> > - MIPS maintained by Ryan >>> >> > - ARMv5 and soon ARMv8 maintained by Tim >>> >> > >>> >> > Each backend requires months of work. >>> >> > >>> >> > Do you want to maintain 10 backends instead of 5 ? Do want to spend >>> >> > time to implement 5 backends or 10 ? >>> >> > >>> >> > I don't. Only the x86 backend is duplicated right now in AsmJIT. >>> >> > >>> >> > So one idea is to reuse Cog's backends from the image instead of >>> >> > AsmJIT. To do so, we can use encodings available in the sista >>> >> > extended instruction set to tell Cog what machine code to generate. >>> A >>> >> > project named uFFI aims, among other things, at doing this, by >>> >> > providing a unified interface between the Cog backends and AsmJIT. >>> >> > >>> >> > >>> >> > 2015-12-15 16:43 GMT+01:00 Eliot Miranda <[email protected]>: >>> >> > > Hi Jan, >>> >> > > >>> >> > > > On Dec 15, 2015, at 3:06 AM, Jan Vrany <[email protected]> >>> >> > > wrote: >>> >> > > > >>> >> > > > Hi guys, >>> >> > > > >>> >> > > > two queations: >>> >> > > > >>> >> > > > (i) Is AsmJit going to be developed any more or it's abandoned >>> >> > > > as well as native boost? >>> >> > > >>> >> > > AsmJIT is effectively being abandoned but NativeBoost is not. >>> >> > > >>> >> > > The key limitation of AsmJIT is that it was not designed to be >>> >> > > cross platform; it is effectively an x86 assembler. As such it's >>> >> > > use gets in the way of ARM and x86_64 (I am currently getting the >>> C >>> >> > > version of 64-bit Cog Spur working on x86_64, given that it is >>> >> > > working in the simulator). >>> >> > > >>> >> > > Another limitation is that it doesn't play that well with the VM's >>> >> > > JIT. Igor and I never managed to work on integrating it better. >>> >> > > The VM's job is managing code and Igor's approach was to hack; >>> >> > > eliminating execution protection in the entire heap, instead of >>> >> > > extending the support that either the Alien plugin's callback >>> >> > > support or the JIT's executable method zone provides. Making the >>> >> > > entire heap executable is /not/ a sensible approach. >>> >> > > >>> >> > > But there is a better way! A key component of the Sista adaptive >>> >> > > optimizer/speculative inlined that Clément is currently >>> stabilizing >>> >> > > (!!) is a set of bytecodes that encode unsafe operations like >>> >> > > at:put: without bounds, type or store checks. For example, the >>> >> > > normal at:put: is about a hundred instructions, checks for >>> >> > > smallinteger indices, differentiates between byte, 32-but long and >>> >> > > pointer objects and does a store check. But one of the Sista codes >>> >> > > for at:put: generates about two instructions, one to adjust the >>> >> > > index, the other to do the store. Distaste job is to analyze code >>> >> > > and inline methods using these unsafe bytecodes where they are >>> >> > > proven to be safe, hence increasing performance. >>> >> > > >>> >> > > Unlike AsmJIT, Sista's unsafe bytecodes are cross platform, and, >>> >> > > being executed by the VM, can work on an interpreter VM or be >>> >> > > converted to machine code by the JIT. >>> >> > > >>> >> > > So our plan is to extend these bytecodes with ones that support >>> >> > > marshaling arguments for NativeBoost calls. Ronie Salgado has >>> >> > > already extended his lowcode scheme to define these instructions >>> >> > > and sometime soon (hopefully 2016) we shall rewrite NativeBoost to >>> >> > > target these bytecodes. >>> >> > > >>> >> > > HTH >>> >> > > >>> >> > > > (ii)Where can I find latest AsmJit? I'm properly confused: >>> >> > > > >>> >> > > > * Is is the one in latest Pharo 4.0 (5.0) image? >>> >> > > > * Is it the one here: >>> http://smalltalkhub.com/#!/~Pharo/AsmJi >>> >> > > t ? >>> >> > > > (the one in the image seem to be based on completely >>> >> > > disjunct >>> >> > > > set of .mcz than those in the repo above). >>> >> > > > >>> >> > > > Best, Jan >>> >> > > >>> >> > > _,,,^..^,,,_ (phone) >>> >> _,,,^..^,,,_ >> best, Eliot >> > _,,,^..^,,,_ flummoxed, Eliot
