Re: Is it possible to use DMD as a library to compile strings at runtime?
On Monday, 10 February 2020 at 12:31:03 UTC, Basile B. wrote: On Friday, 31 January 2020 at 14:25:30 UTC, Basile B. wrote: [...] about [1] (llvm) I've made a better binding this weekend: https://gitlab.com/basile.b/llvmd-d Seriouly I cant believe that at some point in the past I translated by hand. dstep can handle big C libraries. ah ah ah, I meant https://gitlab.com/basile.b/llvm-d
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Friday, 31 January 2020 at 14:25:30 UTC, Basile B. wrote: On Friday, 31 January 2020 at 11:19:37 UTC, Saurabh Das wrote: I see that DUB has DMD as a library package, but I was not able to understand how to use it. Is it possible to use DMD as a library within a D program to compile a string to machine code and run the compiled code at runtime? Thanks, Saurabh Fundamentally DMD as a library is a front-end. Jitting is to the backend side. You'll be able to lex and parse the source to get an AST, to perform the semantic passes on this AST and that's all. Then to run this code you would need to make an AST visitor that will generate the binary code to execute. Even using a specialized library with jitting abilities, such as LLVM-d [1] or libfirm-d [2], this would be *quite* a journey. [1] https://github.com/MoritzMaxeiner/llvm-d [2] https://gitlab.com/basile.b/libfirm-d about [1] (llvm) I've made a better binding this weekend: https://gitlab.com/basile.b/llvmd-d Seriouly I cant believe that at some point in the past I translated by hand. dstep can handle big C libraries.
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Sunday, 2 February 2020 at 06:03:01 UTC, H. S. Teoh wrote: On Sun, Feb 02, 2020 at 03:16:46AM +, Saurabh Das via Digitalmars-d-learn wrote: [...] [...] > [...] [...] [...] [...] It's very simple. Let's say you have your code in some string called 'code'. Since dmd nowadays can take stdin as input (specify "-" as input filename), all you have to do is to assemble your dmd command and use std.process's awesome API to run it: [...] Thanks. Will try this out. Saurabh
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Sun, Feb 02, 2020 at 03:16:46AM +, Saurabh Das via Digitalmars-d-learn
wrote:
> On Saturday, 1 February 2020 at 20:37:03 UTC, H. S. Teoh wrote:
[...]
> > I've actually done this before in an equation grapher program: the
> > user inputs an equation, the program generates D code to compute the
> > equation, then runs dmd to compile it into a shared library, and
> > opens the shared library and looks up the symbol to execute the
> > compiled code. Dmd is fast enough that this actually works fairly
> > well. When the input to dmd is small, it's so fast you don't even
> > notice it.
[...]
> This approach seems more tractable at present. Would you have any
> example code lying around for this?
[...]
It's very simple. Let's say you have your code in some string called
'code'. Since dmd nowadays can take stdin as input (specify "-" as input
filename), all you have to do is to assemble your dmd command and use
std.process's awesome API to run it:
/*
* Step 1: Compile the code
*/
string code = ...;
auto cmd = [
"/usr/bin/dmd", // or wherever your dmd is
"-O", // or whatever other flags you need
"-fPIC", "-shared", // this is important
"-of" ~ soFilename, // specify output filename
"-" // read from stdin
]
// This part is a bit involved because we have to spawn the
// compiler as a child process then write our code string into
// its stdin.
// Alternatively, just write your code into a temporary file and
// pass the filename to dmd, then you can just use
// std.process.execute() which has a much simpler API.
import std.process : pipeProcess, Redirect, wait;
auto pipes = pipeProcess(cmd, Redirect.stdin | Redirect.stdout |
Redirect.stderrToStdout);
// Send code to compiler
pipes.stdin.write(code);
pipes.stdin.flush();
pipes.stdin.close();
// Read compiler output (optional)
auto app = appender!string();
enum chunkSize = 4096;
pipes.stdout.byChunk(chunkSize)
.copy(app);
// Wait for compiler to finish
auto status = wait(pipes.pid);
auto output = app.data;
if (status != 0)
throw new Exception("Failed to compile code:\n" ~ output);
/*
* Step 2: Load the compiled library.
*/
// This is for Posix; replace with Windows equivalent if you're
// on Windows.
auto libhandle = dlopen(soFilename.toStringz, RTLD_LAZY | RTLD_LOCAL);
if (libhandle is null) ... /* handle error here */
// Look up entry point by symbol.
string entryPoint = ...; /* symbol of library entry point */
alias FunType = int function(string); // your function signature here
auto funptr = cast(FunType) dlsym(libhandle, entryPoint.toStringz);
/*
* Step 3: Use the compiled code.
*/
// Call the compiled function with whatever arguments.
int result = funptr("my input");
...
// Cleanup once you're done with the library.
dlclose(libhandle);
std.file.remove(soFilename);
T
--
First Rule of History: History doesn't repeat itself -- historians merely
repeat each other.
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Saturday, 1 February 2020 at 20:37:03 UTC, H. S. Teoh wrote: On Sat, Feb 01, 2020 at 08:01:34PM +, Andre Pany via Digitalmars-d-learn wrote: [...] Another approach: - include the dmd compiler package with your application - within your app call the compiler executable and compile the source code to a dll / so - call the dll / so function [...] I've actually done this before in an equation grapher program: the user inputs an equation, the program generates D code to compute the equation, then runs dmd to compile it into a shared library, and opens the shared library and looks up the symbol to execute the compiled code. Dmd is fast enough that this actually works fairly well. When the input to dmd is small, it's so fast you don't even notice it. T This approach seems more tractable at present. Would you have any example code lying around for this? Saurabh
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Sat, Feb 01, 2020 at 08:01:34PM +, Andre Pany via Digitalmars-d-learn wrote: [...] > Another approach: > - include the dmd compiler package with your application > - within your app call the compiler executable and compile the source > code to a dll / so > - call the dll / so function [...] I've actually done this before in an equation grapher program: the user inputs an equation, the program generates D code to compute the equation, then runs dmd to compile it into a shared library, and opens the shared library and looks up the symbol to execute the compiled code. Dmd is fast enough that this actually works fairly well. When the input to dmd is small, it's so fast you don't even notice it. T -- An imaginary friend squared is a real enemy.
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Friday, 31 January 2020 at 11:19:37 UTC, Saurabh Das wrote: I see that DUB has DMD as a library package, but I was not able to understand how to use it. Is it possible to use DMD as a library within a D program to compile a string to machine code and run the compiled code at runtime? Thanks, Saurabh Another approach: - include the dmd compiler package with your application - within your app call the compiler executable and compile the source code to a dll / so - call the dll / so function Maybe you will get some trouble with AV software with this approach... Kind regards Andre
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Friday, 31 January 2020 at 14:25:30 UTC, Basile B. wrote: On Friday, 31 January 2020 at 11:19:37 UTC, Saurabh Das wrote: [...] Fundamentally DMD as a library is a front-end. Jitting is to the backend side. You'll be able to lex and parse the source to get an AST, to perform the semantic passes on this AST and that's all. Then to run this code you would need to make an AST visitor that will generate the binary code to execute. Even using a specialized library with jitting abilities, such as LLVM-d [1] or libfirm-d [2], this would be *quite* a journey. [1] https://github.com/MoritzMaxeiner/llvm-d [2] https://gitlab.com/basile.b/libfirm-d Thank you. That's enough to get me started tinkering! Saurabh
Re: Is it possible to use DMD as a library to compile strings at runtime?
On Friday, 31 January 2020 at 11:19:37 UTC, Saurabh Das wrote: I see that DUB has DMD as a library package, but I was not able to understand how to use it. Is it possible to use DMD as a library within a D program to compile a string to machine code and run the compiled code at runtime? Thanks, Saurabh Fundamentally DMD as a library is a front-end. Jitting is to the backend side. You'll be able to lex and parse the source to get an AST, to perform the semantic passes on this AST and that's all. Then to run this code you would need to make an AST visitor that will generate the binary code to execute. Even using a specialized library with jitting abilities, such as LLVM-d [1] or libfirm-d [2], this would be *quite* a journey. [1] https://github.com/MoritzMaxeiner/llvm-d [2] https://gitlab.com/basile.b/libfirm-d
