On Wednesday, 20 March 2019 at 14:27:54 UTC, Ethan wrote:
On Tuesday, 19 March 2019 at 19:50:15 UTC, Craig wrote:
For example, with windows I could simply compile to a dll then
extract the code, or just use LoadLibrary and it effectively
does all the work(steps 1 & 2 & 3).
LoadLibrary performs steps 1 and part of step 2. The DllMain
function of a library is called separately by Windows for each
thread in your system, which is where the bulk of step 2 is
handled. Step 3 isn't handled at all by LoadLibrary, and is
instead entirely up to you to deal with via the GetProcAddress
function.
If you want DLLs to operate in that step 1-2-3 manner, then the
compiler can generate a static library that handles that all
for you. But, as you might expect, that removes the hot
reloading capability as that is all handled before WinMain is
entered.
If you expect hot reloading to work without effort, you're
going to have to compile your DLLs with -betterC. The D Runtime
is not yet in a standalone DLL (unless I've missed an
announcement over the last couple of months). Each DLL you
build will require the D runtime to be embedded. Things get
really tricky from there.
If you ever use your DLL for more than calling functions with
basic types, or value aggregates (structs) containing basic
types, then be aware that without -betterC that there is
potential for the moduleinfo and typeinfo systems to go a bit
screwy. Especially once druntime lives in its own DLL, as
druntime expects those systems to be initialised once at
startup. POSIX libraries will have the same issues, but I have
nod had a need to investigate further over the last few months.
LoadLibrary is not portable though and seems excessive to do
something that should be rather simple unless there is
something I'm missing that has to be done that is complex.
LoadLibrary is the equivalent to dlopen on POSIX systems, with
one very important difference: Windows does not provide lazy
symbol evaluation. As such, extra work is required to handle
that. The average usecase is covered by the mentioned static
lib, but any hot reloading is handled in a custom manner by
every Windows codebase you'll encounter.
The static lib I mention, in fact, redirects every cross-DLL
call to a jump table. And cross-DLL data sharing gets hairy in
Windows. Function parameters aren't a problem, but Windows
explicitly disallows thread local variables to cross the DLL
boundary for example (even if you expose it for export and
perform a symbol lookup, it won't work and you'll get garbage
back).
As I said, it's not a simple thing by any means. My work has
attempted to hide all that complexity by boiling it all down to
"include these headers/modules, link this lib, and get on with
life". If you find the resulting code too complex (and I freely
admit that I haven't got the integration in to as foolproof a
manner as I am aiming for) then very definitely read through
the presentations to get a clear idea of why something that
should be so simple really isn't.
You are making this more complicated because you don't understand
that I don't need a general purpose solution. I simply need to
call simple functions that do not do much, not full blown
applications.
For example, Suppose I want to write a fractal display
application and instead of hard coding the function to display,
it's external.
e.g.,
import std.complex;
alias C = Complex!double;
C Mandelbrot(C z, C c) { return z*z + c; }
This function does not require much of anything to run. The
machine code can be copied and pasted at will to any executable
memory location and ran... it can, in fact, be inlined with a
little caution(reserve enough space and ignore the stack code).
No initialization needs to be nor garbage collecting.
For a little more robustness one would want to allow for GC,
which should not be hard, and context passing so more useful
functionality can be had.
If D had a Dstring2Machine that compiled the above code to
machine code then it could be used directly without issue.
Because these functions are deterministic and the host determines
their usage, there is little issues to worry about. Hot swapping
is simply changing a pointer when the function is not being
called.
As long as certain guidelines are followed one can do this quite
simply without too much issue and it allows for a scripting like
solution for D apps. It allows one to inject functionality in to
an application that runs at full speed and is relatively fast to
compile.
The idea is not to be able to connect two full blow external apps
but to connect little pieces of functionality to an app that
allow for vast flexibility. Since each unit of functionality is
essentially pure, there are no major issues.
It's this simple:
1. Take source and compile it in to machine code.
2. App extracts machine code and inserts it in to it's memory
space to be able to execute.
Do you agree that, at least in some cases, this is entirely
feasible and easy?
I'll give you a hint:
1.
module simple;
void foo() { };
compile directly in to machine code(the most raw form possible
containing foo).
2.
App load the machine code, execute, reload and execute if
desired...
Since foo does nothing it's just a nop. We are just copying
around a ret. It's impossible for anything to ever go awry here
excluding buggy code/compiler/etc.
From here one can expand the model to include something that is
more useful.
The idea is to expand it enough to make it generally quite useful
but not so much one has to include the kitchen sink.
I don't need a heavy version of binderoo, I need a light one.
Something that lets me do basic "scripting" so I can modify the
application behavior at runtime without recompiling the
application.
If you think the example is stupid and useless then
int age() { return 43; }
and nothing changes.
of course you will then think that example is stupid because we
could just put it in a text file and read that rather than go
through all this trouble.
so then
C calc(C z, C c) { return z*z*z*z*z*z + c*z + c; }
and to get around this without leveraging the D compiler one has
to make a full blown parser... even though it's still just
relatively basic code like age. It just includes some
calculations.
It's no different from foo though. Just basic code.
But we can go further by allowing more information to be passed
to and from the function. The function can also be impure such as
void foo() { writeln("I'm IMPURE!"); }
which will work without much work(might require fixing up a few
symbols/pointers).
All it is, is delayed compilation. I'm simply wanting to delay
the compilation of a piece of code in an app from the main
compilation. All the other stuff about the OS and TLS are
irrelevant in this context.
One could sort of do this with the object files and just
recompile the functions and relink everything. But with a little
work this step can be avoided since it's not needed and
slower(but more robust).
