On 19.10.2011 01:52, foobar wrote:
Don Wrote:
On 17.10.2011 01:43, foobar wrote:
foobar Wrote:
Don Wrote:
You're assuming that the compiler can run the code it's generating. This
isn't true in general. Suppose you're on x86, compiling for ARM. You
can't run the ARM code from the compiler.
This is quite possible in Nemerle's model of compilation.
This is the same concept as XLST - a macro is a high level transform
from D code to D code.
1. compile the macro ahead of time into a loadable compiler
module/plugin.
the plugin is compiled for the HOST machine (x86) either by a separate
compiler
or by a cross-compiler that can also compile to its HOST target.
YES!!! This is the whole point. That model requires TWO backends. One
for the host, one for the target.
That is, it requires an entire backend PURELY FOR CTFE.
Yes, of course it is POSSIBLE, but it is an incredible burden to place
on a compiler vendor.
How does that differ from the current situation? We already have a separate
implementation of a D interpreter for CTFE.
That's true, but it's quite different from a separate backend.
The CTFE interpreter doesn't have much in common with a backend. It's
more like a glue layer. Most of what's in there at the moment, is doing
marshalling and error checking.
Suppose instead, you made calls into a real backend. You'd still need a
marshalling layer, first to get the syntax trees into native types for
the backend, and secondly to convert the native types back into syntax
trees. The thing is, you can be changing only part of a syntax tree
(just one element of an array, for example) so if you used a native
backend, the majority of the code in the CTFE interpreter would remain.
Yes, there is an actual CTFE backend in there, but it's tiny.
I'm not sure I follow you here. macros as defined in Nemerle are simply
compiler plugins that use hooks in the compiler front-end.
Nemerle supports hooks at several levels, mostly at parsing and semantic
analysis passes but I think also during lexing in order to allows syntax
extensions.
for instance at the parsing stage, the macro's input and output would be token
streams. I don't understand how that requires a separate backend.
It does require APIs/hooks into the compiler passes.
Whatever code the compiler calls (compiler plugin, or JITted code), the
compiler needs to be able to pass parameters to it, and return them. How
can it do that?
For example, all extant D compilers are written in C++ and they can't
call D code directly.
The macro itself is a regular program - implemented in Nemerle and compiled
into a dll.
I disagree with the second point as well - Nothing forces the SAME compiler to
contain two separate implementations as is now the case.
In Fact you could compile the macros with a compiler of a different vendor.
After all that's the purpose of an ABI, isn't it?
In fact it makes the burden on the vendor much smaller since you remove the
need for a separate interpreter.
I forgot to mention an additional aspect of this design - it greatly simplifies
the language which also reduces the burden on the compiler vendor.
You no longer need to support static constructs like "static if", CTFE, is(),
pragma, etc. You also gain more capabilities with less effort,
e.g you could connect to a DB at compile time to validate a SQL query or use
regular IO functions from phobos.
Statements of the form "XXX would make the compiler simpler" seem to
come up quite often, and I can't remember a single one which was made
with much knowledge of where the complexities are!
For example, the most complex thing you listed is is(), because
is(typeof()) accepts all kinds of things which normally wouldn't
compile. This has implications for the whole compiler, and no changes in
how CTFE is done would make it any simpler.
I actually said it would make the _language_ simpler. it makes the language
more regular in that you use the same syntax to implement the macro itself.
No need to maintain an additional set of compile-time syntax.
I don't understand. D has almost no compile-time syntax.
This in turn would simplify the implementation of the compiler.
Syntax is trivial. It's a negligible fraction of the compiler (provided
you retain D's rigid seperation between parsing and semantic pass).
the above list should contain all compile-time features (I didn't mention
templates and traits)
Since a macro has hooks into the semantic pass it can do all sorts of
transforms directly on the abstract syntax tree - modifying the data structures
in the compiler's memory.
this means you use _regular_ code in combination with a compiler API instead of
special syntax.
E.g. traits and is(typof()) would be regular functions (part of the API)
instead of language features.
I don't have a clue how you would do that. Most of the simple traits
could be implemented in D in the existing compiler, but is(typeof()) is
a different story. It practically runs the entire compiler with errors
suppressed. The complexity isn't in is(typeof()) itself, but rather in
the fact that the whole compiler has to cope with errors being suppressed.
> e.g you could connect to a DB at compile time to validate a SQL query
or use regular IO functions from phobos.
You know, I'm yet to be convinced that that it's really desirable. The
ability to put all your source in a repository and say, "the executable
depends on this and nothing else", is IMHO of enormous value. Once you
allow it to depend on the result of external function calls, it depends
on all kinds of hidden variables, which are ephemeral, and it seems to
me much better to completely separate that "information gathering" step
from the compilation step.
Note that since it's a snapshot, you *don't* have a guarantee that your
SQL query is still valid by the time the code runs.
I agree this adds dependencies to the compilation process. However, IIUC, D
already allows adding vendor extensions with pragma.
To date, no vendor extensions introduce additional dependencies.
At the moment this requires to changes the compiler's source. macros simply
extend this notion with shared objects.
Look for example at the pragma for printing at compile-time. instead of
implementing it in as part of the compiler, you could simply use phobos'
writeln.
"simply"? Let me show you the code for pragma(msg). The whole thing, in
all its complexity:
-------------------
Statement *PragmaStatement::semantic(Scope *sc)
{ if (ident == Id::msg) {
if (args) {
for (size_t i = 0; i < args->dim; i++) {
Expression *e = args->tdata()[i];
e = e->semantic(sc);
e = e->optimize(WANTvalue | WANTinterpret);
StringExp *se = e->toString();
if (se)
fprintf(stdmsg, "%.*s", (int)se->len, (char
*)se->string);
else
fprintf(stdmsg, "%s", e->toChars());
}
fprintf(stdmsg, "\n");
}
}
-------------------
BTW there's nothing in the present design which prevents CTFE from being
implemented by doing a JIT to native code. I expect most implementations
will go that way, but they'll be motivated by speed.
Also I'm confused by this term "macros" that keeps popping up. I don't
know why it's being used, and I don't know what it means.
The term comes from lisp.
The word is much older than Lisp (macro assemblers are ancient; CAR and
CDR were asm macros). The uses of the word on these forums are clearly
influenced by Lisp macros, but the mapping of 'macro' from Lisp to the
usage here is by no means clear. I feel that a lot of extra assumptions
are being smuggled in through the word.
Most commonly, it seems to mean "some compile-time programming system,
which is completely undescribed except for the fact that it's almost
perfect in every way" <g>.
Sometimes it seems to mean "macros as implemented by Nemerle". Even
then, it's unclear which aspects of Nemerle macros are considered to be
important.
Occasionally, macro means "any compile-time programming system",
including what we have in D at the moment.
