On Tuesday, 31 July 2012 at 16:10:14 UTC, Kai Meyer wrote:
I know I'm a little late coming into this conversation. This
seems like a nice thread to toss myself into. I've started
working on a generic lexer that is based off of a defined
grammar.
Every helping hand is appreciated :-)
As I read through the thread (I unfortunately don't have enough
time to read every post, but I skimmed through as many as I
could, and read the ones that seemed important), it seems like
we need a parser in D that can lex D, and provide a Range of
tokens that could be consumed.
Yes. To make this statement more precise: We need a lexer that
provides
a range of tokens and we need a parser which makes it possible to
build
an AST. Pegged combines both approaches but imposes an overhead
if you
just need a token list. However I'm not sure if this is a problem.
There are already some working D-Parsers buried in this thread.
With some very minor tweaks, and a properly written Grammar
class, I basically have it already done. D was going to be one
of the first languages I would have written a definition for.
https://github.com/kai4785/firstfront
I've only glimpsed at your code. For most languages lexing is far
more expensive then parsing and thus the lexer has to be very
fast and I wouldn't pursue your approach and instead use
something like ragel. It already has D output but needs a
separate build step.
