On 8/29/14, 10:41 AM, Mike James wrote:
Hi,
Looking at the DMD Source Guide it says "The lexer transforms the file
into an array of tokens."
Why is this step taken instead of, say, just calling a function that
returns the next token (or however many required for the look-ahead)?
Regards,
-=mike=-
I believe this is just an abstract description of how it works. It
actually uses something like next token with a freelist of tokens if it
needs to do some lookahead.
https://github.com/D-Programming-Language/dmd/blob/master/src/lexer.h#L260
https://github.com/D-Programming-Language/dmd/blob/master/src/lexer.h#L261
https://github.com/D-Programming-Language/dmd/blob/master/src/lexer.h#L237
