I have been writing several lexers and parsers. The grammars I need to parse are really complex, and consequently I didn't feel confident about the code quality, especially in the lexers. So I decided to jump on the functional progamming bandwagon to see if that would help. It definitely does help, there are fewer lines of code, and I feel better about the code quality. I started at the high level, and had the input buffer return a range of characters, and the lexer return a range of tokens. But when I got down to the lower levels of building up tokens, I ran into a problem:

First I started with this which worked:

private void getNumber(MCInputStreamRange buf)
{
    while (!buf.empty())
    {
        p++;
        buf.popFront();
        if (buf.front() <= '0' || buf.front() >= '9') break;
        *p = buf.front();
    }
    curTok.kind = Token_t.NUMBER;
curTok.image = cast(string) cbuffer[0 .. (p - cbuffer.ptr)].dup;
}

I thought I could improve this like so:

private void getNumber(MCInputStreamRange buf)
{
    auto s = buf.until("a <= '0' || a >= '9'");
    curTok.kind = Token_t.NUMBER;
    curTok.image = to!string(s);
}

The problem is that "until" seems to not stop at the end of the number, and instead continues until the end of the buffer. Am I doing something wrong here? Also, what is the fastest way to convert a range to a string?

Thanks,

Eric














Reply via email to