On Mon, May 30, 2016 at 2:32 PM, Alexis King <[email protected]> wrote: > >> On May 30, 2016, at 12:23, Alexis King <[email protected]> wrote: >> >> Now the colorer attempts to lex the slash alone as a new token without >> attempting to re-lex “foo”. Since a forward slash is not valid at the >> beginning of an identifier, it is colored as an error. >> >> Basically, when I type a forward slash, I want the preceding token to be >> re-lexed. I don’t need anything more than that... just a single >> character of lookahead is necessary, effectively. > > Actually, I realized just after sending this that my description is not > the full picture. I think the confusion probably stems from that the > colorer does, in fact attempt to re-lex “foo” once the forward slash is > typed. However, since a trailing slash is illegal at the end of an > identifier as well, it’s still lexed as an error. Now, I keep typing: > > foo/bar > ^ > > This presumably causes the slash to be re-lexed, but since it’s still > illegal at the beginning of a token, the lexer still fails. My guess > here was that I needed to include a backup-distance on the error token, > but that doesn’t seem to help, so I’m unsure how to properly indicate > the need to re-lex the entire “foo/bar” token once “bar” is added.
I'm wondering why the entire thing isn't a single token, actually. So, when you type any of these: f fo foo foo/ foo/b foo/ba foo/bar your lexer would always say that they are a single token. The fourth one would be an error token, but still a single token. Robby -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
