"Luke" == Luke Palmer <[EMAIL PROTECTED]> writes:
Luke> But you don't really need to parse to syntax highlight, either. You Luke> just need to tokenize.
Unfortunately, to tokenize, you also have to know the state of the parse. As long as / is both "divide" and "begin regex", you're toasted.
So you're saying that in Perl 6 it will be entirely impossible to determine if / appears as the division operator or as the beginning of a regex from a purely syntactic examination of the source code?
I'm finding that very, very hard to believe. Regexps aren't valid where /-the-operator is, after all.
Please correct me if I'm wrong, but I've got the impression that Perl 6 is tokenisable without requiring BEGIN blocks to be run - provided no grammars which the tokeniser doesn't already know about are used, of course, that one will never be avoidable.
