On Fri, Oct 7, 2011 at 11:17 PM, Benjamin S Wolf <[email protected]> wrote: > I've discovered two bugs in working with the Python target to generate lexers. > > The first is in that when supplying k, the generated code for special > state transitions is invalid: the "elif" keyword is split across lines > as "el\nif", so the python interpreter crashes upon reading "el".
That's been reported before, but I have problem reproducing it. Are you using antlr-3.4-complete.jar (I can only see this problem with that build) or did you build it yourself from antlr-3.4.tar.gz (or something else completely)? > I've > been working around this in vim by running the following command in > the generated file: > > :%s/\( \+\)el\n\1/\1el/ > > (Replaces "el\n" preceded and followed with n spaces by n spaces > followed by "el".) > > The second is in the Python antlr3 library. Calling getTokens() on a > CommonTokenStream will return all but the last token. This is because > the slice notation [start:stop] is inclusive on the left and exclusive > on the right, but stop is set to len(self.tokens) - 1. > http://www.antlr.org/api/Python/antlr3_8py-source.html#l01733 > > This can be fixed by finding the following lines in getTokens() (in > antlr3/streams.py): > > if stop is None or stop >= len(self.tokens): > stop = len(self.tokens) - 1 > > and changing them to > > if stop is None or stop > len(self.tokens): > stop = len(self.tokens) > > or it can be worked around by using the tokens attribute directly. Will have a look at that, thanks for reporting. -Ben > > --Ben > > List: http://www.antlr.org/mailman/listinfo/antlr-interest > Unsubscribe: > http://www.antlr.org/mailman/options/antlr-interest/your-email-address > List: http://www.antlr.org/mailman/listinfo/antlr-interest Unsubscribe: http://www.antlr.org/mailman/options/antlr-interest/your-email-address -- You received this message because you are subscribed to the Google Groups "il-antlr-interest" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/il-antlr-interest?hl=en.
