Wow! You are SOooooo correct. Fixed. Had to add another field...minor tweak, folks. Excellent catch Benjamin...wow, this rewrite engine is a bitch! added new test.
edit //depot/code/antlr/main/CHANGES.txt#80 edit //depot/code/antlr/main/runtime/Java/src/org/antlr/runtime/ TokenRewriteStream.java#11 edit //depot/code/antlr/main/src/org/antlr/test/ TestTokenRewriteStream.java#16 Change 5132 submitted. Ter On Aug 2, 2008, at 12:45 PM, Benjamin Niemann wrote: > Hi Ter, > > I'm just about to port the new TokenRewriteStream to Python. Something > smells fishy in the reduceToSingleOperationPerIndex method... > In lines 456, 464 and 493 you write > rewrites.set(j, null); // delete insert as it's a no-op. > > j is the index into the list returned by getKindOfOps, but you use it > as the index into the rewrites list. In all testcases this seems to be > doing what you intend, but I can't believe this is correct. > E.g. if I modify the test2ReplaceMiddleIndex test to have a > tokens.insertBefore(0, "_") before the replaces, the numbers get > screwed up and the exception in line 517 is triggered. > > I guess what you want is that j is the index within rewrites of those > ops filtered out by getKindOfOps. > My implementation of getKindOfOps now looks like > > for i, op in enumerate(rewrites[:before]): > if op is None: > # ignore deleted > continue > if op.__class__ == kind: > yield i, op > > i.e. for each matching op it returns a tuple (index_in_rewrites, > op). Then I do > > for j, iop in self.getKindOfOps(rewrites, InsertBeforeOp, i): > ... > > and j is what I think it should be. And the modified testcase above > works as intended. > > -Ben > _______________________________________________ > antlr-dev mailing list > [email protected] > http://www.antlr.org:8080/mailman/listinfo/antlr-dev _______________________________________________ antlr-dev mailing list [email protected] http://www.antlr.org:8080/mailman/listinfo/antlr-dev
