On Jun 27, 2012, at 9:33 , Dmitri Gribenko <[email protected]> wrote:
> Why not StringRefs? Per Doug's comment: > + /// Contains text value associated with a token. > + StringRef TextValue1; > + > + /// Contains text value associated with a token. > + StringRef TextValue2; > > It would be nice if Token were a POD type. Could we just store ptr/length for > TextValue1 and TextValue2, so that Token remains a POD? Oops, I forgot that SourceLocation is POD (of course). Never mind. > > I think that it is useful because the extracted text is the semantic > value of the token, which is used a lot in the parser and sema. I > don't think it is a good idea to require them to strip extra > characters -- that's what the lexer is for. Oh, I just meant that for things like commands and literal text, getText() and getCommandName() could do the work on the fly, rather than caching it. But I guess since every token takes the cost for those variables, you might as well use them. (Actually, that's a bit worrisome. Most tokens have an extra pointer-and-size because they don't need the second string argument. But I can't think of a good way around that off the top of my head.) Jordan
_______________________________________________ cfe-commits mailing list [email protected] http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
