ilya-biryukov added a comment.

I've added a new parameter to `EnterToken` and `EnterTokenStream`. There are a 
bunch of interesting cases.

`Preprocessor::AnnotateCachedTokens` is one. It consumes some `CachedTokens` 
(which are always considered re-injected) and replaces them with a single 
annotation token (which we normally don't consider re-injected).
So we end up with an annotation token that is re-injected (and therefore never 
reported). That's ok for our current use-cases, so I won't try to fix it in 
this change. If this actually turns out important at any moment, can fix in a 
follow-up.

Another interesting case is parser "tweaking" tokens and re-introducing them, 
e.g. `FixDigraph` that turns a digraph token into two or various recoveries 
turning `::` into `:` or introducing ephemeral `;` and `}` tokens.
I've decided to mark those as re-injected for the time being, that is fine for 
now and I'm happy to reconsider later if it turns out we need them.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D59885/new/

https://reviews.llvm.org/D59885



_______________________________________________
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

Reply via email to