yronglin wrote: > Previously you said Preprocessor::LexAfterModuleImport won't executed for > C++20 modules.
Yes, I can't reproduce the fix by adding `!isImportingCXXNamedModules()` in `LexAfterModuleImport`. I'm not sure I have missing anything. > And why it was executed in clangd's case and why the current patch fixes that? The root cause is: The import directive `import NonExistent;` will tigger `cutOfParsing` in https://github.com/llvm/llvm-project/blob/93d256b076b12bf30e546910d9ae473ec93146c0/clang/lib/Parse/Parser.cpp#L2447-L2451, and `cutOfParsing` only set the `Tok` member to EOF. Clangd will add a token watcher in https://github.com/llvm/llvm-project/blob/93d256b076b12bf30e546910d9ae473ec93146c0/clang/lib/Tooling/Syntax/Tokens.cpp#L683-L694, and this callback will be called in https://github.com/llvm/llvm-project/blob/93d256b076b12bf30e546910d9ae473ec93146c0/clang/lib/Lex/Preprocessor.cpp#L1024-L1025, but the module import error cutoff parsing and the token watcher will never receive this EOF token callback. SO the last token in the token recording sequence is not an EOF token, but a ';' token. This tigger assertion failure in https://github.com/llvm/llvm-project/blob/d818fa4c55c24b989eb925581e62b284c3c3a461/clang/lib/Tooling/Syntax/Tokens.cpp#L715 This PR propagate and record the EOF token when the parser hit an module load fatal error and try to cutOffParsing. Also the token watcher reject annotation tokens in https://github.com/llvm/llvm-project/blob/d818fa4c55c24b989eb925581e62b284c3c3a461/clang/lib/Tooling/Syntax/Tokens.cpp#L683-L685, and it's also mistakenly rejected `tok::annot_module_name` token, this PR also fix this issue. The test failure is related, I'll fix it later. https://github.com/llvm/llvm-project/pull/187858 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
