| Issue |
203278
|
| Summary |
[clang] MSVC mangler hits UNREACHABLE "Invalid type expected" (mangleAutoReturnType) on lambda conversion to function pointer with deduced return type
|
| Labels |
|
| Assignees |
|
| Reporter |
Fedr
|
When targeting the MSVC ABI, clang crashes while mangling a lambda's conversion operator to function pointer when the lambda has a deduced return type (`-> auto &&`): the conversion operator's return type is a pointer to a *function* type carrying the placeholder, and `MicrosoftCXXNameMangler::mangleAutoReturnType`'s dispatcher only handles `MemberPointer`/`Pointer`/`LValueReference`/`RValueReference`, so it falls into `llvm_unreachable("Invalid type expected")`.
In builds with assertions this is a clean UNREACHABLE abort. In release builds without assertions the unreachable is UB, and we observed it as a hard segfault (0xC0000005 on Windows) or an infinite loop / time-limit kill, depending on the build — which makes it easy to misdiagnose as an OOM or CI timeout.
## Reproducer
https://gcc.godbolt.org/z/vs3MEYP61 (clang assertions trunk, `-std=c++20 --target=x86_64-pc-windows-msvc`)
```cpp
template <typename F> void add(F);
template <typename T>
struct RegisterOneType
{
inline static const decltype(nullptr) register_type = (add(+[](int &x) -> auto && { return x; }), nullptr);
};
inline void bake() { (void)&RegisterOneType<int>::register_type; }
```
```
clang++ -std=c++20 --target=x86_64-pc-windows-msvc -c repro.cpp
```
Note `bake()` is never called; referencing `®ister_type` from it is enough to trigger the end-of-TU instantiation and codegen of the inline variable's dynamic initializer.
## Output (assertions trunk)
```
Invalid type expected
UNREACHABLE executed at /root/llvm-project/llvm/tools/clang/lib/AST/MicrosoftMangle.cpp:2572!
...
2. <source>:6:43: instantiating variable definition 'RegisterOneType<int>::register_type'
3. <source>:6:43: Generating code for declaration 'RegisterOneType<int>::register_type'
4. <source>:6:65: Mangling declaration 'RegisterOneType<int>::(lambda)::operator int &(*)(int &)'
#9 0x000000000835889a (anonymous namespace)::MicrosoftCXXNameMangler::mangleAutoReturnType(clang::QualType, (anonymous namespace)::MicrosoftCXXNameMangler::QualifierMangleMode) MicrosoftMangle.cpp:0:0
#10 0x000000000835866b (anonymous namespace)::MicrosoftCXXNameMangler::mangleAutoReturnType(clang::QualType, (anonymous namespace)::MicrosoftCXXNameMangler::QualifierMangleMode) MicrosoftMangle.cpp:0:0
#11 0x0000000008358db7 (anonymous namespace)::MicrosoftCXXNameMangler::mangleFunctionType(clang::FunctionType const*, clang::FunctionDecl const*, bool, bool) MicrosoftMangle.cpp:0:0
#12 0x0000000008359fe9 (anonymous namespace)::MicrosoftCXXNameMangler::mangleType(clang::QualType, clang::SourceRange, (anonymous namespace)::MicrosoftCXXNameMangler::QualifierMangleMode) MicrosoftMangle.cpp:0:0
#13 0x000000000835a268 (anonymous namespace)::MicrosoftCXXNameMangler::mangleType(clang::QualType, clang::SourceRange, (anonymous namespace)::MicrosoftCXXNameMangler::QualifierMangleMode) MicrosoftMangle.cpp:0:0
#14 0x0000000008359214 (anonymous namespace)::MicrosoftCXXNameMangler::mangleFunctionType(clang::FunctionType const*, clang::FunctionDecl const*, bool, bool) MicrosoftMangle.cpp:0:0
#15 0x0000000008361235 (anonymous namespace)::MicrosoftMangleContextImpl::mangleCXXName(clang::GlobalDecl, llvm::raw_ostream&) MicrosoftMangle.cpp:0:0
#16 0x0000000008342d9c clang::MangleContext::mangleName(clang::GlobalDecl, llvm::raw_ostream&)
#17 0x0000000004aac3b5 getMangledNameImpl(clang::CodeGen::CodeGenModule&, clang::GlobalDecl, clang::NamedDecl const*, bool) CodeGenModule.cpp:0:0
```
## Versions
Verified on Compiler Explorer with `-std=c++20 --target=x86_64-pc-windows-msvc`:
| version | result |
|---|---|
| clang 18.1.0 | compiles OK |
| clang 19.1.0 | compiles OK |
| clang 20.1.0 (assertions) | UNREACHABLE at MicrosoftMangle.cpp:2551 |
| clang 20.1.0 / 21.1.0 (release) | hangs until killed |
| clang 22.1.0 (assertions) | UNREACHABLE at MicrosoftMangle.cpp:2572 |
| clang 22.1.x (release, Windows native) | segfault (0xC0000005) |
| trunk (assertions) | UNREACHABLE at MicrosoftMangle.cpp:2572 |
So this looks like a regression introduced with the placeholder-return-type mangling for MSVC 1920+ (#102848, relanded as #104722, first shipped in clang 20).
The same TU compiles fine with the Itanium mangler (e.g. `--target=x86_64-pc-linux-gnu`).
`-std=c++23` is affected the same way.
## Context
Originally hit in MeshLib's Python-binding CI (pybind11-style registration code), where it appeared only after `-fpch-instantiate-templates` started instantiating these lambdas inside a PCH — every fragment compile then died with 0xC0000005 inside `clang::MicrosoftMangleContext`. The reduced reproducer above needs no PCH.
![]()
_______________________________________________
llvm-bugs mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-bugs
