alexfh wrote:
> > It's all internal code, so sharing it in any way wouldn't be possible. But
> > if you have any ideas of how to stuff Clang with debug logs and/or
> > assertions to make the issue easier to diagnose, I could run Clang and send
> > you logs.
>
> Not really without further understanding what is going wrong under which
> circumstances... Some pointers (not sure how much time you would be able to
> spend on your side debugging the problem):
>
> 1. As a first matter of action, `ASTContext::getASTRecordLayout` completes
> the type. This should trigger template loading which is more "lazy" after
> these fixes.
>
> https://github.com/llvm/llvm-project/blob/bba40ab4bd60e636e77362c46c181eafd377f541/clang/lib/AST/RecordLayoutBuilder.cpp#L3371-L3379
> 2. One thing that could happen is that deserialization triggers merging of
> declarations and another one has the actual definition attached (redecl
> chains are a bit black magic for me).
> 3. Another possibility is that the forward declaration is local and not
> external, thus `D->hasExternalLexicalStorage()` returns `false` and we skip
> the call to `CompleteType`. We might try changing that condition to
> `getExternalSource() && !D->getDefinition()`...
> 4. Finally it would be interesting to know if what type of template class (?)
> we are dealing with: implicit specialization, explicit, involving partial
> specializations? In the past, the "triggering" commit was
> [3a272ba](https://github.com/llvm/llvm-project/commit/3a272ba3e512218976660912ef9354ff5576c95d)
> if you can maybe see if reverting that one avoids the problems (or in
> general which of the four changes does)?
I could allocate some time to debugging this, and here's what I got.
<details>
The crash stack is:
```
...
frame #7: 0x00005555748dba8a clang-dbg`__assert_fail(assertion="D &&
\"Cannot get layout of forward declarations!\"",
file="llvm-project/clang/lib/AST/RecordLayoutBuilder.cpp", line=3377,
function="const ASTRecordLayout &clang::ASTContext::getASTRecordLayout(const
RecordDecl *) const") at logging.cc:61:3
frame #8: 0x000055556a8dd209
clang-dbg`clang::ASTContext::getASTRecordLayout(this=0x000030f1bfc20000,
D=0x0000000000000000) const at RecordLayoutBuilder.cpp:3377:3
* frame #9: 0x000055556976f180
clang-dbg`clang::ASTContext::getTypeInfoImpl(this=0x000030f1bfc20000,
T=0x000030f1ba458cb0) const at ASTContext.cpp:2409:37
frame #10: 0x0000555569770a6c
clang-dbg`clang::ASTContext::getTypeInfo(this=0x000030f1bfc20000,
T=0x000030f1ba458cb0) const at ASTContext.cpp:1969:17
frame #11: 0x00005555644077f8
clang-dbg`clang::ASTContext::getTypeInfo(this=0x000030f1bfc20000, T=QualType @
0x00007ffffffe6dd8) const at ASTContext.h:2645:51
frame #12: 0x0000555564b475c8
clang-dbg`clang::ASTContext::getTypeAlign(this=0x000030f1bfc20000, T=QualType @
0x00007ffffffe6e38) const at ASTContext.h:2682:52
frame #13: 0x00005555697717f8
clang-dbg`clang::ASTContext::getTypeAlignInChars(this=0x000030f1bfc20000,
T=QualType @ 0x00007ffffffe6e70) const at ASTContext.cpp:2574:30
frame #14: 0x0000555564483e7c
clang-dbg`clang::CodeGen::CodeGenFunction::CreateIRTemp(this=0x00007ffffffeb608,
Ty=QualType @ 0x00007ffffffe6ef8, Name=0x00007ffffffe8740) at CGExpr.cpp:183:34
frame #15: 0x0000555564e95a9e
clang-dbg`clang::CodeGen::CodeGenFunction::StartFunction(this=0x00007ffffffeb608,
GD=GlobalDecl @ 0x00007ffffffea560, RetTy=QualType @ 0x00007ffffffea558,
Fn=0x000030f1b47866c8, FnInfo=0x000030f1b477d980, Args=0x00007ffffffeb3c0,
Loc=(ID = 2115544462), StartLoc=(ID = 2115544504)) at
CodeGenFunction.cpp:1248:19
frame #16: 0x0000555564e98bf0
clang-dbg`clang::CodeGen::CodeGenFunction::GenerateCode(this=0x00007ffffffeb608,
GD=GlobalDecl @ 0x00007ffffffeb470, Fn=0x000030f1b47866c8,
FnInfo=0x000030f1b477d980) at CodeGenFunction.cpp:1558:3
frame #17: 0x0000555564eee07c
clang-dbg`clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(this=0x000030f1bfd04000,
GD=GlobalDecl @ 0x00007ffffffecfa0, GV=0x000030f1b47866c8) at
CodeGenModule.cpp:6406:26
frame #18: 0x0000555564eddd64
clang-dbg`clang::CodeGen::CodeGenModule::EmitGlobalDefinition(this=0x000030f1bfd04000,
GD=GlobalDecl @ 0x00007ffffffed150, GV=0x000030f1b47866c8) at
CodeGenModule.cpp:4479:9
frame #19: 0x0000555564ecb704
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3562:5
frame #20: 0x0000555564ecb774
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3568:7
frame #21: 0x0000555564ecb774
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3568:7
frame #22: 0x0000555564ecb774
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3568:7
frame #23: 0x0000555564ecb774
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3568:7
frame #24: 0x0000555564ecb774
clang-dbg`clang::CodeGen::CodeGenModule::EmitDeferred(this=0x000030f1bfd04000)
at CodeGenModule.cpp:3568:7
frame #25: 0x0000555564ec46ed
clang-dbg`clang::CodeGen::CodeGenModule::Release(this=0x000030f1bfd04000) at
CodeGenModule.cpp:965:3
frame #26: 0x00005555652f3165 clang-dbg`(anonymous
namespace)::CodeGeneratorImpl::HandleTranslationUnit(this=0x000030f1bfe07110,
Ctx=0x000030f1bfc20000) at ModuleBuilder.cpp:293:18
frame #27: 0x0000555564350b65
clang-dbg`clang::BackendConsumer::HandleTranslationUnit(this=0x000030f1bfe08540,
C=0x000030f1bfc20000) at CodeGenAction.cpp:240:10
frame #28: 0x0000555566fd16e1
clang-dbg`clang::ParseAST(S=0x000030f1bb597000, PrintStats=false,
SkipFunctionBodies=false) at ParseAST.cpp:183:13
frame #29: 0x00005555669479c5
clang-dbg`clang::ASTFrontendAction::ExecuteAction(this=0x000030f1bfe2a600) at
FrontendAction.cpp:1431:3
frame #30: 0x0000555564356f5d
clang-dbg`clang::CodeGenAction::ExecuteAction(this=0x000030f1bfe2a600) at
CodeGenAction.cpp:1109:30
frame #31: 0x0000555566946f0b
clang-dbg`clang::FrontendAction::Execute(this=0x000030f1bfe2a600) at
FrontendAction.cpp:1311:3
frame #32: 0x00005555667ef49d
clang-dbg`clang::CompilerInstance::ExecuteAction(this=0x000030f1bfe08380,
Act=0x000030f1bfe2a600) at CompilerInstance.cpp:1008:33
frame #33: 0x00005555643456ab
clang-dbg`clang::ExecuteCompilerInvocation(Clang=0x000030f1bfe08380) at
ExecuteCompilerInvocation.cpp:310:25
...
```
Looking closer at `frame #9: 0x000055556976f180
clang-dbg`clang::ASTContext::getTypeInfoImpl(this=0x000030f1bfc20000,
T=0x000030f1ba458cb0) const at ASTContext.cpp:2409:37`:
```
2386 case Type::Record:
2387 case Type::Enum: {
2388 const auto *TT = cast<TagType>(T);
2389 const TagDecl *TD = TT->getDecl()->getDefinitionOrSelf();
2390
2391 if (TD->isInvalidDecl()) {
2392 Width = 8;
2393 Align = 8;
(lldb)
2394 break;
2395 }
2396
2397 if (isa<EnumType>(TT)) {
2398 const EnumDecl *ED = cast<EnumDecl>(TD);
2399 TypeInfo Info =
2400
getTypeInfo(ED->getIntegerType()->getUnqualifiedDesugaredType());
2401 if (unsigned AttrAlign = ED->getMaxAlignment()) {
2402 Info.Align = AttrAlign;
2403 Info.AlignRequirement = AlignRequirementKind::RequiredByEnum;
2404 }
(lldb)
2405 return Info;
2406 }
2407
2408 const auto *RD = cast<RecordDecl>(TD);
2409 const ASTRecordLayout &Layout = getASTRecordLayout(RD);
```
The crash is in the function call above ^^
```
2410 Width = toBits(Layout.getSize());
2411 Align = toBits(Layout.getAlignment());
2412 AlignRequirement = RD->hasAttr<AlignedAttr>()
2413 ? AlignRequirementKind::RequiredByRecord
2414 : AlignRequirementKind::None;
2415 break;
```
The CXXRecordDecl that triggers the crash is:
```
(lldb) p RD->dump()
CXXRecordDecl 0x30f1ba455568 parent 0x30f1babf29d8 prev 0x30f1ba449eb0
<third_party/absl/container/internal/raw_hash_set.h:890:1, col:7> col:7
imported in //third_party/protobuf/util:differencer hidden class const_iterator
instantiated_from 0x30f1ba97cea0
(lldb) p RD->hasExternalLexicalStorage()
(bool) false
(lldb) p RD->getDefinition()
(clang::RecordDecl *) nullptr
```
Just in case a full dump of `*RD`:
```
(lldb) settings set target.max-children-depth 100
(lldb) p *RD
(const clang::RecordDecl) {
clang::TagDecl = {
clang::TypeDecl = {
clang::NamedDecl = {
clang::Decl = {
NextInContextAndBits = {
Value = (Data = "\U00000002")
}
DeclCtx = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::DeclContext
*, clang::Decl::MultipleDC *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *, clang:
:Decl::MultipleDC *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *,
clang::Decl::MultipleDC *> > >, 0, clang::DeclContext *,
clang::Decl::MultipleDC *> = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::DeclContext
*, clang::Decl::MultipleDC *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *, clan
g::Decl::MultipleDC *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *,
clang::Decl::MultipleDC *> > >, 1, clang::Decl::MultipleDC *> = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::DeclContext
*, clang::Decl::MultipleDC *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *, cl
ang::Decl::MultipleDC *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::DeclContext *,
clang::Decl::MultipleDC *> > >, 2> = {
Val = {
Value = (Data = "\U00000014\xb6I\xba\xf10")
}
}
}
}
}
Loc = (ID = 2102307394)
DeclKind = CXXRecord
InvalidDecl = false
HasAttrs = false
Implicit = false
Used = false
Referenced = false
TopLevelDeclInObjCContainer = false
Access = AS_public
FromASTFile = true
IdentifierNamespace = IDNS_Tag | IDNS_Type
= 0
CacheValidAndLinkage = Invalid
}
Name = (Ptr = 53814798516824)
}
TypeForDecl = 0x000030f1ba49ecb0
LocStart = (ID = 2102307388)
}
clang::DeclContext = {
LookupPtr = nullptr
= {
DeclContextBits = {
DeclKind = CXXRecord
ExternalLexicalStorage = false
ExternalVisibleStorage = false
NeedToReconcileExternalVisibleStorage = false
HasLazyLocalLexicalLookups = false
HasLazyExternalLexicalLookups = false
UseQualifiedLookup = false
}
NamespaceDeclBits = (59, IsInline = true, IsNested = true)
TagDeclBits = {
= 59
TagDeclKind = Class
IsCompleteDefinition = false
IsBeingDefined = false
IsEmbeddedInDeclarator = false
IsFreeStanding = false
IsCompleteDefinitionRequired = false
IsThisDeclarationADemotedDefinition = false
}
EnumDeclBits = {
= 24635
NumPositiveBits = 0
NumNegativeBits = 0
IsScoped = false
IsScopedUsingClassTag = false
IsFixed = false
HasODRHash = false
}
RecordDeclBits = {
= 24635
HasFlexibleArrayMember = false
AnonymousStructOrUnion = false
HasObjectMember = false
HasVolatileMember = false
LoadedFieldsFromExternalStorage = false
NonTrivialToPrimitiveDefaultInitialize = false
NonTrivialToPrimitiveCopy = false
NonTrivialToPrimitiveDestroy = false
HasNonTrivialToPrimitiveDefaultInitializeCUnion = false
HasNonTrivialToPrimitiveDestructCUnion = false
HasNonTrivialToPrimitiveCopyCUnion = false
HasUninitializedExplicitInitFields = false
ParamDestroyedInCallee = false
ArgPassingRestrictions = CanPassInRegs
IsRandomized = false
ODRHash = 0
}
OMPDeclareReductionDeclBits = (59, InitializerKind = Direct | Copy)
FunctionDeclBits = {
= 59
SClass = SC_PrivateExtern
IsInline = false
IsInlineSpecified = false
IsVirtualAsWritten = false
IsPureVirtual = false
HasInheritedPrototype = false
HasWrittenPrototype = false
IsDeleted = false
IsTrivial = false
IsTrivialForCall = false
IsDefaulted = false
IsExplicitlyDefaulted = false
HasDefaultedOrDeletedInfo = false
IsIneligibleOrNotSelected = false
HasImplicitReturnZero = false
IsLateTemplateParsed = false
IsInstantiatedFromMemberTemplate = false
ConstexprKind = Unspecified
BodyContainsImmediateEscalatingExpression = false
InstantiationIsPending = false
UsesSEHTry = false
HasSkippedBody = false
WillHaveBody = false
...
}
CXXConstructorDeclBits = {
= 24635
NumCtorInitializers = 0
IsInheritingConstructor = false
HasTrailingExplicitSpecifier = false
IsSimpleExplicit = false
}
ObjCMethodDeclBits = {
= 59
Family = OMF_init
IsInstance = false
IsVariadic = false
IsPropertyAccessor = false
IsSynthesizedAccessorStub = false
IsDefined = false
IsRedeclaration = false
HasRedeclaration = false
DeclImplementation = None
objcDeclQualifier = OBJC_TQ_None
RelatedResultType = false
SelLocsKind = SelLoc_NonStandard
IsOverriding = false
HasSkippedBody = false
}
ObjCContainerDeclBits = {
AtStart = (ID = 0)
}
LinkageSpecDeclBits = (59, Language = C | CXX, HasBraces = false)
BlockDeclBits = {
= 59
IsVariadic = true
CapturesCXXThis = true
BlockMissingReturnType = false
IsConversionFromLambda = false
DoesNotEscape = false
CanAvoidCopyToHeap = false
}
}
FirstDecl = nullptr
LastDecl = nullptr
}
clang::Redeclarable<clang::TagDecl> = {
RedeclLink = {
Link = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> >,
llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> >,
llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> > > >,
0, llvm::PointerUnion<clang::Decl *, const void *>,
clang::LazyGenerationalUpdatePtr<const clang::Decl *, clang::Decl *, void
(clang::ExternalASTSource::*)(const clang::Decl *)> > = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::
Decl *)> >, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> >,
llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> > > >,
1, clang::LazyGenerationalUpdatePtr<const clang::Decl *, clang::Decl *, void
(clang::ExternalASTSource::*)(const clang::Decl *)> > = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang
::Decl *)> >, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void (clang::ExternalASTSource::*)(const clang::Decl *)> >,
llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<llvm::PointerUnion<clang::Decl
*, const void *>, clang::LazyGenerationalUpdatePtr<const clang::Decl *,
clang::Decl *, void
(clang::ExternalASTSource::*)(const clang::Decl *)> > > >, 2> = {
Val = {
Value = (Data = "\xb0\xfeH\xba\xf10")
}
}
}
}
}
}
First = 0x000030f1ba48feb0
}
BraceRange = {
B = (ID = 0)
E = (ID = 0)
}
TypedefNameDeclOrQualifier = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::TypedefNameDecl
*, clang::QualifierInfo *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang::
QualifierInfo *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang::QualifierInfo *> > >, 0, clang::TypedefNameDecl *, clang::QualifierInfo
*> = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::TypedefNameDecl
*, clang::QualifierInfo *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang
::QualifierInfo *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang::QualifierInfo *> > >, 1, clang::QualifierInfo *> = {
llvm::pointer_union_detail::PointerUnionMembers<llvm::PointerUnion<clang::TypedefNameDecl
*, clang::QualifierInfo *>, llvm::PointerIntPair<void *, 1, int,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang::QualifierInfo *>, llvm::PointerIntPairInfo<void *, 1,
llvm::pointer_union_detail::PointerUnionUIntTraits<clang::TypedefNameDecl *,
clang::QualifierInfo *> > >, 2> = {
Val = {
Value = (Data = "")
}
}
}
}
}
}
}
```
</details>
It looks like it happens as you suspected ("`D->hasExternalLexicalStorage()`
returns `false` and we skip the call to `CompleteType`"). I tried changing that
condition to `getExternalSource() && !D->getDefinition()`, but it doesn't help
- I'm still getting the same assertion:
```
assert.h assertion failed at
third_party/llvm/llvm-project/clang/lib/AST/RecordLayoutBuilder.cpp:3377 in
const ASTRecordLayout &clang::ASTContext::getASTRecordLayout(const RecordDecl
*) const: D && "Cannot get layout of forward declarations!"
```
```
(lldb)
3360
3361 /// getASTRecordLayout - Get or compute information about the layout of
the
3362 /// specified record (struct/union/class), which indicates its size and
field
3363 /// position information.
3364 const ASTRecordLayout &
3365 ASTContext::getASTRecordLayout(const RecordDecl *D) const {
3366 // These asserts test different things. A record has a definition
3367 // as soon as we begin to parse the definition. That definition is
3368 // not a complete definition (which is what isDefinition() tests)
3369 // until we *finish* parsing the definition.
(lldb)
3370
3371 if (getExternalSource() && !D->getDefinition())
3372 getExternalSource()->CompleteType(const_cast<RecordDecl*>(D));
3373 // Complete the redecl chain (if necessary).
3374 (void)D->getMostRecentDecl();
3375
3376 D = D->getDefinition();
3377 assert(D && "Cannot get layout of forward declarations!");
```
One more thing I noticed while trying to debug what's happening in the
`getExternalSource()->CompleteType(const_cast<RecordDecl*>(D));` is that the
crash happens after a different number of calls to `getASTRecordLayout()`. It
varies in a wide range (I saw numbers from ~800 to ~15000 hits of a breakpoint
on line 3371 when the crash occurred). This makes me think about a
non-determinism here.
Please let me know, if you have any more ideas I could try.
> > Meanwhile, CVise made some progress (but not much). The input is down to
> > 250MB and ~12.5K files.
>
> Ok, if possible maybe you can keep it running in parallel, we will still need
> a reproducer as a regression test later on.
The test case is down to ~130MB and 9K files, but still nothing I can share
directly.
https://github.com/llvm/llvm-project/pull/133057
_______________________________________________
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
