[PATCH] D159497: [RFC][clangd] Check if SelectionTree is complete

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

> One such heuristic is to check the nodes in the selection tree for the exact 
> line match

It looks like this heuristic doesn't work well in multiline cases.
E.g.

  int
  f^unc() {
return 0;
  }

will give empty result with  this patch applied.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D159497/new/

https://reviews.llvm.org/D159497

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D138546: Clangd: Preserve target flags in system includes extractor

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

It seems this patch can break GCC toolchains support

This patch expects that we never get target-related options in `CommandLine` 
passed to `extractSystemIncludesAndTarget()` for **GCC** toolchain. But seems 
this is not always true because of 
https://github.com/llvm/llvm-project/blob/main/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp#L253

In other words, with this patch we preserve `--target=...`, but this option can 
appear from 
https://github.com/llvm/llvm-project/blob/main/clang-tools-extra/clangd/GlobalCompilationDatabase.cpp#L253
 (but not from `compile_commands.json`). As the result of it, we can call 
**GCC** toolchain with `--target=..` option at trying to query-driver and this 
call will fail (because GCC has no --target option)


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D138546/new/

https://reviews.llvm.org/D138546

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D109239: Add support for floating-option `-ffp-eval-method` and for new `pragma clang fp eval-method`

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

It seems for x86_32 case (e.g. with -m32 in command line) there is a difference 
between clang and GCC:
clang: `__FLT_EVAL_METHOD__` == 0
GCC: `__FLT_EVAL_METHOD__` == 2

Example:
https://godbolt.org/z/EbY8rPYGn

I am not sure, is it a correct behavior from clang?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D109239/new/

https://reviews.llvm.org/D109239

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D135362: [clang] Make variables of undeduced types to have dependent alignment

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG5a42c90b778c: [clang] Make variables of undeduced types to 
have dependent alignment (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D135362/new/

https://reviews.llvm.org/D135362

Files:
  clang/lib/AST/Decl.cpp
  clang/test/Sema/tls_alignment.cpp


Index: clang/test/Sema/tls_alignment.cpp
===
--- clang/test/Sema/tls_alignment.cpp
+++ clang/test/Sema/tls_alignment.cpp
@@ -1,6 +1,6 @@
 // TLS variable cannot be aligned to more than 32 bytes on PS4.
 
-// RUN: %clang_cc1 -triple x86_64-scei-ps4 -fsyntax-only -verify %s
+// RUN: %clang_cc1 -triple x86_64-scei-ps4 -std=c++17 -fsyntax-only -verify %s
 
 
 // A non-aligned type.
@@ -18,6 +18,12 @@
 int some_aligned_data[12] __attribute__(( aligned(64) )); // 48 bytes of 
stuff, aligned to 64.
 };
 
+// A templated type
+template 
+struct templated_struct {};
+// expected-note@-1{{candidate template ignored: couldn't infer template 
argument ''}}
+// expected-note@-2{{candidate function template not viable: requires 1 
argument, but 0 were provided}}
+
 // A typedef of the aligned struct.
 typedef aligned_struct another_aligned_struct;
 
@@ -42,6 +48,9 @@
 // Variable aligned because of typedef, second case.
 __thread yet_another_aligned_structbar5; // expected-error{{alignment 
(64) of thread-local variable}}
 
+// No crash for undeduced type.
+__thread templated_struct  bar6; // expected-error{{no viable 
constructor or deduction guide for deduction of template arguments of 
'templated_struct'}}
+
 int baz ()
 {
 return foo.some_data[0] + bar.some_data[1] + bar2.some_data[2] +
Index: clang/lib/AST/Decl.cpp
===
--- clang/lib/AST/Decl.cpp
+++ clang/lib/AST/Decl.cpp
@@ -2581,7 +2581,7 @@
 
 bool VarDecl::hasDependentAlignment() const {
   QualType T = getType();
-  return T->isDependentType() || T->isUndeducedAutoType() ||
+  return T->isDependentType() || T->isUndeducedType() ||
  llvm::any_of(specific_attrs(), [](const AlignedAttr *AA) 
{
return AA->isAlignmentDependent();
  });


Index: clang/test/Sema/tls_alignment.cpp
===
--- clang/test/Sema/tls_alignment.cpp
+++ clang/test/Sema/tls_alignment.cpp
@@ -1,6 +1,6 @@
 // TLS variable cannot be aligned to more than 32 bytes on PS4.
 
-// RUN: %clang_cc1 -triple x86_64-scei-ps4 -fsyntax-only -verify %s
+// RUN: %clang_cc1 -triple x86_64-scei-ps4 -std=c++17 -fsyntax-only -verify %s
 
 
 // A non-aligned type.
@@ -18,6 +18,12 @@
 int some_aligned_data[12] __attribute__(( aligned(64) )); // 48 bytes of stuff, aligned to 64.
 };
 
+// A templated type
+template 
+struct templated_struct {};
+// expected-note@-1{{candidate template ignored: couldn't infer template argument ''}}
+// expected-note@-2{{candidate function template not viable: requires 1 argument, but 0 were provided}}
+
 // A typedef of the aligned struct.
 typedef aligned_struct another_aligned_struct;
 
@@ -42,6 +48,9 @@
 // Variable aligned because of typedef, second case.
 __thread yet_another_aligned_structbar5; // expected-error{{alignment (64) of thread-local variable}}
 
+// No crash for undeduced type.
+__thread templated_struct  bar6; // expected-error{{no viable constructor or deduction guide for deduction of template arguments of 'templated_struct'}}
+
 int baz ()
 {
 return foo.some_data[0] + bar.some_data[1] + bar2.some_data[2] +
Index: clang/lib/AST/Decl.cpp
===
--- clang/lib/AST/Decl.cpp
+++ clang/lib/AST/Decl.cpp
@@ -2581,7 +2581,7 @@
 
 bool VarDecl::hasDependentAlignment() const {
   QualType T = getType();
-  return T->isDependentType() || T->isUndeducedAutoType() ||
+  return T->isDependentType() || T->isUndeducedType() ||
  llvm::any_of(specific_attrs(), [](const AlignedAttr *AA) {
return AA->isAlignmentDependent();
  });
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D135362: [clang] Make variables of undeduced types to have dependent alignment

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: aaron.ballman, rsmith, mizvekov.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

Without this patch `VarDecl::hasDependent()` checks only undeduced auto types, 
so can give false negatives result for other undeduced types.
This lead to crashes in sequence `!VarDecl::hasDepentent()` => `getDeclAlign()`.

It seems this problem appeared since D105380 


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D135362

Files:
  clang/lib/AST/Decl.cpp
  clang/test/Sema/tls_alignment.cpp


Index: clang/test/Sema/tls_alignment.cpp
===
--- clang/test/Sema/tls_alignment.cpp
+++ clang/test/Sema/tls_alignment.cpp
@@ -1,6 +1,6 @@
 // TLS variable cannot be aligned to more than 32 bytes on PS4.
 
-// RUN: %clang_cc1 -triple x86_64-scei-ps4 -fsyntax-only -verify %s
+// RUN: %clang_cc1 -triple x86_64-scei-ps4 -std=c++17 -fsyntax-only -verify %s
 
 
 // A non-aligned type.
@@ -18,6 +18,12 @@
 int some_aligned_data[12] __attribute__(( aligned(64) )); // 48 bytes of 
stuff, aligned to 64.
 };
 
+// A templated type
+template 
+struct templated_struct {};
+// expected-note@-1{{candidate template ignored: couldn't infer template 
argument ''}}
+// expected-note@-2{{candidate function template not viable: requires 1 
argument, but 0 were provided}}
+
 // A typedef of the aligned struct.
 typedef aligned_struct another_aligned_struct;
 
@@ -42,6 +48,9 @@
 // Variable aligned because of typedef, second case.
 __thread yet_another_aligned_structbar5; // expected-error{{alignment 
(64) of thread-local variable}}
 
+// No crash for undeduced type.
+__thread templated_struct  bar6; // expected-error{{no viable 
constructor or deduction guide for deduction of template arguments of 
'templated_struct'}}
+
 int baz ()
 {
 return foo.some_data[0] + bar.some_data[1] + bar2.some_data[2] +
Index: clang/lib/AST/Decl.cpp
===
--- clang/lib/AST/Decl.cpp
+++ clang/lib/AST/Decl.cpp
@@ -2581,7 +2581,7 @@
 
 bool VarDecl::hasDependentAlignment() const {
   QualType T = getType();
-  return T->isDependentType() || T->isUndeducedAutoType() ||
+  return T->isDependentType() || T->isUndeducedType() ||
  llvm::any_of(specific_attrs(), [](const AlignedAttr *AA) 
{
return AA->isAlignmentDependent();
  });


Index: clang/test/Sema/tls_alignment.cpp
===
--- clang/test/Sema/tls_alignment.cpp
+++ clang/test/Sema/tls_alignment.cpp
@@ -1,6 +1,6 @@
 // TLS variable cannot be aligned to more than 32 bytes on PS4.
 
-// RUN: %clang_cc1 -triple x86_64-scei-ps4 -fsyntax-only -verify %s
+// RUN: %clang_cc1 -triple x86_64-scei-ps4 -std=c++17 -fsyntax-only -verify %s
 
 
 // A non-aligned type.
@@ -18,6 +18,12 @@
 int some_aligned_data[12] __attribute__(( aligned(64) )); // 48 bytes of stuff, aligned to 64.
 };
 
+// A templated type
+template 
+struct templated_struct {};
+// expected-note@-1{{candidate template ignored: couldn't infer template argument ''}}
+// expected-note@-2{{candidate function template not viable: requires 1 argument, but 0 were provided}}
+
 // A typedef of the aligned struct.
 typedef aligned_struct another_aligned_struct;
 
@@ -42,6 +48,9 @@
 // Variable aligned because of typedef, second case.
 __thread yet_another_aligned_structbar5; // expected-error{{alignment (64) of thread-local variable}}
 
+// No crash for undeduced type.
+__thread templated_struct  bar6; // expected-error{{no viable constructor or deduction guide for deduction of template arguments of 'templated_struct'}}
+
 int baz ()
 {
 return foo.some_data[0] + bar.some_data[1] + bar2.some_data[2] +
Index: clang/lib/AST/Decl.cpp
===
--- clang/lib/AST/Decl.cpp
+++ clang/lib/AST/Decl.cpp
@@ -2581,7 +2581,7 @@
 
 bool VarDecl::hasDependentAlignment() const {
   QualType T = getType();
-  return T->isDependentType() || T->isUndeducedAutoType() ||
+  return T->isDependentType() || T->isUndeducedType() ||
  llvm::any_of(specific_attrs(), [](const AlignedAttr *AA) {
return AA->isAlignmentDependent();
  });
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134379: [clangd] IncludeCleaner: handle using namespace

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D134379#3808043 , @kadircet wrote:

> In D134379#3807770 , @ArcsinX wrote:
>
>> Anyway if this is the only concern, we can handle namespace declaration as a 
>> special case inside `ReferencedLocationCrawler::add()`. something like this:
>>
>>   diff
>>   -for (const Decl *Redecl : D->redecls())
>>   -  Result.User.insert(Redecl->getLocation());
>>   +if (llvm::isa(D)) {
>>   +  Result.User.insert(D->getCanonicalDecl()->getLocation());
>>   +} else {
>>   +  for (const Decl *Redecl : D->redecls())
>>   +Result.User.insert(Redecl->getLocation());
>>   +}
>>
>> And in the above example `#include bar.h` will be suggested to remove
>>
>> Could this be a suitable solution?
>
> I don't think picking the canonical declaration would be helpful in general 
> case; it'll likely preserve the first header all the time, which might be 
> unused otherwise.
>
> I feel like the best option here is, diagnosing the unused using directive as 
> well. but I am not sure if changing the lookup semantics would always be 
> "right".

Diagnosing the unused using directive is a good option, but it's not related 
with `IncludeCleaner`. For now `IncludeCleaner` suggests to remove a header, 
but removing it leads to compile errors, which looks wrong.

> The other option would be to consider headers in batches after full analysis 
> is complete, while keeping track of symbols provided by a particular header.
> That way when we have a symbol that's only re-declared in a bunch of headers, 
> we can rank them based on other symbols being provided and only keep the 
> headers that are providing some extra "used" symbols in addition to redecls.
> In the absence of such we get to pick an option between "all/none/forward 
> declare ourselves". I guess picking "all" in this case is less troublesome, 
> the user will get all the "unused include" warnings as soon as they use a 
> symbol from that namespace.

This looks like a general problem, declaration of a used function in every 
header leads to "every header is used", maybe it's ok as far as it's not a 
common case.

So, yes, with the proposed solution we will get rid of some "false positive" 
cases, but we will get some "false negative" cases.
But IWYU handles this 
https://github.com/include-what-you-use/include-what-you-use/commit/97e236c55e5ebbd95b8bb221fd9497851f67c3cc


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134379/new/

https://reviews.llvm.org/D134379

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134379: [clangd] IncludeCleaner: handle using namespace

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

Anyway if this is the only concern, we can handle namespace declaration as a 
special case inside `ReferencedLocationCrawler::add()`. something like this:

  diff
  -for (const Decl *Redecl : D->redecls())
  -  Result.User.insert(Redecl->getLocation());
  +if (llvm::isa(D)) {
  +  Result.User.insert(D->getCanonicalDecl()->getLocation());
  +} else {
  +  for (const Decl *Redecl : D->redecls())
  +Result.User.insert(Redecl->getLocation());
  +}

And in the above example `#include bar.h` will be suggested to remove

Could this be a suitable solution?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134379/new/

https://reviews.llvm.org/D134379

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134379: [clangd] IncludeCleaner: handle using namespace

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D134379#3806664 , @sammccall wrote:

> Is test.h meaningfully used in that example?
> Yes, the code is going to fail to compile without it, but it seems like the 
> "spirit" of IWYU would say to delete both the include and the using directive.

Yes, in this example both should be removed, but from IDE user point of view, I 
expect warning about unused `using namespace ...`, and after it will be 
removed, warning about unused include can appear.

Also:

- I think that in unsure cases, we should keep includes and avoid code with 
errors
- IWYU tool doesn't report that include can be removed

> My concern about marking it used is that namespaces are typically redeclared 
> in *every* header, and this effectively disables the feature on large swaths 
> of code:
>
>   // foo.h
>   namespace myproj { void foo(); }
>   
>   // bar.h
>   namespace myproj { void bar(); }
>   
>   // main.cc
>   #include "foo.h"
>   #include "bar.h" // not meaningfully used
>   
>   using namespace myproj;
>   int main() {
>foo();
>   }

In this case I don't see a difference with other declaration types. E.g. 
function

  // foo.h
  void foo();
  
  // bar.h
  void foo();
  
  // main.cc
  #include "foo.h"
  #include "bar.h"
  
  int main() {
   foo();
  }

both foo.h and bar.h contains prototype of foo(), but bar.h can be removed


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D134379/new/

https://reviews.llvm.org/D134379

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D134379: [clangd] IncludeCleaner: handle using namespace

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: sammccall, kbobyrev.
Herald added subscribers: kadircet, arphaman.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

IncludeCleaner suggests to remove `#include "test.h"` in the following example:

test.h

  namespace ns {}

test.cpp

  include "test.h"
  using namespace ns;

This patch fixes this behavior by handling using-directive declarations


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D134379

Files:
  clang-tools-extra/clangd/IncludeCleaner.cpp
  clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -77,7 +77,7 @@
   "using ns::foo;",
   },
   {
-  "namespace ns { void foo(); void foo() {}; }",
+  "namespace ^ns { void foo(); void foo() {}; }",
   "using namespace ns;",
   },
   {
Index: clang-tools-extra/clangd/IncludeCleaner.cpp
===
--- clang-tools-extra/clangd/IncludeCleaner.cpp
+++ clang-tools-extra/clangd/IncludeCleaner.cpp
@@ -133,6 +133,11 @@
 return true;
   }
 
+  bool VisitUsingDirectiveDecl(UsingDirectiveDecl *D) {
+add(D->getNominatedNamespace());
+return true;
+  }
+
   // Enums may be usefully forward-declared as *complete* types by specifying
   // an underlying type. In this case, the definition should see the 
declaration
   // so they can be checked for compatibility.


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -77,7 +77,7 @@
   "using ns::foo;",
   },
   {
-  "namespace ns { void foo(); void foo() {}; }",
+  "namespace ^ns { void foo(); void foo() {}; }",
   "using namespace ns;",
   },
   {
Index: clang-tools-extra/clangd/IncludeCleaner.cpp
===
--- clang-tools-extra/clangd/IncludeCleaner.cpp
+++ clang-tools-extra/clangd/IncludeCleaner.cpp
@@ -133,6 +133,11 @@
 return true;
   }
 
+  bool VisitUsingDirectiveDecl(UsingDirectiveDecl *D) {
+add(D->getNominatedNamespace());
+return true;
+  }
+
   // Enums may be usefully forward-declared as *complete* types by specifying
   // an underlying type. In this case, the definition should see the declaration
   // so they can be checked for compatibility.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type is dependent

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
ArcsinX marked an inline comment as done.
Closed by commit rG3ce7d256f2d7: [clang][RecoveryExpr] Dont perform 
alignment check if parameter type is… (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133886/new/

https://reviews.llvm.org/D133886

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/SemaCXX/recovery-expr-type.cpp


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,12 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+void f() {
+  struct {
+void m(int (&)[undefined()]) {} // expected-error {{undeclared identifier}}
+  } S;
+  S.m(1); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5690,9 +5690,9 @@
 
   // Find expected alignment, and the actual alignment of the passed object.
   // getTypeAlignInChars requires complete types
-  if (ArgTy.isNull() || ParamTy->isIncompleteType() ||
-  ArgTy->isIncompleteType() || ParamTy->isUndeducedType() ||
-  ArgTy->isUndeducedType())
+  if (ArgTy.isNull() || ParamTy->isDependentType() ||
+  ParamTy->isIncompleteType() || ArgTy->isIncompleteType() ||
+  ParamTy->isUndeducedType() || ArgTy->isUndeducedType())
 return;
 
   CharUnits ParamAlign = Context.getTypeAlignInChars(ParamTy);


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,12 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+void f() {
+  struct {
+void m(int (&)[undefined()]) {} // expected-error {{undeclared identifier}}
+  } S;
+  S.m(1); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5690,9 +5690,9 @@
 
   // Find expected alignment, and the actual alignment of the passed object.
   // getTypeAlignInChars requires complete types
-  if (ArgTy.isNull() || ParamTy->isIncompleteType() ||
-  ArgTy->isIncompleteType() || ParamTy->isUndeducedType() ||
-  ArgTy->isUndeducedType())
+  if (ArgTy.isNull() || ParamTy->isDependentType() ||
+  ParamTy->isIncompleteType() || ArgTy->isIncompleteType() ||
+  ParamTy->isUndeducedType() || ArgTy->isUndeducedType())
 return;
 
   CharUnits ParamAlign = Context.getTypeAlignInChars(ParamTy);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type contains errors

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked 3 inline comments as done.
ArcsinX added inline comments.



Comment at: clang/lib/Sema/SemaChecking.cpp:5779
 QualType ParamTy = Proto->getParamType(ArgIdx);
+if (ParamTy->containsErrors())
+  continue;

hokein wrote:
> ArcsinX wrote:
> > hokein wrote:
> > > It looks like for the failure case the `ParamTy` for the parameter is a 
> > > dependent array type, and it violates the "non-dependent" assumption of 
> > > `clang::ASTContext::getTypeInfoImpl` which is called by  
> > > `getTypeAlignInChars` in `CheckArgAlignment`.
> > > 
> > > so I'd suggest moving the fix to `CheckArgAlignment` line 5685 (adding a 
> > > `ParamTy->isDependentType()` to the `if` condition).
> > When I found this problem I fixed it like you are suggesting. But after 
> > that I replaced it with this check, because it seems there is no reason to 
> > try to check something on code with errors.
> > 
> > I mean that even if we are not crashing, results from `CheckArgAlignment()` 
> > can't be trusted if we are passing  something with errors to it.
> > because it seems there is no reason to try to check something on code with 
> > errors.
> 
> I think this is case-by-case (for this case, it may be true) -- in some cases 
> (see test7 and test8 for example in `recovery-expr-type.cpp`), we do want to 
> check something even on the code with errors to emit useful secondary 
> diagnostics. In general we want to emit a full list of diagnostics and at the 
> same time avoid any suspicious diagnostics, however achieving both goals is 
> hard.
> 
> Depending on how fatal the error is
> 
> - if the error is fatal, RecoveryError is just a dependent-type wrapper, we 
> should not call check* to emit diagnostics (we're less certain about the 
> quality of these diagnostics), this is mostly done by leveraging on the 
> existing template dependent mechanism;
> - if the error is minor, RecoveryError preserves a concrete type, we might 
> want call check* to emit diagnostics as we're more confident; 
> 
> The current solution makes sense to fix the crash, but I think the main 
> reason is that `CheckArgAlignment` now can be invoked with a dependent-type 
> parameter in non-template context (after we extended the "dependent" concept 
> from depending on template parameters to depending on template parameters and 
> errors), so we should fix `CheckArgAlignment`.
> 
Thanks for clarification. Moved check inside CheckArgAlignment() as you 
suggested.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133886/new/

https://reviews.llvm.org/D133886

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type contains errors

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 460358.
ArcsinX added a comment.

- Check for dependent type inside CheckArgAlignment()
- Simplify test


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133886/new/

https://reviews.llvm.org/D133886

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/SemaCXX/recovery-expr-type.cpp


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,12 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+void f() {
+  struct {
+void m(int (&)[undefined()]) {} // expected-error {{undeclared identifier}}
+  } S;
+  S.m(1); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5690,9 +5690,9 @@
 
   // Find expected alignment, and the actual alignment of the passed object.
   // getTypeAlignInChars requires complete types
-  if (ArgTy.isNull() || ParamTy->isIncompleteType() ||
-  ArgTy->isIncompleteType() || ParamTy->isUndeducedType() ||
-  ArgTy->isUndeducedType())
+  if (ArgTy.isNull() || ParamTy->isDependentType() ||
+  ParamTy->isIncompleteType() || ArgTy->isIncompleteType() ||
+  ParamTy->isUndeducedType() || ArgTy->isUndeducedType())
 return;
 
   CharUnits ParamAlign = Context.getTypeAlignInChars(ParamTy);


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,12 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+void f() {
+  struct {
+void m(int (&)[undefined()]) {} // expected-error {{undeclared identifier}}
+  } S;
+  S.m(1); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5690,9 +5690,9 @@
 
   // Find expected alignment, and the actual alignment of the passed object.
   // getTypeAlignInChars requires complete types
-  if (ArgTy.isNull() || ParamTy->isIncompleteType() ||
-  ArgTy->isIncompleteType() || ParamTy->isUndeducedType() ||
-  ArgTy->isUndeducedType())
+  if (ArgTy.isNull() || ParamTy->isDependentType() ||
+  ParamTy->isIncompleteType() || ArgTy->isIncompleteType() ||
+  ParamTy->isUndeducedType() || ArgTy->isUndeducedType())
 return;
 
   CharUnits ParamAlign = Context.getTypeAlignInChars(ParamTy);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type contains errors

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang/lib/Sema/SemaChecking.cpp:5779
 QualType ParamTy = Proto->getParamType(ArgIdx);
+if (ParamTy->containsErrors())
+  continue;

hokein wrote:
> It looks like for the failure case the `ParamTy` for the parameter is a 
> dependent array type, and it violates the "non-dependent" assumption of 
> `clang::ASTContext::getTypeInfoImpl` which is called by  
> `getTypeAlignInChars` in `CheckArgAlignment`.
> 
> so I'd suggest moving the fix to `CheckArgAlignment` line 5685 (adding a 
> `ParamTy->isDependentType()` to the `if` condition).
When I found this problem I fixed it like you are suggesting. But after that I 
replaced it with this check, because it seems there is no reason to try to 
check something on code with errors.

I mean that even if we are not crashing, results from `CheckArgAlignment()` 
can't be trusted if we are passing  something with errors to it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133886/new/

https://reviews.llvm.org/D133886

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type contains errors

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

This looks similar to the problem fixed in D103825 



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133886/new/

https://reviews.llvm.org/D133886

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133886: [clang][RecoveryExpr] Don't perform alignment check if parameter type contains errors

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: hokein, kadircet.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

This patch fixes a crash which appears because of CheckArgAlignment() call with 
error type.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D133886

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/SemaCXX/recovery-expr-type.cpp


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,15 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+const UnknownType SZ = 8; // expected-error {{unknown type}}
+typedef unsigned char UC;
+void f() {
+  struct {
+void m(UC (&)[SZ]) {}
+  } S;
+  unsigned char A[8];
+  S.m(A); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5776,6 +5776,8 @@
   checkAIXMemberAlignment((Arg->getExprLoc()), Arg);
 
 QualType ParamTy = Proto->getParamType(ArgIdx);
+if (ParamTy->containsErrors())
+  continue;
 QualType ArgTy = Arg->getType();
 CheckArgAlignment(Arg->getExprLoc(), FDecl, std::to_string(ArgIdx + 1),
   ArgTy, ParamTy);


Index: clang/test/SemaCXX/recovery-expr-type.cpp
===
--- clang/test/SemaCXX/recovery-expr-type.cpp
+++ clang/test/SemaCXX/recovery-expr-type.cpp
@@ -162,3 +162,15 @@
   b = __builtin_object_size(c, 0); // crash2
 }
 }
+
+namespace test15 {
+const UnknownType SZ = 8; // expected-error {{unknown type}}
+typedef unsigned char UC;
+void f() {
+  struct {
+void m(UC (&)[SZ]) {}
+  } S;
+  unsigned char A[8];
+  S.m(A); // no crash
+}
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -5776,6 +5776,8 @@
   checkAIXMemberAlignment((Arg->getExprLoc()), Arg);
 
 QualType ParamTy = Proto->getParamType(ArgIdx);
+if (ParamTy->containsErrors())
+  continue;
 QualType ArgTy = Arg->getType();
 CheckArgAlignment(Arg->getExprLoc(), FDecl, std::to_string(ArgIdx + 1),
   ArgTy, ParamTy);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D133043: [clangd] Fix tests for implicit C function declaration

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGcc4b86cfc01c: [clangd] Fix tests for implicit C function 
declaration (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D133043/new/

https://reviews.llvm.org/D133043

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -1465,11 +1465,12 @@
 TEST(IncludeFixerTest, HeaderNamedInDiag) {
   Annotations Test(R"cpp(
 $insert[[]]int main() {
-  [[printf]](""); // error-ok
+  [[printf]]("");
 }
   )cpp");
   auto TU = TestTU::withCode(Test.code());
-  TU.ExtraArgs = {"-xc"};
+  TU.ExtraArgs = {"-xc", "-std=c99",
+  "-Wno-error=implicit-function-declaration"};
   auto Index = buildIndexWithSymbol({});
   TU.ExternalIndex = Index.get();
 
@@ -1482,13 +1483,22 @@
  "declarations"),
   withFix(Fix(Test.range("insert"), "#include \n",
   "Include  for symbol printf");
+
+  TU.ExtraArgs = {"-xc", "-std=c89"};
+  EXPECT_THAT(
+  *TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicitly declaring library function 'printf' "
+ "with type 'int (const char *, ...)'"),
+  withFix(Fix(Test.range("insert"), "#include \n",
+  "Include  for symbol printf");
 }
 
 TEST(IncludeFixerTest, CImplicitFunctionDecl) {
-  Annotations Test("void x() { [[foo]](); /* error-ok */ }");
+  Annotations Test("void x() { [[foo]](); }");
   auto TU = TestTU::withCode(Test.code());
   TU.Filename = "test.c";
-  TU.ExtraArgs.push_back("-std=c99");
+  TU.ExtraArgs = {"-std=c99", "-Wno-error=implicit-function-declaration"};
 
   Symbol Sym = func("foo");
   Sym.Flags |= Symbol::IndexedForCodeCompletion;
@@ -1509,6 +1519,13 @@
"support implicit function declarations"),
   withFix(Fix(Range{}, "#include \"foo.h\"\n",
   "Include \"foo.h\" for symbol foo");
+
+  TU.ExtraArgs = {"-std=c89", "-Wall"};
+  EXPECT_THAT(*TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicit declaration of function 'foo'"),
+  withFix(Fix(Range{}, "#include \"foo.h\"\n",
+  "Include \"foo.h\" for symbol foo");
 }
 
 TEST(DiagsInHeaders, DiagInsideHeader) {
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -493,6 +493,7 @@
 // We restore the original severity in the level adjuster.
 // FIXME: It would be better to have a real API for this, but what?
 for (auto ID : {diag::ext_implicit_function_decl_c99,
+diag::ext_implicit_lib_function_decl,
 diag::ext_implicit_lib_function_decl_c99,
 diag::warn_implicit_function_decl}) {
   OverriddenSeverity.try_emplace(


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -1465,11 +1465,12 @@
 TEST(IncludeFixerTest, HeaderNamedInDiag) {
   Annotations Test(R"cpp(
 $insert[[]]int main() {
-  [[printf]](""); // error-ok
+  [[printf]]("");
 }
   )cpp");
   auto TU = TestTU::withCode(Test.code());
-  TU.ExtraArgs = {"-xc"};
+  TU.ExtraArgs = {"-xc", "-std=c99",
+  "-Wno-error=implicit-function-declaration"};
   auto Index = buildIndexWithSymbol({});
   TU.ExternalIndex = Index.get();
 
@@ -1482,13 +1483,22 @@
  "declarations"),
   withFix(Fix(Test.range("insert"), "#include \n",
   "Include  for symbol printf");
+
+  TU.ExtraArgs = {"-xc", "-std=c89"};
+  EXPECT_THAT(
+  *TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicitly declaring library function 'printf' "
+ "with type 'int (const char *, ...)'"),
+  withFix(Fix(Test.range("insert"), "#include \n",
+  "Include  for symbol printf");
 }
 
 TEST(IncludeFixerTest, CImplicitFunctionDecl) {
-  Annotations Test("void x() { [[foo]](); /* error-ok */ }");
+  Annotations Test("void x() { [[foo]](); }");
   auto TU = TestTU::withCode(Test.code());
   TU.Filename = "test.c";
-  TU.ExtraArgs.push_back("-std=c99");
+  TU.ExtraArgs = {"-std=c99", 

[PATCH] D133043: [clangd] Fix tests for implicit C function declaration

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: aaron.ballman, sammccall, kadircet.
Herald added a subscriber: arphaman.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

clangd code fixes at D122983  were not right.
We need to check that clangd provides IncludeFixer fixits for implicit function 
declaration even if this is not an error (e.g. implicit function declaration in 
C89).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D133043

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -1465,11 +1465,12 @@
 TEST(IncludeFixerTest, HeaderNamedInDiag) {
   Annotations Test(R"cpp(
 $insert[[]]int main() {
-  [[printf]](""); // error-ok
+  [[printf]]("");
 }
   )cpp");
   auto TU = TestTU::withCode(Test.code());
-  TU.ExtraArgs = {"-xc"};
+  TU.ExtraArgs = {"-xc", "-std=c99",
+  "-Wno-error=implicit-function-declaration"};
   auto Index = buildIndexWithSymbol({});
   TU.ExternalIndex = Index.get();
 
@@ -1482,13 +1483,22 @@
  "declarations"),
   withFix(Fix(Test.range("insert"), "#include \n",
   "Include  for symbol printf");
+
+  TU.ExtraArgs = {"-xc", "-std=c89"};
+  EXPECT_THAT(
+  *TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicitly declaring library function 'printf' "
+ "with type 'int (const char *, ...)'"),
+  withFix(Fix(Test.range("insert"), "#include \n",
+  "Include  for symbol printf");
 }
 
 TEST(IncludeFixerTest, CImplicitFunctionDecl) {
-  Annotations Test("void x() { [[foo]](); /* error-ok */ }");
+  Annotations Test("void x() { [[foo]](); }");
   auto TU = TestTU::withCode(Test.code());
   TU.Filename = "test.c";
-  TU.ExtraArgs.push_back("-std=c99");
+  TU.ExtraArgs = {"-std=c99", "-Wno-error=implicit-function-declaration"};
 
   Symbol Sym = func("foo");
   Sym.Flags |= Symbol::IndexedForCodeCompletion;
@@ -1509,6 +1519,13 @@
"support implicit function declarations"),
   withFix(Fix(Range{}, "#include \"foo.h\"\n",
   "Include \"foo.h\" for symbol foo");
+
+  TU.ExtraArgs = {"-std=c89", "-Wall"};
+  EXPECT_THAT(*TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicit declaration of function 'foo'"),
+  withFix(Fix(Range{}, "#include \"foo.h\"\n",
+  "Include \"foo.h\" for symbol foo");
 }
 
 TEST(DiagsInHeaders, DiagInsideHeader) {
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -493,6 +493,7 @@
 // We restore the original severity in the level adjuster.
 // FIXME: It would be better to have a real API for this, but what?
 for (auto ID : {diag::ext_implicit_function_decl_c99,
+diag::ext_implicit_lib_function_decl,
 diag::ext_implicit_lib_function_decl_c99,
 diag::warn_implicit_function_decl}) {
   OverriddenSeverity.try_emplace(


Index: clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
===
--- clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
+++ clang-tools-extra/clangd/unittests/DiagnosticsTests.cpp
@@ -1465,11 +1465,12 @@
 TEST(IncludeFixerTest, HeaderNamedInDiag) {
   Annotations Test(R"cpp(
 $insert[[]]int main() {
-  [[printf]](""); // error-ok
+  [[printf]]("");
 }
   )cpp");
   auto TU = TestTU::withCode(Test.code());
-  TU.ExtraArgs = {"-xc"};
+  TU.ExtraArgs = {"-xc", "-std=c99",
+  "-Wno-error=implicit-function-declaration"};
   auto Index = buildIndexWithSymbol({});
   TU.ExternalIndex = Index.get();
 
@@ -1482,13 +1483,22 @@
  "declarations"),
   withFix(Fix(Test.range("insert"), "#include \n",
   "Include  for symbol printf");
+
+  TU.ExtraArgs = {"-xc", "-std=c89"};
+  EXPECT_THAT(
+  *TU.build().getDiagnostics(),
+  ElementsAre(AllOf(
+  Diag(Test.range(), "implicitly declaring library function 'printf' "
+ "with type 'int (const char *, ...)'"),
+  withFix(Fix(Test.range("insert"), "#include \n",
+  "Include  for symbol printf");
 }
 
 TEST(IncludeFixerTest, 

[PATCH] D131091: [clang][index] Index unresolved member expression as reference

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGee648c0ce09b: [clang][index] Index unresolved member 
expression as reference (authored by denis-fatkulin, committed by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131091/new/

https://reviews.llvm.org/D131091

Files:
  clang-tools-extra/clangd/unittests/XRefsTests.cpp
  clang/lib/Index/IndexBody.cpp
  clang/test/Index/Core/index-dependent-source.cpp


Index: clang/test/Index/Core/index-dependent-source.cpp
===
--- clang/test/Index/Core/index-dependent-source.cpp
+++ clang/test/Index/Core/index-dependent-source.cpp
@@ -231,3 +231,12 @@
   foo();
 // CHECK: [[@LINE-1]]:3 | function/C | foo | c:@FT@>1#Tfoo#v# |  | 
Ref,Call,RelCall,RelCont | rel: 1
 }
+
+struct Foo {
+  template  void bar();
+  // CHECK: [[@LINE-1]]:30 | instance-method/C++ | bar | 
c:@S@Foo@FT@>1#Tbar#v# |  | Decl,RelChild | rel: 1
+};
+template  void baz(Foo f) {
+  f.bar();
+  // CHECK: [[@LINE-1]]:5 | instance-method/C++ | bar | c:@S@Foo@FT@>1#Tbar#v# 
|  | Ref,Call,RelCall,RelCont | rel: 1
+}
Index: clang/lib/Index/IndexBody.cpp
===
--- clang/lib/Index/IndexBody.cpp
+++ clang/lib/Index/IndexBody.cpp
@@ -468,7 +468,7 @@
 return true;
   }
 
-  bool VisitUnresolvedLookupExpr(UnresolvedLookupExpr *E) {
+  bool VisitOverloadExpr(OverloadExpr *E) {
 SmallVector Relations;
 SymbolRoleSet Roles = getRolesForRef(E, Relations);
 for (auto *D : E->decls())
Index: clang-tools-extra/clangd/unittests/XRefsTests.cpp
===
--- clang-tools-extra/clangd/unittests/XRefsTests.cpp
+++ clang-tools-extra/clangd/unittests/XRefsTests.cpp
@@ -2090,6 +2090,14 @@
   [[f^oo]](s);
 }
   )cpp",
+  R"cpp(// unresolved member expression
+struct Foo {
+  template  void $decl[[b^ar]](T t); 
+};
+template  void test(Foo F, T t) {
+  F.[[bar]](t);
+}
+  )cpp",
 
   // Enum base
   R"cpp(


Index: clang/test/Index/Core/index-dependent-source.cpp
===
--- clang/test/Index/Core/index-dependent-source.cpp
+++ clang/test/Index/Core/index-dependent-source.cpp
@@ -231,3 +231,12 @@
   foo();
 // CHECK: [[@LINE-1]]:3 | function/C | foo | c:@FT@>1#Tfoo#v# |  | Ref,Call,RelCall,RelCont | rel: 1
 }
+
+struct Foo {
+  template  void bar();
+  // CHECK: [[@LINE-1]]:30 | instance-method/C++ | bar | c:@S@Foo@FT@>1#Tbar#v# |  | Decl,RelChild | rel: 1
+};
+template  void baz(Foo f) {
+  f.bar();
+  // CHECK: [[@LINE-1]]:5 | instance-method/C++ | bar | c:@S@Foo@FT@>1#Tbar#v# |  | Ref,Call,RelCall,RelCont | rel: 1
+}
Index: clang/lib/Index/IndexBody.cpp
===
--- clang/lib/Index/IndexBody.cpp
+++ clang/lib/Index/IndexBody.cpp
@@ -468,7 +468,7 @@
 return true;
   }
 
-  bool VisitUnresolvedLookupExpr(UnresolvedLookupExpr *E) {
+  bool VisitOverloadExpr(OverloadExpr *E) {
 SmallVector Relations;
 SymbolRoleSet Roles = getRolesForRef(E, Relations);
 for (auto *D : E->decls())
Index: clang-tools-extra/clangd/unittests/XRefsTests.cpp
===
--- clang-tools-extra/clangd/unittests/XRefsTests.cpp
+++ clang-tools-extra/clangd/unittests/XRefsTests.cpp
@@ -2090,6 +2090,14 @@
   [[f^oo]](s);
 }
   )cpp",
+  R"cpp(// unresolved member expression
+struct Foo {
+  template  void $decl[[b^ar]](T t); 
+};
+template  void test(Foo F, T t) {
+  F.[[bar]](t);
+}
+  )cpp",
 
   // Enum base
   R"cpp(
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131706: [clangd][unittests][IncludeCleaner] Don't call findReferencedFiles() if the result is not used

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG42ee0d8c16f7: [clangd][unittests][IncludeCleaner] Dont 
call findReferencedFiles() if the… (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131706/new/

https://reviews.llvm.org/D131706

Files:
  clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -571,9 +571,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   EXPECT_THAT(computeUnusedIncludes(AST), IsEmpty());
 }
@@ -596,9 +593,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   // FIXME: This is not correct: foo.h is unused but is not diagnosed as such
   // because we ignore headers with IWYU export pragmas for now.


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -571,9 +571,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   EXPECT_THAT(computeUnusedIncludes(AST), IsEmpty());
 }
@@ -596,9 +593,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   // FIXME: This is not correct: foo.h is unused but is not diagnosed as such
   // because we ignore headers with IWYU export pragmas for now.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D131706: [clangd][unittests][IncludeCleaner] Don't call findReferencedFiles() if the result is not used

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: kbobyrev, sammccall, kadircet.
Herald added subscribers: usaxena95, arphaman.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

IncludeCleaner.RecursiveInclusion and IncludeCleaner.IWYUPragmaExport tests 
don't check referenced files list, so we don't need to call 
findReferencedFiles() there.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D131706

Files:
  clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -571,9 +571,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   EXPECT_THAT(computeUnusedIncludes(AST), IsEmpty());
 }
@@ -596,9 +593,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   // FIXME: This is not correct: foo.h is unused but is not diagnosed as such
   // because we ignore headers with IWYU export pragmas for now.


Index: clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
===
--- clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
+++ clang-tools-extra/clangd/unittests/IncludeCleanerTests.cpp
@@ -571,9 +571,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   EXPECT_THAT(computeUnusedIncludes(AST), IsEmpty());
 }
@@ -596,9 +593,6 @@
   )cpp");
   ParsedAST AST = TU.build();
 
-  auto ReferencedFiles = findReferencedFiles(
-  findReferencedLocations(AST), AST.getIncludeStructure(),
-  AST.getCanonicalIncludes(), AST.getSourceManager());
   EXPECT_THAT(AST.getDiagnostics(), llvm::ValueIs(IsEmpty()));
   // FIXME: This is not correct: foo.h is unused but is not diagnosed as such
   // because we ignore headers with IWYU export pragmas for now.
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D130417: [clang-format] Missing space between trailing return type 'auto' and left brace

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG4977fd2192fc: [clang-format] Missing space between trailing 
return type auto and left brace (authored by denis-fatkulin, 
committed by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D130417/new/

https://reviews.llvm.org/D130417

Files:
  clang/lib/Format/TokenAnnotator.cpp
  clang/unittests/Format/FormatTest.cpp


Index: clang/unittests/Format/FormatTest.cpp
===
--- clang/unittests/Format/FormatTest.cpp
+++ clang/unittests/Format/FormatTest.cpp
@@ -23550,6 +23550,16 @@
   verifyFormat("auto lambda = [ = a]() { a = 2; };", AlignStyle);
 }
 
+TEST_F(FormatTest, TrailingReturnTypeAuto) {
+  FormatStyle Style = getLLVMStyle();
+  verifyFormat("[]() -> auto { return Val; }", Style);
+  verifyFormat("[]() -> auto * { return Val; }", Style);
+  verifyFormat("[]() -> auto & { return Val; }", Style);
+  verifyFormat("auto foo() -> auto { return Val; }", Style);
+  verifyFormat("auto foo() -> auto * { return Val; }", Style);
+  verifyFormat("auto foo() -> auto & { return Val; }", Style);
+}
+
 TEST_F(FormatTest, SpacesInConditionalStatement) {
   FormatStyle Spaces = getLLVMStyle();
   Spaces.IfMacros.clear();
Index: clang/lib/Format/TokenAnnotator.cpp
===
--- clang/lib/Format/TokenAnnotator.cpp
+++ clang/lib/Format/TokenAnnotator.cpp
@@ -3310,6 +3310,11 @@
 }
   }
 
+  // trailing return type 'auto': []() -> auto {}, auto foo() -> auto {}
+  if (Left.is(tok::kw_auto) &&
+  Right.isOneOf(TT_LambdaLBrace, TT_FunctionLBrace))
+return true;
+
   // auto{x} auto(x)
   if (Left.is(tok::kw_auto) && Right.isOneOf(tok::l_paren, tok::l_brace))
 return false;


Index: clang/unittests/Format/FormatTest.cpp
===
--- clang/unittests/Format/FormatTest.cpp
+++ clang/unittests/Format/FormatTest.cpp
@@ -23550,6 +23550,16 @@
   verifyFormat("auto lambda = [ = a]() { a = 2; };", AlignStyle);
 }
 
+TEST_F(FormatTest, TrailingReturnTypeAuto) {
+  FormatStyle Style = getLLVMStyle();
+  verifyFormat("[]() -> auto { return Val; }", Style);
+  verifyFormat("[]() -> auto * { return Val; }", Style);
+  verifyFormat("[]() -> auto & { return Val; }", Style);
+  verifyFormat("auto foo() -> auto { return Val; }", Style);
+  verifyFormat("auto foo() -> auto * { return Val; }", Style);
+  verifyFormat("auto foo() -> auto & { return Val; }", Style);
+}
+
 TEST_F(FormatTest, SpacesInConditionalStatement) {
   FormatStyle Spaces = getLLVMStyle();
   Spaces.IfMacros.clear();
Index: clang/lib/Format/TokenAnnotator.cpp
===
--- clang/lib/Format/TokenAnnotator.cpp
+++ clang/lib/Format/TokenAnnotator.cpp
@@ -3310,6 +3310,11 @@
 }
   }
 
+  // trailing return type 'auto': []() -> auto {}, auto foo() -> auto {}
+  if (Left.is(tok::kw_auto) &&
+  Right.isOneOf(TT_LambdaLBrace, TT_FunctionLBrace))
+return true;
+
   // auto{x} auto(x)
   if (Left.is(tok::kw_auto) && Right.isOneOf(tok::l_paren, tok::l_brace))
 return false;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D130299: [clang-format] FIX: Misannotation 'auto' as trailing return type in lambdas

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG17fb879764dc: [clang-format] FIX: Misannotation 
auto as trailing return type in lambdas (authored by 
denis-fatkulin, committed by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D130299/new/

https://reviews.llvm.org/D130299

Files:
  clang/lib/Format/UnwrappedLineParser.cpp
  clang/unittests/Format/TokenAnnotatorTest.cpp


Index: clang/unittests/Format/TokenAnnotatorTest.cpp
===
--- clang/unittests/Format/TokenAnnotatorTest.cpp
+++ clang/unittests/Format/TokenAnnotatorTest.cpp
@@ -747,6 +747,21 @@
   ASSERT_EQ(Tokens.size(), 8u) << Tokens;
   EXPECT_TOKEN(Tokens[0], tok::l_square, TT_LambdaLSquare);
   EXPECT_TOKEN(Tokens[5], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto {}");
+  ASSERT_EQ(Tokens.size(), 9u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[6], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto & {}");
+  ASSERT_EQ(Tokens.size(), 10u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[7], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto * {}");
+  ASSERT_EQ(Tokens.size(), 10u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[7], tok::l_brace, TT_LambdaLBrace);
 }
 
 } // namespace
Index: clang/lib/Format/UnwrappedLineParser.cpp
===
--- clang/lib/Format/UnwrappedLineParser.cpp
+++ clang/lib/Format/UnwrappedLineParser.cpp
@@ -2108,6 +2108,7 @@
 case tok::l_square:
   parseSquare();
   break;
+case tok::kw_auto:
 case tok::kw_class:
 case tok::kw_template:
 case tok::kw_typename:


Index: clang/unittests/Format/TokenAnnotatorTest.cpp
===
--- clang/unittests/Format/TokenAnnotatorTest.cpp
+++ clang/unittests/Format/TokenAnnotatorTest.cpp
@@ -747,6 +747,21 @@
   ASSERT_EQ(Tokens.size(), 8u) << Tokens;
   EXPECT_TOKEN(Tokens[0], tok::l_square, TT_LambdaLSquare);
   EXPECT_TOKEN(Tokens[5], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto {}");
+  ASSERT_EQ(Tokens.size(), 9u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[6], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto & {}");
+  ASSERT_EQ(Tokens.size(), 10u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[7], tok::l_brace, TT_LambdaLBrace);
+
+  Tokens = annotate("[]() -> auto * {}");
+  ASSERT_EQ(Tokens.size(), 10u) << Tokens;
+  EXPECT_TOKEN(Tokens[4], tok::arrow, TT_LambdaArrow);
+  EXPECT_TOKEN(Tokens[7], tok::l_brace, TT_LambdaLBrace);
 }
 
 } // namespace
Index: clang/lib/Format/UnwrappedLineParser.cpp
===
--- clang/lib/Format/UnwrappedLineParser.cpp
+++ clang/lib/Format/UnwrappedLineParser.cpp
@@ -2108,6 +2108,7 @@
 case tok::l_square:
   parseSquare();
   break;
+case tok::kw_auto:
 case tok::kw_class:
 case tok::kw_template:
 case tok::kw_typename:
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGb2c3ae0b6f05: [Sema] Dont check bounds for function 
pointer (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122748/new/

https://reviews.llvm.org/D122748

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/Sema/unbounded-array-bounds.c


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15495,6 +15495,8 @@
 ND = ME->getMemberDecl();
 
   if (IsUnboundedArray) {
+if (EffectiveType->isFunctionType())
+  return;
 if (index.isUnsigned() || !index.isNegative()) {
   const auto  = getASTContext();
   unsigned AddrBits =


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15495,6 +15495,8 @@
 ND = ME->getMemberDecl();
 
   if (IsUnboundedArray) {
+if (EffectiveType->isFunctionType())
+  return;
 if (index.isUnsigned() || !index.isNegative()) {
   const auto  = getASTContext();
   unsigned AddrBits =
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

Friendly ping


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122748/new/

https://reviews.llvm.org/D122748

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

Friendly ping.

In D122748#3417500 , @erichkeane 
wrote:

> would still like to see the feedback from the original submitter here to see 
> if there is more work to do in here that would be valuable.

Unsure, maybe we can process without him? It seems his last activity here was 
more than six months ago.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122748/new/

https://reviews.llvm.org/D122748

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D122748#3417240 , @erichkeane 
wrote:

> That said, we might not want to early-exist here, I think we can just skip 
> the `IsUnboundedArray` branch?

It seems you are right, thanks, fixed.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122748/new/

https://reviews.llvm.org/D122748

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 419242.
ArcsinX added a comment.

Check for function type only if IsUnboundedArray is true


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D122748/new/

https://reviews.llvm.org/D122748

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/Sema/unbounded-array-bounds.c


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15468,6 +15468,8 @@
 ND = ME->getMemberDecl();
 
   if (IsUnboundedArray) {
+if (EffectiveType->isFunctionType())
+  return;
 if (index.isUnsigned() || !index.isNegative()) {
   const auto  = getASTContext();
   unsigned AddrBits =


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15468,6 +15468,8 @@
 ND = ME->getMemberDecl();
 
   if (IsUnboundedArray) {
+if (EffectiveType->isFunctionType())
+  return;
 if (index.isUnsigned() || !index.isNegative()) {
   const auto  = getASTContext();
   unsigned AddrBits =
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D122748: [Sema] Don't check bounds for function pointer

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: aaron.ballman, erichkeane, abhinavgaba, 
chrish_ericsson_atx.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

Currently, clang crashes with i386 target on the following code:

  void f() {
f + 0xdeadUL;
  }

This problem is similar to the problem fixed in D104424 
, but that fix can't handle function pointer 
case, because `getTypeSizeInCharsIfKnown()` says that size is known and equal 
to 0 for function type.

This patch prevents bounds checking for function pointer, thus fixes the crash.

Fixes https://github.com/llvm/llvm-project/issues/50463


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D122748

Files:
  clang/lib/Sema/SemaChecking.cpp
  clang/test/Sema/unbounded-array-bounds.c


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15447,7 +15447,7 @@
   const Type *BaseType =
   ArrayTy == nullptr ? nullptr : ArrayTy->getElementType().getTypePtr();
   bool IsUnboundedArray = (BaseType == nullptr);
-  if (EffectiveType->isDependentType() ||
+  if (EffectiveType->isDependentType() || EffectiveType->isFunctionType() ||
   (!IsUnboundedArray && BaseType->isDependentType()))
 return;
 


Index: clang/test/Sema/unbounded-array-bounds.c
===
--- clang/test/Sema/unbounded-array-bounds.c
+++ clang/test/Sema/unbounded-array-bounds.c
@@ -80,3 +80,7 @@
   (void *)0 + 0xdeadUL;
   // no array-bounds warning, and no crash
 }
+
+void func() {
+  func + 0xdeadUL; // no crash
+}
Index: clang/lib/Sema/SemaChecking.cpp
===
--- clang/lib/Sema/SemaChecking.cpp
+++ clang/lib/Sema/SemaChecking.cpp
@@ -15447,7 +15447,7 @@
   const Type *BaseType =
   ArrayTy == nullptr ? nullptr : ArrayTy->getElementType().getTypePtr();
   bool IsUnboundedArray = (BaseType == nullptr);
-  if (EffectiveType->isDependentType() ||
+  if (EffectiveType->isDependentType() || EffectiveType->isFunctionType() ||
   (!IsUnboundedArray && BaseType->isDependentType()))
 return;
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG8bd00557e3f4: [clang][parser] Allow GNU attributes before 
namespace identifier (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121245/new/

https://reviews.llvm.org/D121245

Files:
  clang/docs/ReleaseNotes.rst
  clang/lib/Parse/ParseDeclCXX.cpp
  clang/test/Parser/namespace-attributes.cpp

Index: clang/test/Parser/namespace-attributes.cpp
===
--- /dev/null
+++ clang/test/Parser/namespace-attributes.cpp
@@ -0,0 +1,41 @@
+// RUN: %clang_cc1 -std=c++17 -verify %s
+
+namespace __attribute__(()) A
+{
+}
+
+namespace A __attribute__(())
+{
+}
+
+namespace __attribute__(()) [[]] A
+{
+}
+
+namespace [[]] __attribute__(()) A
+{
+}
+
+namespace A __attribute__(()) [[]]
+{
+}
+
+namespace A [[]] __attribute__(())
+{
+}
+
+namespace [[]] A __attribute__(())
+{
+}
+
+namespace __attribute__(()) A [[]]
+{
+}
+
+namespace A::B __attribute__(()) // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
+
+namespace __attribute__(()) A::B // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
Index: clang/lib/Parse/ParseDeclCXX.cpp
===
--- clang/lib/Parse/ParseDeclCXX.cpp
+++ clang/lib/Parse/ParseDeclCXX.cpp
@@ -74,15 +74,27 @@
   SourceLocation FirstNestedInlineLoc;
 
   ParsedAttributesWithRange attrs(AttrFactory);
-  SourceLocation attrLoc;
-  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
-Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
-? diag::warn_cxx14_compat_ns_enum_attribute
-: diag::ext_ns_enum_attribute)
-  << 0 /*namespace*/;
-attrLoc = Tok.getLocation();
-ParseCXX11Attributes(attrs);
-  }
+
+  auto ReadAttributes = [&] {
+bool MoreToParse;
+do {
+  MoreToParse = false;
+  if (Tok.is(tok::kw___attribute)) {
+ParseGNUAttributes(attrs);
+MoreToParse = true;
+  }
+  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
+Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
+? diag::warn_cxx14_compat_ns_enum_attribute
+: diag::ext_ns_enum_attribute)
+<< 0 /*namespace*/;
+ParseCXX11Attributes(attrs);
+MoreToParse = true;
+  }
+} while (MoreToParse);
+  };
+
+  ReadAttributes();
 
   if (Tok.is(tok::identifier)) {
 Ident = Tok.getIdentifierInfo();
@@ -108,16 +120,14 @@
 }
   }
 
+  ReadAttributes();
+
+  SourceLocation attrLoc = attrs.Range.getBegin();
+
   // A nested namespace definition cannot have attributes.
   if (!ExtraNSs.empty() && attrLoc.isValid())
 Diag(attrLoc, diag::err_unexpected_nested_namespace_attribute);
 
-  // Read label attributes, if present.
-  if (Tok.is(tok::kw___attribute)) {
-attrLoc = Tok.getLocation();
-ParseGNUAttributes(attrs);
-  }
-
   if (Tok.is(tok::equal)) {
 if (!Ident) {
   Diag(Tok, diag::err_expected) << tok::identifier;
Index: clang/docs/ReleaseNotes.rst
===
--- clang/docs/ReleaseNotes.rst
+++ clang/docs/ReleaseNotes.rst
@@ -94,6 +94,11 @@
   locations a declaration attribute may appear.
   This fixes `Issue 53805 `_.
 
+- Improved namespace attributes handling:
+  - Handle GNU attributes before a namespace identifier and subsequent
+attributes of different kinds.
+  - Emit error on GNU attributes for a nested namespace definition.
+
 Windows Support
 ---
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D121245#3380346 , @aaron.ballman 
wrote:

> LGTM! Might be worth adding a release note for it (does this close any bugs 
> in the bug database? If so, it'd be worth mentioning those in the commit 
> message and release note).

Thank you for review.

I failed to find any opened bugs related to this. I'm not sure how detailed 
information about this patch should be in release notes. Can you please take a 
look at `ReleaseNotes.rst` modifications?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121245/new/

https://reviews.llvm.org/D121245

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 415470.
ArcsinX added a comment.

Update release notes
Rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121245/new/

https://reviews.llvm.org/D121245

Files:
  clang/docs/ReleaseNotes.rst
  clang/lib/Parse/ParseDeclCXX.cpp
  clang/test/Parser/namespace-attributes.cpp

Index: clang/test/Parser/namespace-attributes.cpp
===
--- /dev/null
+++ clang/test/Parser/namespace-attributes.cpp
@@ -0,0 +1,41 @@
+// RUN: %clang_cc1 -std=c++17 -verify %s
+
+namespace __attribute__(()) A
+{
+}
+
+namespace A __attribute__(())
+{
+}
+
+namespace __attribute__(()) [[]] A
+{
+}
+
+namespace [[]] __attribute__(()) A
+{
+}
+
+namespace A __attribute__(()) [[]]
+{
+}
+
+namespace A [[]] __attribute__(())
+{
+}
+
+namespace [[]] A __attribute__(())
+{
+}
+
+namespace __attribute__(()) A [[]]
+{
+}
+
+namespace A::B __attribute__(()) // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
+
+namespace __attribute__(()) A::B // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
Index: clang/lib/Parse/ParseDeclCXX.cpp
===
--- clang/lib/Parse/ParseDeclCXX.cpp
+++ clang/lib/Parse/ParseDeclCXX.cpp
@@ -74,15 +74,27 @@
   SourceLocation FirstNestedInlineLoc;
 
   ParsedAttributesWithRange attrs(AttrFactory);
-  SourceLocation attrLoc;
-  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
-Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
-? diag::warn_cxx14_compat_ns_enum_attribute
-: diag::ext_ns_enum_attribute)
-  << 0 /*namespace*/;
-attrLoc = Tok.getLocation();
-ParseCXX11Attributes(attrs);
-  }
+
+  auto ReadAttributes = [&] {
+bool MoreToParse;
+do {
+  MoreToParse = false;
+  if (Tok.is(tok::kw___attribute)) {
+ParseGNUAttributes(attrs);
+MoreToParse = true;
+  }
+  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
+Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
+? diag::warn_cxx14_compat_ns_enum_attribute
+: diag::ext_ns_enum_attribute)
+<< 0 /*namespace*/;
+ParseCXX11Attributes(attrs);
+MoreToParse = true;
+  }
+} while (MoreToParse);
+  };
+
+  ReadAttributes();
 
   if (Tok.is(tok::identifier)) {
 Ident = Tok.getIdentifierInfo();
@@ -108,16 +120,14 @@
 }
   }
 
+  ReadAttributes();
+
+  SourceLocation attrLoc = attrs.Range.getBegin();
+
   // A nested namespace definition cannot have attributes.
   if (!ExtraNSs.empty() && attrLoc.isValid())
 Diag(attrLoc, diag::err_unexpected_nested_namespace_attribute);
 
-  // Read label attributes, if present.
-  if (Tok.is(tok::kw___attribute)) {
-attrLoc = Tok.getLocation();
-ParseGNUAttributes(attrs);
-  }
-
   if (Tok.is(tok::equal)) {
 if (!Ident) {
   Diag(Tok, diag::err_expected) << tok::identifier;
Index: clang/docs/ReleaseNotes.rst
===
--- clang/docs/ReleaseNotes.rst
+++ clang/docs/ReleaseNotes.rst
@@ -94,6 +94,11 @@
   locations a declaration attribute may appear.
   This fixes `Issue 53805 `_.
 
+- Improved namespace attributes handling:
+  - Handle GNU attributes before a namespace identifier and subsequent
+attributes of different kinds.
+  - Emit error on GNU attributes for a nested namespace definition.
+
 Windows Support
 ---
 
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked an inline comment as done.
ArcsinX added inline comments.



Comment at: clang/lib/Parse/ParseDeclCXX.cpp:79
+
+  auto ReadLabelAttrubutes = [&] {
+// Read label attributes, if present.

aaron.ballman wrote:
> However, I don't think there's a reason we need this lambda -- it seems we 
> can call `MaybeParseGNUAttributes()` instead, and get the attribute location 
> from the `ParsedAttributesWithRange` object passed in.
`MaybeParseAttributes(PAKM_GNU | PAKM_CXX11, Attrs)` parses different kind of 
attributes in a loop (e.g. `struct __attribute__(()) [[]] [[]] 
__attribute__(()) S {};` is valid for clang), but we can't use it because we 
need to warn about c++11 attributes usage for namespace if c++ < 17. So, I 
added a lambda which works similar to `MaybeParseAttributes()`.

Thus, now clang handles attributes for namespace like it handle attributes for 
structure.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121245/new/

https://reviews.llvm.org/D121245

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 414868.
ArcsinX added a comment.

- get attributes location from ParsedAttributesWithRange object
- parse subsequent attributes of different kinds in a loop


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D121245/new/

https://reviews.llvm.org/D121245

Files:
  clang/lib/Parse/ParseDeclCXX.cpp
  clang/test/Parser/namespace-attributes.cpp

Index: clang/test/Parser/namespace-attributes.cpp
===
--- /dev/null
+++ clang/test/Parser/namespace-attributes.cpp
@@ -0,0 +1,41 @@
+// RUN: %clang_cc1 -std=c++17 -verify %s
+
+namespace __attribute__(()) A
+{
+}
+
+namespace A __attribute__(())
+{
+}
+
+namespace __attribute__(()) [[]] A
+{
+}
+
+namespace [[]] __attribute__(()) A
+{
+}
+
+namespace A __attribute__(()) [[]]
+{
+}
+
+namespace A [[]] __attribute__(())
+{
+}
+
+namespace [[]] A __attribute__(())
+{
+}
+
+namespace __attribute__(()) A [[]]
+{
+}
+
+namespace A::B __attribute__(()) // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
+
+namespace __attribute__(()) A::B // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
Index: clang/lib/Parse/ParseDeclCXX.cpp
===
--- clang/lib/Parse/ParseDeclCXX.cpp
+++ clang/lib/Parse/ParseDeclCXX.cpp
@@ -74,15 +74,27 @@
   SourceLocation FirstNestedInlineLoc;
 
   ParsedAttributesWithRange attrs(AttrFactory);
-  SourceLocation attrLoc;
-  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
-Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
-? diag::warn_cxx14_compat_ns_enum_attribute
-: diag::ext_ns_enum_attribute)
-  << 0 /*namespace*/;
-attrLoc = Tok.getLocation();
-ParseCXX11Attributes(attrs);
-  }
+
+  auto ReadAttributes = [&] {
+bool MoreToParse;
+do {
+  MoreToParse = false;
+  if (Tok.is(tok::kw___attribute)) {
+ParseGNUAttributes(attrs);
+MoreToParse = true;
+  }
+  if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
+Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
+? diag::warn_cxx14_compat_ns_enum_attribute
+: diag::ext_ns_enum_attribute)
+<< 0 /*namespace*/;
+ParseCXX11Attributes(attrs);
+MoreToParse = true;
+  }
+} while (MoreToParse);
+  };
+
+  ReadAttributes();
 
   if (Tok.is(tok::identifier)) {
 Ident = Tok.getIdentifierInfo();
@@ -108,16 +120,14 @@
 }
   }
 
+  ReadAttributes();
+
+  SourceLocation attrLoc = attrs.Range.getBegin();
+
   // A nested namespace definition cannot have attributes.
   if (!ExtraNSs.empty() && attrLoc.isValid())
 Diag(attrLoc, diag::err_unexpected_nested_namespace_attribute);
 
-  // Read label attributes, if present.
-  if (Tok.is(tok::kw___attribute)) {
-attrLoc = Tok.getLocation();
-ParseGNUAttributes(attrs);
-  }
-
   if (Tok.is(tok::equal)) {
 if (!Ident) {
   Diag(Tok, diag::err_expected) << tok::identifier;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D121245: [clang][parser] Allow GNU attributes before namespace identifier

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: aaron.ballman, rsmith.
Herald added a subscriber: jdoerfert.
Herald added a project: All.
ArcsinX requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

GCC supports:

- `namespace  identifier`
- `namespace identifier `

But clang supports only `namespace identifier ` and diagnostics 
for `namespace  identifier` case looks unclear:
Code:

  namespace __attribute__((visibility("hidden"))) A
  {
  }

Diags:

  test.cpp:1:49: error: expected identifier or '{'
  namespace __attribute__((visibility("hidden"))) A
  ^
  test.cpp:1:49: error: C++ requires a type specifier for all declarations
  test.cpp:3:2: error: expected ';' after top level declarator
  }

This patch adds support for `namespace  identifier` and also 
forbids gnu attributes for nested namespaces (this already done for C++ 
attributes).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121245

Files:
  clang/lib/Parse/ParseDeclCXX.cpp
  clang/test/Parser/namespace-attributes.cpp


Index: clang/test/Parser/namespace-attributes.cpp
===
--- /dev/null
+++ clang/test/Parser/namespace-attributes.cpp
@@ -0,0 +1,17 @@
+// RUN: %clang_cc1 -std=c++17 -verify %s
+
+namespace __attribute__((visibility("hidden"))) A
+{
+}
+
+namespace B __attribute__((visibility("hidden")))
+{
+}
+
+namespace C::D __attribute__((visibility("hidden"))) // 
expected-error{{attributes cannot be specified on a nested namespace 
definition}}
+{
+}
+
+namespace __attribute__((visibility("hidden"))) E::F // 
expected-error{{attributes cannot be specified on a nested namespace 
definition}}
+{
+}
Index: clang/lib/Parse/ParseDeclCXX.cpp
===
--- clang/lib/Parse/ParseDeclCXX.cpp
+++ clang/lib/Parse/ParseDeclCXX.cpp
@@ -75,6 +75,17 @@
 
   ParsedAttributesWithRange attrs(AttrFactory);
   SourceLocation attrLoc;
+
+  auto ReadLabelAttrubutes = [&] {
+// Read label attributes, if present.
+if (Tok.is(tok::kw___attribute)) {
+  attrLoc = Tok.getLocation();
+  ParseGNUAttributes(attrs);
+}
+  };
+
+  ReadLabelAttrubutes();
+
   if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
 Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
 ? diag::warn_cxx14_compat_ns_enum_attribute
@@ -108,16 +119,12 @@
 }
   }
 
+  ReadLabelAttrubutes();
+
   // A nested namespace definition cannot have attributes.
   if (!ExtraNSs.empty() && attrLoc.isValid())
 Diag(attrLoc, diag::err_unexpected_nested_namespace_attribute);
 
-  // Read label attributes, if present.
-  if (Tok.is(tok::kw___attribute)) {
-attrLoc = Tok.getLocation();
-ParseGNUAttributes(attrs);
-  }
-
   if (Tok.is(tok::equal)) {
 if (!Ident) {
   Diag(Tok, diag::err_expected) << tok::identifier;


Index: clang/test/Parser/namespace-attributes.cpp
===
--- /dev/null
+++ clang/test/Parser/namespace-attributes.cpp
@@ -0,0 +1,17 @@
+// RUN: %clang_cc1 -std=c++17 -verify %s
+
+namespace __attribute__((visibility("hidden"))) A
+{
+}
+
+namespace B __attribute__((visibility("hidden")))
+{
+}
+
+namespace C::D __attribute__((visibility("hidden"))) // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
+
+namespace __attribute__((visibility("hidden"))) E::F // expected-error{{attributes cannot be specified on a nested namespace definition}}
+{
+}
Index: clang/lib/Parse/ParseDeclCXX.cpp
===
--- clang/lib/Parse/ParseDeclCXX.cpp
+++ clang/lib/Parse/ParseDeclCXX.cpp
@@ -75,6 +75,17 @@
 
   ParsedAttributesWithRange attrs(AttrFactory);
   SourceLocation attrLoc;
+
+  auto ReadLabelAttrubutes = [&] {
+// Read label attributes, if present.
+if (Tok.is(tok::kw___attribute)) {
+  attrLoc = Tok.getLocation();
+  ParseGNUAttributes(attrs);
+}
+  };
+
+  ReadLabelAttrubutes();
+
   if (getLangOpts().CPlusPlus11 && isCXX11AttributeSpecifier()) {
 Diag(Tok.getLocation(), getLangOpts().CPlusPlus17
 ? diag::warn_cxx14_compat_ns_enum_attribute
@@ -108,16 +119,12 @@
 }
   }
 
+  ReadLabelAttrubutes();
+
   // A nested namespace definition cannot have attributes.
   if (!ExtraNSs.empty() && attrLoc.isValid())
 Diag(attrLoc, diag::err_unexpected_nested_namespace_attribute);
 
-  // Read label attributes, if present.
-  if (Tok.is(tok::kw___attribute)) {
-attrLoc = Tok.getLocation();
-ParseGNUAttributes(attrs);
-  }
-
   if (Tok.is(tok::equal)) {
 if (!Ident) {
   Diag(Tok, diag::err_expected) << tok::identifier;
___
cfe-commits mailing list
cfe-commits@lists.llvm.org

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG491c154677bc: [analyzer] Dont specify PLUGIN_TOOL for 
analyzer plugins (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

Files:
  clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
  clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
  clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt


Index: clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
+++ clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE 
${CMAKE_CURRENT_SOURCE_DIR}/SampleAnalyzerPlugin.exports)
-add_llvm_library(SampleAnalyzerPlugin MODULE BUILDTREE_ONLY 
MainCallChecker.cpp PLUGIN_TOOL clang)
+add_llvm_library(SampleAnalyzerPlugin MODULE BUILDTREE_ONLY 
MainCallChecker.cpp)
 
 clang_target_link_libraries(SampleAnalyzerPlugin PRIVATE
   clangAnalysis
Index: clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
+++ clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE 
${CMAKE_CURRENT_SOURCE_DIR}/CheckerOptionHandlingAnalyzerPlugin.exports)
-add_llvm_library(CheckerOptionHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY 
CheckerOptionHandling.cpp PLUGIN_TOOL clang)
+add_llvm_library(CheckerOptionHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY 
CheckerOptionHandling.cpp)
 
 clang_target_link_libraries(CheckerOptionHandlingAnalyzerPlugin PRIVATE
   clangAnalysis
Index: clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
+++ clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE 
${CMAKE_CURRENT_SOURCE_DIR}/CheckerDependencyHandlingAnalyzerPlugin.exports)
-add_llvm_library(CheckerDependencyHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY 
CheckerDependencyHandling.cpp PLUGIN_TOOL clang)
+add_llvm_library(CheckerDependencyHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY 
CheckerDependencyHandling.cpp)
 
 clang_target_link_libraries(CheckerDependencyHandlingAnalyzerPlugin PRIVATE
   clangAnalysis


Index: clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
+++ clang/lib/Analysis/plugins/SampleAnalyzer/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/SampleAnalyzerPlugin.exports)
-add_llvm_library(SampleAnalyzerPlugin MODULE BUILDTREE_ONLY MainCallChecker.cpp PLUGIN_TOOL clang)
+add_llvm_library(SampleAnalyzerPlugin MODULE BUILDTREE_ONLY MainCallChecker.cpp)
 
 clang_target_link_libraries(SampleAnalyzerPlugin PRIVATE
   clangAnalysis
Index: clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
+++ clang/lib/Analysis/plugins/CheckerOptionHandling/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/CheckerOptionHandlingAnalyzerPlugin.exports)
-add_llvm_library(CheckerOptionHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY CheckerOptionHandling.cpp PLUGIN_TOOL clang)
+add_llvm_library(CheckerOptionHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY CheckerOptionHandling.cpp)
 
 clang_target_link_libraries(CheckerOptionHandlingAnalyzerPlugin PRIVATE
   clangAnalysis
Index: clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
===
--- clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
+++ clang/lib/Analysis/plugins/CheckerDependencyHandling/CMakeLists.txt
@@ -3,7 +3,7 @@
   )
 
 set(LLVM_EXPORTED_SYMBOL_FILE ${CMAKE_CURRENT_SOURCE_DIR}/CheckerDependencyHandlingAnalyzerPlugin.exports)
-add_llvm_library(CheckerDependencyHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY CheckerDependencyHandling.cpp PLUGIN_TOOL clang)
+add_llvm_library(CheckerDependencyHandlingAnalyzerPlugin MODULE BUILDTREE_ONLY CheckerDependencyHandling.cpp)
 
 clang_target_link_libraries(CheckerDependencyHandlingAnalyzerPlugin PRIVATE
   clangAnalysis
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D116966#3261479 , @beanz wrote:

> Have you looked at the impact on binary size? PLUGIN_TOOL _should_ cause the 
> plugins to link against the copy of LLVM & Clang in the tool target binary 
> which reduces the binary size of the plugin and avoids duplicate global 
> initializations.

To see this difference we need to build this plugin with and without 
`PLUGIN_TOOL` (and  with `-DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On`)
But with `PLUGIN_TOOL` and `-DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On` 
SampleAnalyzerPlugin can't be built using MSVC because of libraries conflict: 
we link DLL agains clang tool and also link libraries which this tool already 
contains (clangAnalysis,  clangAST,  clangStaticAnalyzerCore,  
clangStaticAnalyzerFrontend). You can check build log in the description.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D116966#3261168 , @aaron.ballman 
wrote:

> However, the changes you've made don't look to be specific to building on 
> Windows; this removes `PLUGIN_TOOL` for all targets. I presume it's still 
> needed for non-Windows targets, isn't it?

As I can see, `PLUGIN_TOOL` is unused for non-Windows OS (at least for now): 
https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/AddLLVM.cmake#L648


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

Friendly ping.

I am not sure that this patch is clear, so I will try to explain it again:

- clang static analyzer plugins are *NOT* a typical clang plugins: instead of 
`llvm::Registry<>::Add` usage, these plugins provides `сlang_registerCheckers` 
and `clang_analyzerAPIVersionString` functions which are called from static 
analyzer. So, these plugins work ok without anything special on Windows (but 
typical clang plugins are not and D18826  was 
a try to fix it).
- `PLUGIN_TOOL` together with `LLVM_EXPORT_SYMBOLS_FOR_PLUGINS` (introduced in 
D18826 ) is a try to add support for a typical 
clang plugins as a DLL on Windows. Thus, `PLUGIN_TOOL` does nothing on 
non-Windows OS.
- `PLUGIN_TOOL` does nothing without  `LLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On`, but 
in clang static analyzer plugins `LLVM_EXPORTED_SYMBOL_FILE`  file is used to 
specify symbols we want to export (`сlang_registerCheckers` and 
`clang_analyzerAPIVersionString`)

Thus, for now the only thing `PLUGIN_TOOL` specification for clang static 
analyzer plugins does is breaking MSVC build when 
`LLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On` (and plugins support enabled with 
`-DCLANG_PLUGINS_SUPPORT=On -DLLVM_ENABLE_PLUGINS=On`).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D116966#3243374 , @aaron.ballman 
wrote:

> To be clear, I'm trying to figure out whether plugins are *actually* 
> supported on Windows or whether they just so happen to work if the stars line 
> up right for you. If they're supported, then awesome, let's go forward with 
> this. If they're not supported, it's less clear whether we want to make it 
> easier for the stars to line up or not.

That how it work for me (extra cmake flags: -DLLVM_ENABLE_PLUGINS=On 
-DCLANG_PLUGINS_SUPPORT=On -DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On)

  d:\work\llvm-all\build>bin\clang.exe -Xclang -plugin -Xclang print-fns 
d:\work\1.cpp -c
  error: unable to find plugin 'print-fns'
  
  d:\work\llvm-project\build>bin\clang.exe -fplugin=bin\PrintFunctionNames.dll 
-Xclang -plugin -Xclang print-fns d:\work\1.cpp -c
  top-level-decl: "main"
  top-level-decl: "a"
  
  d:\work\llvm-project\build>cat CMakeCache.txt | grep BUILD_SHARED_LIB
  BUILD_SHARED_LIBS:BOOL=OFF


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D116966#3243318 , @aaron.ballman 
wrote:

> AFAIK, we don't support plugins on Windows. See: 
> https://reviews.llvm.org/D16761#1441359 I don't think anything has changed in 
> this regard.

I can say that I successfully use clang plugins on Windows (and analyzer 
plugins too, which works in a bit different way). Seems this became possible 
after https://reviews.llvm.org/D18826


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D116966#3242170 , @aaronpuchert 
wrote:

> This is only observable on Windows, right?

Thanks for your reply. Yes, PLUGIN_TOOL argument is used on Windows only, so 
this problems can't be observed on non-Windows systems and this patch affects 
only Windows build.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D116966/new/

https://reviews.llvm.org/D116966

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D116966: [analyzer] Don't specify PLUGIN_TOOL for analyzer plugins

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: aaronpuchert, hintonda, NoQ, Szelethus, lebedev.ri.
Herald added subscribers: manas, wenlei, steakhal, ASDenysPetrov, usaxena95, 
dkrupp, donat.nagy, kadircet, mikhail.ramalho, a.sidorin, szepet, 
baloghadamsoftware, xazax.hun, mgorny.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, sstefan1, ilya-biryukov.
Herald added a reviewer: jdoerfert.
Herald added a project: clang.

Analyzer plugins explicitly export clang_registerCheckers and 
clang_analyzerAPIVersionString symbols, so we don't need to specify a tool to 
link agains.

Also, without this patch MSVC build fails with cmake flags 
-DLLVM_ENABLE_PLUGINS=On -DCLANG_PLUGINS_SUPPORT=On 
-DLLVM_EXPORT_SYMBOLS_FOR_PLUGINS=On

  [936/936] Linking CXX shared module bin\SampleAnalyzerPlugin.dll
  FAILED: bin/SampleAnalyzerPlugin.dll
  cmd.exe /C "cd . && "D:\Program Files\CMake\bin\cmake.exe" -E vs_link_dll 
--intdir=tools\clang\lib\Analysis\plugins\SampleAnalyzer\CMakeFiles\SampleAnalyzerPlugin.dir
 --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe 
--mt=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\mt.exe --manifests  -- 
C:\PROGRA~2\MICROS~4\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe
 /nologo 
tools\clang\lib\Analysis\plugins\SampleAnalyzer\CMakeFiles\SampleAnalyzerPlugin.dir\MainCallChecker.cpp.obj
  /out:bin\SampleAnalyzerPlugin.dll /implib:lib\SampleAnalyzerPlugin.lib 
/pdb:bin\SampleAnalyzerPlugin.pdb /dll /version:0.0 /machine:x64 
/INCREMENTAL:NO  
/DEF:"D:/work/llvm-project-original/build-plugins/tools/clang/lib/Analysis/plugins/SampleAnalyzer/SampleAnalyzerPlugin.def"
  lib\clang.lib  lib\clangAnalysis.lib  lib\clangAST.lib  
lib\clangStaticAnalyzerCore.lib  lib\clangStaticAnalyzerFrontend.lib  
lib\clangStaticAnalyzerCheckers.lib  lib\clangStaticAnalyzerCore.lib  
lib\clangCrossTU.lib  lib\clangIndex.lib  lib\clangFormat.lib  
lib\clangToolingInclusions.lib  lib\clangFrontend.lib  lib\clangDriver.lib  
version.lib  lib\clangParse.lib  lib\clangSerialization.lib  lib\clangSema.lib  
lib\clangAnalysis.lib  lib\clangEdit.lib  lib\LLVMOption.lib  
lib\clangToolingCore.lib  lib\clangRewrite.lib  lib\clangASTMatchers.lib  
lib\clangAST.lib  lib\clangLex.lib  lib\clangBasic.lib  
lib\LLVMFrontendOpenMP.lib  lib\LLVMScalarOpts.lib  
lib\LLVMAggressiveInstCombine.lib  lib\LLVMInstCombine.lib  
lib\LLVMTransformUtils.lib  lib\LLVMAnalysis.lib  lib\LLVMProfileData.lib  
lib\LLVMDebugInfoDWARF.lib  lib\LLVMObject.lib  lib\LLVMBitReader.lib  
lib\LLVMCore.lib  lib\LLVMRemarks.lib  lib\LLVMBitstreamReader.lib  
lib\LLVMMCParser.lib  lib\LLVMMC.lib  lib\LLVMDebugInfoCodeView.lib  
lib\LLVMTextAPI.lib  lib\LLVMBinaryFormat.lib  lib\LLVMSupport.lib  psapi.lib  
shell32.lib  ole32.lib  uuid.lib  advapi32.lib  delayimp.lib  
-delayload:shell32.dll  -delayload:ole32.dll  lib\LLVMDemangle.lib  
kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib 
oleaut32.lib uuid.lib comdlg32.lib advapi32.lib  && cd ."
  LINK: command 
"C:\PROGRA~2\MICROS~4\2019\COMMUN~1\VC\Tools\MSVC\1428~1.299\bin\Hostx64\x64\link.exe
 /nologo 
tools\clang\lib\Analysis\plugins\SampleAnalyzer\CMakeFiles\SampleAnalyzerPlugin.dir\MainCallChecker.cpp.obj
 /out:bin\SampleAnalyzerPlugin.dll /implib:lib\SampleAnalyzerPlugin.lib 
/pdb:bin\SampleAnalyzerPlugin.pdb /dll /version:0.0 /machine:x64 
/INCREMENTAL:NO 
/DEF:D:/work/llvm-project-original/build-plugins/tools/clang/lib/Analysis/plugins/SampleAnalyzer/SampleAnalyzerPlugin.def
 lib\clang.lib lib\clangAnalysis.lib lib\clangAST.lib 
lib\clangStaticAnalyzerCore.lib lib\clangStaticAnalyzerFrontend.lib 
lib\clangStaticAnalyzerCheckers.lib lib\clangStaticAnalyzerCore.lib 
lib\clangCrossTU.lib lib\clangIndex.lib lib\clangFormat.lib 
lib\clangToolingInclusions.lib lib\clangFrontend.lib lib\clangDriver.lib 
version.lib lib\clangParse.lib lib\clangSerialization.lib lib\clangSema.lib 
lib\clangAnalysis.lib lib\clangEdit.lib lib\LLVMOption.lib 
lib\clangToolingCore.lib lib\clangRewrite.lib lib\clangASTMatchers.lib 
lib\clangAST.lib lib\clangLex.lib lib\clangBasic.lib lib\LLVMFrontendOpenMP.lib 
lib\LLVMScalarOpts.lib lib\LLVMAggressiveInstCombine.lib 
lib\LLVMInstCombine.lib lib\LLVMTransformUtils.lib lib\LLVMAnalysis.lib 
lib\LLVMProfileData.lib lib\LLVMDebugInfoDWARF.lib lib\LLVMObject.lib 
lib\LLVMBitReader.lib lib\LLVMCore.lib lib\LLVMRemarks.lib 
lib\LLVMBitstreamReader.lib lib\LLVMMCParser.lib lib\LLVMMC.lib 
lib\LLVMDebugInfoCodeView.lib lib\LLVMTextAPI.lib lib\LLVMBinaryFormat.lib 
lib\LLVMSupport.lib psapi.lib shell32.lib ole32.lib uuid.lib advapi32.lib 
delayimp.lib -delayload:shell32.dll -delayload:ole32.dll lib\LLVMDemangle.lib 
kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib 
oleaut32.lib uuid.lib comdlg32.lib advapi32.lib /MANIFEST 
/MANIFESTFILE:bin\SampleAnalyzerPlugin.dll.manifest" failed (exit code 1169) 
with the following output:
  clangStaticAnalyzerCore.lib(BugReporter.cpp.obj) : error LNK2005: 

[PATCH] D115959: [clangd] Fix undefined behavior when generating error message at rename with an invalid name

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG555eacf75f21: [clangd] Fix undefined behavior when 
generating error message at rename with an… (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D115959/new/

https://reviews.llvm.org/D115959

Files:
  clang-tools-extra/clangd/refactor/Rename.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp


Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1060,6 +1060,11 @@
   )cpp",
"conflict", !HeaderFile, "Conflict"},
 
+  {R"cpp(
+int V^ar;
+  )cpp",
+   "\"const\" is a keyword", !HeaderFile, "const"},
+
   {R"cpp(// Trying to rename into the same name, SameName == SameName.
 void func() {
   int S^ameName;
Index: clang-tools-extra/clangd/refactor/Rename.cpp
===
--- clang-tools-extra/clangd/refactor/Rename.cpp
+++ clang-tools-extra/clangd/refactor/Rename.cpp
@@ -455,7 +455,7 @@
 }
 
 llvm::Error makeError(InvalidName Reason) {
-  auto Message = [](InvalidName Reason) {
+  auto Message = [](const InvalidName ) {
 switch (Reason.K) {
 case InvalidName::Keywords:
   return llvm::formatv("the chosen name \"{0}\" is a keyword",
@@ -733,7 +733,7 @@
 return makeError(ReasonToReject::SameName);
   auto Invalid = checkName(RenameDecl, RInputs.NewName);
   if (Invalid)
-return makeError(*Invalid);
+return makeError(std::move(*Invalid));
 
   auto Reject = renameable(RenameDecl, RInputs.MainFilePath, RInputs.Index);
   if (Reject)


Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1060,6 +1060,11 @@
   )cpp",
"conflict", !HeaderFile, "Conflict"},
 
+  {R"cpp(
+int V^ar;
+  )cpp",
+   "\"const\" is a keyword", !HeaderFile, "const"},
+
   {R"cpp(// Trying to rename into the same name, SameName == SameName.
 void func() {
   int S^ameName;
Index: clang-tools-extra/clangd/refactor/Rename.cpp
===
--- clang-tools-extra/clangd/refactor/Rename.cpp
+++ clang-tools-extra/clangd/refactor/Rename.cpp
@@ -455,7 +455,7 @@
 }
 
 llvm::Error makeError(InvalidName Reason) {
-  auto Message = [](InvalidName Reason) {
+  auto Message = [](const InvalidName ) {
 switch (Reason.K) {
 case InvalidName::Keywords:
   return llvm::formatv("the chosen name \"{0}\" is a keyword",
@@ -733,7 +733,7 @@
 return makeError(ReasonToReject::SameName);
   auto Invalid = checkName(RenameDecl, RInputs.NewName);
   if (Invalid)
-return makeError(*Invalid);
+return makeError(std::move(*Invalid));
 
   auto Reject = renameable(RenameDecl, RInputs.MainFilePath, RInputs.Index);
   if (Reject)
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D115959: [clangd] Fix undefined behavior when generating error message at rename with an invalid name

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: sammccall, kadircet, hokein.
Herald added subscribers: usaxena95, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

`Message()` lambda uses `Reason.Details` as an input parameter for 
`llvm::formatv()`, but `Reason` in `Message()` is a local object.
Return value of `llvm::formatv()` contains references to its input arguments, 
thus `Message()` returns an object which contains a reference to `Details` 
field of the local object `Reason`.
This patch fixes this behavior by passing `Reason` as a reference to 
`Message()` to ensure that return value of `Message()` contains references to 
alive object and also prevents copying of `InvalidName` structure at passing it 
to `makeError()`.

Provided test passes on Linux+GCC with or without this patch, but fails on 
Windows+VisualStudio without this patch.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D115959

Files:
  clang-tools-extra/clangd/refactor/Rename.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp


Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1060,6 +1060,11 @@
   )cpp",
"conflict", !HeaderFile, "Conflict"},
 
+  {R"cpp(
+int V^ar;
+  )cpp",
+   "\"const\" is a keyword", !HeaderFile, "const"},
+
   {R"cpp(// Trying to rename into the same name, SameName == SameName.
 void func() {
   int S^ameName;
Index: clang-tools-extra/clangd/refactor/Rename.cpp
===
--- clang-tools-extra/clangd/refactor/Rename.cpp
+++ clang-tools-extra/clangd/refactor/Rename.cpp
@@ -455,7 +455,7 @@
 }
 
 llvm::Error makeError(InvalidName Reason) {
-  auto Message = [](InvalidName Reason) {
+  auto Message = [](const InvalidName ) {
 switch (Reason.K) {
 case InvalidName::Keywords:
   return llvm::formatv("the chosen name \"{0}\" is a keyword",
@@ -733,7 +733,7 @@
 return makeError(ReasonToReject::SameName);
   auto Invalid = checkName(RenameDecl, RInputs.NewName);
   if (Invalid)
-return makeError(*Invalid);
+return makeError(std::move(*Invalid));
 
   auto Reject = renameable(RenameDecl, RInputs.MainFilePath, RInputs.Index);
   if (Reject)


Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1060,6 +1060,11 @@
   )cpp",
"conflict", !HeaderFile, "Conflict"},
 
+  {R"cpp(
+int V^ar;
+  )cpp",
+   "\"const\" is a keyword", !HeaderFile, "const"},
+
   {R"cpp(// Trying to rename into the same name, SameName == SameName.
 void func() {
   int S^ameName;
Index: clang-tools-extra/clangd/refactor/Rename.cpp
===
--- clang-tools-extra/clangd/refactor/Rename.cpp
+++ clang-tools-extra/clangd/refactor/Rename.cpp
@@ -455,7 +455,7 @@
 }
 
 llvm::Error makeError(InvalidName Reason) {
-  auto Message = [](InvalidName Reason) {
+  auto Message = [](const InvalidName ) {
 switch (Reason.K) {
 case InvalidName::Keywords:
   return llvm::formatv("the chosen name \"{0}\" is a keyword",
@@ -733,7 +733,7 @@
 return makeError(ReasonToReject::SameName);
   auto Invalid = checkName(RenameDecl, RInputs.NewName);
   if (Invalid)
-return makeError(*Invalid);
+return makeError(std::move(*Invalid));
 
   auto Reject = renameable(RenameDecl, RInputs.MainFilePath, RInputs.Index);
   if (Reject)
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX abandoned this revision.
ArcsinX added a comment.

As I can see, no chances for this to get approved. So. closing.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2926310 , @sammccall wrote:

> Clang's current answer is "yes we are GCC >=4, but no we are not GCC >=5".
> This causes the codebase to reject the compiler, because it relies on 
> features/absence of bugs from GCC 5. @ArcSinX any idea what these are, 
> concretely?

The checks about GCC version that I saw were related with some features from 
C++ standard that GCC < 4.X.Y doesn't support and there was a try to implement 
these features for GCC < 4.X.Y, but clang has these features and some kind of 
conflict appears.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2924886 , @kadircet wrote:

> Ok if you think that setting GCC version to higher values by default won't 
> break anything,

I think that GCC version should be the same, which was used to build the 
project: it can be higher or lower than 4.2.1

In D107304#2925220 , @joerg wrote:

> This discussion is becoming pointless.

Yes, seems we can't get agreement here, so I am planning to abandon this.

> Clang is not a 1:1 replacement for later GCC versions. Easiest example is the 
> support for `__builtin_apply` and friends.

Still we have no example when GCC x.y.z can compile something, but clang with 
-fgnuc-version=x.y.z can't process this project with some critical errors (but 
works ok with -fgnuc-version=4.2.1). But it's easy to create an example when 
-fgnuc-version=4.2.1 doesn't work (example in the description).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2922975 , @kadircet wrote:

> But I am not convinced that changing the default behaviour here is going to 
> be beneficial for majority of the users.

I think that majority of users doesn't use `--query-driver` at all.

> In other words, if we keep things as-is today, people that want a specific 
> GNUC version can set clangd to parse their code with that specific version 
> they know about and face the consequences if clang doesn't provide the same 
> extensions GCC does. Or we can change the behaviour to what you propose and 
> make people that has a broken parse to change their GNUC version to something 
> they have no idea about (they won't know that they should set it to 4.2.1 or 
> any other specific version).

I think that setting correct version of GCC wont' break anything for the most 
of users (but I do not know how to prove this), but can fix (I saw this by 
myself), that's why I can't agree with you =)
I bet that most of users which face the problem I described, can't solve this 
by their own, because of knowledge lack (nothing can be found in the 
documentation that can help):
Level 1: I see errors in my code with `--query-driver` specified. Why this 
happened?
Level 2: Ok, I know that this is because of `__GNUC__` macro definition, what 
should I do to make it works?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

@kadircet Thanks for clarification. Now your position is clear for me.

But clang was designed as a drop-in replacement for GCC:

- https://clang.llvm.org/docs/LanguageExtensions.html#introduction
- https://clang.llvm.org/features.html#gcccompat

I do not have any proves that clang is 100% compatible with any GCC version, 
but also I can't find any evidence that the most compatible version of GCC is 
4.2.1

I really faced the problem with that default 4.2.1 on a project: strange errors 
in clangd and it was really hard to understand that the reason was `__GNUC__ == 
4`. `-fgnuc-version=` addition helped me to solve this. 
That's why I think that it can be added automatically (and `.clangd` can be 
used to override this with `4.2.1` (or any other) if user doesn't want this 
automatic GCC version detection =) )


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2921694 , @kadircet wrote:

>> The most confusing thing for me that we already have this implicitly set 
>> -fgnuc-version=4.2.1. So, we already have implicitly set __GNUC__ and other 
>> GCC macros as a default behavior.
>
> It might be because clang actually supports all the GNU extensions up until 
> 4.2.1, I don't know the history here, but if clang can compile with that GNUC 
> macro by default, clangd should be good as well. But there are no promises 
> for anything higher than that.

Seem the background is that it was unable to build chromium with clang: 
https://reviews.llvm.org/D68055
And as I can see there: `this flag does not enable or disable any GCC 
extensions implemented in Clang`. Thus, it just defines some GCC macros. So, 
without this patch we guaranty to user that these macros values will be 
incompatible with his GCC version if it is not 4.2.1, which is not an expected 
behavior for `--query-driver` for me: I expect that it can extract as much 
information from the driver output as possible and help clang with files 
processing.

> I think these are the reasons why this needs to be set explicitly by the 
> user. Because even though it can make things work, it might as well break 
> things implicitly (e.g. you have a compiler version 4.2.1 everything works as 
> expected, but once you update your compiler to gcc-5 clangd breaks all of a 
> sudden)

Can't agree that we have equal probabilities to break something in a project 
which was build with GCC x.y.z when specifying the correct version (x.y.z)  and 
incorrect (4.2.1).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2921588 , @kadircet wrote:

> I am not sure this is a good idea. Surely we can go with this approach and 
> pretend to have the same GNUC version as the toolchain, but we are still 
> using clang underneath as @joerg pointed out and we might not be able to 
> parse language/compiler extensions supported by GNU compilers, which will 
> lead to confusion.

The most confusing thing for me that we already have this implicitly set 
`-fgnuc-version=4.2.1`. So, we already have implicitly set `__GNUC__` and other 
GCC macros as a default behavior.

> I think in such cases people should just have a `.clangd` file in their repo 
> that adds `-fgnuc-version=x.y.z` to compile flags, i.e:
>
>   CompileFlags:
> Add: -fgnuc-version=x.y.z
>
> This will be a more explicit step taken by the user, hence they'll be more 
> likely to know why clangd is not behaving the way they want it to.

Sometimes it's not an easy task to understand that errors in diagnostics 
returned by clangd are the result of incorrect `__GNUC__` value (in many 
projects it's not just `#error ...`). Also it's not convenient to use `.clangd` 
config in this case:

- change compiler version => update `.clangd`
- several build directories (e.g. native and cross with different GCC versions).
- idea was to make it in more automatic way.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D107304#2920960 , @joerg wrote:

> What do you mean with "GCC macros are correct"?

Imagine that we have a project and we build it with GCC, we have 
`compile_commands.json` file which  contains all compile command used to build 
this project.
Now we are trying to use clangd for this project, but as it turns, `__GNUC__` 
is equal to 4, but we expect it to be equal to compiler version which was used 
during the build.

> Clang is *not* GCC

We are talking about clangd -- tool, which provides IDE features, I expect it 
to be compatible (as much as possible) with a project which was built with GCC


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D107304/new/

https://reviews.llvm.org/D107304

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D107304: [clangd][query-driver] Extract GCC version from the driver output

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: sammccall, kadircet.
Herald added subscribers: usaxena95, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Clang uses 4.2.1 as a default GCC version.
For projects that rely on the GCC version, this can be a problem, e.g.

  #if __GNUC__ < 5
  #error "Incompatible compiler"
  #endif

This patch extracts the GCC version from the driver output and adds 
-fgnuc-version= compile option to unsure that the values of 
GCC macros are correct.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D107304

Files:
  clang-tools-extra/clangd/QueryDriverDatabase.cpp
  clang-tools-extra/clangd/test/system-include-extractor.test

Index: clang-tools-extra/clangd/test/system-include-extractor.test
===
--- clang-tools-extra/clangd/test/system-include-extractor.test
+++ clang-tools-extra/clangd/test/system-include-extractor.test
@@ -16,6 +16,7 @@
 # RUN: echo 'echo " $* " | grep " --sysroot /my/sysroot/path " || exit' >> %t.dir/bin/my_driver.sh
 # RUN: echo 'echo line to ignore >&2' >> %t.dir/bin/my_driver.sh
 # RUN: echo 'printf "Target: arm-linux-gnueabihf\r\n" >&2' >> %t.dir/bin/my_driver.sh
+# RUN: echo 'printf "gcc version 8.3.0 (Rev2)\r\n" >&2' >> %t.dir/bin/my_driver.sh
 # RUN: echo 'printf "#include <...> search starts here:\r\n" >&2' >> %t.dir/bin/my_driver.sh
 # RUN: echo 'echo %t.dir/my/dir/ >&2' >> %t.dir/bin/my_driver.sh
 # RUN: echo 'echo %t.dir/my/dir2/ >&2' >> %t.dir/bin/my_driver.sh
@@ -49,7 +50,7 @@
   "uri": "file://INPUT_DIR/the-file.cpp",
   "languageId":"cpp",
   "version":1,
-  "text":"#include \n#include \n#if !defined(__ARM_ARCH) || !defined(__gnu_linux__)\n#error \"Invalid target\"\n#endif"
+  "text":"#include \n#include \n#if !defined(__ARM_ARCH) || !defined(__gnu_linux__)\n#error \"Invalid target\"\n#endif\n#if !defined(__GNUC__) || __GNUC__ != 8\n#error \"Incorrect GCC version\"\n#endif"
 }
   }
 }
Index: clang-tools-extra/clangd/QueryDriverDatabase.cpp
===
--- clang-tools-extra/clangd/QueryDriverDatabase.cpp
+++ clang-tools-extra/clangd/QueryDriverDatabase.cpp
@@ -62,6 +62,7 @@
 struct DriverInfo {
   std::vector SystemIncludes;
   std::string Target;
+  std::string GccVersion;
 };
 
 bool isValidTarget(llvm::StringRef Triple) {
@@ -79,6 +80,7 @@
   const char SIS[] = "#include <...> search starts here:";
   const char SIE[] = "End of search list.";
   const char TS[] = "Target: ";
+  const char VS[] = "gcc version ";
   llvm::SmallVector Lines;
   Output.split(Lines, '\n', /*MaxSplit=*/-1, /*KeepEmpty=*/false);
 
@@ -89,6 +91,7 @@
   } State = Initial;
   bool SeenIncludes = false;
   bool SeenTarget = false;
+  bool SeenVersion = false;
   for (auto *It = Lines.begin(); State != Done && It != Lines.end(); ++It) {
 auto Line = *It;
 switch (State) {
@@ -109,6 +112,17 @@
   vlog("System include extraction: target extracted: \"{0}\"",
TargetLine);
 }
+  } else if (!SeenVersion && Line.trim().startswith(VS)) {
+SeenVersion = true;
+llvm::StringRef GccVersionLine = Line.trim();
+GccVersionLine.consume_front(VS);
+Info.GccVersion = GccVersionLine
+  .take_while([](unsigned char C) {
+return C == '.' || std::isdigit(C);
+  })
+  .str();
+vlog("System include extraction: GCC version extracted: \"{0}\"",
+ Info.GccVersion);
   }
   break;
 case IncludesExtracting:
@@ -231,8 +245,11 @@
   if (!Info)
 return llvm::None;
   log("System includes extractor: successfully executed {0}\n\tgot includes: "
-  "\"{1}\"\n\tgot target: \"{2}\"",
-  Driver, llvm::join(Info->SystemIncludes, ", "), Info->Target);
+  "\"{1}\"\n\tgot target: \"{2}\"{3}",
+  Driver, llvm::join(Info->SystemIncludes, ", "), Info->Target,
+  Info->GccVersion.empty()
+  ? std::string()
+  : llvm::formatv("\n\tgot GCC version: \"{0}\"", Info->GccVersion));
   return Info;
 }
 
@@ -260,6 +277,19 @@
   return Cmd;
 }
 
+tooling::CompileCommand (tooling::CompileCommand ,
+   const std::string ) {
+  if (!GccVersion.empty()) {
+// We do not want to override existing GCC version with extracted one.
+for (llvm::StringRef Arg : Cmd.CommandLine) {
+  if (Arg.startswith("-fgnuc-version"))
+return Cmd;
+}
+Cmd.CommandLine.push_back("-fgnuc-version=" + GccVersion);
+  }
+  return Cmd;
+}
+
 /// Converts a glob containing only ** or * into a regex.
 std::string convertGlobToRegex(llvm::StringRef Glob) {
   std::string RegText;
@@ -345,7 +375,9 @@
   return extractSystemIncludesAndTarget(
 

[PATCH] D104056: [clangd][nfc] Show more information in logs when compiler instance prepare fails

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGa62579fc008e: [clangd][nfc] Show more information in logs 
when compiler instance prepare fails (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D104056/new/

https://reviews.llvm.org/D104056

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+std::vector Diags(ASTDiags.take());
+elog("Failed to prepare a compiler instance: {0}",
+ !Diags.empty() ? static_cast(Diags.back()).Message
+: "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+std::vector Diags(ASTDiags.take());
+elog("Failed to prepare a compiler instance: {0}",
+ !Diags.empty() ? static_cast(Diags.back()).Message
+: "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D104056: [clangd][nfc] Show more information in logs when compiler instance prepare fails

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 355389.
ArcsinX added a comment.

Check ASTDiags.take() for empty instead of ASTDiags.getNumErrors() value check


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D104056/new/

https://reviews.llvm.org/D104056

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+std::vector Diags(ASTDiags.take());
+elog("Failed to prepare a compiler instance: {0}",
+ !Diags.empty() ? static_cast(Diags.back()).Message
+: "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+std::vector Diags(ASTDiags.take());
+elog("Failed to prepare a compiler instance: {0}",
+ !Diags.empty() ? static_cast(Diags.back()).Message
+: "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D104056: [clangd][nfc] Show more information in logs when compiler instance prepare fails

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: sammccall, kadircet.
Herald added subscribers: usaxena95, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang-tools-extra.

Without this patch clangd silently process compiler instance prepare failure 
and only LSP errors "Invalid AST" could be found in logs.
E.g. the reason of the problem https://github.com/clangd/clangd/issues/734 is 
impossible to understand without verbose logs or with disabled background index.
This patch adds more information into logs to help understand the reason of 
such failures.

Logs without this patch:

  E[...] Could not build a preamble for file test version 1

Logs with this patch:

  E[...] Could not build a preamble for file test.cpp version 1: 
CreateTargetInfo() return null
  ..
  E[...] Failed to prepare a compiler instance: unknown target ABI 'lp64'


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D104056

Files:
  clang-tools-extra/clangd/ParsedAST.cpp
  clang-tools-extra/clangd/Preamble.cpp


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+elog("Failed to prepare a compiler instance: {0}",
+ ASTDiags.getNumErrors()
+ ? static_cast(ASTDiags.take().back()).Message
+ : "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];


Index: clang-tools-extra/clangd/Preamble.cpp
===
--- clang-tools-extra/clangd/Preamble.cpp
+++ clang-tools-extra/clangd/Preamble.cpp
@@ -390,8 +390,8 @@
 SerializedDeclsCollector.takeMacros(), std::move(StatCache),
 SerializedDeclsCollector.takeCanonicalIncludes());
   } else {
-elog("Could not build a preamble for file {0} version {1}", FileName,
- Inputs.Version);
+elog("Could not build a preamble for file {0} version {1}: {2}", FileName,
+ Inputs.Version, BuiltPreamble.getError().message());
 return nullptr;
   }
 }
Index: clang-tools-extra/clangd/ParsedAST.cpp
===
--- clang-tools-extra/clangd/ParsedAST.cpp
+++ clang-tools-extra/clangd/ParsedAST.cpp
@@ -289,8 +289,15 @@
   std::move(CI), PreamblePCH,
   llvm::MemoryBuffer::getMemBufferCopy(Inputs.Contents, Filename), VFS,
   ASTDiags);
-  if (!Clang)
+  if (!Clang) {
+// The last diagnostic contains information about the reason of this
+// failure.
+elog("Failed to prepare a compiler instance: {0}",
+ ASTDiags.getNumErrors()
+ ? static_cast(ASTDiags.take().back()).Message
+ : "unknown error");
 return None;
+  }
 
   auto Action = std::make_unique();
   const FrontendInputFile  = Clang->getFrontendOpts().Inputs[0];
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGc4efd04f18c7: [clangd] Use URIs instead of paths in the 
index file list (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/test/memory_tree.test
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -229,7 +229,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
  std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
@@ -506,7 +506,7 @@
   RefSlab DynRefs;
   auto DynSize = DynSymbols.bytes() + DynRefs.bytes();
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
-  llvm::StringSet<> DynFiles = {testPath("foo.cc")};
+  llvm::StringSet<> DynFiles = {"unittest:///foo.cc"};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
 RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
 std::move(DynData), DynSize);
@@ -514,7 +514,7 @@
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> StaticFiles = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex StaticIndex(
   std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
   std::move(StaticFiles), IndexContents::References, std::move(StaticData),
Index: clang-tools-extra/clangd/unittests/FileIndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/FileIndexTests.cpp
+++ clang-tools-extra/clangd/unittests/FileIndexTests.cpp
@@ -352,14 +352,14 @@
   Test.Code = std::string(MainCode.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Index.updateMain(Test.Filename, AST);
+  Index.updateMain(testPath(Test.Filename), AST);
   // Add test2.cc
   TestTU Test2;
   Test2.HeaderCode = HeaderCode;
   Test2.Code = std::string(MainCode.code());
   Test2.Filename = "test2.cc";
   AST = Test2.build();
-  Index.updateMain(Test2.Filename, AST);
+  Index.updateMain(testPath(Test2.Filename), AST);
 
   EXPECT_THAT(getRefs(Index, Foo.ID),
   RefsAre({AllOf(RefRange(MainCode.range("foo")),
@@ -387,7 +387,7 @@
   Test.Code = std::string(MainCode.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Index.updateMain(Test.Filename, AST);
+  Index.updateMain(testPath(Test.Filename), AST);
 
   auto HeaderMacro = findSymbol(Test.headerSymbols(), "HEADER_MACRO");
   EXPECT_THAT(getRefs(Index, HeaderMacro.ID),
Index: clang-tools-extra/clangd/unittests/DexTests.cpp
===
--- clang-tools-extra/clangd/unittests/DexTests.cpp
+++ clang-tools-extra/clangd/unittests/DexTests.cpp
@@ -737,7 +737,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   Dex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
 std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
Index: clang-tools-extra/clangd/test/memory_tree.test
===
--- clang-tools-extra/clangd/test/memory_tree.test
+++ clang-tools-extra/clangd/test/memory_tree.test
@@ -23,7 +23,9 @@
 # CHECK-NEXT: "_total": {{[0-9]+}}
 # CHECK-NEXT:   },
 # CHECK-NEXT:   "slabs": {
-# CHECK-NEXT: "{{.*}}main.cpp": {
+# CHECK-NEXT: "_self": {{[0-9]+}},
+# CHECK-NEXT: "_total": {{[0-9]+}},
+# CHECK-NEXT: "test:///main.cpp": {
 # CHECK-NEXT:   "_self": {{[0-9]+}},
 # CHECK-NEXT:   "_total": 

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

Thank you for review!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 328222.
ArcsinX added a comment.

Call `FileIndex::updateMain()` with an absolute file path in tests to avoid 
test failures in debug mode.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/test/memory_tree.test
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -229,7 +229,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
  std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
@@ -506,7 +506,7 @@
   RefSlab DynRefs;
   auto DynSize = DynSymbols.bytes() + DynRefs.bytes();
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
-  llvm::StringSet<> DynFiles = {testPath("foo.cc")};
+  llvm::StringSet<> DynFiles = {"unittest:///foo.cc"};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
 RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
 std::move(DynData), DynSize);
@@ -514,7 +514,7 @@
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> StaticFiles = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex StaticIndex(
   std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
   std::move(StaticFiles), IndexContents::References, std::move(StaticData),
Index: clang-tools-extra/clangd/unittests/FileIndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/FileIndexTests.cpp
+++ clang-tools-extra/clangd/unittests/FileIndexTests.cpp
@@ -352,14 +352,14 @@
   Test.Code = std::string(MainCode.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Index.updateMain(Test.Filename, AST);
+  Index.updateMain(testPath(Test.Filename), AST);
   // Add test2.cc
   TestTU Test2;
   Test2.HeaderCode = HeaderCode;
   Test2.Code = std::string(MainCode.code());
   Test2.Filename = "test2.cc";
   AST = Test2.build();
-  Index.updateMain(Test2.Filename, AST);
+  Index.updateMain(testPath(Test2.Filename), AST);
 
   EXPECT_THAT(getRefs(Index, Foo.ID),
   RefsAre({AllOf(RefRange(MainCode.range("foo")),
@@ -387,7 +387,7 @@
   Test.Code = std::string(MainCode.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Index.updateMain(Test.Filename, AST);
+  Index.updateMain(testPath(Test.Filename), AST);
 
   auto HeaderMacro = findSymbol(Test.headerSymbols(), "HEADER_MACRO");
   EXPECT_THAT(getRefs(Index, HeaderMacro.ID),
Index: clang-tools-extra/clangd/unittests/DexTests.cpp
===
--- clang-tools-extra/clangd/unittests/DexTests.cpp
+++ clang-tools-extra/clangd/unittests/DexTests.cpp
@@ -737,7 +737,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   Dex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
 std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
Index: clang-tools-extra/clangd/test/memory_tree.test
===
--- clang-tools-extra/clangd/test/memory_tree.test
+++ clang-tools-extra/clangd/test/memory_tree.test
@@ -23,7 +23,9 @@
 # CHECK-NEXT: "_total": {{[0-9]+}}
 # CHECK-NEXT:   },
 # CHECK-NEXT:   "slabs": {
-# CHECK-NEXT: "{{.*}}main.cpp": {
+# CHECK-NEXT: "_self": {{[0-9]+}},
+# CHECK-NEXT: "_total": {{[0-9]+}},
+# CHECK-NEXT: "test:///main.cpp": {
 # CHECK-NEXT:   "_self": {{[0-9]+}},
 # CHECK-NEXT:   "_total": {{[0-9]+}},
 # CHECK-NEXT:   "references": {
@@ -38,9 

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 327853.
ArcsinX added a comment.

- use URIs as keys
- create the file list from the keys


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/test/memory_tree.test
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -229,7 +229,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
  std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
@@ -506,7 +506,7 @@
   RefSlab DynRefs;
   auto DynSize = DynSymbols.bytes() + DynRefs.bytes();
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
-  llvm::StringSet<> DynFiles = {testPath("foo.cc")};
+  llvm::StringSet<> DynFiles = {"unittest:///foo.cc"};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
 RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
 std::move(DynData), DynSize);
@@ -514,7 +514,7 @@
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> StaticFiles = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex StaticIndex(
   std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
   std::move(StaticFiles), IndexContents::References, std::move(StaticData),
Index: clang-tools-extra/clangd/unittests/DexTests.cpp
===
--- clang-tools-extra/clangd/unittests/DexTests.cpp
+++ clang-tools-extra/clangd/unittests/DexTests.cpp
@@ -737,7 +737,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   Dex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
 std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
Index: clang-tools-extra/clangd/test/memory_tree.test
===
--- clang-tools-extra/clangd/test/memory_tree.test
+++ clang-tools-extra/clangd/test/memory_tree.test
@@ -23,7 +23,9 @@
 # CHECK-NEXT: "_total": {{[0-9]+}}
 # CHECK-NEXT:   },
 # CHECK-NEXT:   "slabs": {
-# CHECK-NEXT: "{{.*}}main.cpp": {
+# CHECK-NEXT: "_self": {{[0-9]+}},
+# CHECK-NEXT: "_total": {{[0-9]+}},
+# CHECK-NEXT: "test:///main.cpp": {
 # CHECK-NEXT:   "_self": {{[0-9]+}},
 # CHECK-NEXT:   "_total": {{[0-9]+}},
 # CHECK-NEXT:   "references": {
@@ -38,9 +40,7 @@
 # CHECK-NEXT: "_self": {{[0-9]+}},
 # CHECK-NEXT: "_total": {{[0-9]+}}
 # CHECK-NEXT:   }
-# CHECK-NEXT: },
-# CHECK-NEXT: "_self": {{[0-9]+}},
-# CHECK-NEXT: "_total": {{[0-9]+}}
+# CHECK-NEXT: }
 # CHECK-NEXT:   }
 # CHECK-NEXT: },
 # CHECK-NEXT: "preamble": {
Index: clang-tools-extra/clangd/index/dex/Dex.cpp
===
--- clang-tools-extra/clangd/index/dex/Dex.cpp
+++ clang-tools-extra/clangd/index/dex/Dex.cpp
@@ -316,14 +316,7 @@
 llvm::unique_function
 Dex::indexedFiles() const {
   return [this](llvm::StringRef FileURI) {
-if (Files.empty())
-  return IndexContents::None;
-auto Path = URI::resolve(FileURI, Files.begin()->first());
-if (!Path) {
-  vlog("Failed to resolve the URI {0} : {1}", FileURI, Path.takeError());
-  return IndexContents::None;
-}
-return Files.contains(*Path) ? IdxContents : IndexContents::None;
+return Files.contains(FileURI) ? IdxContents : IndexContents::None;
   };
 }
 
Index: clang-tools-extra/clangd/index/MemIndex.cpp
===
--- 

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX planned changes to this revision.
ArcsinX added a comment.

In D97535#2596992 , @kadircet wrote:

> But until that day, I suppose the best we can do is change the contract of 
> `FileSymbols` to sey keys are always `URIs representing the file producing 
> associated slabs`.

Agree. I will rework this patch.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:292
+for (const auto  : *Slab) {
+  if (Sym.Definition)
+Files.insert(Sym.Definition.FileURI);

kadircet wrote:
> ArcsinX wrote:
> > kadircet wrote:
> > > it feels weird to choose one or the other here, why not just insert both 
> > > (after checking if definition exists, ofc).
> > > 
> > > We are likely to have a merged symbol information anyway, and the logic 
> > > here will likely result in no index owning the header files, unless there 
> > > are some symbols *defined* (not just declared) in it.
> > > 
> > > This will likely result in some overshooting as knowing about a symbols 
> > > declaration location doesn't mean indexing that particular file. But so 
> > > does the current version, as knowing about definition location might be 
> > > because of a merged symbol, rather than indexing of particular file.
> > I will try to explain on example:
> > 
> > - **test.h**
> > ```
> > void f();
> > ```
> > - **test.c**
> > ```
> > #include "test.h"
> > void f() { }
> > ```
> > - compile_commands.json contains a command for **test.c** compilation.
> > 
> > Scenario:
> > - open **test.c**
> > - try to find all references for `f()`
> > 
> > For that scenario result for `find all references` will be incorrect if 
> > both (decl and def) files are in the file list because:
> > - decl location is inside **test.h**
> > - def location is inside **test.c**
> > - the file list for the main file index contains **test.h** and **test.c**
> > - the main file index does not contain references from **test.h**
> > - the background (static) index contains references from **test.c**, but 
> > results from the background index will be skipped, because **test.h** is in 
> > the main file (dynamic) index file list.
> Ah i see. This all seems really fragile :/ We might as well have something 
> like:
> 
> a.h:
> ```
> struct Bar;
> ```
> 
> a.cc:
> ```
> #include "a.h"
> struct Bar;
> ```
> 
> and now as soon as you open a.cc, all the results from a.h will be gone, 
> because canonical declaration location for `Bar` will be in `a.h` and there 
> is no definition. (and i believe this kind of forward-decl madness is quite 
> common in at least LLVM). 
> Even worse, the same will also happen even if you have definition in the 
> header, but a forward decl in the main file, so even accepting the file for 
> definition wouldn't be enough.
> 
> It is currently (i.e. without this patch) working as expected, as main file 
> index only owns the information for the "main file" indexing was initiated 
> for. This feels like a big regression to me (that I didn't notice initially, 
> sorry for that), but I am ready to be convinced otherwise :D
> 
> As Sam mentioned what you do here and partitioning logic in FileShardedIndex 
> is quite similar (yours undershoot, sharding logic overshoots) but in the 
> sharding process we split indexing result of a full TU/preamble, and later on 
> those shards will always be used in a merged fashion (e.g. when a.cc and 
> b.cc, both including a.h gets indexed, the shard produced for a.h from b.cc 
> won't contain any definition locations, but the shard produced for a.cc will 
> know about those symbols definition locations and in a merged view those 
> symbols will have all the necessary information.), whereas in here the 
> resulting information is used in isolation (main file isn't merged with 
> preamble symbols, but mixes Files view). Hence causing such regressions.
> 
> I think the proper thing to do here is to propagate relevant files with slabs 
> on `FileSymbols::update`. What you do here and sharding isn't very different. 
> The question is should we have a:
> - string File
> - optional File
> - vector Files
> 
> Currently we always have exactly one File associated with all of those slabs 
> as we either:
> - always do sharding (preamble and background idx)
> - even though symbol informations tell otherwise, we've only processed a 
> single file (main file idx)
> 
> We would need 3rd option if we were to use filesymbols with a monolithic 
> index, but we don't. And even if we need such a thing in future it shouldn't 
> be a huge change hopefully.
> 
> 
> Sorry for the wall of text, I hope I do make sense, but please tell me if I 
> misunderstood/missing something.
> It is currently (i.e. without this patch) working as expected, as main file 
> index only owns the information for the "main file" indexing was initiated 
> for. This feels like a big regression to me (that I didn't notice initially, 
> sorry for that), but I am ready to be convinced otherwise :D

Currently it works mostly as expected. The only thing which worries me is the 
preamble index update: the first parameter passed to `PreambleSymbols.update()` 
is an URI, but we expect a file path there. Thus, the file list of the preamble 
index contains URIs, which seems incorrect and 

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:292
+for (const auto  : *Slab) {
+  if (Sym.Definition)
+Files.insert(Sym.Definition.FileURI);

kadircet wrote:
> it feels weird to choose one or the other here, why not just insert both 
> (after checking if definition exists, ofc).
> 
> We are likely to have a merged symbol information anyway, and the logic here 
> will likely result in no index owning the header files, unless there are some 
> symbols *defined* (not just declared) in it.
> 
> This will likely result in some overshooting as knowing about a symbols 
> declaration location doesn't mean indexing that particular file. But so does 
> the current version, as knowing about definition location might be because of 
> a merged symbol, rather than indexing of particular file.
I will try to explain on example:

- **test.h**
```
void f();
```
- **test.c**
```
#include "test.h"
void f() { }
```
- compile_commands.json contains a command for **test.c** compilation.

Scenario:
- open **test.c**
- try to find all references for `f()`

For that scenario result for `find all references` will be incorrect if both 
(decl and def) files are in the file list because:
- decl location is inside **test.h**
- def location is inside **test.c**
- the file list for the main file index contains **test.h** and **test.c**
- the main file index does not contain references from **test.h**
- the background (static) index contains references from **test.c**, but 
results from the background index will be skipped, because **test.h** is in the 
main file (dynamic) index file list.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked an inline comment as done.
ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:279
   SymbolSlabs.push_back(FileAndSymbols.second);
+  for (const auto  : *FileAndSymbols.second) {
+if (S.Definition)

kadircet wrote:
> iterating over all the symbols here (and refs below) seems really 
> unfortunate. But looking at the previous discussions that seems to be best of 
> both worlds until we populate a file list in SymbolCollector. However, I 
> think it would be better to do the looping after releasing the lock (we 
> already have the information copied over to our snapshots).
> 
> moreover we seem to be still inserting keys of the Symbol and RefSlabs into 
> Files, but not doing that for RelationSlabs, why? (i believe we shouldn't be 
> inserting the keys at all, if we indeed want to just insert URIs and keep 
> treating the keys as opaque objects).
> However, I think it would be better to do the looping after releasing the 
> lock (we already have the information copied over to our snapshots).
Done

> we seem to be still inserting keys of the Symbol and RefSlabs into Files, but 
> not doing that for RelationSlabs
Forgot to remove these insertions, thanks, fixed.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 326903.
ArcsinX added a comment.

- Do not insert the keys into the file list
- Iterate through refs and symbols to create the file list after releasing the 
lock


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

Files:
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -229,7 +229,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
  std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
@@ -339,21 +339,21 @@
   FileIndex DynamicIndex, StaticIndex;
   MergedIndex Merge(, );
 
-  const char *HeaderCode = "class Foo;";
+  const char *HeaderCode = "class Foo; class Bar;";
   auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
   auto Foo = findSymbol(HeaderSymbols, "Foo");
 
-  // Build static index for test.cc with Foo symbol
+  // Build static index for test.cc with Foo and Bar symbols
   TestTU Test;
   Test.HeaderCode = HeaderCode;
-  Test.Code = "class Foo {};";
+  Test.Code = "class Foo {}; class Bar {};";
   Test.Filename = "test.cc";
   auto AST = Test.build();
   StaticIndex.updateMain(testPath(Test.Filename), AST);
 
-  // Remove Foo symbol, i.e. build dynamic index for test.cc, which is empty.
-  Test.HeaderCode = "";
-  Test.Code = "";
+  // Remove Foo symbol.
+  Test.HeaderCode = "class Bar;";
+  Test.Code = "class Bar {};";
   AST = Test.build();
   DynamicIndex.updateMain(testPath(Test.Filename), AST);
 
@@ -487,7 +487,7 @@
 
   // Remove all refs for test.cc from dynamic index,
   // merged index should not return results from static index for test.cc.
-  Test.Code = "";
+  Test.Code = "int I;"; // test.cc should not be empty
   AST = Test.build();
   Dyn.updateMain(testPath(Test.Filename), AST);
 
@@ -506,7 +506,7 @@
   RefSlab DynRefs;
   auto DynSize = DynSymbols.bytes() + DynRefs.bytes();
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
-  llvm::StringSet<> DynFiles = {testPath("foo.cc")};
+  llvm::StringSet<> DynFiles = {"unittest:///foo.cc"};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
 RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
 std::move(DynData), DynSize);
@@ -514,7 +514,7 @@
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> StaticFiles = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex StaticIndex(
   std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
   std::move(StaticFiles), IndexContents::References, std::move(StaticData),
Index: clang-tools-extra/clangd/unittests/DexTests.cpp
===
--- clang-tools-extra/clangd/unittests/DexTests.cpp
+++ clang-tools-extra/clangd/unittests/DexTests.cpp
@@ -737,7 +737,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   Dex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
 std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
Index: clang-tools-extra/clangd/index/dex/Dex.cpp
===
--- clang-tools-extra/clangd/index/dex/Dex.cpp
+++ clang-tools-extra/clangd/index/dex/Dex.cpp
@@ -316,14 +316,7 @@
 llvm::unique_function
 Dex::indexedFiles() const {
   return [this](llvm::StringRef FileURI) {
-if (Files.empty())
-  return IndexContents::None;
-auto Path = URI::resolve(FileURI, Files.begin()->first());
-if (!Path) {
-  vlog("Failed to resolve the URI {0} : {1}", FileURI, Path.takeError());
-  return IndexContents::None;
-}
-return Files.contains(*Path) ? IdxContents : IndexContents::None;
+return Files.contains(FileURI) ? IdxContents : IndexContents::None;

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added reviewers: sammccall, kadircet, hokein.
ArcsinX added a comment.

Initial discussion D94952#inline-892421 



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97535/new/

https://reviews.llvm.org/D97535

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D97535: [clangd] Use URIs instead of paths in the index file list

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
Herald added subscribers: usaxena95, kadircet, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

Without this patch the file list of the index is derived from the keys, but 
initial `FileSymbols` design was that the keys are arbitrary, so we can not 
assume that the keys are always paths/URIs (e.g. the preamble index uses URIs 
for the keys and other indexes use file paths).
This patch introduces the file list extraction from the index data, thus it 
makes `FileSymbols` usage more flexible (we do not need to think about the 
keys) and helps to avoid a lot of URI <=> path conversions, but from the other 
hand the index file list could be empty if there are no symbols/references 
(empty files seems to be a very rare case).


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D97535

Files:
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -229,7 +229,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
  std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
@@ -339,21 +339,21 @@
   FileIndex DynamicIndex, StaticIndex;
   MergedIndex Merge(, );
 
-  const char *HeaderCode = "class Foo;";
+  const char *HeaderCode = "class Foo; class Bar;";
   auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
   auto Foo = findSymbol(HeaderSymbols, "Foo");
 
-  // Build static index for test.cc with Foo symbol
+  // Build static index for test.cc with Foo and Bar symbols
   TestTU Test;
   Test.HeaderCode = HeaderCode;
-  Test.Code = "class Foo {};";
+  Test.Code = "class Foo {}; class Bar {};";
   Test.Filename = "test.cc";
   auto AST = Test.build();
   StaticIndex.updateMain(testPath(Test.Filename), AST);
 
-  // Remove Foo symbol, i.e. build dynamic index for test.cc, which is empty.
-  Test.HeaderCode = "";
-  Test.Code = "";
+  // Remove Foo symbol.
+  Test.HeaderCode = "class Bar;";
+  Test.Code = "class Bar {};";
   AST = Test.build();
   DynamicIndex.updateMain(testPath(Test.Filename), AST);
 
@@ -487,7 +487,7 @@
 
   // Remove all refs for test.cc from dynamic index,
   // merged index should not return results from static index for test.cc.
-  Test.Code = "";
+  Test.Code = "int I;"; // test.cc should not be empty
   AST = Test.build();
   Dyn.updateMain(testPath(Test.Filename), AST);
 
@@ -506,7 +506,7 @@
   RefSlab DynRefs;
   auto DynSize = DynSymbols.bytes() + DynRefs.bytes();
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
-  llvm::StringSet<> DynFiles = {testPath("foo.cc")};
+  llvm::StringSet<> DynFiles = {"unittest:///foo.cc"};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
 RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
 std::move(DynData), DynSize);
@@ -514,7 +514,7 @@
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> StaticFiles = {"unittest:///foo.cc", "unittest:///bar.cc"};
   MemIndex StaticIndex(
   std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
   std::move(StaticFiles), IndexContents::References, std::move(StaticData),
Index: clang-tools-extra/clangd/unittests/DexTests.cpp
===
--- clang-tools-extra/clangd/unittests/DexTests.cpp
+++ clang-tools-extra/clangd/unittests/DexTests.cpp
@@ -737,7 +737,7 @@
   RefSlab Refs;
   auto Size = Symbols.bytes() + Refs.bytes();
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
-  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  llvm::StringSet<> Files = {"unittest:///foo.cc", "unittest:///bar.cc"};
   Dex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
 std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
Index: clang-tools-extra/clangd/index/dex/Dex.cpp
===
--- 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rG91698fe45f60: [clangd] Take into account what is in the 
index (symbols, references, etc.) at… (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/FileIndex.h
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/BackgroundIndexTests.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp

Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1290,7 +1290,7 @@
   std::string BarPath = testPath("bar.cc");
   // Build the index, the index has "Foo" references from foo.cc and "Bar"
   // references from bar.cc.
-  FileSymbols FSymbols;
+  FileSymbols FSymbols(IndexContents::All);
   FSymbols.update(FooPath, nullptr, buildRefSlab(FooCode, "Foo", FooPath),
   nullptr, false);
   FSymbols.update(BarPath, nullptr, buildRefSlab(BarCode, "Bar", BarPath),
@@ -1367,9 +1367,9 @@
llvm::function_ref
Callback) const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
@@ -1421,9 +1421,9 @@
llvm::function_ref)
 const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -231,11 +231,11 @@
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
   llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
- std::move(Files), std::move(Data), Size);
+ std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexContents::None);
 }
 
 TEST(MemIndexTest, TemplateSpecialization) {
@@ -508,23 +508,24 @@
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
   llvm::StringSet<> DynFiles = {testPath("foo.cc")};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
-RelationSlab(), std::move(DynFiles), std::move(DynData),
-DynSize);
+RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
+std::move(DynData), DynSize);
   SymbolSlab StaticSymbols;
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("bar.cc")};
-  MemIndex StaticIndex(std::move(StaticData.first),
-   std::move(StaticData.second), RelationSlab(),
-   std::move(StaticFiles), std::move(StaticData),
-   StaticSymbols.bytes() + StaticRefs.bytes());
+  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex StaticIndex(
+  std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
+  std::move(StaticFiles), 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked 3 inline comments as done.
ArcsinX added a comment.

Thank you for review.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 321442.
ArcsinX added a comment.

- Remove default value of `IdxContents` in `FileSymbols` constructor.
- Fix index contents for the preamble index (Symbols => Symbols|Relations).
- Rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/FileIndex.h
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/BackgroundIndexTests.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp

Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1269,7 +1269,7 @@
   std::string BarPath = testPath("bar.cc");
   // Build the index, the index has "Foo" references from foo.cc and "Bar"
   // references from bar.cc.
-  FileSymbols FSymbols;
+  FileSymbols FSymbols(IndexContents::All);
   FSymbols.update(FooPath, nullptr, buildRefSlab(FooCode, "Foo", FooPath),
   nullptr, false);
   FSymbols.update(BarPath, nullptr, buildRefSlab(BarCode, "Bar", BarPath),
@@ -1346,9 +1346,9 @@
llvm::function_ref
Callback) const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
@@ -1400,9 +1400,9 @@
llvm::function_ref)
 const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -231,11 +231,11 @@
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
   llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
- std::move(Files), std::move(Data), Size);
+ std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexContents::None);
 }
 
 TEST(MemIndexTest, TemplateSpecialization) {
@@ -508,23 +508,24 @@
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
   llvm::StringSet<> DynFiles = {testPath("foo.cc")};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
-RelationSlab(), std::move(DynFiles), std::move(DynData),
-DynSize);
+RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
+std::move(DynData), DynSize);
   SymbolSlab StaticSymbols;
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("bar.cc")};
-  MemIndex StaticIndex(std::move(StaticData.first),
-   std::move(StaticData.second), RelationSlab(),
-   std::move(StaticFiles), std::move(StaticData),
-   StaticSymbols.bytes() + StaticRefs.bytes());
+  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex StaticIndex(
+  std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
+  std::move(StaticFiles), IndexContents::References, 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:433
 PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
+FilePath ? *FilePath : (consumeError(FilePath.takeError()), Uri),
+std::make_unique(std::move(*IF->Symbols)),

ArcsinX wrote:
> sammccall wrote:
> > ArcsinX wrote:
> > > ArcsinX wrote:
> > > > sammccall wrote:
> > > > > ArcsinX wrote:
> > > > > > sammccall wrote:
> > > > > > > Is this change related? It changes the key scheme for the 
> > > > > > > preamble index from URIs to paths, but I'm not sure why.
> > > > > > > 
> > > > > > > Do we have multiple URIs pointing to the same path? What are they 
> > > > > > > concretely?
> > > > > > This is the main thing in this patch. I will try to explain.
> > > > > > We use these keys to create the file list, which is used by 
> > > > > > `indexedFiles()`.
> > > > > > Currently, the preamble index contains URI's instead of paths (as a 
> > > > > > file list), that leads to the function returned by 
> > > > > > `PreambleIndex::indexedFiles()` always return `false` (because we 
> > > > > > pass to this function paths, not URI's). So, we always take data 
> > > > > > from the preamble index (but maybe we should not in some cases).
> > > > > > 
> > > > > Oooh... I'm not sure how I misunderstood the original so much :-( And 
> > > > > I missed it in this patch description as well, apologies.
> > > > > 
> > > > > My impression was that the file list was derived from the index data, 
> > > > > rather than from the keys, which were always intended to be 
> > > > > opaque/arbitrary.
> > > > > (At various times, these have been filenames, URIs, and other things 
> > > > > IIRC. And until relatively recently, the preamble index keys were the 
> > > > > file the preamble was built from, not the file containing the symbol!)
> > > > > 
> > > > > It feels like using URIs extracted from symbols might not be 
> > > > > *completely* robust. Because having CanonicalDeclaration etc set to a 
> > > > > file might not line up exactly with the idea that we indexed the 
> > > > > file. But we do use this partitioning for FileShardedIndex, so it has 
> > > > > to work well enough.
> > > > > 
> > > > > The advantage of using the extracted URIs would be: also works for 
> > > > > non-file-sharded indexes like --index-file, avoid a bunch of 
> > > > > conversion between URI and path, and we get to keep the 
> > > > > simpler/flexible design for FileSymbols where the key is opaque.
> > > > > 
> > > > > Does this seem feasible to you?
> > > > > that leads to the function returned by 
> > > > > `PreambleIndex::indexedFiles()` always return `false` (because we 
> > > > > pass to this function paths, not URI's)
> > > > 
> > > > This is a bit incorrect.
> > > > We pass to this function URI, but this URI is converted to path. i.e. 
> > > > `MemIndex::indexedFiles()`, `Dex::indexedFiles()` expect that `Files` 
> > > > are paths, but they are URI's for the preamble index. That's why 
> > > > `PreambleIndex::indexedFiles()` always return `false`.
> > > I also do not like these path <=> URI conversions. But what about empty 
> > > files?, e.g.:
> > > - open a file
> > > - remove everything from this file
> > > - the dynamic index has no symbols with definition/declaration from this 
> > > file, so we do not have this file in the dynamic index file list.
> > > - the static index has symbols with definition/declaration from this 
> > > file, so we have this file in the static index file list.
> > > - we will show stale results from the static index for this file.
> > > 
> > > 
> > > Unsure, maybe it's ok to ignore the problem with empty files, seems this 
> > > is the only case when the file was indexed, but we have no symbols 
> > > located there.
> > > 
> > > Overall, I like the idea to use URI's instead of paths. I think we could 
> > > implement it first as a separate patch and after that return to this one.
> > > 
> > > What do you think?
> > Right, completely empty files are going to be mishandled., in the same way 
> > that before your changes we mishandled files that had no results from the 
> > query.
> > 
> > I do still think this is the way to go though, because:
> >  - completely empty files are much rarer
> >  - again this is a peer to an FileShardedIndex issue where empty files get 
> > no shards. Therefore *within* the dynamic index, we'll never clear out the 
> > symbols for a file while the file is emptied
> >  - having SymbolCollector explicitly the files indexed is the principled 
> > solution to both problems, and that seems feasible for us to do at some 
> > point.
> > 
> > > Overall, I like the idea to use URI's instead of paths. I think we could 
> > > implement it first as a separate patch and after that return to this one.
> > 
> > That sounds great if you don't mind!
> > (Is there a reason we can't land the rest of this patch as-is already?)
> > (Is 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked 4 inline comments as done.
ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:433
 PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
+FilePath ? *FilePath : (consumeError(FilePath.takeError()), Uri),
+std::make_unique(std::move(*IF->Symbols)),

sammccall wrote:
> ArcsinX wrote:
> > ArcsinX wrote:
> > > sammccall wrote:
> > > > ArcsinX wrote:
> > > > > sammccall wrote:
> > > > > > Is this change related? It changes the key scheme for the preamble 
> > > > > > index from URIs to paths, but I'm not sure why.
> > > > > > 
> > > > > > Do we have multiple URIs pointing to the same path? What are they 
> > > > > > concretely?
> > > > > This is the main thing in this patch. I will try to explain.
> > > > > We use these keys to create the file list, which is used by 
> > > > > `indexedFiles()`.
> > > > > Currently, the preamble index contains URI's instead of paths (as a 
> > > > > file list), that leads to the function returned by 
> > > > > `PreambleIndex::indexedFiles()` always return `false` (because we 
> > > > > pass to this function paths, not URI's). So, we always take data from 
> > > > > the preamble index (but maybe we should not in some cases).
> > > > > 
> > > > Oooh... I'm not sure how I misunderstood the original so much :-( And I 
> > > > missed it in this patch description as well, apologies.
> > > > 
> > > > My impression was that the file list was derived from the index data, 
> > > > rather than from the keys, which were always intended to be 
> > > > opaque/arbitrary.
> > > > (At various times, these have been filenames, URIs, and other things 
> > > > IIRC. And until relatively recently, the preamble index keys were the 
> > > > file the preamble was built from, not the file containing the symbol!)
> > > > 
> > > > It feels like using URIs extracted from symbols might not be 
> > > > *completely* robust. Because having CanonicalDeclaration etc set to a 
> > > > file might not line up exactly with the idea that we indexed the file. 
> > > > But we do use this partitioning for FileShardedIndex, so it has to work 
> > > > well enough.
> > > > 
> > > > The advantage of using the extracted URIs would be: also works for 
> > > > non-file-sharded indexes like --index-file, avoid a bunch of conversion 
> > > > between URI and path, and we get to keep the simpler/flexible design 
> > > > for FileSymbols where the key is opaque.
> > > > 
> > > > Does this seem feasible to you?
> > > > that leads to the function returned by `PreambleIndex::indexedFiles()` 
> > > > always return `false` (because we pass to this function paths, not 
> > > > URI's)
> > > 
> > > This is a bit incorrect.
> > > We pass to this function URI, but this URI is converted to path. i.e. 
> > > `MemIndex::indexedFiles()`, `Dex::indexedFiles()` expect that `Files` are 
> > > paths, but they are URI's for the preamble index. That's why 
> > > `PreambleIndex::indexedFiles()` always return `false`.
> > I also do not like these path <=> URI conversions. But what about empty 
> > files?, e.g.:
> > - open a file
> > - remove everything from this file
> > - the dynamic index has no symbols with definition/declaration from this 
> > file, so we do not have this file in the dynamic index file list.
> > - the static index has symbols with definition/declaration from this file, 
> > so we have this file in the static index file list.
> > - we will show stale results from the static index for this file.
> > 
> > 
> > Unsure, maybe it's ok to ignore the problem with empty files, seems this is 
> > the only case when the file was indexed, but we have no symbols located 
> > there.
> > 
> > Overall, I like the idea to use URI's instead of paths. I think we could 
> > implement it first as a separate patch and after that return to this one.
> > 
> > What do you think?
> Right, completely empty files are going to be mishandled., in the same way 
> that before your changes we mishandled files that had no results from the 
> query.
> 
> I do still think this is the way to go though, because:
>  - completely empty files are much rarer
>  - again this is a peer to an FileShardedIndex issue where empty files get no 
> shards. Therefore *within* the dynamic index, we'll never clear out the 
> symbols for a file while the file is emptied
>  - having SymbolCollector explicitly the files indexed is the principled 
> solution to both problems, and that seems feasible for us to do at some point.
> 
> > Overall, I like the idea to use URI's instead of paths. I think we could 
> > implement it first as a separate patch and after that return to this one.
> 
> That sounds great if you don't mind!
> (Is there a reason we can't land the rest of this patch as-is already?)
> (Is there a reason we can't land the rest of this patch as-is already?)
Seems we have no reason =)



Comment at: 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 319602.
ArcsinX added a comment.

- Set `IdxContents` at `FileSymbols` object breation instead of at 
`FileSymbols::buildIndex()` call.
- Revert change of the preamble index key scheme
- Add comment for `IndexContents`
- `IndexDataKind` => `IndexContents`


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

Files:
  clang-tools-extra/clangd/index/Background.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/FileIndex.h
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp

Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1244,9 +1244,9 @@
llvm::function_ref
Callback) const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
@@ -1298,9 +1298,9 @@
llvm::function_ref)
 const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexContents::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -231,11 +231,11 @@
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
   llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
- std::move(Files), std::move(Data), Size);
+ std::move(Files), IndexContents::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexContents::All);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexContents::None);
 }
 
 TEST(MemIndexTest, TemplateSpecialization) {
@@ -508,23 +508,24 @@
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
   llvm::StringSet<> DynFiles = {testPath("foo.cc")};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
-RelationSlab(), std::move(DynFiles), std::move(DynData),
-DynSize);
+RelationSlab(), std::move(DynFiles), IndexContents::Symbols,
+std::move(DynData), DynSize);
   SymbolSlab StaticSymbols;
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("bar.cc")};
-  MemIndex StaticIndex(std::move(StaticData.first),
-   std::move(StaticData.second), RelationSlab(),
-   std::move(StaticFiles), std::move(StaticData),
-   StaticSymbols.bytes() + StaticRefs.bytes());
+  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex StaticIndex(
+  std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
+  std::move(StaticFiles), IndexContents::References, std::move(StaticData),
+  StaticSymbols.bytes() + StaticRefs.bytes());
   MergedIndex Merge(, );
 
   auto ContainsFile = Merge.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"),
+IndexContents::Symbols | IndexContents::References);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexContents::References);
+  

[PATCH] D95349: [clangd] Allow diagnostics to be suppressed with configuration

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/ParsedAST.cpp:319
 
-if (!CTChecks.empty()) {
-  ASTDiags.setLevelAdjuster([](DiagnosticsEngine::Level 
DiagLevel,
- const clang::Diagnostic ) {
+ASTDiags.setLevelAdjuster([&, (Config::current())](
+  DiagnosticsEngine::Level DiagLevel,

This line breaks GCC 5.4 build with the following log:
```
llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:319:55: error: binding 
‘const clang::clangd::Config’ to reference of type ‘clang::clangd::Config&’ 
discards qualifiers
 ASTDiags.setLevelAdjuster([&, (Config::current())](
   ^
```
Unsure is it a compiler bug or not. Moving `Cfg` out of the capture list or 
using a cast to `Config &` solves the problem.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D95349/new/

https://reviews.llvm.org/D95349

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:433
 PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
+FilePath ? *FilePath : (consumeError(FilePath.takeError()), Uri),
+std::make_unique(std::move(*IF->Symbols)),

ArcsinX wrote:
> sammccall wrote:
> > ArcsinX wrote:
> > > sammccall wrote:
> > > > Is this change related? It changes the key scheme for the preamble 
> > > > index from URIs to paths, but I'm not sure why.
> > > > 
> > > > Do we have multiple URIs pointing to the same path? What are they 
> > > > concretely?
> > > This is the main thing in this patch. I will try to explain.
> > > We use these keys to create the file list, which is used by 
> > > `indexedFiles()`.
> > > Currently, the preamble index contains URI's instead of paths (as a file 
> > > list), that leads to the function returned by 
> > > `PreambleIndex::indexedFiles()` always return `false` (because we pass to 
> > > this function paths, not URI's). So, we always take data from the 
> > > preamble index (but maybe we should not in some cases).
> > > 
> > Oooh... I'm not sure how I misunderstood the original so much :-( And I 
> > missed it in this patch description as well, apologies.
> > 
> > My impression was that the file list was derived from the index data, 
> > rather than from the keys, which were always intended to be 
> > opaque/arbitrary.
> > (At various times, these have been filenames, URIs, and other things IIRC. 
> > And until relatively recently, the preamble index keys were the file the 
> > preamble was built from, not the file containing the symbol!)
> > 
> > It feels like using URIs extracted from symbols might not be *completely* 
> > robust. Because having CanonicalDeclaration etc set to a file might not 
> > line up exactly with the idea that we indexed the file. But we do use this 
> > partitioning for FileShardedIndex, so it has to work well enough.
> > 
> > The advantage of using the extracted URIs would be: also works for 
> > non-file-sharded indexes like --index-file, avoid a bunch of conversion 
> > between URI and path, and we get to keep the simpler/flexible design for 
> > FileSymbols where the key is opaque.
> > 
> > Does this seem feasible to you?
> > that leads to the function returned by `PreambleIndex::indexedFiles()` 
> > always return `false` (because we pass to this function paths, not URI's)
> 
> This is a bit incorrect.
> We pass to this function URI, but this URI is converted to path. i.e. 
> `MemIndex::indexedFiles()`, `Dex::indexedFiles()` expect that `Files` are 
> paths, but they are URI's for the preamble index. That's why 
> `PreambleIndex::indexedFiles()` always return `false`.
I also do not like these path <=> URI conversions. But what about empty files?, 
e.g.:
- open a file
- remove everything from this file
- the dynamic index has no symbols with definition/declaration from this file, 
so we do not have this file in the dynamic index file list.
- the static index has symbols with definition/declaration from this file, so 
we have this file in the static index file list.
- we will show stale results from the static index for this file.


Unsure, maybe it's ok to ignore the problem with empty files, seems this is the 
only case when the file was indexed, but we have no symbols located there.

Overall, I like the idea to use URI's instead of paths. I think we could 
implement it first as a separate patch and after that return to this one.

What do you think?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:433
 PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
+FilePath ? *FilePath : (consumeError(FilePath.takeError()), Uri),
+std::make_unique(std::move(*IF->Symbols)),

ArcsinX wrote:
> sammccall wrote:
> > Is this change related? It changes the key scheme for the preamble index 
> > from URIs to paths, but I'm not sure why.
> > 
> > Do we have multiple URIs pointing to the same path? What are they 
> > concretely?
> This is the main thing in this patch. I will try to explain.
> We use these keys to create the file list, which is used by `indexedFiles()`.
> Currently, the preamble index contains URI's instead of paths (as a file 
> list), that leads to the function returned by `PreambleIndex::indexedFiles()` 
> always return `false` (because we pass to this function paths, not URI's). 
> So, we always take data from the preamble index (but maybe we should not in 
> some cases).
> 
> that leads to the function returned by `PreambleIndex::indexedFiles()` always 
> return `false` (because we pass to this function paths, not URI's)

This is a bit incorrect.
We pass to this function URI, but this URI is converted to path. i.e. 
`MemIndex::indexedFiles()`, `Dex::indexedFiles()` expect that `Files` are 
paths, but they are URI's for the preamble index. That's why 
`PreambleIndex::indexedFiles()` always return `false`.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added inline comments.



Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:433
 PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
+FilePath ? *FilePath : (consumeError(FilePath.takeError()), Uri),
+std::make_unique(std::move(*IF->Symbols)),

sammccall wrote:
> Is this change related? It changes the key scheme for the preamble index from 
> URIs to paths, but I'm not sure why.
> 
> Do we have multiple URIs pointing to the same path? What are they concretely?
This is the main thing in this patch. I will try to explain.
We use these keys to create the file list, which is used by `indexedFiles()`.
Currently, the preamble index contains URI's instead of paths (as a file list), 
that leads to the function returned by `PreambleIndex::indexedFiles()` always 
return `false` (because we pass to this function paths, not URI's). So, we 
always take data from the preamble index (but maybe we should not in some 
cases).



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D95206: [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG7388c3468595: [clangd][SwapIndex] ensure that the old index 
is alive while we are using it… (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D95206/new/

https://reviews.llvm.org/D95206

Files:
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Merge.cpp


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  auto DynS = Dyn.find(S.ID);
+  ++StaticCount;
+  if (DynS == Dyn.end())
+return Callback(S);
+  ++MergedCount;
+  SeenDynamicSymbols.insert(S.ID);
+  Callback(mergeSymbol(*DynS, S));
+});
+  }
   SPAN_ATTACH(Tracer, "dynamic", DynamicCount);
   SPAN_ATTACH(Tracer, "static", StaticCount);
   SPAN_ATTACH(Tracer, "merged", MergedCount);
@@ -77,20 +79,22 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  Static->lookup(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-const Symbol *Sym = B.find(S.ID);
-RemainingIDs.erase(S.ID);
-if (!Sym)
-  Callback(S);
-else
-  Callback(mergeSymbol(*Sym, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+Static->lookup(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  const Symbol *Sym = B.find(S.ID);
+  RemainingIDs.erase(S.ID);
+  if (!Sym)
+Callback(S);
+  else
+Callback(mergeSymbol(*Sym, S));
+});
+  }
   for (const auto  : RemainingIDs)
 if (const Symbol *Sym = B.find(ID))
   Callback(*Sym);
Index: clang-tools-extra/clangd/index/Index.cpp
===
--- clang-tools-extra/clangd/index/Index.cpp
+++ clang-tools-extra/clangd/index/Index.cpp
@@ -78,7 +78,13 @@
 
 llvm::unique_function
 SwapIndex::indexedFiles() const {
-  return snapshot()->indexedFiles();
+  // The index snapshot should outlive this method return value.
+  auto SnapShot = snapshot();
+  auto IndexedFiles = SnapShot->indexedFiles();
+  return [KeepAlive{std::move(SnapShot)},
+  IndexContainsFile{std::move(IndexedFiles)}](llvm::StringRef File) {
+return IndexContainsFile(File);
+  };
 }
 
 size_t SwapIndex::estimateMemoryUsage() const {


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
- 

[PATCH] D95206: [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 318474.
ArcsinX added a comment.

Fix format


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D95206/new/

https://reviews.llvm.org/D95206

Files:
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Merge.cpp


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  auto DynS = Dyn.find(S.ID);
+  ++StaticCount;
+  if (DynS == Dyn.end())
+return Callback(S);
+  ++MergedCount;
+  SeenDynamicSymbols.insert(S.ID);
+  Callback(mergeSymbol(*DynS, S));
+});
+  }
   SPAN_ATTACH(Tracer, "dynamic", DynamicCount);
   SPAN_ATTACH(Tracer, "static", StaticCount);
   SPAN_ATTACH(Tracer, "merged", MergedCount);
@@ -77,20 +79,22 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  Static->lookup(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-const Symbol *Sym = B.find(S.ID);
-RemainingIDs.erase(S.ID);
-if (!Sym)
-  Callback(S);
-else
-  Callback(mergeSymbol(*Sym, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+Static->lookup(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  const Symbol *Sym = B.find(S.ID);
+  RemainingIDs.erase(S.ID);
+  if (!Sym)
+Callback(S);
+  else
+Callback(mergeSymbol(*Sym, S));
+});
+  }
   for (const auto  : RemainingIDs)
 if (const Symbol *Sym = B.find(ID))
   Callback(*Sym);
Index: clang-tools-extra/clangd/index/Index.cpp
===
--- clang-tools-extra/clangd/index/Index.cpp
+++ clang-tools-extra/clangd/index/Index.cpp
@@ -78,7 +78,13 @@
 
 llvm::unique_function
 SwapIndex::indexedFiles() const {
-  return snapshot()->indexedFiles();
+  // The index snapshot should outlive this method return value.
+  auto SnapShot = snapshot();
+  auto IndexedFiles = SnapShot->indexedFiles();
+  return [KeepAlive{std::move(SnapShot)},
+  IndexContainsFile{std::move(IndexedFiles)}](llvm::StringRef File) {
+return IndexContainsFile(File);
+  };
 }
 
 size_t SwapIndex::estimateMemoryUsage() const {


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-

[PATCH] D95206: [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 318470.
ArcsinX added a comment.

Call snapshot() only once to avoid possible race.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D95206/new/

https://reviews.llvm.org/D95206

Files:
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Merge.cpp


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  auto DynS = Dyn.find(S.ID);
+  ++StaticCount;
+  if (DynS == Dyn.end())
+return Callback(S);
+  ++MergedCount;
+  SeenDynamicSymbols.insert(S.ID);
+  Callback(mergeSymbol(*DynS, S));
+});
+  }
   SPAN_ATTACH(Tracer, "dynamic", DynamicCount);
   SPAN_ATTACH(Tracer, "static", StaticCount);
   SPAN_ATTACH(Tracer, "merged", MergedCount);
@@ -77,20 +79,22 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  Static->lookup(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-const Symbol *Sym = B.find(S.ID);
-RemainingIDs.erase(S.ID);
-if (!Sym)
-  Callback(S);
-else
-  Callback(mergeSymbol(*Sym, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+Static->lookup(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  const Symbol *Sym = B.find(S.ID);
+  RemainingIDs.erase(S.ID);
+  if (!Sym)
+Callback(S);
+  else
+Callback(mergeSymbol(*Sym, S));
+});
+  }
   for (const auto  : RemainingIDs)
 if (const Symbol *Sym = B.find(ID))
   Callback(*Sym);
Index: clang-tools-extra/clangd/index/Index.cpp
===
--- clang-tools-extra/clangd/index/Index.cpp
+++ clang-tools-extra/clangd/index/Index.cpp
@@ -78,7 +78,12 @@
 
 llvm::unique_function
 SwapIndex::indexedFiles() const {
-  return snapshot()->indexedFiles();
+  // The index snapshot should outlive this method return value.
+  auto SnapShot = snapshot();
+  auto IndexedFiles = SnapShot->indexedFiles();
+  return [KeepAlive{std::move(SnapShot)},
+  IndexContainsFile{std::move(IndexedFiles)}](
+ llvm::StringRef File) { return IndexContainsFile(File); };
 }
 
 size_t SwapIndex::estimateMemoryUsage() const {


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-

[PATCH] D95206: [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 318442.
ArcsinX added a comment.

Fix format


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D95206/new/

https://reviews.llvm.org/D95206

Files:
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Merge.cpp


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  auto DynS = Dyn.find(S.ID);
+  ++StaticCount;
+  if (DynS == Dyn.end())
+return Callback(S);
+  ++MergedCount;
+  SeenDynamicSymbols.insert(S.ID);
+  Callback(mergeSymbol(*DynS, S));
+});
+  }
   SPAN_ATTACH(Tracer, "dynamic", DynamicCount);
   SPAN_ATTACH(Tracer, "static", StaticCount);
   SPAN_ATTACH(Tracer, "merged", MergedCount);
@@ -77,20 +79,22 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  Static->lookup(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-const Symbol *Sym = B.find(S.ID);
-RemainingIDs.erase(S.ID);
-if (!Sym)
-  Callback(S);
-else
-  Callback(mergeSymbol(*Sym, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+Static->lookup(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  const Symbol *Sym = B.find(S.ID);
+  RemainingIDs.erase(S.ID);
+  if (!Sym)
+Callback(S);
+  else
+Callback(mergeSymbol(*Sym, S));
+});
+  }
   for (const auto  : RemainingIDs)
 if (const Symbol *Sym = B.find(ID))
   Callback(*Sym);
Index: clang-tools-extra/clangd/index/Index.cpp
===
--- clang-tools-extra/clangd/index/Index.cpp
+++ clang-tools-extra/clangd/index/Index.cpp
@@ -78,7 +78,8 @@
 
 llvm::unique_function
 SwapIndex::indexedFiles() const {
-  return snapshot()->indexedFiles();
+  return [KeepAlive = snapshot(), IndexedFiles = snapshot()->indexedFiles()](
+ llvm::StringRef File) { return IndexedFiles(File); };
 }
 
 size_t SwapIndex::estimateMemoryUsage() const {


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const 

[PATCH] D95206: [clangd][SwapIndex] ensure that the old index is alive while we are using it via the function returned by `SwapIndex::indexedFiles()` call

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added reviewers: sammccall, kadircet.
Herald added subscribers: usaxena95, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

Without this patch the old index could be freed, but there still could be tries 
to access it via the function returned by `SwapIndex::indexedFiles()` call.
This leads to hard to reproduce clangd crashes at code completion.
This patch keeps the old index alive until the function returned by 
`SwapIndex::indexedFiles()` call is alive.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D95206

Files:
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Merge.cpp


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-auto DynS = Dyn.find(S.ID);
-++StaticCount;
-if (DynS == Dyn.end())
-  return Callback(S);
-++MergedCount;
-SeenDynamicSymbols.insert(S.ID);
-Callback(mergeSymbol(*DynS, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  auto DynS = Dyn.find(S.ID);
+  ++StaticCount;
+  if (DynS == Dyn.end())
+return Callback(S);
+  ++MergedCount;
+  SeenDynamicSymbols.insert(S.ID);
+  Callback(mergeSymbol(*DynS, S));
+});
+  }
   SPAN_ATTACH(Tracer, "dynamic", DynamicCount);
   SPAN_ATTACH(Tracer, "static", StaticCount);
   SPAN_ATTACH(Tracer, "merged", MergedCount);
@@ -77,20 +79,22 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  Static->lookup(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
- : S.CanonicalDeclaration.FileURI))
-  return;
-const Symbol *Sym = B.find(S.ID);
-RemainingIDs.erase(S.ID);
-if (!Sym)
-  Callback(S);
-else
-  Callback(mergeSymbol(*Sym, S));
-  });
+  {
+auto DynamicContainsFile = Dynamic->indexedFiles();
+Static->lookup(Req, [&](const Symbol ) {
+  // We expect the definition to see the canonical declaration, so it seems
+  // to be enough to check only the definition if it exists.
+  if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+   : S.CanonicalDeclaration.FileURI))
+return;
+  const Symbol *Sym = B.find(S.ID);
+  RemainingIDs.erase(S.ID);
+  if (!Sym)
+Callback(S);
+  else
+Callback(mergeSymbol(*Sym, S));
+});
+  }
   for (const auto  : RemainingIDs)
 if (const Symbol *Sym = B.find(ID))
   Callback(*Sym);
Index: clang-tools-extra/clangd/index/Index.cpp
===
--- clang-tools-extra/clangd/index/Index.cpp
+++ clang-tools-extra/clangd/index/Index.cpp
@@ -78,7 +78,10 @@
 
 llvm::unique_function
 SwapIndex::indexedFiles() const {
-  return snapshot()->indexedFiles();
+  return [ KeepAlive = snapshot(),
+   IndexedFiles = snapshot()->indexedFiles() ](llvm::StringRef File) {
+return IndexedFiles(File);
+  };
 }
 
 size_t SwapIndex::estimateMemoryUsage() const {


Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -44,21 +44,23 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
-  auto DynamicContainsFile = Dynamic->indexedFiles();
-  More |= Static->fuzzyFind(Req, [&](const Symbol ) {
-// We expect the definition to see the canonical declaration, so it seems
-// to be enough to check only the definition if it exists.
-if 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 317521.
ArcsinX added a comment.

Prevent crash in debug mode.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

Files:
  clang-tools-extra/clangd/index/BackgroundRebuild.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/FileIndex.h
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp

Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1238,9 +1238,9 @@
llvm::function_ref
Callback) const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexDataKind::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
@@ -1292,9 +1292,9 @@
llvm::function_ref)
 const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexDataKind::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -231,11 +231,11 @@
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
   llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
- std::move(Files), std::move(Data), Size);
+ std::move(Files), IndexDataKind::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"), IndexDataKind::All);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexDataKind::All);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexDataKind::None);
 }
 
 TEST(MemIndexTest, TemplateSpecialization) {
@@ -508,23 +508,24 @@
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
   llvm::StringSet<> DynFiles = {testPath("foo.cc")};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
-RelationSlab(), std::move(DynFiles), std::move(DynData),
-DynSize);
+RelationSlab(), std::move(DynFiles), IndexDataKind::Symbols,
+std::move(DynData), DynSize);
   SymbolSlab StaticSymbols;
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("bar.cc")};
-  MemIndex StaticIndex(std::move(StaticData.first),
-   std::move(StaticData.second), RelationSlab(),
-   std::move(StaticFiles), std::move(StaticData),
-   StaticSymbols.bytes() + StaticRefs.bytes());
+  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex StaticIndex(
+  std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
+  std::move(StaticFiles), IndexDataKind::References, std::move(StaticData),
+  StaticSymbols.bytes() + StaticRefs.bytes());
   MergedIndex Merge(, );
 
   auto ContainsFile = Merge.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"),
+IndexDataKind::Symbols | IndexDataKind::References);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexDataKind::References);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexDataKind::None);
 }
 
 TEST(MergeIndexTest, NonDocumentation) {
Index: 

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added reviewers: sammccall, kadircet.
ArcsinX added a comment.

Initial discussion https://reviews.llvm.org/D93683?id=313280#inline-875666


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94952/new/

https://reviews.llvm.org/D94952

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94952: [clangd] Take into account what is in the index (symbols, references, etc.) at indexes merge

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
Herald added subscribers: usaxena95, kadircet, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

Current indexes merge logic skip data from the static index if the file is in 
the dynamic index, but sometimes the dynamic index does not contain references 
(e.g. preamble (dynamic) index vs background (static) index).
This problem was masked by an incorrect file list in the preamble index: the 
preamble file list consists of files URI's and all other indexes file lists 
consist of file paths.
This patch introduces the index data kind (kind of data inside the index), 
which makes indexes merge more flexible and solves the problem described above.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D94952

Files:
  clang-tools-extra/clangd/index/BackgroundRebuild.cpp
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/FileIndex.h
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/FileIndexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp

Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1238,9 +1238,9 @@
llvm::function_ref
Callback) const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexDataKind::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
@@ -1292,9 +1292,9 @@
llvm::function_ref)
 const override {}
 
-llvm::unique_function
+llvm::unique_function
 indexedFiles() const override {
-  return [](llvm::StringRef) { return false; };
+  return [](llvm::StringRef) { return IndexDataKind::None; };
 }
 
 size_t estimateMemoryUsage() const override { return 0; }
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -231,11 +231,11 @@
   auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
   llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
   MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
- std::move(Files), std::move(Data), Size);
+ std::move(Files), IndexDataKind::All, std::move(Data), Size);
   auto ContainsFile = I.indexedFiles();
-  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
-  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
-  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+  EXPECT_EQ(ContainsFile("unittest:///foo.cc"), IndexDataKind::All);
+  EXPECT_EQ(ContainsFile("unittest:///bar.cc"), IndexDataKind::All);
+  EXPECT_EQ(ContainsFile("unittest:///foobar.cc"), IndexDataKind::None);
 }
 
 TEST(MemIndexTest, TemplateSpecialization) {
@@ -508,23 +508,24 @@
   auto DynData = std::make_pair(std::move(DynSymbols), std::move(DynRefs));
   llvm::StringSet<> DynFiles = {testPath("foo.cc")};
   MemIndex DynIndex(std::move(DynData.first), std::move(DynData.second),
-RelationSlab(), std::move(DynFiles), std::move(DynData),
-DynSize);
+RelationSlab(), std::move(DynFiles), IndexDataKind::Symbols,
+std::move(DynData), DynSize);
   SymbolSlab StaticSymbols;
   RefSlab StaticRefs;
   auto StaticData =
   std::make_pair(std::move(StaticSymbols), std::move(StaticRefs));
-  llvm::StringSet<> StaticFiles = {testPath("bar.cc")};
-  MemIndex StaticIndex(std::move(StaticData.first),
-   std::move(StaticData.second), RelationSlab(),
-   std::move(StaticFiles), std::move(StaticData),
-   StaticSymbols.bytes() + StaticRefs.bytes());
+  llvm::StringSet<> StaticFiles = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex StaticIndex(
+  std::move(StaticData.first), std::move(StaticData.second), RelationSlab(),
+  std::move(StaticFiles), IndexDataKind::References, 

[PATCH] D94477: [clangd] Add main file macros into the main-file index.

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG2e25be0b6134: [clangd] Add main file macros into the 
main-file index. (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94477/new/

https://reviews.llvm.org/D94477

Files:
  clang-tools-extra/clangd/CollectMacros.cpp
  clang-tools-extra/clangd/CollectMacros.h
  clang-tools-extra/clangd/SemanticHighlighting.cpp
  clang-tools-extra/clangd/XRefs.cpp
  clang-tools-extra/clangd/index/SymbolCollector.cpp
  clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
  clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
  clang-tools-extra/clangd/unittests/ParsedASTTests.cpp

Index: clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
===
--- clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
+++ clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
@@ -341,10 +341,10 @@
   std::vector MacroExpansionPositions;
   for (const auto  : AST.getMacros().MacroRefs) {
 for (const auto  : SIDToRefs.second)
-  MacroExpansionPositions.push_back(R.start);
+  MacroExpansionPositions.push_back(R.Rng.start);
   }
   for (const auto  : AST.getMacros().UnknownMacros)
-MacroExpansionPositions.push_back(R.start);
+MacroExpansionPositions.push_back(R.Rng.start);
   EXPECT_THAT(MacroExpansionPositions,
   testing::UnorderedElementsAreArray(TestCase.points()));
 }
Index: clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
===
--- clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
+++ clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
@@ -52,17 +52,7 @@
   return *SymbolInfos;
 }
 
-// FIXME: We update two indexes during main file processing:
-//- preamble index (static)
-//- main-file index (dynamic)
-//The macro in this test appears to be in the preamble index and not
-//in the main-file index. According to our logic of indexes merging, we
-//do not take this macro from the static (preamble) index, because it
-//location within the file from the dynamic (main-file) index.
-//
-//Possible solution is to exclude main-file symbols from the preamble
-//index, after that we can enable this test again.
-TEST(WorkspaceSymbols, DISABLED_Macros) {
+TEST(WorkspaceSymbols, Macros) {
   TestTU TU;
   TU.Code = R"cpp(
#define MACRO X
Index: clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
===
--- clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
+++ clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
@@ -95,14 +95,18 @@
   assert(Macro);
   auto SID = getSymbolID(Macro->Name, Macro->Info, SM);
 
-  EXPECT_THAT(ExpectedRefs,
-  UnorderedElementsAreArray(ActualMacroRefs.MacroRefs[SID]))
+  std::vector Ranges;
+  for (const auto  : ActualMacroRefs.MacroRefs[SID])
+Ranges.push_back(Ref.Rng);
+  EXPECT_THAT(ExpectedRefs, UnorderedElementsAreArray(Ranges))
   << "Annotation=" << I << ", MacroName=" << Macro->Name
   << ", Test = " << Test;
 }
 // Unknown macros.
-EXPECT_THAT(AST.getMacros().UnknownMacros,
-UnorderedElementsAreArray(T.ranges("Unknown")))
+std::vector Ranges;
+for (const auto  : AST.getMacros().UnknownMacros)
+  Ranges.push_back(Ref.Rng);
+EXPECT_THAT(Ranges, UnorderedElementsAreArray(T.ranges("Unknown")))
 << "Unknown macros doesn't match in " << Test;
   }
 }
Index: clang-tools-extra/clangd/index/SymbolCollector.cpp
===
--- clang-tools-extra/clangd/index/SymbolCollector.cpp
+++ clang-tools-extra/clangd/index/SymbolCollector.cpp
@@ -386,16 +386,31 @@
   const auto MainFileURI = toURI(SM, MainFileEntry->getName(), Opts);
   // Add macro references.
   for (const auto  : MacroRefsToIndex.MacroRefs) {
-for (const auto  : IDToRefs.second) {
+for (const auto  : IDToRefs.second) {
+  const auto  = MacroRef.Rng;
+  bool IsDefinition = MacroRef.IsDefinition;
   Ref R;
   R.Location.Start.setLine(Range.start.line);
   R.Location.Start.setColumn(Range.start.character);
   R.Location.End.setLine(Range.end.line);
   R.Location.End.setColumn(Range.end.character);
   R.Location.FileURI = MainFileURI.c_str();
-  // FIXME: Add correct RefKind information to MainFileMacros.
-  R.Kind = RefKind::Reference;
+  R.Kind = IsDefinition ? RefKind::Definition : RefKind::Reference;
   Refs.insert(IDToRefs.first, R);
+  if (IsDefinition) {
+Symbol S;
+S.ID = IDToRefs.first;
+auto StartLoc = cantFail(sourceLocationInMainFile(SM, Range.start));
+auto EndLoc = cantFail(sourceLocationInMainFile(SM, 

[PATCH] D94477: [clangd] Add main file macros into the main-file index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 316331.
ArcsinX added a comment.

- std::pair => struct MacroOccurrence
- remove addressed fixme comment
- assert() => cantFail()
- use toSourceCode() to get the macro name


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94477/new/

https://reviews.llvm.org/D94477

Files:
  clang-tools-extra/clangd/CollectMacros.cpp
  clang-tools-extra/clangd/CollectMacros.h
  clang-tools-extra/clangd/SemanticHighlighting.cpp
  clang-tools-extra/clangd/XRefs.cpp
  clang-tools-extra/clangd/index/SymbolCollector.cpp
  clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
  clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
  clang-tools-extra/clangd/unittests/ParsedASTTests.cpp

Index: clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
===
--- clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
+++ clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
@@ -341,10 +341,10 @@
   std::vector MacroExpansionPositions;
   for (const auto  : AST.getMacros().MacroRefs) {
 for (const auto  : SIDToRefs.second)
-  MacroExpansionPositions.push_back(R.start);
+  MacroExpansionPositions.push_back(R.Rng.start);
   }
   for (const auto  : AST.getMacros().UnknownMacros)
-MacroExpansionPositions.push_back(R.start);
+MacroExpansionPositions.push_back(R.Rng.start);
   EXPECT_THAT(MacroExpansionPositions,
   testing::UnorderedElementsAreArray(TestCase.points()));
 }
Index: clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
===
--- clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
+++ clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
@@ -52,17 +52,7 @@
   return *SymbolInfos;
 }
 
-// FIXME: We update two indexes during main file processing:
-//- preamble index (static)
-//- main-file index (dynamic)
-//The macro in this test appears to be in the preamble index and not
-//in the main-file index. According to our logic of indexes merging, we
-//do not take this macro from the static (preamble) index, because it
-//location within the file from the dynamic (main-file) index.
-//
-//Possible solution is to exclude main-file symbols from the preamble
-//index, after that we can enable this test again.
-TEST(WorkspaceSymbols, DISABLED_Macros) {
+TEST(WorkspaceSymbols, Macros) {
   TestTU TU;
   TU.Code = R"cpp(
#define MACRO X
Index: clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
===
--- clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
+++ clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
@@ -95,14 +95,18 @@
   assert(Macro);
   auto SID = getSymbolID(Macro->Name, Macro->Info, SM);
 
-  EXPECT_THAT(ExpectedRefs,
-  UnorderedElementsAreArray(ActualMacroRefs.MacroRefs[SID]))
+  std::vector Ranges;
+  for (const auto  : ActualMacroRefs.MacroRefs[SID])
+Ranges.push_back(Ref.Rng);
+  EXPECT_THAT(ExpectedRefs, UnorderedElementsAreArray(Ranges))
   << "Annotation=" << I << ", MacroName=" << Macro->Name
   << ", Test = " << Test;
 }
 // Unknown macros.
-EXPECT_THAT(AST.getMacros().UnknownMacros,
-UnorderedElementsAreArray(T.ranges("Unknown")))
+std::vector Ranges;
+for (const auto  : AST.getMacros().UnknownMacros)
+  Ranges.push_back(Ref.Rng);
+EXPECT_THAT(Ranges, UnorderedElementsAreArray(T.ranges("Unknown")))
 << "Unknown macros doesn't match in " << Test;
   }
 }
Index: clang-tools-extra/clangd/index/SymbolCollector.cpp
===
--- clang-tools-extra/clangd/index/SymbolCollector.cpp
+++ clang-tools-extra/clangd/index/SymbolCollector.cpp
@@ -386,16 +386,31 @@
   const auto MainFileURI = toURI(SM, MainFileEntry->getName(), Opts);
   // Add macro references.
   for (const auto  : MacroRefsToIndex.MacroRefs) {
-for (const auto  : IDToRefs.second) {
+for (const auto  : IDToRefs.second) {
+  const auto  = MacroRef.Rng;
+  bool IsDefinition = MacroRef.IsDefinition;
   Ref R;
   R.Location.Start.setLine(Range.start.line);
   R.Location.Start.setColumn(Range.start.character);
   R.Location.End.setLine(Range.end.line);
   R.Location.End.setColumn(Range.end.character);
   R.Location.FileURI = MainFileURI.c_str();
-  // FIXME: Add correct RefKind information to MainFileMacros.
-  R.Kind = RefKind::Reference;
+  R.Kind = IsDefinition ? RefKind::Definition : RefKind::Reference;
   Refs.insert(IDToRefs.first, R);
+  if (IsDefinition) {
+Symbol S;
+S.ID = IDToRefs.first;
+auto StartLoc = cantFail(sourceLocationInMainFile(SM, Range.start));
+auto EndLoc = 

[PATCH] D94477: [clangd] Add main file macros into the main-file index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added reviewers: sammccall, kadircet.
ArcsinX added a comment.

I am not happy with my solution, hope to hear an advice.
Initial discussion about this problem: https://reviews.llvm.org/D93683#2468842


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D94477/new/

https://reviews.llvm.org/D94477

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D94477: [clangd] Add main file macros into the main-file index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
Herald added subscribers: usaxena95, kadircet, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

This patch is a try to fix `WorkspaceSymbols.Macros` test after D93796 
.
If a macro definition is in the preamble section, then it appears to be in the 
preamble (static) index and not in the main-file (dynamic) index.
Thus, a such macro could not be found at a symbol search according to the logic 
that we skip symbols from the static index if the location of these symbols is 
inside the dynamic index files.
To fix this behavior this patch adds main file macros into the main-file 
(dynamic) index.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D94477

Files:
  clang-tools-extra/clangd/CollectMacros.cpp
  clang-tools-extra/clangd/CollectMacros.h
  clang-tools-extra/clangd/SemanticHighlighting.cpp
  clang-tools-extra/clangd/XRefs.cpp
  clang-tools-extra/clangd/index/SymbolCollector.cpp
  clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
  clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
  clang-tools-extra/clangd/unittests/ParsedASTTests.cpp

Index: clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
===
--- clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
+++ clang-tools-extra/clangd/unittests/ParsedASTTests.cpp
@@ -341,7 +341,7 @@
   std::vector MacroExpansionPositions;
   for (const auto  : AST.getMacros().MacroRefs) {
 for (const auto  : SIDToRefs.second)
-  MacroExpansionPositions.push_back(R.start);
+  MacroExpansionPositions.push_back(R.first.start);
   }
   for (const auto  : AST.getMacros().UnknownMacros)
 MacroExpansionPositions.push_back(R.start);
Index: clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
===
--- clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
+++ clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
@@ -52,17 +52,7 @@
   return *SymbolInfos;
 }
 
-// FIXME: We update two indexes during main file processing:
-//- preamble index (static)
-//- main-file index (dynamic)
-//The macro in this test appears to be in the preamble index and not
-//in the main-file index. According to our logic of indexes merging, we
-//do not take this macro from the static (preamble) index, because it
-//location within the file from the dynamic (main-file) index.
-//
-//Possible solution is to exclude main-file symbols from the preamble
-//index, after that we can enable this test again.
-TEST(WorkspaceSymbols, DISABLED_Macros) {
+TEST(WorkspaceSymbols, Macros) {
   TestTU TU;
   TU.Code = R"cpp(
#define MACRO X
Index: clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
===
--- clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
+++ clang-tools-extra/clangd/unittests/CollectMacrosTests.cpp
@@ -95,8 +95,10 @@
   assert(Macro);
   auto SID = getSymbolID(Macro->Name, Macro->Info, SM);
 
-  EXPECT_THAT(ExpectedRefs,
-  UnorderedElementsAreArray(ActualMacroRefs.MacroRefs[SID]))
+  std::vector MacroRefs;
+  for (const auto  : ActualMacroRefs.MacroRefs[SID])
+MacroRefs.push_back(RangeAndIsDefinition.first);
+  EXPECT_THAT(ExpectedRefs, UnorderedElementsAreArray(MacroRefs))
   << "Annotation=" << I << ", MacroName=" << Macro->Name
   << ", Test = " << Test;
 }
Index: clang-tools-extra/clangd/index/SymbolCollector.cpp
===
--- clang-tools-extra/clangd/index/SymbolCollector.cpp
+++ clang-tools-extra/clangd/index/SymbolCollector.cpp
@@ -386,7 +386,9 @@
   const auto MainFileURI = toURI(SM, MainFileEntry->getName(), Opts);
   // Add macro references.
   for (const auto  : MacroRefsToIndex.MacroRefs) {
-for (const auto  : IDToRefs.second) {
+for (const auto  : IDToRefs.second) {
+  const auto  = RangeAndIsDefinition.first;
+  bool IsDefinition = RangeAndIsDefinition.second;
   Ref R;
   R.Location.Start.setLine(Range.start.line);
   R.Location.Start.setColumn(Range.start.character);
@@ -394,8 +396,26 @@
   R.Location.End.setColumn(Range.end.character);
   R.Location.FileURI = MainFileURI.c_str();
   // FIXME: Add correct RefKind information to MainFileMacros.
-  R.Kind = RefKind::Reference;
+  R.Kind = IsDefinition ? RefKind::Definition : RefKind::Reference;
   Refs.insert(IDToRefs.first, R);
+  if (IsDefinition) {
+Symbol S;
+S.ID = IDToRefs.first;
+auto StartLoc = sourceLocationInMainFile(SM, Range.start);
+assert(StartLoc);
+auto EndLoc = sourceLocationInMainFile(SM, Range.end);
+

[PATCH] D93796: [clangd][fuzzyFind] Do not show stale symbols in the result.

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG979228f120f4: [clangd][fuzzyFind] Do not show stale symbols 
in the result. (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93796/new/

https://reviews.llvm.org/D93796

Files:
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -335,6 +335,39 @@
   UnorderedElementsAre("ns::A", "ns::B", "ns::C"));
 }
 
+TEST(MergeIndexTest, FuzzyFindRemovedSymbol) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo symbol
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo symbol, i.e. build dynamic index for test.cc, which is empty.
+  Test.HeaderCode = "";
+  Test.Code = "";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return removed symbol.
+  FuzzyFindRequest Req;
+  Req.AnyScope = true;
+  Req.Query = "Foo";
+  unsigned SymbolCounter = 0;
+  bool IsIncomplete =
+  Merge.fuzzyFind(Req, [&](const Symbol &) { ++SymbolCounter; });
+  EXPECT_FALSE(IsIncomplete);
+  EXPECT_EQ(SymbolCounter, 0u);
+}
+
 TEST(MergeTest, Merge) {
   Symbol L, R;
   L.ID = R.ID = SymbolID("hello");
Index: clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
===
--- clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
+++ clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
@@ -52,7 +52,17 @@
   return *SymbolInfos;
 }
 
-TEST(WorkspaceSymbols, Macros) {
+// FIXME: We update two indexes during main file processing:
+//- preamble index (static)
+//- main-file index (dynamic)
+//The macro in this test appears to be in the preamble index and not
+//in the main-file index. According to our logic of indexes merging, we
+//do not take this macro from the static (preamble) index, because it
+//location within the file from the dynamic (main-file) index.
+//
+//Possible solution is to exclude main-file symbols from the preamble
+//index, after that we can enable this test again.
+TEST(WorkspaceSymbols, DISABLED_Macros) {
   TestTU TU;
   TU.Code = R"cpp(
#define MACRO X
Index: clang-tools-extra/clangd/index/Merge.h
===
--- clang-tools-extra/clangd/index/Merge.h
+++ clang-tools-extra/clangd/index/Merge.h
@@ -23,10 +23,6 @@
 //  - the Dynamic index covers few files, but is relatively up-to-date.
 //  - the Static index covers a bigger set of files, but is relatively stale.
 // The returned index attempts to combine results, and avoid duplicates.
-//
-// FIXME: We don't have a mechanism in Index to track deleted symbols and
-// refs in dirty files, so the merged index may return stale symbols
-// and refs from Static index.
 class MergedIndex : public SymbolIndex {
   const SymbolIndex *Dynamic, *Static;
 
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -22,11 +22,6 @@
 namespace clang {
 namespace clangd {
 
-// FIXME: Deleted symbols in dirty files are still returned (from Static).
-//To identify these eliminate these, we should:
-//  - find the generating file from each Symbol which is Static-only
-//  - ask Dynamic if it has that file (needs new SymbolIndex method)
-//  - if so, drop the Symbol.
 bool MergedIndex::fuzzyFind(
 const FuzzyFindRequest ,
 llvm::function_ref Callback) const {
@@ -49,7 +44,13 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition if it exists.
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 auto DynS = Dyn.find(S.ID);
 ++StaticCount;
 

[PATCH] D93796: [clangd][fuzzyFind] Do not show stale symbols in the result.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
ArcsinX added a reviewer: sammccall.
Herald added subscribers: usaxena95, kadircet, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

This is follow up to D93393 .
Without this patch `MergedIndex::fuzzyFind()` returns stale symbols from the 
static index even if these symbols were removed.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D93796

Files:
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp

Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -335,6 +335,39 @@
   UnorderedElementsAre("ns::A", "ns::B", "ns::C"));
 }
 
+TEST(MergeIndexTest, FuzzyFindRemovedSymbol) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo symbol
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo symbol, i.e. build dynamic index for test.cc, which is empty.
+  Test.HeaderCode = "";
+  Test.Code = "";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return removed symbol.
+  FuzzyFindRequest Req;
+  Req.AnyScope = true;
+  Req.Query = "Foo";
+  unsigned SymbolCounter = 0;
+  bool IsIncomplete =
+  Merge.fuzzyFind(Req, [&](const Symbol &) { ++SymbolCounter; });
+  EXPECT_FALSE(IsIncomplete);
+  EXPECT_EQ(SymbolCounter, 0u);
+}
+
 TEST(MergeTest, Merge) {
   Symbol L, R;
   L.ID = R.ID = SymbolID("hello");
Index: clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
===
--- clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
+++ clang-tools-extra/clangd/unittests/FindSymbolsTests.cpp
@@ -52,7 +52,17 @@
   return *SymbolInfos;
 }
 
-TEST(WorkspaceSymbols, Macros) {
+// FIXME: We update two indexes during main file processing:
+//- preamble index (static)
+//- main-file index (dynamic)
+//The macro in this test appears to be in the preamble index and not
+//in the main-file index. According to our logic of indexes merging, we
+//do not take this macro from the static (preamble) index, because it
+//location within the file from the dynamic (main-file) index.
+//
+//Possible solution is to exclude main-file symbols from the preamble
+//index, after that we can enable this test again.
+TEST(WorkspaceSymbols, DISABLED_Macros) {
   TestTU TU;
   TU.Code = R"cpp(
#define MACRO X
Index: clang-tools-extra/clangd/index/Merge.h
===
--- clang-tools-extra/clangd/index/Merge.h
+++ clang-tools-extra/clangd/index/Merge.h
@@ -23,10 +23,6 @@
 //  - the Dynamic index covers few files, but is relatively up-to-date.
 //  - the Static index covers a bigger set of files, but is relatively stale.
 // The returned index attempts to combine results, and avoid duplicates.
-//
-// FIXME: We don't have a mechanism in Index to track deleted symbols and
-// refs in dirty files, so the merged index may return stale symbols
-// and refs from Static index.
 class MergedIndex : public SymbolIndex {
   const SymbolIndex *Dynamic, *Static;
 
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -22,11 +22,6 @@
 namespace clang {
 namespace clangd {
 
-// FIXME: Deleted symbols in dirty files are still returned (from Static).
-//To identify these eliminate these, we should:
-//  - find the generating file from each Symbol which is Static-only
-//  - ask Dynamic if it has that file (needs new SymbolIndex method)
-//  - if so, drop the Symbol.
 bool MergedIndex::fuzzyFind(
 const FuzzyFindRequest ,
 llvm::function_ref Callback) const {
@@ -49,7 +44,13 @@
   SymbolSlab Dyn = std::move(DynB).build();
 
   llvm::DenseSet SeenDynamicSymbols;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   More |= Static->fuzzyFind(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition if it exists.
+if 

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rG2522fa053b62: [clangd] Do not take stale definition from the 
static index. (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

Files:
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,40 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  unsigned SymbolCounter = 0;
+  Merge.lookup(LookupReq, [&](const Symbol ) {
+++SymbolCounter;
+EXPECT_FALSE(Sym.Definition);
+  });
+  EXPECT_EQ(SymbolCounter, 1u);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,13 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition if it exists.
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 const Symbol *Sym = B.find(S.ID);
 RemainingIDs.erase(S.ID);
 if (!Sym)


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,40 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  unsigned SymbolCounter = 0;
+  Merge.lookup(LookupReq, [&](const Symbol ) {
+++SymbolCounter;
+EXPECT_FALSE(Sym.Definition);
+  });
+  EXPECT_EQ(SymbolCounter, 1u);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,13 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition 

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX marked an inline comment as done.
ArcsinX added a comment.

In D93683#2469584 , @sammccall wrote:

> In D93683#2469553 , @ArcsinX wrote:
>
>> In D93683#2468842 , @sammccall 
>> wrote:
>>
>>> I'm not 100% sure this bug would block adding the equivalent change for 
>>> `fuzzyFind` though - it'll affect documentSymbol but not code-completion 
>>> IIUC (because of the workaround I mentioned above). And these symbols are 
>>> pretty rare.
>>
>> With the similar change for `fuzzyFind()`, `WorkspaceSymbols.Macros` test 
>> fails. But maybe I can create one more review for this and we will discuss 
>> it there.
>
> Sorry, incomplete thought. I meant workspace/symbols would be broken for such 
> macros, but maybe that was acceptable (it's not a terrible common feature nor 
> symbol kind). Not great but we're trading one bug for another.
> In particular, if we plan to fix both I don't think the *sequencing* matters.

Ok. I will propose a separate patch for `fuzzyFind()` similar to this one.

Thank you for review.




Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:434
+  PreambleSymbols.update(
+  *FilePath, std::make_unique(std::move(*IF->Symbols)),
+  std::make_unique(),

sammccall wrote:
> ArcsinX wrote:
> > sammccall wrote:
> > > ArcsinX wrote:
> > > > We do not need this change to fix behaviour for removed definition, but 
> > > > this is the only one place where we use URI instead of path as a key.
> > > Yeah... nevertheless the key is totally arbitrary here, so we might as 
> > > well use whatever is most convenient/cheapest.
> > It's not arbitrary after D93393 (we use these keys to create the list of 
> > indexed files).
> > 
> > But as it turns, the problem is deeper... I will try to explain.
> > Preamble index does not contain references, so with that change our logic 
> > about "file X is in the dynamic index, so toss references with location in 
> > X from the static index" breaks here, seems we can not think about preamble 
> > index that it is a part of the dynamic index for the background index 
> > (static). I wonder why tests do not catch this, but with this change 
> > find-all-references does not contain refs from headers for which we have 
> > preamble index, that's why I reverted this. Maybe we need to add some 
> > comment here about this behavior.
> > Example:
> > `test.h`
> > ```
> > void test();
> > ```
> > `test.c`
> > ```
> > #include "test.h"
> > void te^st {}
> > ```
> > 
> > - didOpen `test.c`
> > - references (where ^ is)
> > With this change only 1 element will be returned (definition of `test` in 
> > `test.c` and nothing from `test.h`)
> > 
> > I am not sure what we can do here, but I think we need some special merge 
> > for preamble and main file indexes (another merge class for this case 
> > instead of `MergedIndex`?)
> > Preamble index does not contain references, so with that change our logic 
> > about "file X is in the dynamic index, so toss references with location in 
> > X from the static index" breaks
> 
> Hmm, what if we made the return value of the indexedFiles functor a bitmask 
> instead of a single boolean (contains symbols, contains refs, ...)?
This sounds reasonable. I will try to implement a solution for this problem 
based on your suggestion (in a separate patch)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 313504.
ArcsinX added a comment.

Add comment.
Fix possible `MergeIndexTest.LookupRemovedDefinition` test failure.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

Files:
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,40 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  unsigned SymbolCounter = 0;
+  Merge.lookup(LookupReq, [&](const Symbol ) {
+++SymbolCounter;
+EXPECT_FALSE(Sym.Definition);
+  });
+  EXPECT_EQ(SymbolCounter, 1u);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,13 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition if it exists.
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 const Symbol *Sym = B.find(S.ID);
 RemainingIDs.erase(S.ID);
 if (!Sym)


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,40 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  unsigned SymbolCounter = 0;
+  Merge.lookup(LookupReq, [&](const Symbol ) {
+++SymbolCounter;
+EXPECT_FALSE(Sym.Definition);
+  });
+  EXPECT_EQ(SymbolCounter, 1u);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,13 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+// We expect the definition to see the canonical declaration, so it seems
+// to be enough to check only the definition if it exists.
+if 

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added a comment.

In D93683#2468842 , @sammccall wrote:

> I'm not 100% sure this bug would block adding the equivalent change for 
> `fuzzyFind` though - it'll affect documentSymbol but not code-completion IIUC 
> (because of the workaround I mentioned above). And these symbols are pretty 
> rare.

With the similar change for `fuzzyFind()`, `WorkspaceSymbols.Macros` test 
fails. But maybe I can create one more review for this and we will discuss it 
there.




Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:434
+  PreambleSymbols.update(
+  *FilePath, std::make_unique(std::move(*IF->Symbols)),
+  std::make_unique(),

sammccall wrote:
> ArcsinX wrote:
> > We do not need this change to fix behaviour for removed definition, but 
> > this is the only one place where we use URI instead of path as a key.
> Yeah... nevertheless the key is totally arbitrary here, so we might as well 
> use whatever is most convenient/cheapest.
It's not arbitrary after D93393 (we use these keys to create the list of 
indexed files).

But as it turns, the problem is deeper... I will try to explain.
Preamble index does not contain references, so with that change our logic about 
"file X is in the dynamic index, so toss references with location in X from the 
static index" breaks here, seems we can not think about preamble index that it 
is a part of the dynamic index for the background index (static). I wonder why 
tests do not catch this, but with this change find-all-references does not 
contain refs from headers for which we have preamble index, that's why I 
reverted this. Maybe we need to add some comment here about this behavior.
Example:
`test.h`
```
void test();
```
`test.c`
```
#include "test.h"
void te^st {}
```

- didOpen `test.c`
- references (where ^ is)
With this change only 1 element will be returned (definition of `test` in 
`test.c` and nothing from `test.h`)

I am not sure what we can do here, but I think we need some special merge for 
preamble and main file indexes (another merge class for this case instead of 
`MergedIndex`?)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 313320.
ArcsinX added a comment.

- Simplify test.
- Revert PreambleSymbols.update() first argument change.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

Files:
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,38 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  std::vector Symbols;
+  Merge.lookup(LookupReq, [&](const Symbol ) { Symbols.push_back(Sym); });
+  ASSERT_EQ(Symbols.size(), 1u);
+  EXPECT_FALSE(Symbols.front().Definition);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,11 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 const Symbol *Sym = B.find(S.ID);
 RemainingIDs.erase(S.ID);
 if (!Sym)


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,38 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex, StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  std::vector Symbols;
+  Merge.lookup(LookupReq, [&](const Symbol ) { Symbols.push_back(Sym); });
+  ASSERT_EQ(Symbols.size(), 1u);
+  EXPECT_FALSE(Symbols.front().Definition);
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,11 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 const Symbol *Sym = B.find(S.ID);
 RemainingIDs.erase(S.ID);
 if (!Sym)
___
cfe-commits mailing 

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX added reviewers: sammccall, kadircet.
ArcsinX added a comment.

I also want to propose the similar patch for fuzzyFind() (as a part of this 
patch or a separate one), but I faced the following problem:

`Test.cpp`

  #define FOO 1

This example creates empty dynamic index for main file symbols and static index 
for preamble symbols (`FOO`). The location of `FOO` is inside `Test.cpp`, so 
(according to the logic that the static index could be stale) we should toss 
`FOO` from the static index (i.e. preamble symbols), but this behaviour is 
incorrect... Any advice?




Comment at: clang-tools-extra/clangd/index/FileIndex.cpp:434
+  PreambleSymbols.update(
+  *FilePath, std::make_unique(std::move(*IF->Symbols)),
+  std::make_unique(),

We do not need this change to fix behaviour for removed definition, but this is 
the only one place where we use URI instead of path as a key.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93683/new/

https://reviews.llvm.org/D93683

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D93683: [clangd] Do not take stale definition from the static index.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX created this revision.
Herald added subscribers: usaxena95, kadircet, arphaman.
ArcsinX requested review of this revision.
Herald added subscribers: cfe-commits, MaskRay, ilya-biryukov.
Herald added a project: clang.

This is follow up to D93393 .
Without this patch clangd takes the symbol definition from the static index if 
this definition was removed from the dynamic index.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D93683

Files:
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,42 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex;
+  FileIndex StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. build dynamic index for test.cc
+  // without Foo definition.
+  Test.Code = "class Foo;";
+  AST = Test.build();
+  DynamicIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Merged index should not return the symbol definition if this definition
+  // location is inside a file from the dynamic index.
+  LookupRequest LookupReq;
+  LookupReq.IDs = {Foo.ID};
+  std::vector Symbols;
+  Merge.lookup(LookupReq, [&](const Symbol ) { Symbols.push_back(Sym); });
+  ASSERT_EQ(Symbols.size(), 1u);
+  const auto  = Symbols.front();
+  EXPECT_FALSE(Sym.Definition);
+  EXPECT_EQ(std::string(Sym.CanonicalDeclaration.FileURI),
+std::string("unittest:///TestTU.h"));
+}
+
 TEST(MergeIndexTest, FuzzyFind) {
   auto I = MemIndex::build(generateSymbols({"ns::A", "ns::B"}), RefSlab(),
RelationSlab()),
Index: clang-tools-extra/clangd/index/Merge.cpp
===
--- clang-tools-extra/clangd/index/Merge.cpp
+++ clang-tools-extra/clangd/index/Merge.cpp
@@ -76,7 +76,11 @@
   Dynamic->lookup(Req, [&](const Symbol ) { B.insert(S); });
 
   auto RemainingIDs = Req.IDs;
+  auto DynamicContainsFile = Dynamic->indexedFiles();
   Static->lookup(Req, [&](const Symbol ) {
+if (DynamicContainsFile(S.Definition ? S.Definition.FileURI
+ : S.CanonicalDeclaration.FileURI))
+  return;
 const Symbol *Sym = B.find(S.ID);
 RemainingIDs.erase(S.ID);
 if (!Sym)
Index: clang-tools-extra/clangd/index/FileIndex.cpp
===
--- clang-tools-extra/clangd/index/FileIndex.cpp
+++ clang-tools-extra/clangd/index/FileIndex.cpp
@@ -428,11 +428,17 @@
 // We are using the key received from ShardedIndex, so it should always
 // exist.
 assert(IF);
-PreambleSymbols.update(
-Uri, std::make_unique(std::move(*IF->Symbols)),
-std::make_unique(),
-std::make_unique(std::move(*IF->Relations)),
-/*CountReferences=*/false);
+auto FilePath = URI::resolve(Uri, Path);
+if (FilePath) {
+  PreambleSymbols.update(
+  *FilePath, std::make_unique(std::move(*IF->Symbols)),
+  std::make_unique(),
+  std::make_unique(std::move(*IF->Relations)),
+  /*CountReferences=*/false);
+} else {
+  elog("Update preamble: could not resolve URI {0}: {1}", Uri,
+   FilePath.takeError());
+}
   }
   size_t IndexVersion = 0;
   auto NewIndex =


Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -290,6 +290,42 @@
   EXPECT_THAT(lookup(M, {}), UnorderedElementsAre());
 }
 
+TEST(MergeIndexTest, LookupRemovedDefinition) {
+  FileIndex DynamicIndex;
+  FileIndex StaticIndex;
+  MergedIndex Merge(, );
+
+  const char *HeaderCode = "class Foo;";
+  auto HeaderSymbols = TestTU::withHeaderCode(HeaderCode).headerSymbols();
+  auto Foo = findSymbol(HeaderSymbols, "Foo");
+
+  // Build static index for test.cc with Foo definition
+  TestTU Test;
+  Test.HeaderCode = HeaderCode;
+  Test.Code = "class Foo {};";
+  Test.Filename = "test.cc";
+  auto AST = Test.build();
+  StaticIndex.updateMain(testPath(Test.Filename), AST);
+
+  // Remove Foo definition from test.cc, i.e. 

[PATCH] D93393: [clangd] Ignore the static index refs from the dynamic index files.

[Aleksandr Platonov via Phabricator via cfe-commits]

This revision was landed with ongoing or failed builds.
This revision was automatically updated to reflect the committed changes.
Closed by commit rGe35f9229dcb2: [clangd] Ignore the static index refs from the 
dynamic index files. (authored by ArcsinX).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93393/new/

https://reviews.llvm.org/D93393

Files:
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.cpp

Index: clang-tools-extra/clangd/unittests/TestFS.cpp
===
--- clang-tools-extra/clangd/unittests/TestFS.cpp
+++ clang-tools-extra/clangd/unittests/TestFS.cpp
@@ -99,8 +99,9 @@
   llvm::Expected
   getAbsolutePath(llvm::StringRef /*Authority*/, llvm::StringRef Body,
   llvm::StringRef HintPath) const override {
-if (!HintPath.startswith(testRoot()))
-  return error("Hint path doesn't start with test root: {0}", HintPath);
+if (!HintPath.empty() && !HintPath.startswith(testRoot()))
+  return error("Hint path is not empty and doesn't start with {0}: {1}",
+   testRoot(), HintPath);
 if (!Body.consume_front("/"))
   return error("Body of an unittest: URI must start with '/'");
 llvm::SmallString<16> Path(Body.begin(), Body.end());
Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1237,6 +1237,12 @@
 void relations(const RelationsRequest ,
llvm::function_ref
Callback) const override {}
+
+llvm::unique_function
+indexedFiles() const override {
+  return [](llvm::StringRef) { return false; };
+}
+
 size_t estimateMemoryUsage() const override { return 0; }
   } PIndex;
   Results = rename({MainCode.point(),
@@ -1285,6 +1291,12 @@
 void relations(const RelationsRequest &,
llvm::function_ref)
 const override {}
+
+llvm::unique_function
+indexedFiles() const override {
+  return [](llvm::StringRef) { return false; };
+}
+
 size_t estimateMemoryUsage() const override { return 0; }
 Ref ReturnedRef;
   } DIndex(XRefInBarCC);
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -224,6 +224,20 @@
   EXPECT_THAT(lookup(*I, SymbolID("ns::nonono")), UnorderedElementsAre());
 }
 
+TEST(MemIndexTest, IndexedFiles) {
+  SymbolSlab Symbols;
+  RefSlab Refs;
+  auto Size = Symbols.bytes() + Refs.bytes();
+  auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
+  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
+ std::move(Files), std::move(Data), Size);
+  auto ContainsFile = I.indexedFiles();
+  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
+  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
+  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+}
+
 TEST(MemIndexTest, TemplateSpecialization) {
   SymbolSlab::Builder B;
 
@@ -367,7 +381,7 @@
   Test.Code = std::string(Test1Code.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Dyn.updateMain(Test.Filename, AST);
+  Dyn.updateMain(testPath(Test.Filename), AST);
 
   // Build static index for test.cc.
   Test.HeaderCode = HeaderCode;
@@ -375,7 +389,7 @@
   Test.Filename = "test.cc";
   auto StaticAST = Test.build();
   // Add stale refs for test.cc.
-  StaticIndex.updateMain(Test.Filename, StaticAST);
+  StaticIndex.updateMain(testPath(Test.Filename), StaticAST);
 
   // Add refs for test2.cc
   Annotations Test2Code(R"(class $Foo[[Foo]] {};)");
@@ -384,7 +398,7 @@
   Test2.Code = std::string(Test2Code.code());
   Test2.Filename = "test2.cc";
   StaticAST = Test2.build();
-  StaticIndex.updateMain(Test2.Filename, StaticAST);
+  StaticIndex.updateMain(testPath(Test2.Filename), StaticAST);
 
   RefsRequest Request;
   Request.IDs = {Foo.ID};
@@ -403,10 +417,47 @@
   RefSlab::Builder Results2;
   

[PATCH] D93393: [clangd] Ignore the static index refs from the dynamic index files.

[Aleksandr Platonov via Phabricator via cfe-commits]

ArcsinX updated this revision to Diff 312742.
ArcsinX added a comment.

Fix format
Rebase


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93393/new/

https://reviews.llvm.org/D93393

Files:
  clang-tools-extra/clangd/index/FileIndex.cpp
  clang-tools-extra/clangd/index/Index.cpp
  clang-tools-extra/clangd/index/Index.h
  clang-tools-extra/clangd/index/MemIndex.cpp
  clang-tools-extra/clangd/index/MemIndex.h
  clang-tools-extra/clangd/index/Merge.cpp
  clang-tools-extra/clangd/index/Merge.h
  clang-tools-extra/clangd/index/ProjectAware.cpp
  clang-tools-extra/clangd/index/dex/Dex.cpp
  clang-tools-extra/clangd/index/dex/Dex.h
  clang-tools-extra/clangd/index/remote/Client.cpp
  clang-tools-extra/clangd/unittests/CodeCompleteTests.cpp
  clang-tools-extra/clangd/unittests/DexTests.cpp
  clang-tools-extra/clangd/unittests/IndexTests.cpp
  clang-tools-extra/clangd/unittests/RenameTests.cpp
  clang-tools-extra/clangd/unittests/TestFS.cpp

Index: clang-tools-extra/clangd/unittests/TestFS.cpp
===
--- clang-tools-extra/clangd/unittests/TestFS.cpp
+++ clang-tools-extra/clangd/unittests/TestFS.cpp
@@ -99,8 +99,9 @@
   llvm::Expected
   getAbsolutePath(llvm::StringRef /*Authority*/, llvm::StringRef Body,
   llvm::StringRef HintPath) const override {
-if (!HintPath.startswith(testRoot()))
-  return error("Hint path doesn't start with test root: {0}", HintPath);
+if (!HintPath.empty() && !HintPath.startswith(testRoot()))
+  return error("Hint path is not empty and doesn't start with {0}: {1}",
+   testRoot(), HintPath);
 if (!Body.consume_front("/"))
   return error("Body of an unittest: URI must start with '/'");
 llvm::SmallString<16> Path(Body.begin(), Body.end());
Index: clang-tools-extra/clangd/unittests/RenameTests.cpp
===
--- clang-tools-extra/clangd/unittests/RenameTests.cpp
+++ clang-tools-extra/clangd/unittests/RenameTests.cpp
@@ -1237,6 +1237,12 @@
 void relations(const RelationsRequest ,
llvm::function_ref
Callback) const override {}
+
+llvm::unique_function
+indexedFiles() const override {
+  return [](llvm::StringRef) { return false; };
+}
+
 size_t estimateMemoryUsage() const override { return 0; }
   } PIndex;
   Results = rename({MainCode.point(),
@@ -1285,6 +1291,12 @@
 void relations(const RelationsRequest &,
llvm::function_ref)
 const override {}
+
+llvm::unique_function
+indexedFiles() const override {
+  return [](llvm::StringRef) { return false; };
+}
+
 size_t estimateMemoryUsage() const override { return 0; }
 Ref ReturnedRef;
   } DIndex(XRefInBarCC);
Index: clang-tools-extra/clangd/unittests/IndexTests.cpp
===
--- clang-tools-extra/clangd/unittests/IndexTests.cpp
+++ clang-tools-extra/clangd/unittests/IndexTests.cpp
@@ -224,6 +224,20 @@
   EXPECT_THAT(lookup(*I, SymbolID("ns::nonono")), UnorderedElementsAre());
 }
 
+TEST(MemIndexTest, IndexedFiles) {
+  SymbolSlab Symbols;
+  RefSlab Refs;
+  auto Size = Symbols.bytes() + Refs.bytes();
+  auto Data = std::make_pair(std::move(Symbols), std::move(Refs));
+  llvm::StringSet<> Files = {testPath("foo.cc"), testPath("bar.cc")};
+  MemIndex I(std::move(Data.first), std::move(Data.second), RelationSlab(),
+ std::move(Files), std::move(Data), Size);
+  auto ContainsFile = I.indexedFiles();
+  EXPECT_TRUE(ContainsFile("unittest:///foo.cc"));
+  EXPECT_TRUE(ContainsFile("unittest:///bar.cc"));
+  EXPECT_FALSE(ContainsFile("unittest:///foobar.cc"));
+}
+
 TEST(MemIndexTest, TemplateSpecialization) {
   SymbolSlab::Builder B;
 
@@ -367,7 +381,7 @@
   Test.Code = std::string(Test1Code.code());
   Test.Filename = "test.cc";
   auto AST = Test.build();
-  Dyn.updateMain(Test.Filename, AST);
+  Dyn.updateMain(testPath(Test.Filename), AST);
 
   // Build static index for test.cc.
   Test.HeaderCode = HeaderCode;
@@ -375,7 +389,7 @@
   Test.Filename = "test.cc";
   auto StaticAST = Test.build();
   // Add stale refs for test.cc.
-  StaticIndex.updateMain(Test.Filename, StaticAST);
+  StaticIndex.updateMain(testPath(Test.Filename), StaticAST);
 
   // Add refs for test2.cc
   Annotations Test2Code(R"(class $Foo[[Foo]] {};)");
@@ -384,7 +398,7 @@
   Test2.Code = std::string(Test2Code.code());
   Test2.Filename = "test2.cc";
   StaticAST = Test2.build();
-  StaticIndex.updateMain(Test2.Filename, StaticAST);
+  StaticIndex.updateMain(testPath(Test2.Filename), StaticAST);
 
   RefsRequest Request;
   Request.IDs = {Foo.ID};
@@ -403,10 +417,47 @@
   RefSlab::Builder Results2;
   EXPECT_TRUE(
   Merge.refs(Request, [&](const Ref ) { Results2.insert(Foo.ID, O); }));
-  EXPECT_THAT(std::move(Results2).build(),
-