[PATCH] D70446: [clangd] Treat UserDefinedLiteral as a leaf in SelectionTree, sidestepping tokenization issues

[Sam McCall via Phabricator via cfe-commits]

This revision was automatically updated to reflect the committed changes.
Closed by commit rGe18ab2a0b801: [clangd] Treat UserDefinedLiteral as a leaf in 
SelectionTree, sidestepping… (authored by sammccall).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D70446/new/

https://reviews.llvm.org/D70446

Files:
  clang-tools-extra/clangd/Selection.cpp
  clang-tools-extra/clangd/unittests/SelectionTests.cpp


Index: clang-tools-extra/clangd/unittests/SelectionTests.cpp
===
--- clang-tools-extra/clangd/unittests/SelectionTests.cpp
+++ clang-tools-extra/clangd/unittests/SelectionTests.cpp
@@ -304,6 +304,16 @@
 }
   )cpp",
   "CallExpr"},
+
+  // User-defined literals are tricky: is 12_i one token or two?
+  // For now we treat it as one, and the UserDefinedLiteral as a leaf.
+  {
+  R"cpp(
+struct Foo{};
+Foo operator""_ud(unsigned long long);
+Foo x = [[^12_ud]];
+  )cpp",
+  "UserDefinedLiteral"},
   };
   for (const Case &C : Cases) {
 Annotations Test(C.Code);
Index: clang-tools-extra/clangd/Selection.cpp
===
--- clang-tools-extra/clangd/Selection.cpp
+++ clang-tools-extra/clangd/Selection.cpp
@@ -12,6 +12,7 @@
 #include "clang/AST/ASTTypeTraits.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Expr.h"
+#include "clang/AST/ExprCXX.h"
 #include "clang/AST/PrettyPrinter.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/TypeLoc.h"
@@ -245,6 +246,10 @@
 if (canSafelySkipNode(N))
   return false;
 push(std::move(N));
+if (shouldSkipChildren(X)) {
+  pop();
+  return false;
+}
 return true;
   }
   bool dataTraverseStmtPost(Stmt *X) {
@@ -355,6 +360,15 @@
 return true;
   }
 
+  // There are certain nodes we want to treat as leaves in the SelectionTree,
+  // although they do have children.
+  bool shouldSkipChildren(const Stmt *X) const {
+// UserDefinedLiteral (e.g. 12_i) has two children (12 and _i).
+// Unfortunately TokenBuffer sees 12_i as one token and can't split it.
+// So we treat UserDefinedLiteral as a leaf node, owning the token.
+return llvm::isa(X);
+  }
+
   // Pushes a node onto the ancestor stack. Pairs with pop().
   // Performs early hit detection for some nodes (on the earlySourceRange).
   void push(DynTypedNode Node) {


Index: clang-tools-extra/clangd/unittests/SelectionTests.cpp
===
--- clang-tools-extra/clangd/unittests/SelectionTests.cpp
+++ clang-tools-extra/clangd/unittests/SelectionTests.cpp
@@ -304,6 +304,16 @@
 }
   )cpp",
   "CallExpr"},
+
+  // User-defined literals are tricky: is 12_i one token or two?
+  // For now we treat it as one, and the UserDefinedLiteral as a leaf.
+  {
+  R"cpp(
+struct Foo{};
+Foo operator""_ud(unsigned long long);
+Foo x = [[^12_ud]];
+  )cpp",
+  "UserDefinedLiteral"},
   };
   for (const Case &C : Cases) {
 Annotations Test(C.Code);
Index: clang-tools-extra/clangd/Selection.cpp
===
--- clang-tools-extra/clangd/Selection.cpp
+++ clang-tools-extra/clangd/Selection.cpp
@@ -12,6 +12,7 @@
 #include "clang/AST/ASTTypeTraits.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Expr.h"
+#include "clang/AST/ExprCXX.h"
 #include "clang/AST/PrettyPrinter.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/TypeLoc.h"
@@ -245,6 +246,10 @@
 if (canSafelySkipNode(N))
   return false;
 push(std::move(N));
+if (shouldSkipChildren(X)) {
+  pop();
+  return false;
+}
 return true;
   }
   bool dataTraverseStmtPost(Stmt *X) {
@@ -355,6 +360,15 @@
 return true;
   }
 
+  // There are certain nodes we want to treat as leaves in the SelectionTree,
+  // although they do have children.
+  bool shouldSkipChildren(const Stmt *X) const {
+// UserDefinedLiteral (e.g. 12_i) has two children (12 and _i).
+// Unfortunately TokenBuffer sees 12_i as one token and can't split it.
+// So we treat UserDefinedLiteral as a leaf node, owning the token.
+return llvm::isa(X);
+  }
+
   // Pushes a node onto the ancestor stack. Pairs with pop().
   // Performs early hit detection for some nodes (on the earlySourceRange).
   void push(DynTypedNode Node) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D70446: [clangd] Treat UserDefinedLiteral as a leaf in SelectionTree, sidestepping tokenization issues

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet accepted this revision.
kadircet added a comment.
This revision is now accepted and ready to land.

LGTM


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D70446/new/

https://reviews.llvm.org/D70446



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D70446: [clangd] Treat UserDefinedLiteral as a leaf in SelectionTree, sidestepping tokenization issues

[pre-merge checks [bot] via Phabricator via cfe-commits]

merge_guards_bot added a comment.

Build result: pass - 60179 tests passed, 0 failed and 732 were skipped.
Log files: console-log.txt 
,
 CMakeCache.txt 



Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D70446/new/

https://reviews.llvm.org/D70446



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D70446: [clangd] Treat UserDefinedLiteral as a leaf in SelectionTree, sidestepping tokenization issues

[Sam McCall via Phabricator via cfe-commits]

sammccall created this revision.
sammccall added a reviewer: kadircet.
Herald added subscribers: cfe-commits, usaxena95, arphaman, jkorous, MaskRay, 
ilya-biryukov.
Herald added a project: clang.

Fixes https://github.com/clangd/clangd/issues/203


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D70446

Files:
  clang-tools-extra/clangd/Selection.cpp
  clang-tools-extra/clangd/unittests/SelectionTests.cpp


Index: clang-tools-extra/clangd/unittests/SelectionTests.cpp
===
--- clang-tools-extra/clangd/unittests/SelectionTests.cpp
+++ clang-tools-extra/clangd/unittests/SelectionTests.cpp
@@ -304,6 +304,16 @@
 }
   )cpp",
   "CallExpr"},
+
+  // User-defined literals are tricky: is 12_i one token or two?
+  // For now we treat it as one, and the UserDefinedLiteral as a leaf.
+  {
+  R"cpp(
+struct Foo{};
+Foo operator""_ud(unsigned long long);
+Foo x = [[^12_ud]];
+  )cpp",
+  "UserDefinedLiteral"},
   };
   for (const Case &C : Cases) {
 Annotations Test(C.Code);
Index: clang-tools-extra/clangd/Selection.cpp
===
--- clang-tools-extra/clangd/Selection.cpp
+++ clang-tools-extra/clangd/Selection.cpp
@@ -12,6 +12,7 @@
 #include "clang/AST/ASTTypeTraits.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Expr.h"
+#include "clang/AST/ExprCXX.h"
 #include "clang/AST/PrettyPrinter.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/TypeLoc.h"
@@ -245,6 +246,10 @@
 if (canSafelySkipNode(N))
   return false;
 push(std::move(N));
+if (shouldSkipChildren(X)) {
+  pop();
+  return false;
+}
 return true;
   }
   bool dataTraverseStmtPost(Stmt *X) {
@@ -355,6 +360,15 @@
 return true;
   }
 
+  // There are certain nodes we want to treat as leaves in the SelectionTree,
+  // although they do have children.
+  bool shouldSkipChildren(const Stmt *X) const {
+// UserDefinedLiteral (e.g. 12_i) has two children (12 and _i).
+// Unfortunately TokenBuffer sees 12_i as one token and can't split it.
+// So we treat UserDefinedLiteral as a leaf node, owning the token.
+return llvm::isa(X);
+  }
+
   // Pushes a node onto the ancestor stack. Pairs with pop().
   // Performs early hit detection for some nodes (on the earlySourceRange).
   void push(DynTypedNode Node) {


Index: clang-tools-extra/clangd/unittests/SelectionTests.cpp
===
--- clang-tools-extra/clangd/unittests/SelectionTests.cpp
+++ clang-tools-extra/clangd/unittests/SelectionTests.cpp
@@ -304,6 +304,16 @@
 }
   )cpp",
   "CallExpr"},
+
+  // User-defined literals are tricky: is 12_i one token or two?
+  // For now we treat it as one, and the UserDefinedLiteral as a leaf.
+  {
+  R"cpp(
+struct Foo{};
+Foo operator""_ud(unsigned long long);
+Foo x = [[^12_ud]];
+  )cpp",
+  "UserDefinedLiteral"},
   };
   for (const Case &C : Cases) {
 Annotations Test(C.Code);
Index: clang-tools-extra/clangd/Selection.cpp
===
--- clang-tools-extra/clangd/Selection.cpp
+++ clang-tools-extra/clangd/Selection.cpp
@@ -12,6 +12,7 @@
 #include "clang/AST/ASTTypeTraits.h"
 #include "clang/AST/DeclCXX.h"
 #include "clang/AST/Expr.h"
+#include "clang/AST/ExprCXX.h"
 #include "clang/AST/PrettyPrinter.h"
 #include "clang/AST/RecursiveASTVisitor.h"
 #include "clang/AST/TypeLoc.h"
@@ -245,6 +246,10 @@
 if (canSafelySkipNode(N))
   return false;
 push(std::move(N));
+if (shouldSkipChildren(X)) {
+  pop();
+  return false;
+}
 return true;
   }
   bool dataTraverseStmtPost(Stmt *X) {
@@ -355,6 +360,15 @@
 return true;
   }
 
+  // There are certain nodes we want to treat as leaves in the SelectionTree,
+  // although they do have children.
+  bool shouldSkipChildren(const Stmt *X) const {
+// UserDefinedLiteral (e.g. 12_i) has two children (12 and _i).
+// Unfortunately TokenBuffer sees 12_i as one token and can't split it.
+// So we treat UserDefinedLiteral as a leaf node, owning the token.
+return llvm::isa(X);
+  }
+
   // Pushes a node onto the ancestor stack. Pairs with pop().
   // Performs early hit detection for some nodes (on the earlySourceRange).
   void push(DynTypedNode Node) {
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits