[PATCH] D157990: [clangd] Add --query-driver flag to clangd-indexer

2023-08-24 Thread Nathan Ridge via Phabricator via cfe-commits
nridge added a comment.

Committed, thanks!


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D157990/new/

https://reviews.llvm.org/D157990

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[PATCH] D157990: [clangd] Add --query-driver flag to clangd-indexer

2023-08-24 Thread Nathan Ridge via Phabricator via cfe-commits
This revision was automatically updated to reflect the committed changes.
Closed by commit rG109bc024c8d7: [clangd] Add --query-driver flag to 
clangd-indexer (authored by nridge).

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D157990/new/

https://reviews.llvm.org/D157990

Files:
  clang-tools-extra/clangd/indexer/IndexerMain.cpp


Index: clang-tools-extra/clangd/indexer/IndexerMain.cpp
===
--- clang-tools-extra/clangd/indexer/IndexerMain.cpp
+++ clang-tools-extra/clangd/indexer/IndexerMain.cpp
@@ -38,6 +38,16 @@
"binary RIFF format")),
llvm::cl::init(IndexFileFormat::RIFF));
 
+static llvm::cl::list QueryDriverGlobs{
+"query-driver",
+llvm::cl::desc(
+"Comma separated list of globs for white-listing gcc-compatible "
+"drivers that are safe to execute. Drivers matching any of these globs 
"
+"will be used to extract system includes. e.g. "
+"/usr/bin/**/clang-*,/path/to/repo/**/g++-*"),
+llvm::cl::CommaSeparated,
+};
+
 class IndexActionFactory : public tooling::FrontendActionFactory {
 public:
   IndexActionFactory(IndexFileIn ) : Result(Result) {}
@@ -144,12 +154,16 @@
 
   // Collect symbols found in each translation unit, merging as we go.
   clang::clangd::IndexFileIn Data;
+  auto Mangler = std::make_shared(
+  clang::clangd::CommandMangler::detect());
+  Mangler->SystemIncludeExtractor = clang::clangd::getSystemIncludeExtractor(
+  static_cast>(
+  clang::clangd::QueryDriverGlobs));
   auto Err = Executor->get()->execute(
   std::make_unique(Data),
   clang::tooling::ArgumentsAdjuster(
-  [Mangler = std::make_shared(
-   clang::clangd::CommandMangler::detect())](
-  const std::vector , llvm::StringRef File) {
+  [Mangler = std::move(Mangler)](const std::vector ,
+ llvm::StringRef File) {
 clang::tooling::CompileCommand Cmd;
 Cmd.CommandLine = Args;
 Mangler->operator()(Cmd, File);


Index: clang-tools-extra/clangd/indexer/IndexerMain.cpp
===
--- clang-tools-extra/clangd/indexer/IndexerMain.cpp
+++ clang-tools-extra/clangd/indexer/IndexerMain.cpp
@@ -38,6 +38,16 @@
"binary RIFF format")),
llvm::cl::init(IndexFileFormat::RIFF));
 
+static llvm::cl::list QueryDriverGlobs{
+"query-driver",
+llvm::cl::desc(
+"Comma separated list of globs for white-listing gcc-compatible "
+"drivers that are safe to execute. Drivers matching any of these globs "
+"will be used to extract system includes. e.g. "
+"/usr/bin/**/clang-*,/path/to/repo/**/g++-*"),
+llvm::cl::CommaSeparated,
+};
+
 class IndexActionFactory : public tooling::FrontendActionFactory {
 public:
   IndexActionFactory(IndexFileIn ) : Result(Result) {}
@@ -144,12 +154,16 @@
 
   // Collect symbols found in each translation unit, merging as we go.
   clang::clangd::IndexFileIn Data;
+  auto Mangler = std::make_shared(
+  clang::clangd::CommandMangler::detect());
+  Mangler->SystemIncludeExtractor = clang::clangd::getSystemIncludeExtractor(
+  static_cast>(
+  clang::clangd::QueryDriverGlobs));
   auto Err = Executor->get()->execute(
   std::make_unique(Data),
   clang::tooling::ArgumentsAdjuster(
-  [Mangler = std::make_shared(
-   clang::clangd::CommandMangler::detect())](
-  const std::vector , llvm::StringRef File) {
+  [Mangler = std::move(Mangler)](const std::vector ,
+ llvm::StringRef File) {
 clang::tooling::CompileCommand Cmd;
 Cmd.CommandLine = Args;
 Mangler->operator()(Cmd, File);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[PATCH] D157990: [clangd] Add --query-driver flag to clangd-indexer

2023-08-23 Thread Alex Cameron via Phabricator via cfe-commits
tetsuo-cpp added a comment.

Thanks for the review @nridge! I'd really appreciate if you could help me with 
the commit (I don't have commit access).


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D157990/new/

https://reviews.llvm.org/D157990

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[PATCH] D157990: [clangd] Add --query-driver flag to clangd-indexer

2023-08-20 Thread Nathan Ridge via Phabricator via cfe-commits
nridge accepted this revision.
nridge added a comment.
This revision is now accepted and ready to land.

Thanks for the patch!

My thoughts are:

- As mentioned in the issue, I think this fills a logical gap: clangd-indexer 
is an alternative way of generating a project index to clangd's background 
indexer, so if clangd has a configuration feature needed for some projects 
(such as `--query-driver`), clangd-indexer should have an equivalent ability. 
(Processing `.clangd` config files is another such gap that I think would make 
sense to fill in the future.)
- The implementation is straightforward. I don't think it's interesting enough 
to warrant spending time writing an automated test for this (especially given 
that our existing `system-include-extractor.test` is not particularly easy / 
pleasant to work with.)

I would suggest waiting a few days to see if Sam has any input, otherwise I 
think this should be good to merge. Let me know if you need me to commit it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D157990/new/

https://reviews.llvm.org/D157990

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits


[PATCH] D157990: [clangd] Add --query-driver flag to clangd-indexer

2023-08-15 Thread Alex Cameron via Phabricator via cfe-commits
tetsuo-cpp created this revision.
tetsuo-cpp added reviewers: sammccall, nridge.
tetsuo-cpp added projects: clang-tools-extra, All.
Herald added subscribers: kadircet, arphaman.
tetsuo-cpp requested review of this revision.
Herald added subscribers: MaskRay, ilya-biryukov.

When using `clangd` for cross-compiled projects, it's necessary to use the 
`--query-driver` flag so that `clangd` can extract the compiler's built-in 
header paths. However, there's no such flag for `clangd-indexer` so we're 
unable to build a working static index for these projects.

This patch adds a `--query-driver` flag to `clangd-indexer` for this scenario.

I saw some tests under `clang-tools-extra/clangd/test/` but I think the 
cross-compilation case is a bit more complex to test. Let me know if you'd like 
me to look into this further.

Resolves: https://github.com/clangd/clangd/issues/1717


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D157990

Files:
  clang-tools-extra/clangd/indexer/IndexerMain.cpp


Index: clang-tools-extra/clangd/indexer/IndexerMain.cpp
===
--- clang-tools-extra/clangd/indexer/IndexerMain.cpp
+++ clang-tools-extra/clangd/indexer/IndexerMain.cpp
@@ -38,6 +38,16 @@
"binary RIFF format")),
llvm::cl::init(IndexFileFormat::RIFF));
 
+static llvm::cl::list QueryDriverGlobs{
+"query-driver",
+llvm::cl::desc(
+"Comma separated list of globs for white-listing gcc-compatible "
+"drivers that are safe to execute. Drivers matching any of these globs 
"
+"will be used to extract system includes. e.g. "
+"/usr/bin/**/clang-*,/path/to/repo/**/g++-*"),
+llvm::cl::CommaSeparated,
+};
+
 class IndexActionFactory : public tooling::FrontendActionFactory {
 public:
   IndexActionFactory(IndexFileIn ) : Result(Result) {}
@@ -144,12 +154,16 @@
 
   // Collect symbols found in each translation unit, merging as we go.
   clang::clangd::IndexFileIn Data;
+  auto Mangler = std::make_shared(
+  clang::clangd::CommandMangler::detect());
+  Mangler->SystemIncludeExtractor = clang::clangd::getSystemIncludeExtractor(
+  static_cast>(
+  clang::clangd::QueryDriverGlobs));
   auto Err = Executor->get()->execute(
   std::make_unique(Data),
   clang::tooling::ArgumentsAdjuster(
-  [Mangler = std::make_shared(
-   clang::clangd::CommandMangler::detect())](
-  const std::vector , llvm::StringRef File) {
+  [Mangler = std::move(Mangler)](const std::vector ,
+ llvm::StringRef File) {
 clang::tooling::CompileCommand Cmd;
 Cmd.CommandLine = Args;
 Mangler->operator()(Cmd, File);


Index: clang-tools-extra/clangd/indexer/IndexerMain.cpp
===
--- clang-tools-extra/clangd/indexer/IndexerMain.cpp
+++ clang-tools-extra/clangd/indexer/IndexerMain.cpp
@@ -38,6 +38,16 @@
"binary RIFF format")),
llvm::cl::init(IndexFileFormat::RIFF));
 
+static llvm::cl::list QueryDriverGlobs{
+"query-driver",
+llvm::cl::desc(
+"Comma separated list of globs for white-listing gcc-compatible "
+"drivers that are safe to execute. Drivers matching any of these globs "
+"will be used to extract system includes. e.g. "
+"/usr/bin/**/clang-*,/path/to/repo/**/g++-*"),
+llvm::cl::CommaSeparated,
+};
+
 class IndexActionFactory : public tooling::FrontendActionFactory {
 public:
   IndexActionFactory(IndexFileIn ) : Result(Result) {}
@@ -144,12 +154,16 @@
 
   // Collect symbols found in each translation unit, merging as we go.
   clang::clangd::IndexFileIn Data;
+  auto Mangler = std::make_shared(
+  clang::clangd::CommandMangler::detect());
+  Mangler->SystemIncludeExtractor = clang::clangd::getSystemIncludeExtractor(
+  static_cast>(
+  clang::clangd::QueryDriverGlobs));
   auto Err = Executor->get()->execute(
   std::make_unique(Data),
   clang::tooling::ArgumentsAdjuster(
-  [Mangler = std::make_shared(
-   clang::clangd::CommandMangler::detect())](
-  const std::vector , llvm::StringRef File) {
+  [Mangler = std::move(Mangler)](const std::vector ,
+ llvm::StringRef File) {
 clang::tooling::CompileCommand Cmd;
 Cmd.CommandLine = Args;
 Mangler->operator()(Cmd, File);
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits