from:"Artem Chikin via cfe\-commits"

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-06-03 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/93588

>From f69fd6e79ca7232672b4b3759253e818e8b2cec7 Mon Sep 17 00:00:00 2001
From: artemcm 
Date: Fri, 24 May 2024 15:13:45 -0700
Subject: [PATCH] [clang][deps] Collect discovered module dependencies' Link
 Libraries

This will allow scanner clients to be able to compute e.g. auto-linking 
dependencies of the scanned translation unit.
---
 .../DependencyScanning/ModuleDepCollector.h   |   5 +
 .../DependencyScanning/ModuleDepCollector.cpp |   4 +
 clang/test/ClangScanDeps/diagnostics.c|   1 +
 .../header-search-pruning-transitive.c|   4 +
 .../ClangScanDeps/header-search-pruning.cpp   |   3 +
 clang/test/ClangScanDeps/link-libraries.c | 121 ++
 .../modules-canononical-module-map-case.c |   6 +
 .../test/ClangScanDeps/modules-context-hash.c |   2 +
 clang/test/ClangScanDeps/modules-dep-args.c   |   2 +
 .../ClangScanDeps/modules-excluded-header.m   |   1 +
 .../ClangScanDeps/modules-extern-submodule.c  |   3 +
 .../ClangScanDeps/modules-extern-unrelated.m  |   4 +
 .../modules-file-path-isolation.c |   1 +
 .../modules-fmodule-name-no-module-built.m|   1 +
 .../ClangScanDeps/modules-full-by-mod-name.c  |   3 +
 clang/test/ClangScanDeps/modules-full.cpp |   3 +
 .../modules-implementation-private.m  |   6 +
 .../modules-implicit-dot-private.m|  12 ++
 .../modules-incomplete-umbrella.c |  25 
 clang/test/ClangScanDeps/modules-inferred.m   |   6 +
 .../modules-no-undeclared-includes.c  |   1 +
 .../modules-pch-common-submodule.c|   2 +
 .../modules-pch-common-via-submodule.c|   2 +
 clang/test/ClangScanDeps/modules-pch.c|   5 +
 .../ClangScanDeps/modules-priv-fw-from-pub.m  |  13 ++
 .../test/ClangScanDeps/modules-redefinition.m |   1 +
 .../ClangScanDeps/modules-symlink-dir-vfs.c   |   6 +
 clang/test/ClangScanDeps/modules-transitive.c |   1 +
 .../optimize-canonicalize-macros.m|   2 +
 .../test/ClangScanDeps/optimize-fmodulemap.m  |   2 +
 .../ClangScanDeps/optimize-system-warnings.m  |   3 +
 clang/test/ClangScanDeps/optimize-vfs-leak.m  |   5 +-
 clang/test/ClangScanDeps/optimize-vfs.m   |   3 +
 clang/test/ClangScanDeps/removed-args.c   |   2 +
 clang/test/ClangScanDeps/working-dir.m|   4 +
 clang/tools/clang-scan-deps/ClangScanDeps.cpp |  29 +++--
 36 files changed, 285 insertions(+), 9 deletions(-)
 create mode 100644 clang/test/ClangScanDeps/link-libraries.c

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h 
b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
index da51292296a90..3de19d756da00 100644
--- a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
@@ -10,6 +10,7 @@
 #define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H
 
 #include "clang/Basic/LLVM.h"
+#include "clang/Basic/Module.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/Utils.h"
@@ -138,6 +139,10 @@ struct ModuleDeps {
   /// determined that the differences are benign for this compilation.
   std::vector ClangModuleDeps;
 
+  /// The set of libraries or frameworks to link against when
+  /// an entity from this module is used.
+  llvm::SmallVector LinkLibraries;
+
   /// Get (or compute) the compiler invocation that can be used to build this
   /// module. Does not include argv[0].
   const std::vector &getBuildArguments();
diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp 
b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
index f46324ee9989e..370d834846859 100644
--- a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
+++ b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
@@ -578,6 +578,10 @@ ModuleDepCollectorPP::handleTopLevelModule(const Module 
*M) {
 
   MD.ID.ModuleName = M->getFullModuleName();
   MD.IsSystem = M->IsSystem;
+  // For modules which use export_as link name, the linked product that of the
+  // corresponding export_as-named module.
+  if (!M->UseExportAsModuleLinkName)
+MD.LinkLibraries = M->LinkLibraries;
 
   ModuleMap &ModMapInfo =
   MDC.ScanInstance.getPreprocessor().getHeaderSearchInfo().getModuleMap();
diff --git a/clang/test/ClangScanDeps/diagnostics.c 
b/clang/test/ClangScanDeps/diagnostics.c
index 42aeca16cddc4..bc05594f7828d 100644
--- a/clang/test/ClangScanDeps/diagnostics.c
+++ b/clang/test/ClangScanDeps/diagnostics.c
@@ -37,6 +37,7 @@ module mod { header "mod.h" }
 // CHECK-NEXT: "[[PREFIX]]/mod.h"
 // CHECK-NEXT: "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:   ],
+// CHECK-NEXT:   "link-libraries": [],
 // CHECK-NEXT:   "name": "mod"
 // CHECK-NEXT: }
 // CHECK-NEXT:   ],
diff --git a/clang/test/ClangScanDeps/header-search-pruning-transitive.c 
b/cl

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-06-03 Thread Artem Chikin via cfe-commits


https://github.com/artemcm edited 
https://github.com/llvm/llvm-project/pull/93588
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-06-03 Thread Artem Chikin via cfe-commits



@@ -138,6 +139,14 @@ struct ModuleDeps {
   /// determined that the differences are benign for this compilation.
   std::vector ClangModuleDeps;
 
+  /// The set of libraries or frameworks to link against when
+  /// an entity from this module is used.
+  llvm::SmallVector LinkLibraries;
+
+  /// Autolinking uses the framework name for linking purposes
+  /// when this is false and the export_as name otherwise.
+  bool UseExportAsModuleLinkName;

artemcm wrote:

It seems that `LinkLibraries` are still collected for a `Module` separately 
just in case it gets discovered/imported by a module matching a `export_as` 
name. So I can see why a separate flag was introduced. 

The logic for adding linker flags in CodeGen omits these modules if this flag 
(`UseExportAsModuleLinkName`) is set, and that matching the intent behind this 
change here, I think it would make sense to model the same in the scanner. i.e. 
omit `LinkLibraries` of such module, because by the time we've discovered all 
the dependencies we are guaranteed to have the required knowledge to make this 
decision. 

https://github.com/llvm/llvm-project/pull/93588
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-06-03 Thread Artem Chikin via cfe-commits

@@ -138,6 +139,14 @@ struct ModuleDeps {
   /// determined that the differences are benign for this compilation.
   std::vector ClangModuleDeps;

+  /// The set of libraries or frameworks to link against when
+  /// an entity from this module is used.
+  llvm::SmallVector LinkLibraries;
+
+  /// Autolinking uses the framework name for linking purposes
+  /// when this is false and the export_as name otherwise.
+  bool UseExportAsModuleLinkName;

artemcm wrote:

>If the idea is to drop the autolink of the current module, should we rework 
>that so that we remove it from the link libraries of the importer instead?

Yeah, that's the thinking behind adding it here - the scanner is aggregating a 
set of link libraries for the root importer, and an assumption is made that a 
module with export_as link name set will be covered by a `LinkLibrary` of one 
of its dependents. e.g. `SomeKit` depends on `SomeKitCore` and the latter sets 
`export_as SomeKit`. And the scanner's client may only see `ModuleDeps`, and 
not this info, so it can use this bit to know to avoid processing its 
`LinkLibraries`.

>From what I can tell the above scenario matches the circumstance where this 
>bit is set on a `Module` -> when a module with the `export_as` name has 
>already been found. 

Given this, I do not really see a reason to report `LinkLibraries` for such 
modules at all, what do you think that instead of having this bit here we 
instead just use it in the scanner to know to avoid populating `LinkLibraries` 
added here? 

https://github.com/llvm/llvm-project/pull/93588
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-05-28 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/93588

>From 034f567024c6766f68dcc73ddad9b0fe09b05baf Mon Sep 17 00:00:00 2001
From: artemcm 
Date: Fri, 24 May 2024 15:13:45 -0700
Subject: [PATCH] [clang][deps] Collect discovered module dependencies' Link
 Libraries

This will allow scanner clients to be able to compute e.g. auto-linking 
dependencies of the scanned translation unit.
---
 .../DependencyScanning/ModuleDepCollector.h   |   9 ++
 .../DependencyScanning/ModuleDepCollector.cpp |   2 +
 clang/test/ClangScanDeps/diagnostics.c|   1 +
 .../header-search-pruning-transitive.c|   4 +
 .../ClangScanDeps/header-search-pruning.cpp   |   3 +
 clang/test/ClangScanDeps/link-libraries.c | 121 ++
 .../modules-canononical-module-map-case.c |   6 +
 .../test/ClangScanDeps/modules-context-hash.c |   2 +
 clang/test/ClangScanDeps/modules-dep-args.c   |   2 +
 .../ClangScanDeps/modules-excluded-header.m   |   1 +
 .../ClangScanDeps/modules-extern-submodule.c  |   3 +
 .../ClangScanDeps/modules-extern-unrelated.m  |   4 +
 .../modules-file-path-isolation.c |   1 +
 .../modules-fmodule-name-no-module-built.m|   1 +
 .../ClangScanDeps/modules-full-by-mod-name.c  |   3 +
 clang/test/ClangScanDeps/modules-full.cpp |   3 +
 .../modules-implementation-private.m  |   6 +
 .../modules-implicit-dot-private.m|  12 ++
 .../modules-incomplete-umbrella.c |  25 
 clang/test/ClangScanDeps/modules-inferred.m   |   6 +
 .../modules-no-undeclared-includes.c  |   1 +
 .../modules-pch-common-submodule.c|   2 +
 .../modules-pch-common-via-submodule.c|   2 +
 clang/test/ClangScanDeps/modules-pch.c|   5 +
 .../ClangScanDeps/modules-priv-fw-from-pub.m  |  13 ++
 .../test/ClangScanDeps/modules-redefinition.m |   1 +
 .../ClangScanDeps/modules-symlink-dir-vfs.c   |   6 +
 clang/test/ClangScanDeps/modules-transitive.c |   1 +
 .../optimize-canonicalize-macros.m|   2 +
 .../test/ClangScanDeps/optimize-fmodulemap.m  |   2 +
 .../ClangScanDeps/optimize-system-warnings.m  |   3 +
 clang/test/ClangScanDeps/optimize-vfs-leak.m  |   5 +-
 clang/test/ClangScanDeps/optimize-vfs.m   |   3 +
 clang/test/ClangScanDeps/removed-args.c   |   2 +
 clang/test/ClangScanDeps/working-dir.m|   4 +
 clang/tools/clang-scan-deps/ClangScanDeps.cpp |  29 +++--
 36 files changed, 287 insertions(+), 9 deletions(-)
 create mode 100644 clang/test/ClangScanDeps/link-libraries.c

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h 
b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
index da51292296a90..6c5ddef46e24d 100644
--- a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
@@ -10,6 +10,7 @@
 #define LLVM_CLANG_TOOLING_DEPENDENCYSCANNING_MODULEDEPCOLLECTOR_H
 
 #include "clang/Basic/LLVM.h"
+#include "clang/Basic/Module.h"
 #include "clang/Basic/SourceManager.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/Utils.h"
@@ -138,6 +139,14 @@ struct ModuleDeps {
   /// determined that the differences are benign for this compilation.
   std::vector ClangModuleDeps;
 
+  /// The set of libraries or frameworks to link against when
+  /// an entity from this module is used.
+  llvm::SmallVector LinkLibraries;
+
+  /// Autolinking uses the framework name for linking purposes
+  /// when this is false and the export_as name otherwise.
+  bool UseExportAsModuleLinkName;
+
   /// Get (or compute) the compiler invocation that can be used to build this
   /// module. Does not include argv[0].
   const std::vector &getBuildArguments();
diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp 
b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
index f46324ee9989e..6b2ea1a67b96b 100644
--- a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
+++ b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
@@ -578,6 +578,8 @@ ModuleDepCollectorPP::handleTopLevelModule(const Module *M) 
{
 
   MD.ID.ModuleName = M->getFullModuleName();
   MD.IsSystem = M->IsSystem;
+  MD.LinkLibraries = M->LinkLibraries;
+  MD.UseExportAsModuleLinkName = M->UseExportAsModuleLinkName;
 
   ModuleMap &ModMapInfo =
   MDC.ScanInstance.getPreprocessor().getHeaderSearchInfo().getModuleMap();
diff --git a/clang/test/ClangScanDeps/diagnostics.c 
b/clang/test/ClangScanDeps/diagnostics.c
index 42aeca16cddc4..bc05594f7828d 100644
--- a/clang/test/ClangScanDeps/diagnostics.c
+++ b/clang/test/ClangScanDeps/diagnostics.c
@@ -37,6 +37,7 @@ module mod { header "mod.h" }
 // CHECK-NEXT: "[[PREFIX]]/mod.h"
 // CHECK-NEXT: "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:   ],
+// CHECK-NEXT:   "link-libraries": [],
 // CHECK-NEXT:   "name": "mod"
 // CHECK-NEXT: }
 // CHECK-NEXT:   ],
diff --git a/clan

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

2024-05-28 Thread Artem Chikin via cfe-commits


https://github.com/artemcm created 
https://github.com/llvm/llvm-project/pull/93588

This will allow scanner clients to be able to compute e.g. auto-linking 
dependencies of the scanned translation unit.

>From 763e8e1ba75dd8122eba2ebcab77c45fc4dcb7c0 Mon Sep 17 00:00:00 2001
From: artemcm 
Date: Fri, 24 May 2024 15:13:45 -0700
Subject: [PATCH] [clang][deps] Collect discovered module dependencies' Link
 Libraries

This will allow scanner clients to be able to compute e.g. auto-linking 
dependencies of the scanned translation unit.
---
 .../DependencyScanning/ModuleDepCollector.h   |   9 ++
 .../DependencyScanning/ModuleDepCollector.cpp |   2 +
 clang/test/ClangScanDeps/diagnostics.c|   1 +
 .../header-search-pruning-transitive.c|   4 +
 .../ClangScanDeps/header-search-pruning.cpp   |   3 +
 clang/test/ClangScanDeps/link-libraries.c | 121 ++
 .../modules-canononical-module-map-case.c |   6 +
 .../test/ClangScanDeps/modules-context-hash.c |   2 +
 clang/test/ClangScanDeps/modules-dep-args.c   |   2 +
 .../ClangScanDeps/modules-excluded-header.m   |   1 +
 .../ClangScanDeps/modules-extern-submodule.c  |   3 +
 .../ClangScanDeps/modules-extern-unrelated.m  |   4 +
 .../modules-file-path-isolation.c |   1 +
 .../modules-fmodule-name-no-module-built.m|   1 +
 .../ClangScanDeps/modules-full-by-mod-name.c  |   3 +
 clang/test/ClangScanDeps/modules-full.cpp |   3 +
 .../modules-implementation-private.m  |   6 +
 .../modules-implicit-dot-private.m|  12 ++
 .../modules-incomplete-umbrella.c |  25 
 clang/test/ClangScanDeps/modules-inferred.m   |   6 +
 .../modules-no-undeclared-includes.c  |   1 +
 .../modules-pch-common-submodule.c|   2 +
 .../modules-pch-common-via-submodule.c|   2 +
 clang/test/ClangScanDeps/modules-pch.c|   5 +
 .../ClangScanDeps/modules-priv-fw-from-pub.m  |  13 ++
 .../test/ClangScanDeps/modules-redefinition.m |   1 +
 .../ClangScanDeps/modules-symlink-dir-vfs.c   |   6 +
 clang/test/ClangScanDeps/modules-transitive.c |   1 +
 .../optimize-canonicalize-macros.m|   2 +
 .../test/ClangScanDeps/optimize-fmodulemap.m  |   2 +
 .../ClangScanDeps/optimize-system-warnings.m  |   3 +
 clang/test/ClangScanDeps/optimize-vfs-leak.m  |   5 +-
 clang/test/ClangScanDeps/optimize-vfs.m   |   3 +
 clang/test/ClangScanDeps/removed-args.c   |   2 +
 clang/test/ClangScanDeps/working-dir.m|   4 +
 clang/tools/clang-scan-deps/ClangScanDeps.cpp |  15 +++
 36 files changed, 281 insertions(+), 1 deletion(-)
 create mode 100644 clang/test/ClangScanDeps/link-libraries.c

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h 
b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
index da51292296a90..5afb373b3462e 100644
--- a/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
+++ b/clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
@@ -11,6 +11,7 @@
 
 #include "clang/Basic/LLVM.h"
 #include "clang/Basic/SourceManager.h"
+#include "clang/Basic/Module.h"
 #include "clang/Frontend/CompilerInvocation.h"
 #include "clang/Frontend/Utils.h"
 #include "clang/Lex/HeaderSearch.h"
@@ -138,6 +139,14 @@ struct ModuleDeps {
   /// determined that the differences are benign for this compilation.
   std::vector ClangModuleDeps;
 
+  /// The set of libraries or frameworks to link against when
+  /// an entity from this module is used.
+  llvm::SmallVector LinkLibraries;
+
+  /// Autolinking uses the framework name for linking purposes
+  /// when this is false and the export_as name otherwise.
+  bool UseExportAsModuleLinkName;
+
   /// Get (or compute) the compiler invocation that can be used to build this
   /// module. Does not include argv[0].
   const std::vector &getBuildArguments();
diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp 
b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
index f46324ee9989e..6b2ea1a67b96b 100644
--- a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
+++ b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
@@ -578,6 +578,8 @@ ModuleDepCollectorPP::handleTopLevelModule(const Module *M) 
{
 
   MD.ID.ModuleName = M->getFullModuleName();
   MD.IsSystem = M->IsSystem;
+  MD.LinkLibraries = M->LinkLibraries;
+  MD.UseExportAsModuleLinkName = M->UseExportAsModuleLinkName;
 
   ModuleMap &ModMapInfo =
   MDC.ScanInstance.getPreprocessor().getHeaderSearchInfo().getModuleMap();
diff --git a/clang/test/ClangScanDeps/diagnostics.c 
b/clang/test/ClangScanDeps/diagnostics.c
index 42aeca16cddc4..bc05594f7828d 100644
--- a/clang/test/ClangScanDeps/diagnostics.c
+++ b/clang/test/ClangScanDeps/diagnostics.c
@@ -37,6 +37,7 @@ module mod { header "mod.h" }
 // CHECK-NEXT: "[[PREFIX]]/mod.h"
 // CHECK-NEXT: "[[PREFIX]]/module.modulemap"
 // CHECK-NEXT:   ],
+// CHECK-NEXT:   "link-libraries": [],
 // CHECK-

[clang] [NFC] Parameterize Initialization of `clang::CodeGenerator` on a `TargetInfo` instance which may differ from the one in the `ASTContext` (PR #88977)

2024-04-16 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88977

>From 04e826688a504e141f3407567616bcf5cce9e2cc Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Wed, 20 Dec 2023 10:56:42 -0800
Subject: [PATCH] [NFC] Parameterize Initialization of 'clang::CodeGenerator'
 on a TargetInfo instance which may differ from the one in the ASTContext

As per https://github.com/apple/swift/pull/65930, Swift compiler's built-in 
Clang instance may require to perform type-checking against one OS version and 
compilation/code-generation against an earlier version. This change allows 
Swift to configure it's built-in Clang code-generator with a custom 
'TargetInfo'.
---
 clang/include/clang/AST/ASTConsumer.h |  5 
 clang/lib/CodeGen/CodeGenModule.cpp   | 29 +--
 clang/lib/CodeGen/CodeGenModule.h |  3 +-
 clang/lib/CodeGen/ModuleBuilder.cpp   | 21 +-
 .../ObjectFilePCHContainerOperations.cpp  |  5 ++--
 5 files changed, 36 insertions(+), 27 deletions(-)

diff --git a/clang/include/clang/AST/ASTConsumer.h 
b/clang/include/clang/AST/ASTConsumer.h
index ebcd8059284d8d..774d19565e57b0 100644
--- a/clang/include/clang/AST/ASTConsumer.h
+++ b/clang/include/clang/AST/ASTConsumer.h
@@ -26,6 +26,7 @@ namespace clang {
   class VarDecl;
   class FunctionDecl;
   class ImportDecl;
+  class TargetInfo;
 
 /// ASTConsumer - This is an abstract interface that should be implemented by
 /// clients that read ASTs.  This abstraction layer allows the client to be
@@ -46,6 +47,10 @@ class ASTConsumer {
   /// ASTContext.
   virtual void Initialize(ASTContext &Context) {}
 
+  /// Initialize - This is called to initialize the consumer, providing the
+  /// ASTContext.
+  virtual void Initialize(ASTContext &Context, const TargetInfo 
&CodeGenTargetInfo) {}
+
   /// HandleTopLevelDecl - Handle the specified top-level declaration.  This is
   /// called by the parser to process every top-level Decl*.
   ///
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp 
b/clang/lib/CodeGen/CodeGenModule.cpp
index 0c447b20cef40d..8380b71ababe5b 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -333,14 +333,13 @@ CodeGenModule::CodeGenModule(ASTContext &C,
  IntrusiveRefCntPtr FS,
  const HeaderSearchOptions &HSO,
  const PreprocessorOptions &PPO,
- const CodeGenOptions &CGO, llvm::Module &M,
- DiagnosticsEngine &diags,
+ const CodeGenOptions &CGO, const TargetInfo &CGTI,
+ llvm::Module &M, DiagnosticsEngine &diags,
  CoverageSourceInfo *CoverageInfo)
 : Context(C), LangOpts(C.getLangOpts()), FS(FS), HeaderSearchOpts(HSO),
   PreprocessorOpts(PPO), CodeGenOpts(CGO), TheModule(M), Diags(diags),
-  Target(C.getTargetInfo()), ABI(createCXXABI(*this)),
-  VMContext(M.getContext()), Types(*this), VTables(*this),
-  SanitizerMD(new SanitizerMetadata(*this)) {
+  Target(CGTI), ABI(createCXXABI(*this)), VMContext(M.getContext()),
+  Types(*this), VTables(*this), SanitizerMD(new SanitizerMetadata(*this)) {
 
   // Initialize the type cache.
   llvm::LLVMContext &LLVMContext = M.getContext();
@@ -353,20 +352,18 @@ CodeGenModule::CodeGenModule(ASTContext &C,
   BFloatTy = llvm::Type::getBFloatTy(LLVMContext);
   FloatTy = llvm::Type::getFloatTy(LLVMContext);
   DoubleTy = llvm::Type::getDoubleTy(LLVMContext);
-  PointerWidthInBits = C.getTargetInfo().getPointerWidth(LangAS::Default);
+  PointerWidthInBits = Target.getPointerWidth(LangAS::Default);
   PointerAlignInBytes =
-  C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(LangAS::Default))
+  C.toCharUnitsFromBits(Target.getPointerAlign(LangAS::Default))
   .getQuantity();
   SizeSizeInBytes =
-
C.toCharUnitsFromBits(C.getTargetInfo().getMaxPointerWidth()).getQuantity();
-  IntAlignInBytes =
-C.toCharUnitsFromBits(C.getTargetInfo().getIntAlign()).getQuantity();
-  CharTy =
-llvm::IntegerType::get(LLVMContext, C.getTargetInfo().getCharWidth());
-  IntTy = llvm::IntegerType::get(LLVMContext, C.getTargetInfo().getIntWidth());
-  IntPtrTy = llvm::IntegerType::get(LLVMContext,
-C.getTargetInfo().getMaxPointerWidth());
-  Int8PtrTy = llvm::PointerType::get(LLVMContext, 0);
+  C.toCharUnitsFromBits(Target.getMaxPointerWidth()).getQuantity();
+  IntAlignInBytes = C.toCharUnitsFromBits(Target.getIntAlign()).getQuantity();
+  CharTy = llvm::IntegerType::get(LLVMContext, Target.getCharWidth());
+  IntTy = llvm::IntegerType::get(LLVMContext, Target.getIntWidth());
+  IntPtrTy = llvm::IntegerType::get(LLVMContext, Target.getMaxPointerWidth());
+  Int8PtrTy = Int8Ty->getPointerTo(0);
+  Int8PtrPtrTy = Int8PtrTy->getPointerTo(0);
   const llvm::DataLayout &DL = M.getDataLayout();
   AllocaInt8PtrTy =

[clang] [NFC] Parameterize Initialization of `clang::CodeGenerator` on a `TargetInfo` instance which may differ from the one in the `ASTContext` (PR #88977)

2024-04-16 Thread Artem Chikin via cfe-commits


https://github.com/artemcm created 
https://github.com/llvm/llvm-project/pull/88977

A client's Clang instance may require to perform type-checking against one OS 
version and compilation/code-generation against an earlier version. This change 
allows such clients to configure their built-in Clang code-generator with a 
custom `TargetInfo`.

>From c86ee8d6fc2b0d3995f0dee4e7bc206ef6e05c8b Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Wed, 20 Dec 2023 10:56:42 -0800
Subject: [PATCH] [NFC] Parameterize Initialization of 'clang::CodeGenerator'
 on a TargetInfo instance which may differ from the one in the ASTContext

As per https://github.com/apple/swift/pull/65930, Swift compiler's built-in 
Clang instance may require to perform type-checking against one OS version and 
compilation/code-generation against an earlier version. This change allows 
Swift to configure it's built-in Clang code-generator with a custom 
'TargetInfo'.
---
 clang/include/clang/AST/ASTConsumer.h |  5 
 clang/lib/CodeGen/CodeGenModule.cpp   | 23 +++
 clang/lib/CodeGen/CodeGenModule.h |  4 +++-
 clang/lib/CodeGen/ModuleBuilder.cpp   | 17 +-
 .../ObjectFilePCHContainerOperations.cpp  |  3 ++-
 5 files changed, 34 insertions(+), 18 deletions(-)

diff --git a/clang/include/clang/AST/ASTConsumer.h 
b/clang/include/clang/AST/ASTConsumer.h
index ebcd8059284d8d..774d19565e57b0 100644
--- a/clang/include/clang/AST/ASTConsumer.h
+++ b/clang/include/clang/AST/ASTConsumer.h
@@ -26,6 +26,7 @@ namespace clang {
   class VarDecl;
   class FunctionDecl;
   class ImportDecl;
+  class TargetInfo;
 
 /// ASTConsumer - This is an abstract interface that should be implemented by
 /// clients that read ASTs.  This abstraction layer allows the client to be
@@ -46,6 +47,10 @@ class ASTConsumer {
   /// ASTContext.
   virtual void Initialize(ASTContext &Context) {}
 
+  /// Initialize - This is called to initialize the consumer, providing the
+  /// ASTContext.
+  virtual void Initialize(ASTContext &Context, const TargetInfo 
&CodeGenTargetInfo) {}
+
   /// HandleTopLevelDecl - Handle the specified top-level declaration.  This is
   /// called by the parser to process every top-level Decl*.
   ///
diff --git a/clang/lib/CodeGen/CodeGenModule.cpp 
b/clang/lib/CodeGen/CodeGenModule.cpp
index 0c447b20cef40d..52ff65c0986931 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -333,12 +333,14 @@ CodeGenModule::CodeGenModule(ASTContext &C,
  IntrusiveRefCntPtr FS,
  const HeaderSearchOptions &HSO,
  const PreprocessorOptions &PPO,
- const CodeGenOptions &CGO, llvm::Module &M,
+ const CodeGenOptions &CGO,
+ const TargetInfo &CGTI,
+ llvm::Module &M,
  DiagnosticsEngine &diags,
  CoverageSourceInfo *CoverageInfo)
 : Context(C), LangOpts(C.getLangOpts()), FS(FS), HeaderSearchOpts(HSO),
   PreprocessorOpts(PPO), CodeGenOpts(CGO), TheModule(M), Diags(diags),
-  Target(C.getTargetInfo()), ABI(createCXXABI(*this)),
+  Target(CGTI), ABI(createCXXABI(*this)),
   VMContext(M.getContext()), Types(*this), VTables(*this),
   SanitizerMD(new SanitizerMetadata(*this)) {
 
@@ -353,20 +355,21 @@ CodeGenModule::CodeGenModule(ASTContext &C,
   BFloatTy = llvm::Type::getBFloatTy(LLVMContext);
   FloatTy = llvm::Type::getFloatTy(LLVMContext);
   DoubleTy = llvm::Type::getDoubleTy(LLVMContext);
-  PointerWidthInBits = C.getTargetInfo().getPointerWidth(LangAS::Default);
+  PointerWidthInBits = Target.getPointerWidth(LangAS::Default);
   PointerAlignInBytes =
-  C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(LangAS::Default))
+  C.toCharUnitsFromBits(Target.getPointerAlign(LangAS::Default))
   .getQuantity();
   SizeSizeInBytes =
-
C.toCharUnitsFromBits(C.getTargetInfo().getMaxPointerWidth()).getQuantity();
+C.toCharUnitsFromBits(Target.getMaxPointerWidth()).getQuantity();
   IntAlignInBytes =
-C.toCharUnitsFromBits(C.getTargetInfo().getIntAlign()).getQuantity();
+C.toCharUnitsFromBits(Target.getIntAlign()).getQuantity();
   CharTy =
-llvm::IntegerType::get(LLVMContext, C.getTargetInfo().getCharWidth());
-  IntTy = llvm::IntegerType::get(LLVMContext, C.getTargetInfo().getIntWidth());
+llvm::IntegerType::get(LLVMContext, Target.getCharWidth());
+  IntTy = llvm::IntegerType::get(LLVMContext, Target.getIntWidth());
   IntPtrTy = llvm::IntegerType::get(LLVMContext,
-C.getTargetInfo().getMaxPointerWidth());
-  Int8PtrTy = llvm::PointerType::get(LLVMContext, 0);
+Target.getMaxPointerWidth());
+  Int8PtrTy = Int8Ty->getPointerTo(0);
+  Int8PtrPtrTy = Int8PtrTy->getPointerTo(0);
   const llvm::DataLayout &DL = M.getDataLayout();
   AllocaInt8PtrTy =

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From 1989bbbdb1d575e2e938ec319ad9ad23db6af6fc Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|  4 +++
 .../DependencyScanningFilesystem.cpp  | 11 
 .../DependencyScanningFilesystemTest.cpp  | 27 +++
 3 files changed, 42 insertions(+)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..87bb67cfd9327c 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -18,6 +18,7 @@ struct InstrumentingFilesystem
 : llvm::RTTIExtends {
   unsigned NumStatusCalls = 0;
   unsigned NumGetRealPathCalls = 0;
+  unsigned NumExistsCalls = 0;
 
   using llvm::RTTIExtends::RTTIExtends;
@@ -32,6 +33,11 @@ struct InstrumentingFilesystem
 ++NumGetRealPathCalls;
 return ProxyFileSystem::getRealPath(Path, Output);
   }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return ProxyFileSystem::exists(Path);
+  }
 };
 } // namespace
 
@@ -147,3 +153,24 @@ TEST(DependencyScanningFilesystem, 
RealPathAndStatusInvariants) {
 DepFS.status("/bar");
   }
 }
+
+TEST(DependencyScanningFilesystem, CacheStatOnExists) {
+  auto InMemoryFS = llvm::makeIntrusiveRefCnt();
+  auto InstrumentingFS =
+  llvm::makeIntrusiveRefCnt(InMemoryFS);
+  InMemoryFS->setCurrentWorkingDirectory("/");
+  InMemoryFS->addFile("/foo", 0, llvm::MemoryBuffer::getMemBuffer(""));
+  InMemoryFS->addFile("/bar", 0, llvm::MemoryBuffer::getMemBuffer(""));
+  DependencyScanningFilesystemSharedCache SharedCache;
+  DependencyScanningWorkerFilesystem DepFS(SharedCache, InstrumentingFS);
+
+  DepFS.status("/foo");
+  DepFS.status("/foo");
+  DepFS.status("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatusCalls, 2u);
+
+  DepFS.exists("/foo");
+  DepFS.exists("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatusCalls, 2u);
+  EXPECT_EQ(InstrumentingFS->NumExistsCalls, 0u);
+}

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/c

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From b7c693011abd785bec0a58fe4b6011a84f1ea329 Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|  4 +++
 .../DependencyScanningFilesystem.cpp  | 11 +++
 .../DependencyScanningFilesystemTest.cpp  | 30 +++
 3 files changed, 45 insertions(+)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..71447f43f1e6ff 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -18,6 +18,7 @@ struct InstrumentingFilesystem
 : llvm::RTTIExtends {
   unsigned NumStatusCalls = 0;
   unsigned NumGetRealPathCalls = 0;
+  unsigned NumExistsCalls = 0;
 
   using llvm::RTTIExtends::RTTIExtends;
@@ -32,6 +33,11 @@ struct InstrumentingFilesystem
 ++NumGetRealPathCalls;
 return ProxyFileSystem::getRealPath(Path, Output);
   }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return ProxyFileSystem::exists(Path);
+  }
 };
 } // namespace
 
@@ -147,3 +153,27 @@ TEST(DependencyScanningFilesystem, 
RealPathAndStatusInvariants) {
 DepFS.status("/bar");
   }
 }
+
+TEST(DependencyScanningFilesystem, CacheStatOnExists) {
+  auto InMemoryFS = llvm::makeIntrusiveRefCnt();
+  auto InstrumentingFS =
+  llvm::makeIntrusiveRefCnt(InMemoryFS);
+  InMemoryFS->setCurrentWorkingDirectory("/");
+  InMemoryFS->addFile("/foo", 0,
+  llvm::MemoryBuffer::getMemBuffer(""));
+  InMemoryFS->addFile("/bar", 0,
+  llvm::MemoryBuffer::getMemBuffer(""));
+  DependencyScanningFilesystemSharedCache SharedCache;
+  DependencyScanningWorkerFilesystem DepFS(SharedCache,
+   InstrumentingFS);
+
+  DepFS.status("/foo");
+  DepFS.status("/foo");
+  DepFS.status("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatusCalls, 2u);
+
+  DepFS.exists("/foo");
+  DepFS.exists("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatusCalls, 2u);
+  EXPECT_EQ(InstrumentingFS->NumExistsCalls, 0u);
+}

___
cfe-commits

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From 1dd15b331795daaf2ac70cbd013f56f2d7677041 Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|  4 ++
 .../DependencyScanningFilesystem.cpp  | 11 
 .../DependencyScanningFilesystemTest.cpp  | 53 ++-
 3 files changed, 55 insertions(+), 13 deletions(-)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..256dd980b2118f 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -8,6 +8,7 @@
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "gtest/gtest.h"
 
@@ -15,31 +16,36 @@ using namespace clang::tooling::dependencies;
 
 namespace {
 struct InstrumentingFilesystem
-: llvm::RTTIExtends {
+: llvm::RTTIExtends {
   unsigned NumStatusCalls = 0;
   unsigned NumGetRealPathCalls = 0;
+  unsigned NumExistsCalls = 0;
 
   using llvm::RTTIExtends::RTTIExtends;
+  llvm::vfs::InMemoryFileSystem>::RTTIExtends;
 
   llvm::ErrorOr status(const llvm::Twine &Path) override {
 ++NumStatusCalls;
-return ProxyFileSystem::status(Path);
+return InMemoryFileSystem::status(Path);
   }
 
   std::error_code getRealPath(const llvm::Twine &Path,
   llvm::SmallVectorImpl &Output) override {
 ++NumGetRealPathCalls;
-return ProxyFileSystem::getRealPath(Path, Output);
+return InMemoryFileSystem::getRealPath(Path, Output);
+  }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return InMemoryFileSystem::exists(Path);
   }
 };
+
 } // namespace
 
 TEST(DependencyScanningWorkerFilesystem, CacheStatusFailures) {
-  auto InMemoryFS = llvm::makeIntrusiveRefCnt();
-
   auto InstrumentingFS =
-  llvm::makeIntrusiveRefCnt(InMemoryFS);
+  llvm::makeIntrusiveRefCnt();
 
   DependencyScanningFilesystemSharedCache SharedCache;
   DependencyScanningWorkerFilesystem DepFS(SharedC

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


artemcm wrote:

> Clang test now looks good to me. Might be nice to drop 
> `InstrumentingInMemoryFilesystem` in favor of the existing 
> `InstrumentingFilesystem` (that I added just moments ago, sorry!) wrapped 
> around a normal `InMemoryFileSystem`, but I'm happy to do that myself in a 
> follow-up commit.

I had to update the commit anyway. Unified on `InstrumentingFilesystem`. 

https://github.com/llvm/llvm-project/pull/88152
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From 8164aaf2e93dde4761789e6d574088d1cb0e97fc Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|   4 +
 .../DependencyScanningFilesystem.cpp  |  11 ++
 .../DependencyScanningFilesystemTest.cpp  |  53 +--
 .../Support/VirtualFileSystemTest.cpp | 140 +-
 4 files changed, 125 insertions(+), 83 deletions(-)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..256dd980b2118f 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -8,6 +8,7 @@
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "gtest/gtest.h"
 
@@ -15,31 +16,36 @@ using namespace clang::tooling::dependencies;
 
 namespace {
 struct InstrumentingFilesystem
-: llvm::RTTIExtends {
+: llvm::RTTIExtends {
   unsigned NumStatusCalls = 0;
   unsigned NumGetRealPathCalls = 0;
+  unsigned NumExistsCalls = 0;
 
   using llvm::RTTIExtends::RTTIExtends;
+  llvm::vfs::InMemoryFileSystem>::RTTIExtends;
 
   llvm::ErrorOr status(const llvm::Twine &Path) override {
 ++NumStatusCalls;
-return ProxyFileSystem::status(Path);
+return InMemoryFileSystem::status(Path);
   }
 
   std::error_code getRealPath(const llvm::Twine &Path,
   llvm::SmallVectorImpl &Output) override {
 ++NumGetRealPathCalls;
-return ProxyFileSystem::getRealPath(Path, Output);
+return InMemoryFileSystem::getRealPath(Path, Output);
+  }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return InMemoryFileSystem::exists(Path);
   }
 };
+
 } // namespace
 
 TEST(DependencyScanningWorkerFilesystem, CacheStatusFailures) {
-  auto InMemoryFS = llvm::makeIntrusiveRefCnt();
-
   auto InstrumentingFS =
-  llvm::makeIntrusiveRefCnt(InMemoryFS);
+  llvm::makeIntrusiveRefCnt();
 
   DependencyScanningFilesystemSharedCache Sha

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From e010a761a34692e5ca1959fc9022d0950e669664 Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|   4 +
 .../DependencyScanningFilesystem.cpp  |  11 ++
 .../DependencyScanningFilesystemTest.cpp  |  45 ++
 .../Support/VirtualFileSystemTest.cpp | 140 +-
 4 files changed, 130 insertions(+), 70 deletions(-)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..ccc02b7b2f09dd 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -8,6 +8,7 @@
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "gtest/gtest.h"
 
@@ -33,6 +34,27 @@ struct InstrumentingFilesystem
 return ProxyFileSystem::getRealPath(Path, Output);
   }
 };
+
+struct InstrumentingInMemoryFilesystem
+: llvm::RTTIExtends {
+  unsigned NumStatCalls = 0;
+  unsigned NumExistsCalls = 0;
+
+  using llvm::RTTIExtends::RTTIExtends;
+
+  llvm::ErrorOr status(const llvm::Twine &Path) override {
+++NumStatCalls;
+return InMemoryFileSystem::status(Path);
+  }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return InMemoryFileSystem::exists(Path);
+  }
+};
+
 } // namespace
 
 TEST(DependencyScanningWorkerFilesystem, CacheStatusFailures) {
@@ -147,3 +169,26 @@ TEST(DependencyScanningFilesystem, 
RealPathAndStatusInvariants) {
 DepFS.status("/bar");
   }
 }
+
+TEST(DependencyScanningFilesystem, CacheStatOnExists) {
+  auto InMemoryInstrumentingFS =
+  llvm::makeIntrusiveRefCnt();
+  InMemoryInstrumentingFS->setCurrentWorkingDirectory("/");
+  InMemoryInstrumentingFS->addFile("/foo", 0,
+   llvm::MemoryBuffer::getMemBuffer(""));
+  InMemoryInstrumentingFS->addFile("/bar", 0,
+   llvm::MemoryBuffer::getMemBuffer(""));
+  DependencyScanningFilesystemSharedCache SharedCache;
+  DependencyS

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From 24e869df273b9d75bb4fdf85f4ee8ab2ddbccc2c Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanningFilesystem.h|   4 +
 .../DependencyScanningFilesystem.cpp  |  11 ++
 .../DependencyScanningFilesystemTest.cpp  |  46 ++
 .../Support/VirtualFileSystemTest.cpp | 140 +-
 4 files changed, 131 insertions(+), 70 deletions(-)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 8b6f149c7cb266..f7b4510d7f7beb 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
   /// false if not (i.e. this entry is not a file or its scan fails).
   bool ensureDirectiveTokensArePopulated(EntryRef Entry);
 
+  /// Check whether \p Path exists. By default checks cached result of \c
+  /// status(), and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
 private:
   /// For a filename that's not yet associated with any entry in the caches,
   /// uses the underlying filesystem to either look up the entry based in the
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 84185c931b552c..0cab17a3424406 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -300,6 +300,17 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  // While some VFS overlay filesystems may implement more-efficient
+  // mechanisms for `exists` queries, `DependencyScanningWorkerFilesystem`
+  // typically wraps `RealFileSystem` which does not specialize `exists`,
+  // so it is not likely to benefit from such optimizations. Instead,
+  // it is more-valuable to have this query go through the
+  // cached-`status` code-path of the `DependencyScanningWorkerFilesystem`.
+  llvm::ErrorOr Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using
diff --git 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
index 697b7d70ff035a..67f9ee1a3f97ce 100644
--- 
a/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
+++ 
b/clang/unittests/Tooling/DependencyScanning/DependencyScanningFilesystemTest.cpp
@@ -8,6 +8,7 @@
 
 #include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h"
 #include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/Twine.h"
 #include "llvm/Support/VirtualFileSystem.h"
 #include "gtest/gtest.h"
 
@@ -33,6 +34,28 @@ struct InstrumentingFilesystem
 return ProxyFileSystem::getRealPath(Path, Output);
   }
 };
+
+
+struct InstrumentingInMemoryFilesystem
+: llvm::RTTIExtends {
+  unsigned NumStatCalls = 0;
+  unsigned NumExistsCalls = 0;
+
+  using llvm::RTTIExtends::RTTIExtends;
+
+  llvm::ErrorOr status(const llvm::Twine &Path) override {
+++NumStatCalls;
+return InMemoryFileSystem::status(Path);
+  }
+
+  bool exists(const llvm::Twine &Path) override {
+++NumExistsCalls;
+return InMemoryFileSystem::exists(Path);
+  }
+};
+
 } // namespace
 
 TEST(DependencyScanningWorkerFilesystem, CacheStatusFailures) {
@@ -147,3 +170,26 @@ TEST(DependencyScanningFilesystem, 
RealPathAndStatusInvariants) {
 DepFS.status("/bar");
   }
 }
+
+TEST(DependencyScanningFilesystem, CacheStatOnExists) {
+  auto InMemoryInstrumentingFS =
+  llvm::makeIntrusiveRefCnt();
+  InMemoryInstrumentingFS->setCurrentWorkingDirectory("/");
+  InMemoryInstrumentingFS->addFile("/foo", 0,
+   llvm::MemoryBuffer::getMemBuffer(""));
+  InMemoryInstrumentingFS->addFile("/bar", 0,
+   llvm::MemoryBuffer::getMemBuffer(""));
+  DependencyScanningFilesystemSharedCache SharedCache;
+  Dependenc

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From 12621dc6344335375ac1a5c806d778f1ed15a5ce Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 4b1ca0c3d262b6..0113d6b7da25d3 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output);
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 32b480028e71b4..921af30bfcde9f 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -438,6 +438,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2428,6 +2437,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an overriden

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-12 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Artem Chikin via cfe-commits



@@ -270,6 +270,12 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool
+DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  llvm::ErrorOr Status = status(Path);

artemcm wrote:

Which optimization are you referring to? 

https://github.com/llvm/llvm-project/pull/88152
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Artem Chikin via cfe-commits



@@ -0,0 +1,51 @@
+//===- DependencyScanningFilesystemTest.cpp 
---===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM 
Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===--===//
+
+#include "clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h"
+#include "llvm/ADT/Twine.h"
+#include "llvm/Support/VirtualFileSystem.h"
+#include "gtest/gtest.h"
+
+using namespace clang::tooling::dependencies;
+
+namespace {
+ struct InstrumentingFilesystem
+ : llvm::RTTIExtends {
+   unsigned NumStatCalls = 0;
+
+   using llvm::RTTIExtends::RTTIExtends;
+
+   llvm::ErrorOr status(const llvm::Twine &Path) override {
+ ++NumStatCalls;
+ return ProxyFileSystem::status(Path);
+   }
+ };
+ } // namespace
+
+
+TEST(DependencyScanningFilesystem, CacheStatOnExists) {
+  auto InMemoryFS = llvm::makeIntrusiveRefCnt();
+  InMemoryFS->setCurrentWorkingDirectory("/");
+  InMemoryFS->addFile("/foo", 0, llvm::MemoryBuffer::getMemBuffer(""));
+  InMemoryFS->addFile("/bar", 0, llvm::MemoryBuffer::getMemBuffer(""));
+
+  auto InstrumentingFS =
+llvm::makeIntrusiveRefCnt(InMemoryFS);
+  DependencyScanningFilesystemSharedCache SharedCache;
+  DependencyScanningWorkerFilesystem DepFS(SharedCache, InstrumentingFS);
+
+  DepFS.status("/foo");
+  DepFS.status("/foo");
+  DepFS.status("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatCalls, 2u);
+
+  DepFS.exists("/foo");
+  DepFS.exists("/bar");
+  EXPECT_EQ(InstrumentingFS->NumStatCalls, 2u);  

artemcm wrote:

Gah, was silly to use a ProxyFilesystem as the underlying FS. Fixed, thank you 
very much for catching. 

https://github.com/llvm/llvm-project/pull/88152
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-11 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Artem Chikin via cfe-commits


artemcm wrote:

> I'd like to see a unit test specific to `DependencyScanningFilesystem`, 
> similar to what I have here: #68645.

Done! Added one in `DependencyScanningFilesystemTest.cpp`. Thank you.

https://github.com/llvm/llvm-project/pull/88152
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-10 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-09 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-09 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-09 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From c2b2cd03e1d8f92b1df814e6312158b8820b790d Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Tue, 9 Apr 2024 11:22:44 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  56 ++
 .../Support/VirtualFileSystemTest.cpp | 165 ++
 3 files changed, 227 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..272880ba602b84 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,53 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> Path;
+  OriginalPath.toVector(Path);
+
+  if (makeAbsolute(Path))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(Path))
+  return true;
+  }
+
+  ErrorOr Result = lookupPath(Path);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(Path);
+return false;
+  }
+
+  std::optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> RemappedPath((*ExtRedirect).str());
+  if (makeAbsolute(RemappedPath))
+return false;
+
+  if (ExternalFS->exists(RemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Path);
+  }
+
+  return false;
+}
+
 namespace {
 
 /// Provide a file wrapper with an over

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-09 Thread Artem Chikin via cfe-commits


https://github.com/artemcm updated 
https://github.com/llvm/llvm-project/pull/88152

>From fdf95d6c9186b91cf39a988e3baaa21a9cabe6a1 Mon Sep 17 00:00:00 2001
From: Ben Langmuir 
Date: Thu, 6 Oct 2022 15:58:23 -0700
Subject: [PATCH 1/2] [llvm][vfs] Make vfs::FileSystem::exists() virtual NFC

Allow a vfs::FileSystem to provide a more efficient implementation of
exists() if they are able to. The existing FileSystem implementations
continue to default to using status() except that overlay, proxy, and
redirecting filesystems are taught to forward calls to exists()
correctly to their wrapped/external filesystem.
---
 llvm/include/llvm/Support/VirtualFileSystem.h |   8 +-
 llvm/lib/Support/VirtualFileSystem.cpp|  57 +++
 .../Support/VirtualFileSystemTest.cpp | 160 ++
 3 files changed, 223 insertions(+), 2 deletions(-)

diff --git a/llvm/include/llvm/Support/VirtualFileSystem.h 
b/llvm/include/llvm/Support/VirtualFileSystem.h
index 770ca8764426a4..bdc98bfd7d2571 100644
--- a/llvm/include/llvm/Support/VirtualFileSystem.h
+++ b/llvm/include/llvm/Support/VirtualFileSystem.h
@@ -300,8 +300,9 @@ class FileSystem : public 
llvm::ThreadSafeRefCountedBase,
   virtual std::error_code getRealPath(const Twine &Path,
   SmallVectorImpl &Output) const;
 
-  /// Check whether a file exists. Provided for convenience.
-  bool exists(const Twine &Path);
+  /// Check whether \p Path exists. By default this uses \c status(), but
+  /// filesystems may provide a more efficient implementation if available.
+  virtual bool exists(const Twine &Path);
 
   /// Is the file mounted on a local filesystem?
   virtual std::error_code isLocal(const Twine &Path, bool &Result);
@@ -386,6 +387,7 @@ class OverlayFileSystem : public 
RTTIExtends {
   void pushOverlay(IntrusiveRefCntPtr FS);
 
   llvm::ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override;
   directory_iterator dir_begin(const Twine &Dir, std::error_code &EC) override;
@@ -439,6 +441,7 @@ class ProxyFileSystem : public RTTIExtends {
   llvm::ErrorOr status(const Twine &Path) override {
 return FS->status(Path);
   }
+  bool exists(const Twine &Path) override { return FS->exists(Path); }
   llvm::ErrorOr>
   openFileForRead(const Twine &Path) override {
 return FS->openFileForRead(Path);
@@ -1044,6 +1047,7 @@ class RedirectingFileSystem
  bool UseExternalNames, FileSystem &ExternalFS);
 
   ErrorOr status(const Twine &Path) override;
+  bool exists(const Twine &Path) override;
   ErrorOr> openFileForRead(const Twine &Path) override;
 
   std::error_code getRealPath(const Twine &Path,
diff --git a/llvm/lib/Support/VirtualFileSystem.cpp 
b/llvm/lib/Support/VirtualFileSystem.cpp
index 057f8eae0552c6..ee679f14bf2a37 100644
--- a/llvm/lib/Support/VirtualFileSystem.cpp
+++ b/llvm/lib/Support/VirtualFileSystem.cpp
@@ -439,6 +439,15 @@ ErrorOr OverlayFileSystem::status(const Twine 
&Path) {
   return make_error_code(llvm::errc::no_such_file_or_directory);
 }
 
+bool OverlayFileSystem::exists(const Twine &Path) {
+  // FIXME: handle symlinks that cross file systems
+  for (iterator I = overlays_begin(), E = overlays_end(); I != E; ++I) {
+if ((*I)->exists(Path))
+  return true;
+  }
+  return false;
+}
+
 ErrorOr>
 OverlayFileSystem::openFileForRead(const llvm::Twine &Path) {
   // FIXME: handle symlinks that cross file systems
@@ -2431,6 +2440,54 @@ ErrorOr RedirectingFileSystem::status(const 
Twine &OriginalPath) {
   return S;
 }
 
+bool RedirectingFileSystem::exists(const Twine &OriginalPath) {
+  SmallString<256> CanonicalPath;
+  OriginalPath.toVector(CanonicalPath);
+
+  if (makeCanonical(CanonicalPath))
+return false;
+
+  if (Redirection == RedirectKind::Fallback) {
+// Attempt to find the original file first, only falling back to the
+// mapped file if that fails.
+if (ExternalFS->exists(CanonicalPath))
+  return true;
+  }
+
+  ErrorOr Result =
+  lookupPath(CanonicalPath);
+  if (!Result) {
+// Was not able to map file, fallthrough to using the original path if
+// that was the specified redirection type.
+if (Redirection == RedirectKind::Fallthrough &&
+isFileNotFound(Result.getError()))
+  return ExternalFS->exists(CanonicalPath);
+return false;
+  }
+
+  Optional ExtRedirect = Result->getExternalRedirect();
+  if (!ExtRedirect) {
+assert(isa(Result->E));
+return true;
+  }
+
+  SmallString<256> CanonicalRemappedPath((*ExtRedirect).str());
+  if (makeCanonical(CanonicalRemappedPath))
+return false;
+
+  if (ExternalFS->exists(CanonicalRemappedPath))
+return true;
+
+  if (Redirection == RedirectKind::Fallthrough) {
+// Mapped the file but it wasn't found in the underlying filesystem,
+// fallthrough to using the original path if that was the specified
+// redirection type.
+return ExternalFS->exists(Canon

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

2024-04-09 Thread Artem Chikin via cfe-commits


https://github.com/artemcm created 
https://github.com/llvm/llvm-project/pull/88152

As-is, calls to `exists()` fallback on the implementation in 
`ProxyFileSystem::exists` which explicitly calls out to the underlying `FS`, 
which for the `DependencyScanningFilesystem` (overlay) is the real underlying 
filesystem.

Instead, directly overloading `exists` allows us to have it rely on the cached 
`status` behavior used elsewhere by the `DependencyScanningFilesystem`.

>From 8be194ac570d702b5df2fdcd42868122b0ae16dd Mon Sep 17 00:00:00 2001
From: Artem Chikin 
Date: Tue, 9 Apr 2024 09:37:09 -0700
Subject: [PATCH] [clang][deps] Overload 'Filesystem::exists' in
 'DependencyScanningFilesystem' to have it use cached `status`

As-is, calls to `exists()` fallback on the implementation in 
'ProxyFileSystem::exists' which explicitly calls out to the underlying `FS`, 
which for the 'DependencyScanningFilesystem' (overlay) is the real underlying 
filesystem.

Instead, directly overloading 'exists' allows us to have it rely on the cached 
`status` behavior used elsewhere by the 'DependencyScanningFilesystem'.
---
 .../DependencyScanning/DependencyScanningFilesystem.h   | 4 
 .../DependencyScanning/DependencyScanningFilesystem.cpp | 6 ++
 2 files changed, 10 insertions(+)

diff --git 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
index 9a522a3e2fe252..4fdfebada5b7f7 100644
--- 
a/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
+++ 
b/clang/include/clang/Tooling/DependencyScanning/DependencyScanningFilesystem.h
@@ -365,6 +365,10 @@ class DependencyScanningWorkerFilesystem
 return LocalCache.insertEntryForFilename(Filename, Entry);
   }
 
+  /// Check whether \p Path exists. By default checks cached result of \c 
status(),
+  /// and falls back on FS if unable to do so.
+  bool exists(const Twine &Path) override;
+
   /// Returns entry associated with the filename in the shared cache if there 
is
   /// some. Otherwise, constructs new one with the given error code, associates
   /// it with the filename and returns the result.
diff --git 
a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp 
b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
index 9b7812a1adb9e3..174edc98da5877 100644
--- a/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
+++ b/clang/lib/Tooling/DependencyScanning/DependencyScanningFilesystem.cpp
@@ -270,6 +270,12 @@ DependencyScanningWorkerFilesystem::status(const Twine 
&Path) {
   return Result->getStatus();
 }
 
+bool
+DependencyScanningWorkerFilesystem::exists(const Twine &Path) {
+  auto Status = status(Path);
+  return Status && Status->exists();
+}
+
 namespace {
 
 /// The VFS that is used by clang consumes the \c CachedFileSystemEntry using

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][DependencyScanner] Remove unused -fmodule-map-file arguments (PR #80090)

2024-01-31 Thread Artem Chikin via cfe-commits


https://github.com/artemcm approved this pull request.

Nice. 

https://github.com/llvm/llvm-project/pull/80090
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [clang][deps] Collect discovered module dependencies' Link Libraries (PR #93588)

[clang] [NFC] Parameterize Initialization of `clang::CodeGenerator` on a `TargetInfo` instance which may differ from the one in the `ASTContext` (PR #88977)

[clang] [NFC] Parameterize Initialization of `clang::CodeGenerator` on a `TargetInfo` instance which may differ from the one in the `ASTContext` (PR #88977)

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [llvm] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [clang][deps] Overload `Filesystem::exists` in `DependencyScanningFilesystem` to have it use cached `status` (PR #88152)

[clang] [clang][DependencyScanner] Remove unused -fmodule-map-file arguments (PR #80090)

30 matches

Site Navigation

Mail list logo

Footer information