https://github.com/jansvoboda11 created https://github.com/llvm/llvm-project/pull/217061
The search-path analogue of -fmodules-ignore-macro=<macro>: the named header search path is dropped from the context hash of every module and physically removed from every module-build invocation, and kept only for the translation unit itself. The motivating case is a build system that hands every compile a per-target, per-architecture generated-headers directory (Xcode's DerivedSources, say) even though only a handful of TUs include anything from it. Under -fmodules-strict-context-hash those directories feed every module's context hash, so an SDK module like Foundation is rebuilt once per target rather than once per architecture. Ignoring the directories for modules lets those builds share a single module cache while the TUs that really do need them keep working. This is only sound when no module needs the path: a lookup that would have resolved through an ignored path simply fails, exactly as ignoring a macro a module needs silently changes what that module sees. An empty path is rejected. The context hash filters UserEntries through make_filter_range and still hands the result to HashBuilder::addRange, which hashes the element count first; hashing the survivors individually instead would change every strict context hash even when nothing is ignored, invalidating existing module caches. Verified that a TU's module cache directory is unchanged from before this patch when the option is not used. Keeping the entries out of the hash is also what keeps the "Module hash mismatch!" assertion in cloneForModuleCompileImpl() honest, since the clone physically drops them. The dependency scanner prunes in makeCommonInvocationForModuleBuild, on the common invocation every module descends from, so the pruning is uniform and the search-path bit indices optimizeHeaderSearchOpts() works with still line up with what the real implicit builds used. >From a9315049c4923400f3ff5632cb8a3ddc21b4371f Mon Sep 17 00:00:00 2001 From: Jan Svoboda <[email protected]> Date: Tue, 18 Aug 2026 15:06:29 +0200 Subject: [PATCH] [clang][modules] Add -fmodules-ignore-search-path The search-path analogue of -fmodules-ignore-macro=<macro>: the named header search path is dropped from the context hash of every module and physically removed from every module-build invocation, and kept only for the translation unit itself. The motivating case is a build system that hands every compile a per-target, per-architecture generated-headers directory (Xcode's DerivedSources, say) even though only a handful of TUs include anything from it. Under -fmodules-strict-context-hash those directories feed every module's context hash, so an SDK module like Foundation is rebuilt once per target rather than once per architecture. Ignoring the directories for modules lets those builds share a single module cache while the TUs that really do need them keep working. This is only sound when no module needs the path: a lookup that would have resolved through an ignored path simply fails, exactly as ignoring a macro a module needs silently changes what that module sees. An empty path is rejected. The context hash filters UserEntries through make_filter_range and still hands the result to HashBuilder::addRange, which hashes the element count first; hashing the survivors individually instead would change every strict context hash even when nothing is ignored, invalidating existing module caches. Verified that a TU's module cache directory is unchanged from before this patch when the option is not used. Keeping the entries out of the hash is also what keeps the "Module hash mismatch!" assertion in cloneForModuleCompileImpl() honest, since the clone physically drops them. The dependency scanner prunes in makeCommonInvocationForModuleBuild, on the common invocation every module descends from, so the pruning is uniform and the search-path bit indices optimizeHeaderSearchOpts() works with still line up with what the real implicit builds used. --- clang/docs/ReleaseNotes.md | 8 ++ clang/include/clang/Lex/HeaderSearchOptions.h | 7 +- clang/include/clang/Options/Options.td | 3 + .../DependencyScanning/ModuleDepCollector.cpp | 12 ++ clang/lib/Driver/ToolChains/Clang.cpp | 2 +- clang/lib/Frontend/CompilerInstance.cpp | 11 ++ clang/lib/Frontend/CompilerInvocation.cpp | 22 ++- .../modules-context-hash-ignore-search-path.c | 129 ++++++++++++++++++ .../test/Driver/modules-ignore-search-path.c | 7 + .../test/Modules/modules-ignore-search-path.c | 103 ++++++++++++++ 10 files changed, 300 insertions(+), 4 deletions(-) create mode 100644 clang/test/ClangScanDeps/modules-context-hash-ignore-search-path.c create mode 100644 clang/test/Driver/modules-ignore-search-path.c create mode 100644 clang/test/Modules/modules-ignore-search-path.c diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md index f5f9958543e34..16b040371f00d 100644 --- a/clang/docs/ReleaseNotes.md +++ b/clang/docs/ReleaseNotes.md @@ -209,6 +209,14 @@ features cannot lower the translation-unit ABI level; `-fsanitize=shadow-call-stack`. The selected register must also be reserved with the matching `-ffixed-<reg>`. +- Added `-fmodules-ignore-search-path=<path>`, the search-path analogue of + `-fmodules-ignore-macro=<macro>`: the path is dropped from the context hash of + every module and physically removed from every module build, and kept only for + the translation unit itself. This lets builds that differ only in a per-target + generated-headers directory share one module cache, and is only sound when no + module needs the path -- a lookup that would have resolved through an ignored + path simply fails. + ### Deprecated Compiler Flags ### Modified Compiler Flags diff --git a/clang/include/clang/Lex/HeaderSearchOptions.h b/clang/include/clang/Lex/HeaderSearchOptions.h index fd46ea223bae1..a521f38e28233 100644 --- a/clang/include/clang/Lex/HeaderSearchOptions.h +++ b/clang/include/clang/Lex/HeaderSearchOptions.h @@ -180,10 +180,13 @@ class HeaderSearchOptions { /// loading. uint64_t BuildSessionTimestamp = 0; - /// The set of macro names that should be ignored for the purposes - /// of computing the module hash. + /// The set of macro names that should be ignored by implicitly-built modules. llvm::SmallSetVector<llvm::CachedHashString, 16> ModulesIgnoreMacros; + /// The set of header search paths that should be ignored by implicitly-built + /// modules. + llvm::SmallSetVector<llvm::CachedHashString, 16> ModulesIgnoreSearchPaths; + /// The set of user-provided virtual filesystem overlay files. std::vector<std::string> VFSOverlayFiles; diff --git a/clang/include/clang/Options/Options.td b/clang/include/clang/Options/Options.td index adc4224dd561c..ff085e374e3f9 100644 --- a/clang/include/clang/Options/Options.td +++ b/clang/include/clang/Options/Options.td @@ -3883,6 +3883,9 @@ def fmodule_file : Joined<["-"], "fmodule-file=">, def fmodules_ignore_macro : Joined<["-"], "fmodules-ignore-macro=">, Group<f_Group>, Visibility<[ClangOption, CC1Option, CLOption]>, HelpText<"Ignore the definition of the given macro when building and loading modules">; +def fmodules_ignore_search_path : Joined<["-"], "fmodules-ignore-search-path=">, Group<f_Group>, + Visibility<[ClangOption, CC1Option, CLOption]>, + HelpText<"Ignore the given header search path when building and loading modules">; def fmodules_strict_decluse : Flag <["-"], "fmodules-strict-decluse">, Group<f_Group>, Visibility<[ClangOption, CC1Option, CLOption]>, HelpText<"Like -fmodules-decluse but requires all headers to be in modules">, diff --git a/clang/lib/DependencyScanning/ModuleDepCollector.cpp b/clang/lib/DependencyScanning/ModuleDepCollector.cpp index c7b7fcc8750c3..58b68a69cf571 100644 --- a/clang/lib/DependencyScanning/ModuleDepCollector.cpp +++ b/clang/lib/DependencyScanning/ModuleDepCollector.cpp @@ -294,6 +294,18 @@ makeCommonInvocationForModuleBuild(CompilerInvocation CI) { CI.getHeaderSearchOpts().ModulesIgnoreMacros.clear(); } + // Remove any header search paths that are explicitly ignored. + if (!CI.getHeaderSearchOpts().ModulesIgnoreSearchPaths.empty()) { + llvm::erase_if(CI.getHeaderSearchOpts().UserEntries, + [&CI](const HeaderSearchOptions::Entry &E) { + return CI.getHeaderSearchOpts() + .ModulesIgnoreSearchPaths.contains( + llvm::CachedHashString(E.Path)); + }); + // Remove the now unused option. + CI.getHeaderSearchOpts().ModulesIgnoreSearchPaths.clear(); + } + return CI; } diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp index 54583fe3abbd8..8731188dab706 100644 --- a/clang/lib/Driver/ToolChains/Clang.cpp +++ b/clang/lib/Driver/ToolChains/Clang.cpp @@ -4214,8 +4214,8 @@ static bool RenderModulesOptions(Compilation &C, const Driver &D, if (HaveClangModules) Args.AddLastArg(CmdArgs, options::OPT_fmodules_user_build_path); - // Pass through all -fmodules-ignore-macro arguments. Args.AddAllArgs(CmdArgs, options::OPT_fmodules_ignore_macro); + Args.AddAllArgs(CmdArgs, options::OPT_fmodules_ignore_search_path); Args.AddLastArg(CmdArgs, options::OPT_fmodules_prune_interval); Args.AddLastArg(CmdArgs, options::OPT_fmodules_prune_after); diff --git a/clang/lib/Frontend/CompilerInstance.cpp b/clang/lib/Frontend/CompilerInstance.cpp index 66662a786e6fc..2a981b8e015c9 100644 --- a/clang/lib/Frontend/CompilerInstance.cpp +++ b/clang/lib/Frontend/CompilerInstance.cpp @@ -1165,6 +1165,17 @@ std::unique_ptr<CompilerInstance> CompilerInstance::cloneForModuleCompileImpl( llvm::CachedHashString(MacroDef.split('=').first)); }); + // Likewise for header search paths explicitly ignored by every module + // (-fmodules-ignore-search-path). ModulesIgnoreSearchPaths itself is + // deliberately left in place, so a module built from this clone keeps ignoring + // the same paths in its own, further-nested module builds. + if (!HSOpts.ModulesIgnoreSearchPaths.empty()) + llvm::erase_if(HSOpts.UserEntries, + [&HSOpts](const HeaderSearchOptions::Entry &E) { + return HSOpts.ModulesIgnoreSearchPaths.contains( + llvm::CachedHashString(E.Path)); + }); + // If the original compiler invocation had -fmodule-name, pass it through. Invocation->getLangOpts().ModuleName = getInvocation().getLangOpts().ModuleName; diff --git a/clang/lib/Frontend/CompilerInvocation.cpp b/clang/lib/Frontend/CompilerInvocation.cpp index 9359848d3a29b..c18b17dd57737 100644 --- a/clang/lib/Frontend/CompilerInvocation.cpp +++ b/clang/lib/Frontend/CompilerInvocation.cpp @@ -3352,6 +3352,9 @@ static void GenerateHeaderSearchArgs(const HeaderSearchOptions &Opts, for (const auto &Macro : Opts.ModulesIgnoreMacros) GenerateArg(Consumer, OPT_fmodules_ignore_macro, Macro.val()); + for (const auto &Path : Opts.ModulesIgnoreSearchPaths) + GenerateArg(Consumer, OPT_fmodules_ignore_search_path, Path.val()); + auto Matches = [](const HeaderSearchOptions::Entry &Entry, llvm::ArrayRef<frontend::IncludeDirGroup> Groups, std::optional<bool> IsFramework, @@ -3476,6 +3479,10 @@ static bool ParseHeaderSearchArgs(HeaderSearchOptions &Opts, ArgList &Args, llvm::CachedHashString(MacroDef.split('=').first)); } + for (const auto *A : Args.filtered(OPT_fmodules_ignore_search_path)) + Opts.ModulesIgnoreSearchPaths.insert( + llvm::CachedHashString(A->getValue())); + // Add -I... and -F... options in order. bool IsSysrootSpecified = Args.hasArg(OPT__sysroot_EQ) || Args.hasArg(OPT_isysroot); @@ -5324,7 +5331,20 @@ std::string CompilerInvocation::computeContextHash() const { if (hsOpts.ModulesStrictContextHash) { HBuilder.addRange(hsOpts.SystemHeaderPrefixes); - HBuilder.addRange(hsOpts.UserEntries); + + for (const auto &UserEntry : hsOpts.UserEntries) { + // If we're supposed to ignore this search path for the purposes of + // modules, don't put it into the hash. + if (!hsOpts.ModulesIgnoreSearchPaths.empty()) { + // Check whether we're ignoring this search path. + StringRef Path = UserEntry.Path; + if (hsOpts.ModulesIgnoreSearchPaths.count(llvm::CachedHashString(Path))) + continue; + } + + HBuilder.add(UserEntry); + } + HBuilder.addRange(hsOpts.VFSOverlayFiles); const DiagnosticOptions &diagOpts = getDiagnosticOpts(); diff --git a/clang/test/ClangScanDeps/modules-context-hash-ignore-search-path.c b/clang/test/ClangScanDeps/modules-context-hash-ignore-search-path.c new file mode 100644 index 0000000000000..1b378f72cc481 --- /dev/null +++ b/clang/test/ClangScanDeps/modules-context-hash-ignore-search-path.c @@ -0,0 +1,129 @@ +// Ensure -fmodules-ignore-search-path drops the path from the module build and +// from its context hash, so two TUs that differ only in such a path share one +// module, while the path itself is still available to the TUs themselves. + +// RUN: rm -rf %t +// RUN: split-file %s %t +// RUN: sed "s|DIR|%/t|g" %t/cdb.json.template > %t/cdb.json + +// -optimize-args=none so that the search-path pruning under test is the flag's +// own, not -optimize-args=header-search's usage-based pruning (which would drop +// an unused path from every module's command line regardless). +// +// RUN: clang-scan-deps -compilation-database %t/cdb.json -j 1 \ +// RUN: -optimize-args=none -format experimental-full > %t/deps.json +// RUN: cat %t/deps.json | sed 's:\\\\\?:/:g' \ +// RUN: | %scan-deps-filter --fields=command-line,context-hash,file-deps,input-file,name \ +// RUN: | FileCheck -DPREFIX=%/t %s + +// tu1 and tu2 differ only in the generated-headers path and get two distinct +// 'Mod' variants; tu3 and tu4 ignore theirs for modules and collapse onto a +// third, shared one, whose command line mentions neither the path nor the flag. + +// CHECK: { +// CHECK-NEXT: "modules": [ +// CHECK: { +// CHECK: "command-line": [ +// CHECK: "-I" +// CHECK-NEXT: "[[PREFIX]]/sdk" +// CHECK-NEXT: "-I" +// CHECK-NEXT: "[[PREFIX]]/gen1" +// CHECK: ] +// CHECK: "context-hash": "[[HASH_GEN1:.*]]" +// CHECK: "name": "Mod" +// CHECK-NEXT: } +// CHECK-NEXT: { +// CHECK: "command-line": [ +// CHECK: "-I" +// CHECK-NEXT: "[[PREFIX]]/sdk" +// CHECK-NEXT: "-I" +// CHECK-NEXT: "[[PREFIX]]/gen2" +// CHECK: ] +// CHECK: "context-hash": "[[HASH_GEN2:.*]]" +// CHECK: "name": "Mod" +// CHECK-NEXT: } +// CHECK-NEXT: { +// CHECK: "command-line": [ +// CHECK-NOT: "-fmodules-ignore-search-path +// CHECK: "-I" +// CHECK-NEXT: "[[PREFIX]]/sdk" +// CHECK-NOT: "[[PREFIX]]/gen +// CHECK: ] +// CHECK: "context-hash": "[[HASH_SHARED:.*]]" +// CHECK: "name": "Mod" +// CHECK-NEXT: } +// CHECK-NEXT: ] + +// Each TU keeps its own -I path -- the flag only ever prunes module builds, so +// the TU still reads the generated header through it -- and points at the +// variant computed above. + +// CHECK: "context-hash": "[[HASH_GEN1]]" +// CHECK: "[[PREFIX]]/gen1" +// CHECK: "[[PREFIX]]/gen1/gen.h" +// CHECK: "input-file": "[[PREFIX]]/tu1.c" +// CHECK: "context-hash": "[[HASH_GEN2]]" +// CHECK: "[[PREFIX]]/gen2" +// CHECK: "[[PREFIX]]/gen2/gen.h" +// CHECK: "input-file": "[[PREFIX]]/tu2.c" +// CHECK: "context-hash": "[[HASH_SHARED]]" +// CHECK: "-fmodules-ignore-search-path=[[PREFIX]]/gen1" +// CHECK: "[[PREFIX]]/gen1" +// CHECK: "[[PREFIX]]/gen1/gen.h" +// CHECK: "input-file": "[[PREFIX]]/tu3.c" +// CHECK: "context-hash": "[[HASH_SHARED]]" +// CHECK: "-fmodules-ignore-search-path=[[PREFIX]]/gen2" +// CHECK: "[[PREFIX]]/gen2" +// CHECK: "[[PREFIX]]/gen2/gen.h" +// CHECK: "input-file": "[[PREFIX]]/tu4.c" + +//--- cdb.json.template +[ + { + "directory": "DIR", + "command": "clang -fsyntax-only DIR/tu1.c -I DIR/sdk -I DIR/gen1 -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache", + "file": "DIR/tu1.c" + }, + { + "directory": "DIR", + "command": "clang -fsyntax-only DIR/tu2.c -I DIR/sdk -I DIR/gen2 -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache", + "file": "DIR/tu2.c" + }, + { + "directory": "DIR", + "command": "clang -fsyntax-only DIR/tu3.c -I DIR/sdk -I DIR/gen1 -fmodules-ignore-search-path=DIR/gen1 -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache", + "file": "DIR/tu3.c" + }, + { + "directory": "DIR", + "command": "clang -fsyntax-only DIR/tu4.c -I DIR/sdk -I DIR/gen2 -fmodules-ignore-search-path=DIR/gen2 -fmodules -fimplicit-module-maps -fmodules-cache-path=DIR/cache", + "file": "DIR/tu4.c" + }, +] + +//--- sdk/module.modulemap +module Mod { header "Mod.h" } + +//--- sdk/Mod.h + +//--- gen1/gen.h +int gen(void); + +//--- gen2/gen.h +int gen(void); + +//--- tu1.c +#include "Mod.h" +#include "gen.h" + +//--- tu2.c +#include "Mod.h" +#include "gen.h" + +//--- tu3.c +#include "Mod.h" +#include "gen.h" + +//--- tu4.c +#include "Mod.h" +#include "gen.h" diff --git a/clang/test/Driver/modules-ignore-search-path.c b/clang/test/Driver/modules-ignore-search-path.c new file mode 100644 index 0000000000000..c032af9ab32cf --- /dev/null +++ b/clang/test/Driver/modules-ignore-search-path.c @@ -0,0 +1,7 @@ +// Check that -fmodules-ignore-search-path reaches cc1 through the driver. + +// RUN: %clang -### -c -fmodules -fmodules-ignore-search-path=/tmp/gen1 \ +// RUN: -fmodules-ignore-search-path=/tmp/gen2 %s 2>&1 | FileCheck %s +// +// CHECK: "-fmodules-ignore-search-path=/tmp/gen1" +// CHECK-SAME: "-fmodules-ignore-search-path=/tmp/gen2" diff --git a/clang/test/Modules/modules-ignore-search-path.c b/clang/test/Modules/modules-ignore-search-path.c new file mode 100644 index 0000000000000..fb7e13f0d052c --- /dev/null +++ b/clang/test/Modules/modules-ignore-search-path.c @@ -0,0 +1,103 @@ +// Tests -fmodules-ignore-search-path=P for implicitly-built modules: the path is +// dropped from the context hash of every module and physically removed from +// every module build, and kept only for the translation unit itself. It is the +// -fmodules-ignore-macro treatment applied to a search path rather than a macro, +// and like it, it applies to the regular context hash, so builds differing only +// in that path share modules. + +// RUN: rm -rf %t && split-file %s %t + +// The option round-trips through cc1 argument generation. +// +// RUN: %clang_cc1 -round-trip-args -fsyntax-only -fmodules \ +// RUN: -fimplicit-module-maps -fmodules-cache-path=%t/rtcache -I %t/sdk \ +// RUN: -I %t/gen1 -fmodules-ignore-search-path=%t/gen1 %t/tu.c + +// Three "projects" that see the same SDK but stage per-build generated headers +// at a different path each. Only the TU includes them; no module does. Under +// strict context hash those -I paths alone would fragment every module's hash. +// +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen1 -fmodules-ignore-search-path=%t/gen1 %t/tu.c +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen2 -fmodules-ignore-search-path=%t/gen2 %t/tu.c +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen3 -fmodules-ignore-search-path=%t/gen3 %t/tu.c + +// All three projects share a single context hash for every module: exactly one +// pcm of each exists in the whole cache. +// +// RUN: ls %t/cache/*/Ins-*.pcm | count 1 +// RUN: ls %t/cache/*/Dep-*.pcm | count 1 + +// Without the annotation the -I path feeds every module's context hash, so both +// modules fragment: one per project. +// +// RUN: rm -rf %t/cache +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen1 %t/tu.c +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen2 %t/tu.c +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen3 %t/tu.c +// RUN: ls %t/cache/*/Ins-*.pcm | count 3 +// RUN: ls %t/cache/*/Dep-*.pcm | count 3 + +// The path really is gone from the module builds, not merely unhashed: a module +// that does reach for a header only that path provides fails. That failure is the +// documented cost of asserting a path no module needs. +// +// RUN: rm -rf %t/cache +// RUN: not %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen1 -fmodules-ignore-search-path=%t/gen1 \ +// RUN: %t/tu-module-needs-it.c 2>&1 | FileCheck -check-prefix=NEEDED %s +// +// NEEDED: 'gen.h' file not found + +// And it is only the *ignored* path that goes: an unannotated -I path still +// works from inside a module, and still keys it. +// +// RUN: rm -rf %t/cache +// RUN: %clang_cc1 -fsyntax-only -fmodules -fimplicit-module-maps \ +// RUN: -fmodules-strict-context-hash -fmodules-cache-path=%t/cache \ +// RUN: -I %t/sdk -I %t/gen1 %t/tu-module-needs-it.c + +//--- sdk/module.modulemap +module Ins { header "ins.h" } +module Dep { header "dep.h" } +module NeedsGen { header "needsgen.h" } + +//--- sdk/ins.h +int ins(void); + +//--- sdk/dep.h +#include "ins.h" +int dep(void); + +//--- sdk/needsgen.h +#include "gen.h" +int needsgen(void); + +//--- gen1/gen.h +int gen(void); + +//--- gen2/gen.h +int gen(void); + +//--- gen3/gen.h +int gen(void); + +//--- tu.c +// Only the TU consumes the ignored path. +#include "gen.h" +#include "dep.h" + +//--- tu-module-needs-it.c +#include "needsgen.h" _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
