steakhal wrote:
Hey, I know it's kinda late to report this, but it took me some time to debug
this issue downstream.
@HerrCai0907 @vbvictor
The `clang-tools-extra/test/clang-tidy/infrastructure/custom-query-check.cpp`
test breaks when building on MacOS with the
`CMAKE_OSX_ARCHITECTURES='x86_64;arm64;arm64e'` make option.
Here are the exact steps to reproduce the test failure:
```
git checkout 292c9e3d198249e329be355d1513484a559c2f8a
xcrun cmake -S llvm -B build/tmp
-DCMAKE_OSX_ARCHITECTURES='x86_64;arm64;arm64e'
-DLLVM_ENABLE_PROJECTS='clang;clang-tools-extra' -DLLVM_ENABLE_ZSTD=OFF
-DCMAKE_BUILD_TYPE=Release
xcrun ninja -C build/tmp clang-tidy FileCheck split-file
xcrun build/tmp/bin/llvm-lit -a
clang-tools-extra/test/clang-tidy/infrastructure/custom-query-check.cpp
```
This should result the following test failure:
```
# | error: unable to handle compilation, expected exactly one compiler job in '
"/Applications/XCODE-REDACTED/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++"
"-cc1" "-triple" "x86_64-apple-macosxREDACTED" "-O2"
"-Wundef-prefix=TARGET_OS_" "-Werror=undef-prefix"
"-Wdeprecated-objc-isa-usage" "-Werror=deprecated-objc-isa-usage"
"-fsyntax-only" "-disable-free" "-clear-ast-before-backend" "-main-file-name"
"main.cpp" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=all"
"-ffp-contract=on" "-fno-rounding-math" "-funwind-tables=2"
"-target-sdk-version=REDACTED"
"-fcompatibility-qualified-id-block-type-checking" "-fvisibili
ty-inlines-hidden-static-local-var" "-fdefine-target-os-macros"
"-fno-modulemap-allow-subdirectory-search" "-tune-cpu" "generic"
"-gkey-instructions" "-debug-info-kind=standalone" "-dwarf-vers
ion=5" "-debugger-tuning=lldb" "-fdebug-compilation-dir=build/tmp"
"-target-linker-version" "REDACTED" "-fcoverage-compilation-dir=build/custom"
"-isysroot" "/Library/Developer/CommandLineTools/SDKs/MacOSXREDACTED" "-D"
"GTEST_HAS_RTTI=0" "-D" "LLVM_EXPORTS" "-D" "_DEBUG" "-D" "_GLIBCXX_ASSERTIONS
... "; "/Applications/XCODE..."; "/Applications/XCODE..."; '
[clang-diagnostic-error]
```
From what I could see is that since `CMAKE_OSX_ARCHITECTURES` is set, make will
generate a `build/tmp/compile_commands.json` containing build commands building
fat archives, holding multiple architectures such as `'x86_64;arm64;arm64e'`.
This is materialised in the commands as multiple `-arch x86_64 -arch arm64
-arch arm64e`
For example:
```
{
"directory": "build/tmp",
"command":
"/Applications/Xcode-REDACTED/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-DGTEST_HAS_RTTI=0 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS
-D__STDC_LIMIT_MACROS -Ibuild/tmp/lib/Demangle -Illvm/lib/Demangle
-Ibuild/tmp/include -Illvm/include -fPIC -fvisibility-inlines-hidden
-Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra
-Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
-pedantic -Wno-long-long -Wc++98-compat-extra-semi -Wimplicit-fallthrough
-Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor
-Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion
-Wno-pass-failed -Wmisleading-indentation -Wctad-maybe-unsupported
-fdiagnostics-color -O3 -DNDEBUG -std=c++17 -arch x86_64 -arch arm64 -arch
arm64e -isysroot
/Library/Developer/CommandLineTools/SDKs/MacOSXREDACTED-fno-exceptions
-funwind-tables -fno-rtti -o
lib/Demangle/CMakeFiles/LLVMDemangle.dir/Demangle.cpp.o -c
llvm/lib/Demangle/Demangle.cpp",
"file": "llvm/lib/Demangle/Demangle.cpp",
"output": "lib/Demangle/CMakeFiles/LLVMDemangle.dir/Demangle.cpp.o"
},...
```
What puzzled me for a while was the following: how could this possibly affect
the outcome of the
`clang-tools-extra/test/clang-tidy/infrastructure/custom-query-check.cpp` test.
Well, as it turns out, in the test, as tidy starts up, it will look for a
`compile_commands.json`. And start looking from the test file, which is a VFS
path:
`build/tmp/tools/clang/tools/extra/test/clang-tidy/infrastructure/Output/custom-query-check.cpp.tmp/subdir/main.cpp`.
As it doesn't find the compile_commands.json near that file, it will start
walking up the path, until it finds one.
As it happens, the first compile_commands.json file is at
`build/tmp/compile_commands.json` that doesn't have an exact entry for the
desired `..../subdir/main.cpp`, so the
`InterpolatingCompilationDatabase::chooseProxy()` will try to find a suitable
candidate by "scoring" them. For us, it basically means it will randomly pick
one, and that one will have multiple `-arch` options in the command.
So `tidy` would eventually go into and trigger the
`diag::err_fe_expected_compiler_job` warning: `"unable to handle compilation,
expected exactly one compiler job in '%0'"` inside `getCC1Arguments()` in
`clang/lib/Tooling/Tooling.cpp`.
I am not exactly sure how to fix this, but a workaround I had in mind was to
supply a mock `compile_commands.json` alongside the test so that it picks that
up in the `CompilationDatabase::autoDetectFromSource()`. However, the path walk
uses the `JSONCompilationDatabase::loadFromFile()` which in turn uses
`llvm::MemoryBuffer::getFile()` which does not honour VFS, so using the
`vfsoverlay` trick from the test is not suitable for this.
However, the test could be transformed to use `split-file` and also inject a
mock `compile_commands.json`.
This works and passes for me, using `split-file`:
```
// sed command does not work as-is on Windows.
// UNSUPPORTED: system-windows
// REQUIRES: custom-check
// RUN: rm -rf %t
// RUN: split-file %s %t
// RUN: sed -e "s:OUT_DIR:%t:g" %t/compile_commands.json.template >
%t/compile_commands.json
// RUN: mkdir %t/subdir
// RUN: mkdir %t/subdir-override
// RUN: mkdir %t/subdir-empty
// RUN: mkdir %t/subdir-append
// RUN: cp %t/main.cpp %t/subdir/main.cpp
// RUN: cp %t/main.cpp %t/subdir-override/main.cpp
// RUN: cp %t/main.cpp %t/subdir-empty/main.cpp
// RUN: cp %t/main.cpp %t/subdir-append/main.cpp
// RUN: clang-tidy --experimental-custom-checks %t/main.cpp
-checks='-*,custom-*' | FileCheck %t/main.cpp --check-prefix=CHECK-SAME-DIR
// RUN: clang-tidy --experimental-custom-checks %t/subdir/main.cpp
-checks='-*,custom-*' | FileCheck %t/main.cpp --check-prefix=CHECK-SUB-DIR-BASE
// RUN: clang-tidy --experimental-custom-checks %t/subdir-override/main.cpp
-checks='-*,custom-*' | FileCheck %t/main.cpp
--check-prefix=CHECK-SUB-DIR-OVERRIDE
// RUN: clang-tidy --experimental-custom-checks %t/subdir-empty/main.cpp
-checks='-*,custom-*' --allow-no-checks | FileCheck %t/main.cpp
--check-prefix=CHECK-SUB-DIR-EMPTY
// RUN: clang-tidy --experimental-custom-checks %t/subdir-append/main.cpp
-checks='-*,custom-*' | FileCheck %t/main.cpp
--check-prefix=CHECK-SUB-DIR-APPEND
// RUN: clang-tidy --experimental-custom-checks %t/subdir-append/main.cpp
-checks='-*,custom-*' --list-checks | FileCheck %t/main.cpp
--check-prefix=LIST-CHECK
// RUN: clang-tidy --experimental-custom-checks %t/subdir-append/main.cpp
-checks='-*,custom-*' --dump-config | FileCheck %t/main.cpp
--check-prefix=DUMP-CONFIG
//--- compile_commands.json.template
[{
"directory": "OUT_DIR",
"command": "clang++ main.cpp -fsyntax-only",
"file": "main.cpp",
"output": ""
}]
//--- .clang-tidy
CustomChecks:
- Name: avoid-long-type
Query: |
match varDecl(
hasType(asString("long")),
hasTypeLoc(typeLoc().bind("long"))
)
Diagnostic:
- BindName: long
Message: use 'int' instead of 'long'
Level: Warning
//--- subdir-override/.clang-tidy
CustomChecks:
- Name: avoid-long-type
Query: |
match varDecl(
hasType(asString("long")),
hasTypeLoc(typeLoc().bind("long"))
)
Diagnostic:
- BindName: long
Message: use 'int' instead of 'long' override
Level: Warning
//--- subdir-empty/.clang-tidy
InheritParentConfig: false
//--- subdir-append/.clang-tidy
InheritParentConfig: true
CustomChecks:
- Name: function-decl
Query: match functionDecl().bind("func")
Diagnostic:
- BindName: func
Message: find function decl
Level: Warning
//--- main.cpp
long V;
// CHECK-SAME-DIR: [[@LINE-1]]:1: warning: use 'int' instead of 'long'
[custom-avoid-long-type]
// CHECK-SUB-DIR-BASE: [[@LINE-2]]:1: warning: use 'int' instead of 'long'
[custom-avoid-long-type]
// CHECK-SUB-DIR-OVERRIDE: [[@LINE-3]]:1: warning: use 'int' instead of 'long'
override [custom-avoid-long-type]
// CHECK-SUB-DIR-EMPTY: No checks enabled.
// CHECK-SUB-DIR-APPEND: [[@LINE-5]]:1: warning: use 'int' instead of 'long'
[custom-avoid-long-type]
void f();
// CHECK-SUB-DIR-APPEND: [[@LINE-1]]:1: warning: find function decl
[custom-function-decl]
// LIST-CHECK: Enabled checks:
// LIST-CHECK: custom-avoid-long-type
// LIST-CHECK: custom-function-decl
// DUMP-CONFIG: CustomChecks:
// DUMP-CONFIG: - Name: avoid-long-type
// DUMP-CONFIG: Query: |
// DUMP-CONFIG: match varDecl(
// DUMP-CONFIG: hasType(asString("long")),
// DUMP-CONFIG: hasTypeLoc(typeLoc().bind("long"))
// DUMP-CONFIG: )
// DUMP-CONFIG: Diagnostic:
// DUMP-CONFIG: - BindName: long
// DUMP-CONFIG: Message: |
// DUMP-CONFIG: use 'int' instead of 'long'
// DUMP-CONFIG: Level: Warning
// DUMP-CONFIG: - Name: function-decl
// DUMP-CONFIG: Query: |
// DUMP-CONFIG: match functionDecl().bind("func")
// DUMP-CONFIG: Diagnostic:
// DUMP-CONFIG: - BindName: func
// DUMP-CONFIG: Message: |
// DUMP-CONFIG: find function decl
// DUMP-CONFIG: Level: Warning
```
Despite patching the test would sidestep this failure, I definitely think that
we should instead figure out why did this auto compile-command.json discovery
kick in because that made this debugging experience terrible; so I think there
is some room for improvement here.
https://github.com/llvm/llvm-project/pull/131804
_______________________________________________
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
