Author: Yury Plyakhin Date: 2026-05-29T08:08:27-07:00 New Revision: fa6f7426a39977661a100547690c30e5a4b3691e
URL: https://github.com/llvm/llvm-project/commit/fa6f7426a39977661a100547690c30e5a4b3691e DIFF: https://github.com/llvm/llvm-project/commit/fa6f7426a39977661a100547690c30e5a4b3691e.diff LOG: [clang-sycl-linker] Decouple link step from SYCL; adopt -L / --bc-library options (#199777) The bitcode link phase in `clang-sycl-linker` is not SYCL-specific — it links input bitcode, then links library bitcode with `LinkOnlyNeeded`. This PR makes the implementation reflect that and aligns the option surface with `clang-nvlink-wrapper`. - Rename `linkDeviceCode` / `getSYCLDeviceLibs` and strip SYCL/device wording from verbose output, errors, and doc-comments. The `runSYCLLink` doc-comment is expanded to describe the full link → split → codegen → AOT → pack pipeline it actually drives. - Replace `--device-libs=a.bc,b.bc` with repeatable `--bc-library`, resolved against `-L` search paths. `-L` becomes the canonical form; `--library-path` / `--library-path=` are kept as hidden aliases and now accumulate (previously last-wins). - Consolidate the four `Inputs/SYCL/*.ll` fixtures and `link-device-code.test` into one `split-file`-based `link.ll`; refresh affected check prefixes. New checks cover `-L` joined form, multi-`-L` fall-through, and first-match-wins ordering. - ClangSYCLLinker.rst got a substantive refresh beyond doc-comments: intro paragraphs rewritten, help-text block updated, dropped options (-g, -is-windows-msvc-env, -llvm-spirv-options=, --llvm-spirv-path=) and added options (--module-split-mode, --ocloc-options, --opencl-aot-options) so the doc matches current .td reality. Assisted by claude. Added: clang/test/OffloadTools/clang-sycl-linker/link.ll Modified: clang/docs/ClangSYCLLinker.rst clang/test/Driver/sycl-link-spirv-target.cpp clang/test/OffloadTools/clang-sycl-linker/basic.ll clang/test/OffloadTools/clang-sycl-linker/split-mode.ll clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp clang/tools/clang-sycl-linker/SYCLLinkOpts.td Removed: clang/test/Driver/Inputs/SYCL/bar.ll clang/test/Driver/Inputs/SYCL/baz.ll clang/test/Driver/Inputs/SYCL/foo.ll clang/test/Driver/Inputs/SYCL/libLLVMSYCL.ll clang/test/Driver/link-device-code.test ################################################################################ diff --git a/clang/docs/ClangSYCLLinker.rst b/clang/docs/ClangSYCLLinker.rst index c1a794a2f65f6..c28c9fefaace3 100644 --- a/clang/docs/ClangSYCLLinker.rst +++ b/clang/docs/ClangSYCLLinker.rst @@ -12,9 +12,8 @@ Introduction This tool works as a wrapper around the SYCL device code linking process. The purpose of this tool is to provide an interface to link SYCL device bitcode -in LLVM IR format, SYCL device bitcode in SPIR-V IR format, and native binary -objects, and then use the SPIR-V LLVM Translator tool on fully linked device -objects to produce the final output. +in LLVM IR format together with LLVM bitcode libraries, and then run SPIR-V +code generation on the fully linked device code to produce the final output. After the linking stage, the fully linked device code in LLVM IR format may undergo several SYCL-specific finalization steps before the SPIR-V code generation step. @@ -23,49 +22,44 @@ compilation is the process of invoking the back-end at compile time to produce the final binary, as opposed to just-in-time (JIT) compilation when final code generation is deferred until application runtime. -Device code linking for SYCL offloading has several known quirks that -make it diff icult to use in a unified offloading setting. Two of the primary -issues are: -1. Several finalization steps are required to be run on the fully linked LLVM +Device code linking for SYCL offloading has known quirks that make it + diff icult to use in a unified offloading setting. The primary issue is that +several finalization steps are required to be run on the fully linked LLVM IR bitcode to guarantee conformance to SYCL standards. This step is unique to the SYCL offloading compilation flow. -2. The SPIR-V LLVM Translator tool is an external tool and hence SPIR-V IR code -generation cannot be done as part of LTO. This limitation can be lifted once -the SPIR-V backend is available as a viable LLVM backend. -This tool has been proposed to work around these issues. +This tool has been proposed to work around this issue. Usage ===== This tool can be used with the following options. Several of these options will -be passed down to downstream tools like 'llvm-link', 'llvm-spirv', etc. +be passed down to downstream AOT compilation tools like 'ocloc' and 'opencl-aot'. .. code-block:: console OVERVIEW: A utility that wraps around the SYCL device code linking process. - This enables linking and code generation for SPIR-V JIT targets and AOT - targets. + This enables LLVM IR linking, post-linking and code generation for SPIR-V + JIT and AOT targets. - USAGE: clang-sycl-linker [options] + USAGE: clang-sycl-linker [options] <input bitcode files> OPTIONS: --arch <value> Specify the name of the target architecture. --dry-run Print generated commands without running. - -g Specify that this was a debug compile. -help-hidden Display all available options -help Display available options (--help-hidden for more) - --library-path=<dir> Set the library path for SYCL device libraries - --device-libs=<value> A comma separated list of device libraries that are linked during the device link + -L <dir> Add <dir> to the library search path + --bc-library <name> Add LLVM bitcode library <name> (with extension) to the link. A relative <name> is resolved against -L paths; an absolute path is taken as-is. + --module-split-mode=<mode> Module split mode: 'source' (default), 'kernel', or 'none' + --ocloc-options=<value> Options passed to ocloc for Intel GPU AOT compilation + --opencl-aot-options=<value> Options passed to opencl-aot for Intel CPU AOT compilation -o <path> Path to file to write output --save-temps Save intermediate results --triple <value> Specify the target triple. --version Display the version number and exit -v Print verbose information -spirv-dump-device-code=<dir> Directory to dump SPIR-V IR code into - -is-windows-msvc-env Specify if we are compiling under windows environment - -llvm-spirv-options=<value> Pass options to llvm-spirv tool - --llvm-spirv-path=<dir> Set the system llvm-spirv path Example ======= @@ -79,4 +73,4 @@ generate the final executable. .. code-block:: console - clang-sycl-linker --triple spirv64 --arch native input.bc + clang-sycl-linker --triple spirv64 --arch bmg_g21 input.bc diff --git a/clang/test/Driver/Inputs/SYCL/bar.ll b/clang/test/Driver/Inputs/SYCL/bar.ll deleted file mode 100644 index 9f86b8aa54827..0000000000000 --- a/clang/test/Driver/Inputs/SYCL/bar.ll +++ /dev/null @@ -1,8 +0,0 @@ -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @bar_func1(i32 %a, i32 %b) { -entry: - %res = add nsw i32 %b, %a - ret i32 %res -} diff --git a/clang/test/Driver/Inputs/SYCL/baz.ll b/clang/test/Driver/Inputs/SYCL/baz.ll deleted file mode 100644 index 1fd7e44881981..0000000000000 --- a/clang/test/Driver/Inputs/SYCL/baz.ll +++ /dev/null @@ -1,16 +0,0 @@ -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @bar_func1(i32 %a, i32 %b) { -entry: - %mul = shl nsw i32 %a, 1 - %res = add nsw i32 %mul, %b - ret i32 %res -} - -define spir_func i32 @baz_func1(i32 %a) { -entry: - %add = add nsw i32 %a, 5 - %res = tail call spir_func i32 @bar_func1(i32 %a, i32 %add) - ret i32 %res -} diff --git a/clang/test/Driver/Inputs/SYCL/foo.ll b/clang/test/Driver/Inputs/SYCL/foo.ll deleted file mode 100644 index fbfd8c53bff9c..0000000000000 --- a/clang/test/Driver/Inputs/SYCL/foo.ll +++ /dev/null @@ -1,20 +0,0 @@ -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @foo_func1(i32 %a, i32 %b) { -entry: - %call = tail call spir_func i32 @addFive(i32 %b) - %res = tail call spir_func i32 @bar_func1(i32 %a, i32 %call) - ret i32 %res -} - -declare spir_func i32 @bar_func1(i32, i32) - -declare spir_func i32 @addFive(i32) - -define spir_func i32 @foo_func2(i32 %c, i32 %d, i32 %e) { -entry: - %call = tail call spir_func i32 @foo_func1(i32 %c, i32 %d) - %res = mul nsw i32 %call, %e - ret i32 %res -} diff --git a/clang/test/Driver/Inputs/SYCL/libLLVMSYCL.ll b/clang/test/Driver/Inputs/SYCL/libLLVMSYCL.ll deleted file mode 100644 index b161bde3b0c1e..0000000000000 --- a/clang/test/Driver/Inputs/SYCL/libLLVMSYCL.ll +++ /dev/null @@ -1,14 +0,0 @@ -target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" -target triple = "spirv64" - -define spir_func i32 @addFive(i32 %a) { -entry: - %res = add nsw i32 %a, 5 - ret i32 %res -} - -define spir_func i32 @unusedFunc(i32 %a) { -entry: - %res = mul nsw i32 %a, 5 - ret i32 %res -} diff --git a/clang/test/Driver/link-device-code.test b/clang/test/Driver/link-device-code.test deleted file mode 100644 index eb75eaf1fe5c2..0000000000000 --- a/clang/test/Driver/link-device-code.test +++ /dev/null @@ -1,25 +0,0 @@ -# REQUIRES: spirv-registered-target - -# RUN: llvm-as %S/Inputs/SYCL/foo.ll -o %t.foo.bc -# RUN: llvm-as %S/Inputs/SYCL/bar.ll -o %t.bar.bc -# RUN: llvm-as %S/Inputs/SYCL/baz.ll -o %t.baz.bc -# RUN: llvm-as %S/Inputs/SYCL/libLLVMSYCL.ll -o %t.libLLVMSYCL.bc -# RUN: clang-sycl-linker %t.foo.bc %t.bar.bc -triple=spirv64 --dry-run -o a.spv --print-linked-module 2>&1 | FileCheck %s --check-prefix=CHECK-SIMPLE - -# RUN: not clang-sycl-linker %t.bar.bc %t.baz.bc -triple=spirv64 --dry-run -o a.spv --print-linked-module 2>&1 | FileCheck %s --check-prefix=CHECK-MULTIPLE-DEFS - -# RUN: clang-sycl-linker %t.foo.bc %t.bar.bc -device-libs=%t.libLLVMSYCL.bc -library-path= -triple=spirv64 --dry-run -o a.spv --print-linked-module 2>&1 | FileCheck %s --check-prefix=CHECK-DEVICE-LIB - -; CHECK-SIMPLE: define {{.*}}foo_func1{{.*}} -; CHECK-SIMPLE: define {{.*}}foo_func2{{.*}} -; CHECK-SIMPLE: define {{.*}}bar_func1{{.*}} -; CHECK-SIMPLE-NOT: define {{.*}}addFive{{.*}} -; CHECK-SIMPLE-NOT: define {{.*}}unusedFunc{{.*}} - -; CHECK-MULTIPLE-DEFS: error: Linking globals named {{.*}}bar_func1{{.*}} symbol multiply defined! - -; CHECK-DEVICE-LIB: define {{.*}}foo_func1{{.*}} -; CHECK-DEVICE-LIB: define {{.*}}foo_func2{{.*}} -; CHECK-DEVICE-LIB: define {{.*}}bar_func1{{.*}} -; CHECK-DEVICE-LIB: define {{.*}}addFive{{.*}} -; CHECK-DEVICE-LIB-NOT: define {{.*}}unusedFunc{{.*}} diff --git a/clang/test/Driver/sycl-link-spirv-target.cpp b/clang/test/Driver/sycl-link-spirv-target.cpp index d27c59d59b637..885621140a679 100644 --- a/clang/test/Driver/sycl-link-spirv-target.cpp +++ b/clang/test/Driver/sycl-link-spirv-target.cpp @@ -5,10 +5,9 @@ // // Test that -Xlinker options are being passed to clang-sycl-linker. // RUN: touch %t.bc -// RUN: %clangxx -### --target=spirv64 --sycl-link -Xlinker -triple=spirv64 -Xlinker --library-path=/tmp \ -// RUN: -Xlinker --device-libs=lib1.bc,lib2.bc %t.bc 2>&1 \ +// RUN: %clangxx -### --target=spirv64 --sycl-link -Xlinker --test-arg-1 -Xlinker --test-arg-2=value1,value2 %t.bc 2>&1 \ // RUN: | FileCheck %s -check-prefix=XLINKEROPTS -// XLINKEROPTS: "{{.*}}clang-sycl-linker{{.*}}" "-triple=spirv64" "--library-path=/tmp" "--device-libs=lib1.bc,lib2.bc" "{{.*}}.bc" "-o" "a.out" +// XLINKEROPTS: "{{.*}}clang-sycl-linker{{.*}}" "--test-arg-1" "--test-arg-2=value1,value2" "{{.*}}.bc" "-o" "a.out" // Test that -v is forwarded to clang-sycl-linker when --sycl-link is used. // RUN: touch %t.bc diff --git a/clang/test/OffloadTools/clang-sycl-linker/basic.ll b/clang/test/OffloadTools/clang-sycl-linker/basic.ll index 5bb15b3ca9b4f..9a338ae5215d3 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/basic.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/basic.ll @@ -9,7 +9,7 @@ ; Test the dry run of a simple case to link two input files. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc -o %t/spirv.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SIMPLE-FO -; SIMPLE-FO: sycl-device-link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; SIMPLE-FO: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SIMPLE-FO-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: {{.*}}_0.spv ; SIMPLE-FO-NOT: {{.+}} ; @@ -21,11 +21,29 @@ ; RUN: mkdir -p %t/libs ; RUN: touch %t/libs/lib1.bc ; RUN: touch %t/libs/lib2.bc -; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc --library-path=%t/libs --device-libs=lib1.bc,lib2.bc -o a.spv 2>&1 \ +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc %t/input2.bc --library-path=%t/libs --bc-library lib1.bc --bc-library lib2.bc -o a.spv 2>&1 \ ; RUN: | FileCheck %s --check-prefix=DEVLIBS -; DEVLIBS: sycl-device-link: inputs: {{.*}}.bc libfiles: {{.*}}lib1.bc, {{.*}}lib2.bc output: [[LLVMLINKOUT:.*]].bc +; DEVLIBS: link: inputs: {{.*}}.bc libfiles: {{.*}}lib1.bc, {{.*}}lib2.bc output: [[LLVMLINKOUT:.*]].bc ; DEVLIBS-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: a_0.spv ; +; Test -L short form (joined) and --bc-library= joined form. +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L%t/libs --bc-library=lib1.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=DEVLIBS-SHORT +; DEVLIBS-SHORT: link: inputs: {{.*}}.bc libfiles: {{.*}}libs{{[/\\]}}lib1.bc output: {{.*}}.bc +; +; Test that search continues past the first -L when the library is not found there. lib1.bc exists only in %t/libs (the second -L). +; RUN: mkdir -p %t/empty +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L %t/empty -L %t/libs --bc-library lib1.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=DEVLIBS-FALLTHROUGH +; DEVLIBS-FALLTHROUGH: link: inputs: {{.*}}.bc libfiles: {{.*}}libs{{[/\\]}}lib1.bc output: {{.*}}.bc +; +; Test that -L paths are searched in order: when the same name exists in multiple -L dirs, the first one wins. +; RUN: mkdir -p %t/libs2 +; RUN: touch %t/libs/shadow.bc %t/libs2/shadow.bc +; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t/input1.bc -L %t/libs2 -L %t/libs --bc-library shadow.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=DEVLIBS-ORDER +; DEVLIBS-ORDER: link: inputs: {{.*}}.bc libfiles: {{.*}}libs2{{[/\\]}}shadow.bc output: {{.*}}.bc +; ; Test a simple case with a random file (not bitcode) as input. ; RUN: touch %t/dummy.o ; RUN: not clang-sycl-linker %t/dummy.o -o a.spv 2>&1 \ @@ -33,18 +51,28 @@ ; FILETYPEERROR: Unsupported file type ; ; Test to see if device library related errors are emitted. -; RUN: not clang-sycl-linker --dry-run %t/input1.bc %t/input2.bc --library-path=%t/libs --device-libs= -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=DEVLIBSERR1 -; DEVLIBSERR1: Number of device library files cannot be zero -; RUN: not clang-sycl-linker --dry-run %t/input1.bc %t/input2.bc --library-path=%t/libs --device-libs=lib1.bc,lib2.bc,lib3.bc -o a.spv 2>&1 \ -; RUN: | FileCheck %s --check-prefix=DEVLIBSERR2 -; DEVLIBSERR2: '{{.*}}lib3.bc' SYCL device library file is not found +; RUN: not clang-sycl-linker --dry-run %t/input1.bc %t/input2.bc --library-path=%t/libs --bc-library lib1.bc --bc-library lib2.bc --bc-library lib3.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=DEVLIBSERR +; DEVLIBSERR: '{{.*}}lib3.bc' library file not found +; +; Test that there is no implicit CWD search: a bare bitcode name without any -L +; must fail to resolve, even if a same-named file exists in the CWD. +; RUN: cd %t && not clang-sycl-linker --dry-run input1.bc --bc-library input1.bc -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=NO-CWD-SEARCH +; NO-CWD-SEARCH: 'input1.bc' library file not found +; +; Test that a directory matching the requested name is not accepted as a library: +; %t/libs is a directory created above; resolving --bc-library libs against -L %t +; would otherwise pick it up and fail later with a confusing bitcode-reader error. +; RUN: not clang-sycl-linker --dry-run %t/input1.bc -L %t --bc-library libs -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=NO-DIR-AS-LIB +; NO-DIR-AS-LIB: 'libs' library file not found ; ; Test AOT compilation for an Intel GPU. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none -arch=bmg_g21 %t/input1.bc %t/input2.bc -o %t/aot-gpu.out 2>&1 \ ; RUN: --ocloc-options="-a -b" \ ; RUN: | FileCheck %s --check-prefix=AOT-INTEL-GPU -; AOT-INTEL-GPU: sycl-device-link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; AOT-INTEL-GPU: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; AOT-INTEL-GPU-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: [[SPIRVTRANSLATIONOUT:.*]]_0.spv ; AOT-INTEL-GPU-NEXT: "{{.*}}ocloc{{.*}}" {{.*}}-device bmg_g21 -a -b {{.*}}-output [[SPIRVTRANSLATIONOUT]]_0.out -file [[SPIRVTRANSLATIONOUT]]_0.spv ; @@ -56,7 +84,7 @@ ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none -arch=graniterapids %t/input1.bc %t/input2.bc -o %t/aot-cpu.out 2>&1 \ ; RUN: --opencl-aot-options="-a -b" \ ; RUN: | FileCheck %s --check-prefix=AOT-INTEL-CPU -; AOT-INTEL-CPU: sycl-device-link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; AOT-INTEL-CPU: link: inputs: {{.*}}.bc, {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; AOT-INTEL-CPU-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: [[SPIRVTRANSLATIONOUT:.*]]_0.spv ; AOT-INTEL-CPU-NEXT: "{{.*}}opencl-aot{{.*}}" {{.*}}--device=cpu -a -b {{.*}}-o [[SPIRVTRANSLATIONOUT]]_0.out [[SPIRVTRANSLATIONOUT]]_0.spv ; diff --git a/clang/test/OffloadTools/clang-sycl-linker/link.ll b/clang/test/OffloadTools/clang-sycl-linker/link.ll new file mode 100644 index 0000000000000..4114f0a3f3fb1 --- /dev/null +++ b/clang/test/OffloadTools/clang-sycl-linker/link.ll @@ -0,0 +1,95 @@ +; Tests clang-sycl-linker linking behavior. +; +; REQUIRES: spirv-registered-target +; +; RUN: rm -rf %t && split-file %s %t +; RUN: llvm-as %t/foo.ll -o %t/foo.bc +; RUN: llvm-as %t/bar.ll -o %t/bar.bc +; RUN: llvm-as %t/baz.ll -o %t/baz.bc +; RUN: llvm-as %t/libfoo.ll -o %t/libfoo.bc +; +; Test linking two input files. +; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --dry-run -o a.spv --print-linked-module 2>&1 \ +; RUN: | FileCheck %s --check-prefix=CHECK-SIMPLE +; CHECK-SIMPLE: define {{.*}}foo_func1{{.*}} +; CHECK-SIMPLE: define {{.*}}foo_func2{{.*}} +; CHECK-SIMPLE: define {{.*}}bar_func1{{.*}} +; CHECK-SIMPLE-NOT: define {{.*}}addFive{{.*}} +; CHECK-SIMPLE-NOT: define {{.*}}unusedFunc{{.*}} +; +; Test that multiply defined symbols are reported as errors. +; RUN: not clang-sycl-linker %t/bar.bc %t/baz.bc --dry-run -o a.spv 2>&1 \ +; RUN: | FileCheck %s --check-prefix=CHECK-MULTIPLE-DEFS +; CHECK-MULTIPLE-DEFS: error: Linking globals named {{.*}}bar_func1{{.*}} symbol multiply defined! +; +; Test linking with a BC library file resolved through -L. +; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --bc-library libfoo.bc -L %t --dry-run -o a.spv --print-linked-module 2>&1 \ +; RUN: | FileCheck %s --check-prefix=CHECK-DEVICE-LIB +; CHECK-DEVICE-LIB: define {{.*}}foo_func1{{.*}} +; CHECK-DEVICE-LIB: define {{.*}}foo_func2{{.*}} +; CHECK-DEVICE-LIB: define {{.*}}bar_func1{{.*}} +; CHECK-DEVICE-LIB: define {{.*}}addFive{{.*}} +; CHECK-DEVICE-LIB-NOT: define {{.*}}unusedFunc{{.*}} +; +; Test that an absolute path to --bc-library is taken as-is, with no -L required. +; RUN: clang-sycl-linker %t/foo.bc %t/bar.bc --bc-library %t/libfoo.bc --dry-run -o a.spv --print-linked-module 2>&1 \ +; RUN: | FileCheck %s --check-prefix=CHECK-DEVICE-LIB + +;--- foo.ll +target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" +target triple = "spirv64" + +define spir_func i32 @foo_func1(i32 %a, i32 %b) { +entry: + %call = tail call spir_func i32 @addFive(i32 %b) + %res = tail call spir_func i32 @bar_func1(i32 %a, i32 %call) + ret i32 %res +} + +declare spir_func i32 @bar_func1(i32, i32) + +declare spir_func i32 @addFive(i32) + +define spir_func i32 @foo_func2(i32 %c, i32 %d) { +entry: + %res = add nsw i32 %c, %d + ret i32 %res +} + +;--- bar.ll +target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" +target triple = "spirv64" + +define spir_func i32 @bar_func1(i32 %a, i32 %b) { +entry: + %res = add nsw i32 %b, %a + ret i32 %res +} + +;--- baz.ll +target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" +target triple = "spirv64" + +define spir_func i32 @bar_func1(i32 %a, i32 %b) { +entry: + %mul = shl nsw i32 %a, 1 + %res = add nsw i32 %mul, %b + ret i32 %res +} + + +;--- libfoo.ll +target datalayout = "e-i64:64-v16:16-v24:32-v32:32-v48:64-v96:128-v192:256-v256:256-v512:512-v1024:1024-n8:16:32:64-G1" +target triple = "spirv64" + +define spir_func i32 @addFive(i32 %a) { +entry: + %res = add nsw i32 %a, 5 + ret i32 %res +} + +define spir_func i32 @unusedFunc(i32 %a) { +entry: + %res = mul nsw i32 %a, 5 + ret i32 %res +} diff --git a/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll b/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll index ab6ccfaf64e70..56a6b3e082550 100644 --- a/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll +++ b/clang/test/OffloadTools/clang-sycl-linker/split-mode.ll @@ -12,14 +12,14 @@ ; Test the split mode ("none"): kernels from diff erent TUs are not split into separate images. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=none %t.bc -o %t-none.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-NONE -; SPLIT-NONE: sycl-device-link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; SPLIT-NONE: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-NONE-NEXT: LLVM backend: input: [[LLVMLINKOUT]].bc, output: {{.*}}_0.spv ; SPLIT-NONE-NOT: {{.+}} ; ; Test the split mode ("kernel"): each SPIR_KERNEL function produces its own device image. ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=kernel %t.bc -o %t-split-kernel.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-KERNEL -; SPLIT-KERNEL: sycl-device-link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; SPLIT-KERNEL: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-KERNEL-NEXT: sycl-module-split: input: [[LLVMLINKOUT]].bc, mode: kernel ; SPLIT-KERNEL-NEXT: [[SPLIT0:.*]].bc [kernel_c ] ; SPLIT-KERNEL-NEXT: [[SPLIT1:.*]].bc [kernel_b ] @@ -39,7 +39,7 @@ ; Test per-TU split ('source' explicitly provided) ; RUN: clang-sycl-linker --dry-run -v --module-split-mode=source %t.bc -o %t-src.out 2>&1 \ ; RUN: | FileCheck %s --check-prefix=SPLIT-SRC -; SPLIT-SRC: sycl-device-link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc +; SPLIT-SRC: link: inputs: {{.*}}.bc libfiles: output: [[LLVMLINKOUT:.*]].bc ; SPLIT-SRC-NEXT: sycl-module-split: input: [[LLVMLINKOUT]].bc, mode: source ; SPLIT-SRC-NEXT: [[S0:.*]].bc [kernel_b kernel_c ] ; SPLIT-SRC-NEXT: [[S1:.*]].bc [kernel_a ] diff --git a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp index 88a09d0a3ecc7..fbaba07b7296e 100644 --- a/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp +++ b/clang/tools/clang-sycl-linker/ClangSYCLLinker.cpp @@ -8,7 +8,7 @@ // // This tool executes a sequence of steps required to link device code in SYCL // device images. SYCL device code linking requires a complex sequence of steps -// that include linking of llvm bitcode files, linking device library files +// that include linking of llvm bitcode files, linking bitcode library files // with the fully linked source bitcode file(s), running several SYCL specific // post-link steps on the fully linked bitcode file(s), and finally generating // target-specific device code. @@ -184,7 +184,7 @@ Error executeCommands(StringRef ExecutablePath, ArrayRef<StringRef> Args) { } Expected<SmallVector<std::string>> getInput(const ArgList &Args) { - // Collect all input bitcode files to be passed to the device linking stage. + // Collect all input bitcode files to be passed to the linking stage. SmallVector<std::string> BitcodeFiles; for (const opt::Arg *Arg : Args.filtered(OPT_INPUT)) { std::optional<std::string> Filename = std::string(Arg->getValue()); @@ -217,29 +217,47 @@ Expected<std::unique_ptr<Module>> getBitcodeModule(StringRef File, return createStringError(Err.getMessage()); } -/// Gather all SYCL device library files that will be linked with input device -/// files. -/// The list of files and its location are passed from driver. -Expected<SmallVector<std::string>> getSYCLDeviceLibs(const ArgList &Args) { - SmallVector<std::string> DeviceLibFiles; - StringRef LibraryPath; - if (Arg *A = Args.getLastArg(OPT_library_path_EQ)) - LibraryPath = A->getValue(); - if (Arg *A = Args.getLastArg(OPT_device_libs_EQ)) { - if (A->getValues().size() == 0) - return createStringError( - "Number of device library files cannot be zero."); - for (StringRef Val : A->getValues()) { - SmallString<128> LibName(LibraryPath); - llvm::sys::path::append(LibName, Val); - if (llvm::sys::fs::exists(LibName)) - DeviceLibFiles.push_back(std::string(LibName)); - else - return createStringError("'" + LibName + - "' SYCL device library file is not found."); - } +std::optional<std::string> findFile(StringRef Dir, const Twine &Name) { + SmallString<128> Path(Dir); + llvm::sys::path::append(Path, Name); + if (sys::fs::exists(Path) && !sys::fs::is_directory(Path)) + return std::string(Path); + return std::nullopt; +} + +std::optional<std::string> searchLibrary(StringRef Name, + ArrayRef<StringRef> SearchPaths) { + // An absolute path is taken as-is; -L paths are only consulted for relative + // names. + if (sys::path::is_absolute(Name)) { + if (sys::fs::exists(Name) && !sys::fs::is_directory(Name)) + return std::string(Name); + return std::nullopt; } - return DeviceLibFiles; + for (StringRef Dir : SearchPaths) + if (std::optional<std::string> File = findFile(Dir, Name)) + return File; + return std::nullopt; +} + +/// Gather all library files. The list of files and its location are passed from +/// driver. +Expected<SmallVector<std::string>> getBCLibraryNames(const ArgList &Args) { + SmallVector<StringRef> LibraryPaths; + for (const opt::Arg *Arg : Args.filtered(OPT_library_path)) + LibraryPaths.push_back(Arg->getValue()); + + SmallVector<std::string> LibraryFiles; + for (const opt::Arg *Arg : Args.filtered(OPT_bc_library)) { + std::optional<std::string> LibName = + searchLibrary(Arg->getValue(), LibraryPaths); + if (!LibName) + return createStringError("'" + Twine(Arg->getValue()) + + "' library file not found"); + LibraryFiles.push_back(std::move(*LibName)); + } + + return LibraryFiles; } struct LinkResult { @@ -248,32 +266,26 @@ struct LinkResult { llvm::Triple TargetTriple; }; -/// TODO: There is nothing SYCL specific in linkDeviceCode function. Remove: -/// 1. Mentions of SYCL. -/// 2. "fat" file formats. This linker shouldn't refer to input files as -/// "device code"/"device libraries". - /// Following tasks are performed: /// 1. Resolve the target triple: use --triple= when given, otherwise take the /// first input that supplies a triple as canonical. Issue an error if any /// triple inputs disagree. -/// 2. Link all SYCL device bitcode images into one image. Device linking is -/// performed using the linkInModule API. -/// 3. Gather all SYCL device library bitcode images. +/// 2. Link all input bitcode images into one image using the linkInModule API. +/// 3. Gather all library bitcode images. /// 4. Link all the images gathered in Step 3 with the output of Step 2 using /// linkInModule API. LinkOnlyNeeded flag is used. -Expected<LinkResult> linkDeviceCode(ArrayRef<std::string> InputFiles, - const ArgList &Args, LLVMContext &C) { - llvm::TimeTraceScope TimeScope("SYCL link device code"); +Expected<LinkResult> linkInputs(ArrayRef<std::string> InputFiles, + const ArgList &Args, LLVMContext &C) { + llvm::TimeTraceScope TimeScope("Link code"); assert(InputFiles.size() && "No inputs to link"); - // Get all SYCL device library files, if any. - auto SYCLDeviceLibFiles = getSYCLDeviceLibs(Args); - if (!SYCLDeviceLibFiles) - return SYCLDeviceLibFiles.takeError(); + // Get all library files. + Expected<SmallVector<std::string>> BCLibFiles = getBCLibraryNames(Args); + if (!BCLibFiles) + return BCLibFiles.takeError(); - // Create a new file to write the linked device file to. + // Create a new file to write the linked file to. auto BitcodeOutput = createTempFile(Args, sys::path::filename(OutputFile), "bc"); if (!BitcodeOutput) @@ -281,17 +293,16 @@ Expected<LinkResult> linkDeviceCode(ArrayRef<std::string> InputFiles, if (Verbose || DryRun) { std::string Inputs = llvm::join(InputFiles.begin(), InputFiles.end(), ", "); - std::string LibInputs = llvm::join((*SYCLDeviceLibFiles).begin(), - (*SYCLDeviceLibFiles).end(), ", "); - errs() << formatv( - "sycl-device-link: inputs: {0} libfiles: {1} output: {2}\n", Inputs, - LibInputs, *BitcodeOutput); + std::string LibInputs = + llvm::join((*BCLibFiles).begin(), (*BCLibFiles).end(), ", "); + errs() << formatv("link: inputs: {0} libfiles: {1} output: {2}\n", Inputs, + LibInputs, *BitcodeOutput); } - // Link SYCL device input files. Resolve the target triple. + // Link input files. Resolve the target triple. llvm::Triple TargetTriple(Args.getLastArgValue(OPT_triple_EQ)); StringRef TripleSource = TargetTriple.empty() ? "" : "--triple="; - auto LinkerOutput = std::make_unique<Module>("sycl-device-link", C); + auto LinkerOutput = std::make_unique<Module>("linker-output", C); Linker L(*LinkerOutput); for (auto &File : InputFiles) { @@ -319,8 +330,8 @@ Expected<LinkResult> linkDeviceCode(ArrayRef<std::string> InputFiles, return createStringError( "Target triple must be specified or inferable from inputs"); - // Link in SYCL device library files. - for (auto &File : *SYCLDeviceLibFiles) { + // Link in library files. + for (auto &File : *BCLibFiles) { auto LibMod = getBitcodeModule(File, C); if (!LibMod) return LibMod.takeError(); @@ -417,7 +428,7 @@ static Error runCodeGen(StringRef File, const llvm::Triple &TargetTriple, /// \param OutputFile The output file name. /// \param Args Encompasses all arguments required for linking and wrapping /// device code and will be parsed to generate options required to be passed -/// into the SYCL AOT compilation step. +/// into the AOT compilation step. static Error runAOTCompileIntelCPU(StringRef InputFile, StringRef OutputFile, const ArgList &Args) { SmallVector<StringRef, 8> CmdArgs; @@ -445,7 +456,7 @@ static Error runAOTCompileIntelCPU(StringRef InputFile, StringRef OutputFile, /// \param OutputFile The output file name. /// \param Args Encompasses all arguments required for linking and wrapping /// device code and will be parsed to generate options required to be passed -/// into the SYCL AOT compilation step. +/// into the AOT compilation step. static Error runAOTCompileIntelGPU(StringRef InputFile, StringRef OutputFile, const ArgList &Args) { SmallVector<StringRef, 8> CmdArgs; @@ -481,7 +492,7 @@ static Error runAOTCompileIntelGPU(StringRef InputFile, StringRef OutputFile, /// \param OutputFile The output file name. /// \param Args Encompasses all arguments required for linking and wrapping /// device code and will be parsed to generate options required to be passed -/// into the SYCL AOT compilation step. +/// into the AOT compilation step. static Error runAOTCompile(StringRef InputFile, StringRef OutputFile, const ArgList &Args) { StringRef Arch = Args.getLastArgValue(OPT_arch_EQ); @@ -653,15 +664,20 @@ static bool canSkipModuleSplit(IRSplitMode Mode, const Module &M, } /// Performs the following steps: -/// 1. Link input device code (user code and SYCL device library code). -/// 2. Run SPIR-V code generation. +/// 1. Link all input bitcode files together with library files. +/// 2. Optionally split the linked module according to the requested +/// IRSplitMode. +/// 3. Run SPIR-V code generation on each (split) module. +/// 4. Optionally run AOT compilation when targeting an Intel HW arch. +/// 5. Pack the resulting images into a single OffloadBinary written to the +/// output file. Error runSYCLLink(ArrayRef<std::string> Files, const ArgList &Args) { - llvm::TimeTraceScope TimeScope("SYCL device linking"); + llvm::TimeTraceScope TimeScope("SYCL linking"); LLVMContext C; - // Link all input bitcode files and SYCL device library files, if any. - Expected<LinkResult> LinkedOrErr = linkDeviceCode(Files, Args, C); + // Link all input bitcode files and library files. + Expected<LinkResult> LinkedOrErr = linkInputs(Files, Args, C); if (!LinkedOrErr) return LinkedOrErr.takeError(); LinkResult &Result = *LinkedOrErr; @@ -777,11 +793,10 @@ int main(int argc, char **argv) { if (Args.hasArg(OPT_help) || Args.hasArg(OPT_help_hidden)) { Tbl.printHelp( - outs(), "clang-sycl-linker [options] <options to sycl link steps>", - "A utility that wraps around several steps required to link SYCL " - "device files.\n" + outs(), "clang-sycl-linker [options] <input bitcode files>", + "A utility that wraps around the SYCL device code linking process.\n" "This enables LLVM IR linking, post-linking and code generation for " - "SYCL targets.", + "SPIR-V JIT and AOT targets.", Args.hasArg(OPT_help_hidden), Args.hasArg(OPT_help_hidden)); return EXIT_SUCCESS; } diff --git a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td index 171915c29bd93..e00e63aa1767d 100644 --- a/clang/tools/clang-sycl-linker/SYCLLinkOpts.td +++ b/clang/tools/clang-sycl-linker/SYCLLinkOpts.td @@ -17,12 +17,19 @@ def o : JoinedOrSeparate<["-"], "o">, MetaVarName<"<path>">, def output : Separate<["--"], "output-file">, Alias<o>, Flags<[HelpHidden]>, HelpText<"Alias for -o">; -def library_path_EQ : Joined<["--", "-"], "library-path=">, - Flags<[HelpHidden]>, HelpText<"Add <dir> to the library search path">; +def library_path : JoinedOrSeparate<["-"], "L">, MetaVarName<"<dir>">, + HelpText<"Add <dir> to the library search path">; +def library_path_S : Separate<["--", "-"], "library-path">, Flags<[HelpHidden]>, + Alias<library_path>; +def library_path_EQ : Joined<["--", "-"], "library-path=">, Flags<[HelpHidden]>, + Alias<library_path>; -def device_libs_EQ : CommaJoined<["--", "-"], "device-libs=">, - Flags<[LinkerOnlyOption]>, - HelpText<"A comma separated list of device libraries that are linked during the device link.">; +def bc_library : Separate<["--", "-"], "bc-library">, MetaVarName<"<name>">, + HelpText<"Add LLVM bitcode library <name> (with extension) to the link. A " + "relative <name> is resolved against -L paths; an absolute path is " + "taken as-is.">; +def bc_library_EQ : Joined<["--", "-"], "bc-library=">, Flags<[HelpHidden]>, + Alias<bc_library>; def arch_EQ : Joined<["--", "-"], "arch=">, Flags<[LinkerOnlyOption]>, _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
