llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT--> @llvm/pr-subscribers-backend-nvptx Author: Alex Duran (adurang) <details> <summary>Changes</summary> Currently OpenMP doesn't generate a Kernel Enviroment for host kernels or ompx_bare kernels. This causes the runtime to have special handling for this cases. This patch ensures that a Kernel Environment is always generated which will allow to simplify the RTL handling. Done mostly by Claude. --- Patch is 13.70 MiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/223772.diff 136 Files Affected: - (modified) clang/lib/CodeGen/CGOpenMPRuntime.cpp (+20-3) - (modified) clang/lib/CodeGen/CGOpenMPRuntime.h (+6) - (modified) clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp (+15) - (modified) clang/lib/CodeGen/CGOpenMPRuntimeGPU.h (+5) - (modified) clang/test/OpenMP/bug60602.cpp (+12-12) - (modified) clang/test/OpenMP/distribute_codegen.cpp (+180-180) - (modified) clang/test/OpenMP/distribute_firstprivate_codegen.cpp (+93-93) - (modified) clang/test/OpenMP/distribute_lastprivate_codegen.cpp (+93-93) - (modified) clang/test/OpenMP/distribute_parallel_for_codegen.cpp (+693-693) - (modified) clang/test/OpenMP/distribute_parallel_for_firstprivate_codegen.cpp (+136-136) - (modified) clang/test/OpenMP/distribute_parallel_for_if_codegen.cpp (+79-79) - (modified) clang/test/OpenMP/distribute_parallel_for_lastprivate_codegen.cpp (+144-144) - (modified) clang/test/OpenMP/distribute_parallel_for_num_threads_codegen.cpp (+360-360) - (modified) clang/test/OpenMP/distribute_parallel_for_private_codegen.cpp (+88-88) - (modified) clang/test/OpenMP/distribute_parallel_for_proc_bind_codegen.cpp (+34-34) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_codegen.cpp (+1403-1403) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_firstprivate_codegen.cpp (+256-256) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_if_codegen.cpp (+390-390) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_lastprivate_codegen.cpp (+245-245) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_num_threads_codegen.cpp (+360-360) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_private_codegen.cpp (+178-178) - (modified) clang/test/OpenMP/distribute_parallel_for_simd_proc_bind_codegen.cpp (+100-100) - (modified) clang/test/OpenMP/distribute_private_codegen.cpp (+68-68) - (modified) clang/test/OpenMP/distribute_simd_codegen.cpp (+593-593) - (modified) clang/test/OpenMP/distribute_simd_firstprivate_codegen.cpp (+172-172) - (modified) clang/test/OpenMP/distribute_simd_lastprivate_codegen.cpp (+170-170) - (modified) clang/test/OpenMP/distribute_simd_private_codegen.cpp (+149-149) - (modified) clang/test/OpenMP/distribute_simd_reduction_codegen.cpp (+85-85) - (modified) clang/test/OpenMP/nvptx_lambda_capturing.cpp (+75-75) - (added) clang/test/OpenMP/nvptx_target_teams_ompx_bare_kernel_environment_codegen.cpp (+29) - (modified) clang/test/OpenMP/reduction_implicit_map.cpp (+67-67) - (modified) clang/test/OpenMP/target_default_codegen.cpp (+13-13) - (modified) clang/test/OpenMP/target_dyn_groupprivate_codegen.cpp (+138-138) - (modified) clang/test/OpenMP/target_map_member_expr_codegen.cpp (+1-1) - (modified) clang/test/OpenMP/target_ompx_dyn_cgroup_mem_codegen.cpp (+138-138) - (modified) clang/test/OpenMP/target_parallel_codegen.cpp (+56-56) - (modified) clang/test/OpenMP/target_parallel_for_codegen.cpp (+338-338) - (modified) clang/test/OpenMP/target_parallel_for_simd_codegen.cpp (+873-873) - (modified) clang/test/OpenMP/target_parallel_generic_loop_codegen-1.cpp (+65-65) - (modified) clang/test/OpenMP/target_parallel_num_threads_codegen.cpp (+8-8) - (modified) clang/test/OpenMP/target_parallel_num_threads_strict_codegen.cpp (+43-43) - (modified) clang/test/OpenMP/target_teams_codegen.cpp (+119-119) - (modified) clang/test/OpenMP/target_teams_distribute_codegen.cpp (+245-245) - (modified) clang/test/OpenMP/target_teams_distribute_collapse_codegen.cpp (+42-42) - (modified) clang/test/OpenMP/target_teams_distribute_dist_schedule_codegen.cpp (+149-149) - (modified) clang/test/OpenMP/target_teams_distribute_firstprivate_codegen.cpp (+46-46) - (modified) clang/test/OpenMP/target_teams_distribute_lastprivate_codegen.cpp (+79-79) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_codegen.cpp (+88-88) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_collapse_codegen.cpp (+68-68) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_dist_schedule_codegen.cpp (+200-200) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_firstprivate_codegen.cpp (+200-200) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_if_codegen.cpp (+94-94) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_lastprivate_codegen.cpp (+123-123) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_order_codegen.cpp (+14-14) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_private_codegen.cpp (+126-126) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_proc_bind_codegen.cpp (+34-34) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_reduction_codegen.cpp (+89-89) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_schedule_codegen.cpp (+796-796) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_codegen.cpp (+216-216) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_collapse_codegen.cpp (+182-182) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_dist_schedule_codegen.cpp (+482-482) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_firstprivate_codegen.cpp (+398-398) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_if_codegen.cpp (+394-394) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_if_openmp52_codegen.cpp (+572-572) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_lastprivate_codegen.cpp (+244-244) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_private_codegen.cpp (+284-284) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_proc_bind_codegen.cpp (+100-100) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_reduction_codegen.cpp (+165-165) - (modified) clang/test/OpenMP/target_teams_distribute_parallel_for_simd_schedule_codegen.cpp (+1413-1413) - (modified) clang/test/OpenMP/target_teams_distribute_private_codegen.cpp (+27-27) - (modified) clang/test/OpenMP/target_teams_distribute_reduction_codegen.cpp (+560-560) - (modified) clang/test/OpenMP/target_teams_distribute_simd_codegen.cpp (+1031-1031) - (modified) clang/test/OpenMP/target_teams_distribute_simd_collapse_codegen.cpp (+126-126) - (modified) clang/test/OpenMP/target_teams_distribute_simd_dist_schedule_codegen.cpp (+284-284) - (modified) clang/test/OpenMP/target_teams_distribute_simd_firstprivate_codegen.cpp (+118-118) - (modified) clang/test/OpenMP/target_teams_distribute_simd_lastprivate_codegen.cpp (+154-154) - (modified) clang/test/OpenMP/target_teams_distribute_simd_private_codegen.cpp (+96-96) - (modified) clang/test/OpenMP/target_teams_distribute_simd_reduction_codegen.cpp (+108-108) - (modified) clang/test/OpenMP/target_teams_generic_loop_codegen-1.cpp (+80-80) - (modified) clang/test/OpenMP/target_teams_generic_loop_collapse_codegen.cpp (+68-68) - (modified) clang/test/OpenMP/target_teams_generic_loop_if_codegen.cpp (+55-55) - (modified) clang/test/OpenMP/target_teams_generic_loop_order_codegen.cpp (+14-14) - (modified) clang/test/OpenMP/target_teams_generic_loop_private_codegen.cpp (+78-78) - (modified) clang/test/OpenMP/target_teams_generic_loop_reduction_codegen.cpp (+89-89) - (modified) clang/test/OpenMP/target_teams_generic_loop_uses_allocators_codegen.cpp (+3-3) - (added) clang/test/OpenMP/target_teams_host_kernel_environment_codegen.cpp (+27) - (modified) clang/test/OpenMP/target_teams_map_codegen.cpp (+105-105) - (modified) clang/test/OpenMP/target_teams_num_teams_codegen.cpp (+44-44) - (modified) clang/test/OpenMP/target_teams_thread_limit_codegen.cpp (+12-12) - (modified) clang/test/OpenMP/teams_codegen.cpp (+67-67) - (modified) clang/test/OpenMP/teams_distribute_codegen.cpp (+92-92) - (modified) clang/test/OpenMP/teams_distribute_collapse_codegen.cpp (+44-44) - (modified) clang/test/OpenMP/teams_distribute_dist_schedule_codegen.cpp (+152-152) - (modified) clang/test/OpenMP/teams_distribute_firstprivate_codegen.cpp (+48-48) - (modified) clang/test/OpenMP/teams_distribute_lastprivate_codegen.cpp (+90-90) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_codegen.cpp (+149-149) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_collapse_codegen.cpp (+72-72) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_copyin_codegen.cpp (+68-68) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_dist_schedule_codegen.cpp (+206-206) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_firstprivate_codegen.cpp (+77-77) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_if_codegen.cpp (+93-93) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_lastprivate_codegen.cpp (+141-141) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_num_threads_codegen.cpp (+174-174) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_private_codegen.cpp (+47-47) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_proc_bind_codegen.cpp (+34-34) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_reduction_codegen.cpp (+86-86) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_schedule_codegen.cpp (+816-816) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_codegen.cpp (+223-223) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_collapse_codegen.cpp (+180-180) - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_dist_schedule_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_firstprivate_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_if_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_if_openmp52_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_lastprivate_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_num_threads_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_private_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_proc_bind_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_reduction_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_parallel_for_simd_schedule_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_private_codegen.cpp () - (modified) clang/test/OpenMP/teams_distribute_reduction_codegen.cpp (+53-53) - (modified) clang/test/OpenMP/teams_distribute_simd_codegen.cpp (+298-298) - (modified) clang/test/OpenMP/teams_distribute_simd_collapse_codegen.cpp (+128-128) - (modified) clang/test/OpenMP/teams_distribute_simd_dist_schedule_codegen.cpp (+287-287) - (modified) clang/test/OpenMP/teams_distribute_simd_firstprivate_codegen.cpp (+120-120) - (modified) clang/test/OpenMP/teams_distribute_simd_lastprivate_codegen.cpp (+165-165) - (modified) clang/test/OpenMP/teams_distribute_simd_private_codegen.cpp (+96-96) - (modified) clang/test/OpenMP/teams_distribute_simd_reduction_codegen.cpp (+105-105) - (modified) clang/test/OpenMP/teams_firstprivate_codegen.cpp (+80-80) - (modified) clang/test/OpenMP/teams_generic_loop_codegen-1.cpp (+92-92) - (modified) clang/test/OpenMP/teams_generic_loop_collapse_codegen.cpp (+44-44) - (modified) clang/test/OpenMP/teams_generic_loop_private_codegen.cpp (+27-27) - (modified) clang/test/OpenMP/teams_generic_loop_reduction_codegen.cpp (+48-48) - (modified) clang/test/OpenMP/teams_private_codegen.cpp (+56-56) - (modified) llvm/include/llvm/Frontend/OpenMP/OMPIRBuilder.h (+13) - (modified) llvm/lib/Frontend/OpenMP/OMPIRBuilder.cpp (+37-21) ``````````diff diff --git a/clang/lib/CodeGen/CGOpenMPRuntime.cpp b/clang/lib/CodeGen/CGOpenMPRuntime.cpp index 1ad07936e6f05..862274bf89355 100644 --- a/clang/lib/CodeGen/CGOpenMPRuntime.cpp +++ b/clang/lib/CodeGen/CGOpenMPRuntime.cpp @@ -6256,11 +6256,14 @@ namespace { /// Cleanup action for uses_allocators support. class OMPUsesAllocatorsActionTy final : public PrePostActionTy { ArrayRef<std::pair<const Expr *, const Expr *>> Allocators; + const OMPExecutableDirective &D; + bool IsOffloadEntry; public: OMPUsesAllocatorsActionTy( - ArrayRef<std::pair<const Expr *, const Expr *>> Allocators) - : Allocators(Allocators) {} + ArrayRef<std::pair<const Expr *, const Expr *>> Allocators, + const OMPExecutableDirective &D, bool IsOffloadEntry) + : Allocators(Allocators), D(D), IsOffloadEntry(IsOffloadEntry) {} void Enter(CodeGenFunction &CGF) override { if (!CGF.HaveInsertPoint()) return; @@ -6268,6 +6271,12 @@ class OMPUsesAllocatorsActionTy final : public PrePostActionTy { CGF.CGM.getOpenMPRuntime().emitUsesAllocatorsInit( CGF, AllocatorData.first, AllocatorData.second); } + // This kernel does not go through the device-side runtime + // init/deinit sequence (that is GPU-only), but the runtime still + // needs a '<kernel>_kernel_environment' global to know how the + // kernel was configured, so emit it directly here. + if (IsOffloadEntry) + CGF.CGM.getOpenMPRuntime().emitHostKernelEnvironment(D, CGF); } void Exit(CodeGenFunction &CGF) override { if (!CGF.HaveInsertPoint()) @@ -6280,6 +6289,14 @@ class OMPUsesAllocatorsActionTy final : public PrePostActionTy { }; } // namespace +void CGOpenMPRuntime::emitHostKernelEnvironment(const OMPExecutableDirective &D, + CodeGenFunction &CGF) { + llvm::OpenMPIRBuilder::TargetKernelDefaultAttrs Attrs; + Attrs.ExecFlags = llvm::omp::OMPTgtExecModeFlags::OMP_TGT_EXEC_MODE_GENERIC; + computeMinAndMaxThreadsAndTeams(D, CGF, Attrs); + OMPBuilder.emitKernelEnvironment(CGF.Builder, Attrs); +} + void CGOpenMPRuntime::emitTargetOutlinedFunction( const OMPExecutableDirective &D, StringRef ParentName, llvm::Function *&OutlinedFn, llvm::Constant *&OutlinedFnID, @@ -6295,7 +6312,7 @@ void CGOpenMPRuntime::emitTargetOutlinedFunction( Allocators.emplace_back(D.Allocator, D.AllocatorTraits); } } - OMPUsesAllocatorsActionTy UsesAllocatorAction(Allocators); + OMPUsesAllocatorsActionTy UsesAllocatorAction(Allocators, D, IsOffloadEntry); CodeGen.setAction(UsesAllocatorAction); emitTargetOutlinedFunctionHelper(D, ParentName, OutlinedFn, OutlinedFnID, IsOffloadEntry, CodeGen); diff --git a/clang/lib/CodeGen/CGOpenMPRuntime.h b/clang/lib/CodeGen/CGOpenMPRuntime.h index 73bbcc05f515a..ae55295cadc14 100644 --- a/clang/lib/CodeGen/CGOpenMPRuntime.h +++ b/clang/lib/CodeGen/CGOpenMPRuntime.h @@ -1397,6 +1397,12 @@ class CGOpenMPRuntime { const Expr *IfCond, OpenMPDirectiveKind CancelRegion); + /// Emit the '<kernel>_kernel_environment' global for a target region that + /// is compiled for a non-GPU (host-style) offload target, which does not + /// go through the GPU device-side runtime init/deinit sequence. + void emitHostKernelEnvironment(const OMPExecutableDirective &D, + CodeGenFunction &CGF); + /// Emit outilined function for 'target' directive. /// \param D Directive to emit. /// \param ParentName Name of the function that encloses the target region. diff --git a/clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp b/clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp index e785ee635af70..4daba2547b65c 100644 --- a/clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp +++ b/clang/lib/CodeGen/CGOpenMPRuntimeGPU.cpp @@ -747,6 +747,20 @@ void CGOpenMPRuntimeGPU::emitNonSPMDKernel(const OMPExecutableDirective &D, IsInTTDRegion = false; } +void CGOpenMPRuntimeGPU::emitBareKernelEnvironment( + const OMPExecutableDirective &D, CodeGenFunction &CGF) { + // Bare kernels manage their own initialization and never call + // __kmpc_target_init, but the runtime still needs a + // '<kernel>_kernel_environment' global to know how the kernel was + // configured, so emit it directly here. + llvm::OpenMPIRBuilder::TargetKernelDefaultAttrs Attrs; + Attrs.ExecFlags = llvm::omp::OMPTgtExecModeFlags::OMP_TGT_EXEC_MODE_BARE; + computeMinAndMaxThreadsAndTeams(D, CGF, Attrs); + + CGBuilderTy &Bld = CGF.Builder; + OMPBuilder.emitKernelEnvironment(Bld, Attrs); +} + void CGOpenMPRuntimeGPU::emitKernelInit(const OMPExecutableDirective &D, CodeGenFunction &CGF, EntryFunctionState &EST, bool IsSPMD) { @@ -825,6 +839,7 @@ void CGOpenMPRuntimeGPU::emitSPMDKernel(const OMPExecutableDirective &D, void Enter(CodeGenFunction &CGF) override { if (IsBareKernel) { RT.CurrentDataSharingMode = DataSharingMode::DS_CUDA; + RT.emitBareKernelEnvironment(D, CGF); return; } RT.emitKernelInit(D, CGF, EST, /* IsSPMD */ true); diff --git a/clang/lib/CodeGen/CGOpenMPRuntimeGPU.h b/clang/lib/CodeGen/CGOpenMPRuntimeGPU.h index 3a7ee5456a9d2..50efbfd1577a5 100644 --- a/clang/lib/CodeGen/CGOpenMPRuntimeGPU.h +++ b/clang/lib/CodeGen/CGOpenMPRuntimeGPU.h @@ -59,6 +59,11 @@ class CGOpenMPRuntimeGPU : public CGOpenMPRuntime { /// Get barrier to synchronize all threads in a block. void syncCTAThreads(CodeGenFunction &CGF); + /// Helper to emit the kernel environment global for an `ompx_bare` kernel, + /// which does not go through emitKernelInit/emitKernelDeinit. + void emitBareKernelEnvironment(const OMPExecutableDirective &D, + CodeGenFunction &CGF); + /// Helper for target directive initialization. void emitKernelInit(const OMPExecutableDirective &D, CodeGenFunction &CGF, EntryFunctionState &EST, bool IsSPMD); diff --git a/clang/test/OpenMP/bug60602.cpp b/clang/test/OpenMP/bug60602.cpp index 3c21349142575..e804bfc4145b0 100644 --- a/clang/test/OpenMP/bug60602.cpp +++ b/clang/test/OpenMP/bug60602.cpp @@ -144,7 +144,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: store [3 x i32] zeroinitializer, ptr [[TMP48]], align 4 // CHECK-NEXT: [[TMP49:%.*]] = getelementptr inbounds nuw [[STRUCT___TGT_KERNEL_ARGUMENTS]], ptr [[KERNEL_ARGS]], i32 0, i32 12 // CHECK-NEXT: store i32 0, ptr [[TMP49]], align 4 -// CHECK-NEXT: [[TMP50:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB2:[0-9]+]], i64 -1, i32 1, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l9.region_id, ptr [[KERNEL_ARGS]]) +// CHECK-NEXT: [[TMP50:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB1:[0-9]+]], i64 -1, i32 1, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l9.region_id, ptr [[KERNEL_ARGS]]) // CHECK-NEXT: [[TMP51:%.*]] = icmp ne i32 [[TMP50]], 0 // CHECK-NEXT: br i1 [[TMP51]], label [[OMP_OFFLOAD_FAILED:%.*]], label [[OMP_OFFLOAD_CONT:%.*]] // CHECK: omp_offload.failed: @@ -248,7 +248,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: store [3 x i32] zeroinitializer, ptr [[TMP102]], align 4 // CHECK-NEXT: [[TMP103:%.*]] = getelementptr inbounds nuw [[STRUCT___TGT_KERNEL_ARGUMENTS]], ptr [[KERNEL_ARGS14]], i32 0, i32 12 // CHECK-NEXT: store i32 0, ptr [[TMP103]], align 4 -// CHECK-NEXT: [[TMP104:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB2]], i64 -1, i32 0, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.region_id, ptr [[KERNEL_ARGS14]]) +// CHECK-NEXT: [[TMP104:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB1]], i64 -1, i32 0, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.region_id, ptr [[KERNEL_ARGS14]]) // CHECK-NEXT: [[TMP105:%.*]] = icmp ne i32 [[TMP104]], 0 // CHECK-NEXT: br i1 [[TMP105]], label [[OMP_OFFLOAD_FAILED15:%.*]], label [[OMP_OFFLOAD_CONT16:%.*]] // CHECK: omp_offload.failed15: @@ -260,7 +260,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: [[TMP106:%.*]] = load i32, ptr [[I]], align 4 // CHECK-NEXT: [[INC:%.*]] = add nsw i32 [[TMP106]], 1 // CHECK-NEXT: store i32 [[INC]], ptr [[I]], align 4 -// CHECK-NEXT: br label [[FOR_COND]], !llvm.loop [[LOOP6:![0-9]+]] +// CHECK-NEXT: br label [[FOR_COND]], !llvm.loop [[LOOP7:![0-9]+]] // CHECK: for.end: // CHECK-NEXT: [[TMP107:%.*]] = load ptr, ptr [[A_ADDR]], align 8 // CHECK-NEXT: [[TMP108:%.*]] = load i32, ptr [[N_ADDR]], align 4 @@ -288,7 +288,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: [[TMP1:%.*]] = load i64, ptr [[N_CASTED]], align 8 // CHECK-NEXT: [[TMP2:%.*]] = load ptr, ptr [[A_ADDR]], align 8 // CHECK-NEXT: [[TMP3:%.*]] = load ptr, ptr [[B_ADDR]], align 8 -// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_call(ptr @[[GLOB2]], i32 3, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l9.omp_outlined, i64 [[TMP1]], ptr [[TMP2]], ptr [[TMP3]]) +// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_call(ptr @[[GLOB1]], i32 3, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l9.omp_outlined, i64 [[TMP1]], ptr [[TMP2]], ptr [[TMP3]]) // CHECK-NEXT: ret void // // @@ -334,7 +334,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: store i32 0, ptr [[DOTOMP_IS_LAST]], align 4 // CHECK-NEXT: [[TMP4:%.*]] = load ptr, ptr [[DOTGLOBAL_TID__ADDR]], align 8 // CHECK-NEXT: [[TMP5:%.*]] = load i32, ptr [[TMP4]], align 4 -// CHECK-NEXT: call void @__kmpc_for_static_init_4(ptr @[[GLOB1:[0-9]+]], i32 [[TMP5]], i32 34, ptr [[DOTOMP_IS_LAST]], ptr [[DOTOMP_LB]], ptr [[DOTOMP_UB]], ptr [[DOTOMP_STRIDE]], i32 1, i32 1) +// CHECK-NEXT: call void @__kmpc_for_static_init_4(ptr @[[GLOB2:[0-9]+]], i32 [[TMP5]], i32 34, ptr [[DOTOMP_IS_LAST]], ptr [[DOTOMP_LB]], ptr [[DOTOMP_UB]], ptr [[DOTOMP_STRIDE]], i32 1, i32 1) // CHECK-NEXT: [[TMP6:%.*]] = load i32, ptr [[DOTOMP_UB]], align 4 // CHECK-NEXT: [[TMP7:%.*]] = load i32, ptr [[DOTCAPTURE_EXPR_1]], align 4 // CHECK-NEXT: [[CMP4:%.*]] = icmp sgt i32 [[TMP6]], [[TMP7]] @@ -384,14 +384,14 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK: omp.loop.exit: // CHECK-NEXT: [[TMP20:%.*]] = load ptr, ptr [[DOTGLOBAL_TID__ADDR]], align 8 // CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[TMP20]], align 4 -// CHECK-NEXT: call void @__kmpc_for_static_fini(ptr @[[GLOB1]], i32 [[TMP21]]) +// CHECK-NEXT: call void @__kmpc_for_static_fini(ptr @[[GLOB2]], i32 [[TMP21]]) // CHECK-NEXT: br label [[OMP_PRECOND_END]] // CHECK: omp.precond.end: // CHECK-NEXT: ret void // // // CHECK-LABEL: define internal void @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13 -// CHECK-SAME: (i64 noundef [[N:%.*]], ptr noundef [[A:%.*]], ptr noundef [[B:%.*]], ptr noalias noundef [[DYN_PTR:%.*]]) #[[ATTR1]] { +// CHECK-SAME: (i64 noundef [[N:%.*]], ptr noundef [[A:%.*]], ptr noundef [[B:%.*]], ptr noalias noundef [[DYN_PTR:%.*]]) #[[ATTR5:[0-9]+]] { // CHECK-NEXT: entry: // CHECK-NEXT: [[N_ADDR:%.*]] = alloca i64, align 8 // CHECK-NEXT: [[A_ADDR:%.*]] = alloca ptr, align 8 @@ -407,12 +407,12 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: [[TMP1:%.*]] = load i64, ptr [[N_CASTED]], align 8 // CHECK-NEXT: [[TMP2:%.*]] = load ptr, ptr [[A_ADDR]], align 8 // CHECK-NEXT: [[TMP3:%.*]] = load ptr, ptr [[B_ADDR]], align 8 -// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_teams(ptr @[[GLOB2]], i32 3, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.omp_outlined, i64 [[TMP1]], ptr [[TMP2]], ptr [[TMP3]]) +// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_teams(ptr @[[GLOB1]], i32 3, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.omp_outlined, i64 [[TMP1]], ptr [[TMP2]], ptr [[TMP3]]) // CHECK-NEXT: ret void // // // CHECK-LABEL: define internal void @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.omp_outlined -// CHECK-SAME: (ptr noalias noundef [[DOTGLOBAL_TID_:%.*]], ptr noalias noundef [[DOTBOUND_TID_:%.*]], i64 noundef [[N:%.*]], ptr noundef [[A:%.*]], ptr noundef [[B:%.*]]) #[[ATTR1]] { +// CHECK-SAME: (ptr noalias noundef [[DOTGLOBAL_TID_:%.*]], ptr noalias noundef [[DOTBOUND_TID_:%.*]], i64 noundef [[N:%.*]], ptr noundef [[A:%.*]], ptr noundef [[B:%.*]]) #[[ATTR6:[0-9]+]] { // CHECK-NEXT: entry: // CHECK-NEXT: [[DOTGLOBAL_TID__ADDR:%.*]] = alloca ptr, align 8 // CHECK-NEXT: [[DOTBOUND_TID__ADDR:%.*]] = alloca ptr, align 8 @@ -487,7 +487,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: [[TMP18:%.*]] = load i64, ptr [[N_CASTED]], align 8 // CHECK-NEXT: [[TMP19:%.*]] = load ptr, ptr [[A_ADDR]], align 8 // CHECK-NEXT: [[TMP20:%.*]] = load ptr, ptr [[B_ADDR]], align 8 -// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_call(ptr @[[GLOB2]], i32 5, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.omp_outlined.omp_outlined, i64 [[TMP14]], i64 [[TMP16]], i64 [[TMP18]], ptr [[TMP19]], ptr [[TMP20]]) +// CHECK-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_call(ptr @[[GLOB1]], i32 5, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z18kernel_within_loopPiS_ii_l13.omp_outlined.omp_outlined, i64 [[TMP14]], i64 [[TMP16]], i64 [[TMP18]], ptr [[TMP19]], ptr [[TMP20]]) // CHECK-NEXT: br label [[OMP_INNER_FOR_INC:%.*]] // CHECK: omp.inner.for.inc: // CHECK-NEXT: [[TMP21:%.*]] = load i32, ptr [[DOTOMP_IV]], align 4 @@ -558,7 +558,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK-NEXT: store i32 0, ptr [[DOTOMP_IS_LAST]], align 4 // CHECK-NEXT: [[TMP6:%.*]] = load ptr, ptr [[DOTGLOBAL_TID__ADDR]], align 8 // CHECK-NEXT: [[TMP7:%.*]] = load i32, ptr [[TMP6]], align 4 -// CHECK-NEXT: call void @__kmpc_for_static_init_4u(ptr @[[GLOB1]], i32 [[TMP7]], i32 34, ptr [[DOTOMP_IS_LAST]], ptr [[DOTOMP_LB]], ptr [[DOTOMP_UB]], ptr [[DOTOMP_STRIDE]], i32 1, i32 1) +// CHECK-NEXT: call void @__kmpc_for_static_init_4u(ptr @[[GLOB2]], i32 [[TMP7]], i32 34, ptr [[DOTOMP_IS_LAST]], ptr [[DOTOMP_LB]], ptr [[DOTOMP_UB]], ptr [[DOTOMP_STRIDE]], i32 1, i32 1) // CHECK-NEXT: [[TMP8:%.*]] = load i32, ptr [[DOTOMP_UB]], align 4 // CHECK-NEXT: [[TMP9:%.*]] = load i32, ptr [[DOTCAPTURE_EXPR_1]], align 4 // CHECK-NEXT: [[CMP5:%.*]] = icmp ugt i32 [[TMP8]], [[TMP9]] @@ -610,7 +610,7 @@ int kernel_within_loop(int *a, int *b, int N, int num_iters) { // CHECK: omp.loop.exit: // CHECK-NEXT: [[TMP22:%.*]] = load ptr, ptr [[DOTGLOBAL_TID__ADDR]], align 8 // CHECK-NEXT: [[TMP23:%.*]] = load i32, ptr [[TMP22]], align 4 -// CHECK-NEXT: call void @__kmpc_for_static_fini(ptr @[[GLOB1]], i32 [[TMP23]]) +// CHECK-NEXT: call void @__kmpc_for_static_fini(ptr @[[GLOB2]], i32 [[TMP23]]) // CHECK-NEXT: br label [[OMP_PRECOND_END]] // CHECK: omp.precond.end: // CHECK-NEXT: ret void diff --git a/clang/test/OpenMP/distribute_codegen.cpp b/clang/test/OpenMP/distribute_codegen.cpp index f5b9a3a5d7c9e..3216dddc2bbf4 100644 --- a/clang/test/OpenMP/distribute_codegen.cpp +++ b/clang/test/OpenMP/distribute_codegen.cpp @@ -194,11 +194,11 @@ int fint(void) { return ftemplate<int>(); } // CHECK1-NEXT: store [3 x i32] zeroinitializer, ptr [[TMP32]], align 4 // CHECK1-NEXT: [[TMP33:%.*]] = getelementptr inbounds nuw [[STRUCT___TGT_KERNEL_ARGUMENTS]], ptr [[KERNEL_ARGS]], i32 0, i32 12 // CHECK1-NEXT: store i32 0, ptr [[TMP33]], align 4 -// CHECK1-NEXT: [[TMP34:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB2:[0-9]+]], i64 -1, i32 0, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56.region_id, ptr [[KERNEL_ARGS]]) +// CHECK1-NEXT: [[TMP34:%.*]] = call i32 @__tgt_target_kernel(ptr @[[GLOB1:[0-9]+]], i64 -1, i32 0, i32 0, ptr @.{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56.region_id, ptr [[KERNEL_ARGS]]) // CHECK1-NEXT: [[TMP35:%.*]] = icmp ne i32 [[TMP34]], 0 // CHECK1-NEXT: br i1 [[TMP35]], label [[OMP_OFFLOAD_FAILED:%.*]], label [[OMP_OFFLOAD_CONT:%.*]] // CHECK1: omp_offload.failed: -// CHECK1-NEXT: call void @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56(ptr [[TMP0]], ptr [[TMP1]], ptr [[TMP2]], ptr [[TMP3]], ptr null) #[[ATTR2:[0-9]+]] +// CHECK1-NEXT: call void @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56(ptr [[TMP0]], ptr [[TMP1]], ptr [[TMP2]], ptr [[TMP3]], ptr null) #[[ATTR3:[0-9]+]] // CHECK1-NEXT: br label [[OMP_OFFLOAD_CONT]] // CHECK1: omp_offload.cont: // CHECK1-NEXT: ret void @@ -217,12 +217,12 @@ int fint(void) { return ftemplate<int>(); } // CHECK1-NEXT: store ptr [[C]], ptr [[C_ADDR]], align 8 // CHECK1-NEXT: store ptr [[D]], ptr [[D_ADDR]], align 8 // CHECK1-NEXT: store ptr [[DYN_PTR]], ptr [[DYN_PTR_ADDR]], align 8 -// CHECK1-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_teams(ptr @[[GLOB2]], i32 4, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56.omp_outlined, ptr [[A_ADDR]], ptr [[B_ADDR]], ptr [[C_ADDR]], ptr [[D_ADDR]]) +// CHECK1-NEXT: call void (ptr, i32, ptr, ...) @__kmpc_fork_teams(ptr @[[GLOB1]], i32 4, ptr @{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56.omp_outlined, ptr [[A_ADDR]], ptr [[B_ADDR]], ptr [[C_ADDR]], ptr [[D_ADDR]]) // CHECK1-NEXT: ret void // // // CHECK1-LABEL: define {{[^@]+}}@{{__omp_offloading_[0-9a-z]+_[0-9a-z]+}}__Z23without_schedule_clausePfS_S_S__l56.omp_outlined -// CHECK1-SAME: (ptr noalias noundef [[DOTGLOBAL_TID_:%.*]], ptr noalias noundef [[DOTBOUND_TID_:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[A:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[B:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[C:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[D:%.*]]) #[[ATTR1]] { +// CHECK1-SAME: (ptr noalias noundef [[DOTGLOBAL_TID_:%.*]], ptr noalias noundef [[DOTBOUND_TID_:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[A:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[B:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[C:%.*]], ptr noundef nonnull align 8 dereferenceable(8) [[D:%.*]]) #[[ATTR2:[0-9]+]] { // CHECK1-NEXT: entry: // CHECK1-NEXT: [[DOTGLOBAL_TID__ADDR:%.*]] = alloca ptr, align 8 // CHECK1-NEXT: [[DOTBOUND_TID__ADDR:%.*]] = alloca ptr, align 8 @@ -243,17 +243,17 @@ int fint(void) { return ftemplate<int>(); } // CHECK1-NEXT: store ptr [[B]], ptr [[B_ADDR]], align 8 // CHECK1-NEXT: store ptr [[C]], ptr [[C_ADDR]], align 8 // CHECK1-NEXT: store ptr [[D]], ptr [[D_ADDR]], align 8 -// CHECK1-NEXT: [[TMP0:%.*]] = load ptr, ptr [[A_ADDR]], align 8, !nonnull [[META12:![0-9]+]], !align [[META13:![0-9]+]] -// CHECK1-NEXT: [[TMP1:%.*]] = load ptr, ptr [[B_ADDR]], align 8, !nonnull [[META12]], !align [[META13]] -// CHECK1-NEXT: [[TMP2:%.*]] = load ptr, ptr [[C_ADDR]], align 8, !nonnull [[META12]], !align [[META13]] -// CHECK1-NEXT: [[TMP3:%.*]] = load ptr, ptr [[D_ADDR]], align 8, !nonnull [[META12]], !align [[META13]] +// CHECK1-NEXT: [[TMP0:%.*]] = load ptr, ptr [[A_ADDR]], align 8, !nonnull [[META13:![0-9]+]], !align [[META14:![0-9]+]] +// CHECK1-NEXT: [[TMP1:%.*]] = load ptr, ptr [[B_ADDR]], align 8, !nonnull [[META13]], !align [[META14]] +// CHECK1-NEXT: [[TMP2:%.*]] = load ptr, ptr [[C_ADDR]], align 8, !nonnull [[META13]], !align [[META14]] +// CHECK1-NEXT: [[TMP3:%.*]] = load ptr, ptr [[D_ADDR]], align 8, !nonnull [[META13]], !align [[META14]] // CHECK1-NEXT: store i32 0, ptr [[DOTOMP_LB]], align 4 // CHECK1-NEXT: store i32 4571423, ptr [[DOTOMP_UB]], align 4 // CHECK1-NEXT: store i32 1, ptr [[DOTOMP_STRIDE]], align 4 // CHECK1-NEXT: store i32 0, ptr [[DOTOMP_IS_LAST]], align 4 // CHECK1-NEXT: [[TMP4:%.*]] = load ptr, ptr [[DOTG... [truncated] `````````` </details> https://github.com/llvm/llvm-project/pull/223772 _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
