[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-05 Thread Yaxun Liu via cfe-commits
yxsamliu wrote: [AMD Official Use Only - General] Thanks. I will fix it by memcpy instead of dereferencing int* casted from char*. Sam From: Joseph Huber ***@***.***> Sent: Thursday, October 5, 2023 9:23 AM To: llvm/llvm-project ***@***.***> Cc: Liu, Yaxun (Sam) ***@***.***>; State change

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-05 Thread Joseph Huber via cfe-commits
jhuber6 wrote: For reference, I've had similar problems in the past. Usually it happens because the header needs to be aligned on a `uint64_t` to be read. The ELF section has 8-byte alignment, but that can be incorrect when extracting from something like an archive. I just make a copy in that

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-05 Thread Yaxun Liu via cfe-commits
yxsamliu wrote: [AMD Official Use Only - General] I have reverted the commit. Will fix and reland. Thanks. Sam From: mikaelholmen ***@***.***> Sent: Thursday, October 5, 2023 4:05 AM To: llvm/llvm-project ***@***.***> Cc: Liu, Yaxun (Sam) ***@***.***>; State change ***@***.***> Subject: Re:

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-05 Thread via cfe-commits
mikaelholmen wrote: Hi, I noticed that with UBSan this new testcase fails ``` 09:59:08 Failed Tests (1): 09:59:08 Clang :: Driver/clang-offload-bundler-zlib.c ``` with ``` 09:39:53 ../../clang/lib/Driver/OffloadBundler.cpp:1012:25: runtime error: load of misaligned address 0x55ceca16e46c for

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-04 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu closed https://github.com/llvm/llvm-project/pull/67162 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-04 Thread Joseph Huber via cfe-commits
https://github.com/jhuber6 approved this pull request. I think in the long term we need to move away from the bundler, which would suggest we probably want something similar in the "new driver". But overall the changes here are pretty self contained and the concept is straightforward.

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/67162 >From c2d5f809ab0f1be50cee4baaffdf3994807e5ef2 Mon Sep 17 00:00:00 2001 From: "Yaxun (Sam) Liu" Date: Thu, 21 Sep 2023 13:52:16 -0400 Subject: [PATCH] [HIP] Support compressing device binary Add option

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
@@ -1183,6 +1183,10 @@ def fgpu_inline_threshold_EQ : Joined<["-"], "fgpu-inline-threshold=">, def fgpu_sanitize : Flag<["-"], "fgpu-sanitize">, Group, HelpText<"Enable sanitizer for supported offloading devices">; def fno_gpu_sanitize : Flag<["-"], "fno-gpu-sanitize">,

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Johannes Doerfert via cfe-commits
@@ -1183,6 +1183,10 @@ def fgpu_inline_threshold_EQ : Joined<["-"], "fgpu-inline-threshold=">, def fgpu_sanitize : Flag<["-"], "fgpu-sanitize">, Group, HelpText<"Enable sanitizer for supported offloading devices">; def fno_gpu_sanitize : Flag<["-"], "fno-gpu-sanitize">,

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
@@ -881,6 +900,184 @@ CreateFileHandler(MemoryBuffer , "'" + FilesType + "': invalid file type specified"); } +OffloadBundlerConfig::OffloadBundlerConfig() { + auto IgnoreEnvVarOpt = +

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/67162 >From b4b44809dcbc487239b4430751a232563c5ac590 Mon Sep 17 00:00:00 2001 From: "Yaxun (Sam) Liu" Date: Thu, 21 Sep 2023 13:52:16 -0400 Subject: [PATCH] [HIP] Support compressing device binary Add option

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Joseph Huber via cfe-commits
@@ -881,6 +900,184 @@ CreateFileHandler(MemoryBuffer , "'" + FilesType + "': invalid file type specified"); } +OffloadBundlerConfig::OffloadBundlerConfig() { + auto IgnoreEnvVarOpt = +

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
yxsamliu wrote: > > We need to support compression/uncompression of standalone bundled bitcode, > > and standalone bundled code objects that are not embedded in ELF or COFF > > binaries. An OS-neutral binary format is preferred. > > I think my ideal solution would be LLVM supporting some

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/67162 >From b1c9550da1074414228a45594478fec7ecbf1779 Mon Sep 17 00:00:00 2001 From: "Yaxun (Sam) Liu" Date: Thu, 21 Sep 2023 13:52:16 -0400 Subject: [PATCH] [HIP] Support compressing device binary Add option

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Joseph Huber via cfe-commits
jhuber6 wrote: > > We need to support compression/uncompression of standalone bundled bitcode, > and standalone bundled code objects that are not embedded in ELF or COFF > binaries. An OS-neutral binary format is preferred. I think my ideal solution would be LLVM supporting some module

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
@@ -1183,6 +1183,10 @@ def fgpu_inline_threshold_EQ : Joined<["-"], "fgpu-inline-threshold=">, def fgpu_sanitize : Flag<["-"], "fgpu-sanitize">, Group, HelpText<"Enable sanitizer for supported offloading devices">; def fno_gpu_sanitize : Flag<["-"], "fno-gpu-sanitize">,

[clang] [HIP] Support compressing device binary (PR #67162)

2023-10-02 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/67162 >From 71dc0a3afcc873d1afeb3ae265bc006860da1d82 Mon Sep 17 00:00:00 2001 From: "Yaxun (Sam) Liu" Date: Thu, 21 Sep 2023 13:52:16 -0400 Subject: [PATCH] [HIP] Support compressing device binary Add option

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-29 Thread Yaxun Liu via cfe-commits
yxsamliu wrote: > Unrelated, I need to hurry up and try to make the new driver the default for > CUDA and HIP upstream at some point. > > I've had some thoughts about how to accomplish this in the past. For the new > driver, `.llvm.offloading` is a single section that contains all of the >

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-29 Thread Joseph Huber via cfe-commits
@@ -1183,6 +1183,10 @@ def fgpu_inline_threshold_EQ : Joined<["-"], "fgpu-inline-threshold=">, def fgpu_sanitize : Flag<["-"], "fgpu-sanitize">, Group, HelpText<"Enable sanitizer for supported offloading devices">; def fno_gpu_sanitize : Flag<["-"], "fno-gpu-sanitize">,

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-29 Thread Joseph Huber via cfe-commits
jhuber6 wrote: Unrelated, I need to hurry up and try to make the new driver the default for CUDA and HIP upstream at some point. I've had some thoughts about how to accomplish this in the past. For the new driver, `.llvm.offloading` is a single section that contains all of the embedded

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-28 Thread Yaxun Liu via cfe-commits
yxsamliu wrote: ping A little background: some HIP applications face size restrictions for their packages. Compressing can significantly reduce the bundle size with acceptable overhead of uncompressing. Therefore they need this feature. https://github.com/llvm/llvm-project/pull/67162

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-22 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu updated https://github.com/llvm/llvm-project/pull/67162 >From aa1aee8695a062c472dbb9efd1e87a2ba7cdc58b Mon Sep 17 00:00:00 2001 From: "Yaxun (Sam) Liu" Date: Thu, 21 Sep 2023 13:52:16 -0400 Subject: [PATCH] [HIP] Support compressing device binary Add option

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-22 Thread via cfe-commits
llvmbot wrote: @llvm/pr-subscribers-clang-driver Changes Add option -f[no-]offload-compress to clang to enable/disable compression of device binary for HIP. By default it is disabled. Add option -compress to clang-offload-bundler to enable compression of offload bundle. By default it is

[clang] [HIP] Support compressing device binary (PR #67162)

2023-09-22 Thread Yaxun Liu via cfe-commits
https://github.com/yxsamliu created https://github.com/llvm/llvm-project/pull/67162 Add option -f[no-]offload-compress to clang to enable/disable compression of device binary for HIP. By default it is disabled. Add option -compress to clang-offload-bundler to enable compression of offload