[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #10 from Deepthi H --- As suggested, We've updated the patch to place the arguments in a file instead of passing it from an env variable. When the COLLECT_GCC_OPTIONS is <128kb the env variable is used and when the size >128kb arguments will be moved to a file. In Collect2: We copy back the contents of the arguments from file into a buffer and pass it on to the Linker. See the updated patch : With_Files_Workaround_for_GCC_driver_LongArgumentList.patch The output of the example program in descripton section as below: = $big_100k_var=$(printf "%0*d" 10 0) $<>/gcc -c a.c -DA=$big_100k_var -DB=$big_100k_var -v Using built-in specs. COLLECT_GCC=/home/sunild/Gcc_Driver_with_FILE/install/home/Gcc_Driver_with_FILE/build/bin/gcc Target: x86_64-pc-linux-gnu Configured with: ../gcc/configure --prefix=/home/Gcc_Driver_with_FILE/build --enable-languages=c --disable-multilib Thread model: posix Supported LTO compression algorithms: zlib gcc version 15.0.0 20240513 (experimental) (GCC) /home/Gcc_Driver_with_FILE/install/home/Gcc_Driver_with_FILE/build/bin/../libexec/gcc/x86_64-pc-linux-gnu/15.0.0/cc1 -quiet -v -imultiarch x86_64-linux-gnu -iprefix /home/Gcc_Driver_with_FILE/install/home/Gcc_Driver_with_FILE/build/bin/../lib/gcc/x86_64-pc-linux-gnu/15.0.0/ -D A=000 <> 0 -D B=000... <> 000 a.c -quiet -dumpbase a.c -dumpbase-ext .c -mtune=generic -march=x86-64 -version -o /tmp/cc7Ox5mH.s = Please let us know this solution is ok.
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #9 from Deepthi H --- Created attachment 58212 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58212=edit With_Files_Workaround_for_GCC_ArgumentLongList
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #8 from Andrew Pinski --- (In reply to Deepthi H from comment #7) > > Let us know your comments on this solution. Such a solution is acceptable to > change the gcc driver? Seems better to place the arguments in a file instead and just pass around that file name instead of passing all arguments via an env variable.
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527
--- Comment #7 from Deepthi H ---
We've a solution for this issue.
When gcc/g++ is called using the @response-file.rsp syntax, gcc should forward
the argument to its subprocesses. Previously the files were expanded which
could lead to excessively long argument lists and 'cc1: execv: Argument list
too long' errors.
In particular, CMake and Ninja tend to create a lot of '-isystem' include
directories, requiring allowing the forwarding in spec files by using %@{i*}.
In xputenv method, If the ENV variable size greater then 128kb then we split
the ENV variable(i.e COLLECT_GCC_OPTIONS) where each chunk will be 128kb in
length.
GCC passes the entire command line, including expanded @rsp-files to the
collect2 in environment variable COLLECT_GCC_OPTIONS. This can exceed the build
environment's kernel's environment variable length limit. In this workaround,
environment variables longer than 128kb are split into multiple variables and
stitched back together in collect2.
The patch is attached here.
And, the patch output of the example code given in 'Description' as below:
===
sunild@BFT-LPT-I-051:~$ $GCC_PATH/gcc -c a.c -DA=$big_100k_var
-DB=$big_100k_var -v
Using built-in specs.
COLLECT_GCC=/home/sunild/GCC_Driver/bin/home/sunild/GCC_Driver/build/bin/gcc
Target: x86_64-pc-linux-gnu
Configured with: ../gcc/configure --prefix=/home/sunild/GCC_Driver/build
--enable-languages=c --disable-multilib
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 14.0.1 20240426 (experimental) (GCC)
COLLECT_GCC_OPTIONS_0='-c' '-D'
'A=
COLLECT_GCC_OPTIONS_1=0
COLLECT_GCC_OPTIONS_COUNT=2
===
Let us know your comments on this solution. Such a solution is acceptable to
change the gcc driver?
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #6 from Deepthi H --- Created attachment 58107 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=58107=edit workaround for gcc driver long argument list error
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #5 from Sergei Trofimovich --- (In reply to Deepthi H from comment #4) > I have been investigating this issue further. Hence checking the source code > and debugging the gcc sources. However, I wasn't able to find where the > COLLECT_GCC_OPTION has been set to 128kb > > I couldn't find it being set in gcc. Can you please let us know how can we > increase the limit of collect options? The 128K limit against a single environment variable is a linux kernel limitation set by this define in include/uapi/linux/binfmts.h: #define MAX_ARG_STRLEN (PAGE_SIZE * 32) https://trofi.github.io/posts/299-maximum-argument-count-on-linux-and-in-gcc.html has more words on that.
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #4 from Deepthi H --- I have been investigating this issue further. Hence checking the source code and debugging the gcc sources. However, I wasn't able to find where the COLLECT_GCC_OPTION has been set to 128kb I couldn't find it being set in gcc. Can you please let us know how can we increase the limit of collect options?
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 Deepthi H changed: What|Removed |Added CC||deepadeepthi98 at gmail dot com --- Comment #3 from Deepthi H --- I have been investigating this issue further. Hence checking the source code and debugging the gcc sources. However, I wasn't able to find where the COLLECT_GCC_OPTION has been set to 128kb I couldn't find it being set in gcc. Can you please let us know how can we increase the limit of collect options?
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #2 from Sergei Trofimovich --- (In reply to Richard Biener from comment #1) > Hm, but the COLLECT_GCC_OPTIONS variable is only used for communicating > between the driver and the linker, the options therein are individually > passed to the program execved? AFAIU the driver sets `COLLECT_GCC_OPTIONS` variable and never unsets it. As a result it affects all the `exevce() calls. Be it `cc1`, `as` or anything else regardless of the fact if it uses the variable or not. `cc1` is probably the first casualty. As a simplistic example here we break `ls` with too large environment file: $ COLLECT_GCC_OPTIONS=$(printf "%0*d" 20 0) ls -bash: /run/current-system/sw/bin/ls: Argument list too long > You are maybe looking for the -f*-map options to take a file as input > containing multiple mappings? `NixOS` is also occasionally hottong the same limit by passing too many include an library paths: -I/nix/store/hash1-foo/include -I/nix/store/hash2-bar/include ... -L/nix/store/hash1-foo/lib -L/nix/store/hash2-bar/lib ... -Wl,-rpath,/nix/store/hash1-foo/lib -Wl,-rpath,/nix/store/hash2-bar/lib I wonder if we could solve all of these limitations here by at least avoiding `COLLECT_GCC_OPTIONS`. But otherwise if generic fix is too invasive change then passing a mapping file should do as well. What would be an acceptable for of the file? A new option, like? -fmacro-prefix-map-file=./foo with entries of exactly the same form $ cat foo /nix/store/hash1-foo=/nix/store/-foo /nix/store/hash2-bar=/nix/store/-bar ... Or maybe reuse existing -fmacro-prefix-map= and use response-style file input? Like -fmacro-prefix-map=@./foo. clang would probably need the same handling if we were to extend the driver.
[Bug driver/111527] COLLECT_GCC_OPTIONS option hits single-variable limits too early
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111527 --- Comment #1 from Richard Biener --- Hm, but the COLLECT_GCC_OPTIONS variable is only used for communicating between the driver and the linker, the options therein are individually passed to the program execved? You are maybe looking for the -f*-map options to take a file as input containing multiple mappings?
