[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #20 from John Soo --- I think that in order to really rid gcc of the E2BIG problem on linux, COLLECT_*_OPTIONS will have to be deprecated and removed. This is particularly a problem when executing spec files, it seems.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #19 from John Soo --- I verified the proposed patch sent argv through @file, but COLLECT_GCC_OPTIONS still caused E2BIG. In the failing execve, COLLECT_GCC_OPTIONS was 134227 characters long.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #18 from John Soo --- And actually the proposed patch is not conservative enough, because the size of the strings in argv/env also matter.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #17 from John Soo --- Created attachment 55910 --> https://gcc.gnu.org/bugzilla/attachment.cgi?id=55910=edit libiberty, Unix: pass argv over ARG_MAX through an @file This does not handle environ being too large, but it is an adaptation of the argv fix in pex-win32.c.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #16 from John Soo --- It is actually somewhat likely that ARG_MAX will be hit when running on linux because it is hit when the stack can't contain enough pointers to contain argv and environ (see exec.c in the kernel https://github.com/torvalds/linux/blob/master/fs/exec.c#L509). The bad news is that response files can only mitigate the problem since environ must also be small enough to fit into an execv* call. Is there a reasonable way to keep only the env vars required to exec subprocesses?
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #15 from John Soo --- Just for some context on what limit is hit: under man sysconf you will find a description of ARG_MAX (https://www.man7.org/linux/man-pages/man3/sysconf.3.html) > ARG_MAX - _SC_ARG_MAX > The maximum length of the arguments to the exec(3) family > of functions. Must not be less than _POSIX_ARG_MAX > (4096). Exceeding ARG_MAX will result in E2BIG (https://man7.org/linux/man-pages/man3/errno.3.html). > E2BIG Argument list too long (POSIX.1-2001).
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #14 from John Soo --- > Here though it seems that you are dealing with another sort of limit which is > much larger (I have seen 128K being mentioned on the GH page).If this > somehow corrupts the command line, it wouldn't help if that command line went > into a response file because it would still be wrong.To my knowledge, > Linux-based systems don't have a command line length limitation, so I can't > see how a response file approach would be useful at the point where the > subprocess is spawned.Whether something similar can be used at an earlier > point to save it from the 128K limit, whatever it is, is unknown to me. It is a much larger limit (ARG_MAX resulting in E2BIG), but it is fundamentally the same problem. I think we should assume that the command line is correct and still respect ARG_MAX on linux/unix systems, too. It seems to me that the temporary response file is the best way to do this.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #12 from John Soo --- I think the general problem in that issue is that ARG_MAX is not respected when the driver (or any subprocess) execs things on linux. I think that it is not the same as the original issue here, though. > I don't know if its going to be helpful to see that patch as a guide Do you think using response files like in pex-win32.c in pex-unix.c could help? I tried this out and it seems like this may not solve all the ARG_MAX problems.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #10 from John Soo --- I'm also not sure https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=180ebb8a24d24fc5b105f2257d6216f6dfde62df fixes the collect bug because collect uses collect_execute instead of the pex_* exec functions.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #9 from John Soo --- Would a patch for unix doing something similar to https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=180ebb8a24d24fc5b105f2257d6216f6dfde62df be accepted? I am happy to start working on something like it but I have no gcc contributions yet and would like to know ahead of time if it is desired.
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 --- Comment #8 from John Soo --- > Also, it is typically Windows that suffers from this limitation of command > line length. Ok that may be true but I am effected by this on linux as are quite a few others in this issue https://github.com/NixOS/nixpkgs/issues/41340
[Bug driver/86030] specs file processing does not create response files for input directories
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=86030 John Soo changed: What|Removed |Added CC||john.soo+gcc-bugzilla@arist ||a.com --- Comment #6 from John Soo --- This is not a Windows-only bug, so I don't think it is fixed.
