On Wed, Jul 29, 2009 at 8:03 AM, Ralf Wildenhues<ralf.wildenh...@gmx.de> wrote: > Hello Xan, > > thanks for the report. This is an Automake issue more than an Autoconf > one. >
Agh, silly mistake. I googled "autotools mailing list" and got carried away, didn't stop a minute to look for an automake specific one, sorry. > * Xan Lopez wrote on Tue, Jul 28, 2009 at 09:29:29PM CEST: >> we are using a non-recursive automake setup at the WebKitGTK+ project >> (http://www.webkitgtk.org). Today, when adding two new files to the >> sources list, I started getting this error when doing a 'make >> distcheck': >> >> niraikanai:~/git/WebKit/build/normal%make distcheck >> { test ! -d "webkit-1.1.12" || { find "webkit-1.1.12" -type d ! -perm >> -200 -exec chmod u+w {} ';' && rm -fr "webkit-1.1.12"; }; } >> test -d "webkit-1.1.12" || mkdir "webkit-1.1.12" >> make: execvp: /bin/sh: Argument list too long >> make: *** [distdir] Error 127 >> >> The problem is somewhere in the 'distdir' rule, as the output >> suggests, where we are passing the whole source files list of WebKit >> to a command, which overflows the reserved space in Linux for such >> issues. > > On Linux before 2.6.23(?), the default limit for command line plus > environment when executing a process is something like 128KB, but this > can be modified in the kernel (or /proc, I don't remember). On newer > Linux kernels, this limit has been lifted to not exist any more. So one > workaround would be to build your distribution tarball under a recent > kernel only. That's right, this limit can be modified recompiling the kernel (it's a #define IIRC), although that workaround is not very practical. It's good to know this is going away in recent kernels though. > >> I'd be happy to hear of any workaround we could apply in our setup to >> avoid this, but I suspect this needs fixing in the distdir.am file. > > The Automake manual has a section on avoiding long lists for this > reason, > <http://www.gnu.org/software/automake/manual/html_node/Length-Limitations.html> > but in your case that would mean using recursive makefiles (at least > without changes to distdir.am it would). I'd still need to think about > how practical an distdir.am adjustment would be. At least it would lead > to some code duplication. Right. I don't think it's practical for us to change to a recursive setup as long as there's some kind of workaround the maintainers can apply to be able to run make distcheck, as that would involve a lot of work (not to mention we like non-recursive automake better). > >> I've tried to look at it, but couldn't really figure out where it's >> failing exactly (my lack of shell skills is not helping here...). > > This failure is generaated by the kernel, it fails to start the shell > command that is that part of the rule. You can debug such make rules > with > make SHELL=/bin/sh\ -x That indeed seems to give more output, although in this case it does not really help much (you still can't see the command that's failing). I also tried make -d, with similar results. In the end I just went ahead and looked directly in my Makefile; luckily the distdir rule is fairly short. > >> There's an odd thing, in that we iterate over the DISTFILES (of the >> form 'file1 file2 file3...') with for file in list-of-files; do... >> construct; > > That's likely the culprit. > >> AFAIK that will iterate through the lines in the input, > > No; it does something like > long_list="$(MAKE_VARIABLES_CONTAINING_LONG_LIST)"; > for file in $$long_list; do ... done > for file in $$long_list; do ... done Sorry, what I meant with 'input' is that the for will iterate over each line in the contents of $$long_list, but the file list that's there does not seem to have lines at all, only space separated filenames. I'm pretty sure I'm missing something anyway, because this does indeed work in general, so I'm glad I found the place where it's failing even if I'm not sure why it's failing. > > for two reasons: > - this way, the command passed from make contains the long list string > only once, in the variable assignment; this helps keep below the limit > if possible. > - if the list happens to be empty, it avoids a shell syntax error that > would happen with > for file in $(LONG_LIST); ... > > Hope that helps. I think what would help me most is if you sketched a possible solution in distdir.am, and I'll try to get that done so we can 'fix' automake for people with older kernels (2.6.32 isn't likely to be widely distributed for at least a year or two I'd say). And even if you don't want to fix it upstream, I think probably asking anyone interested in running distcheck to apply a small patch to distdir.am is easier than asking them to rebuild their kernel. > > Man, your package *is* big. I started the 'svn co' before writing this > email, and it's still not done. Speaks for svn slowness, too, but your > git repo seems unavailable ATM. Yeah, it has *lots* of tests, that does not help. Fortunately we do not distribute or do anything with those, so we don't have to add them in our automake files (otherwise we'd have discovered this problem from day zero I suspect). Cheers, Xan > > Cheers, > Ralf >
