bug#48113: Module suggestion: timeout
Just to mention further evidence supporting Simon's final idea of "choosing the default timeout setting is difficult". This report recently came into bug-dejagnu precisely about different timeouts being needed on different systems: https://lists.gnu.org/archive/html/bug-dejagnu/2021-05/msg00011.html And the replies by the maintainer reject the idea of a single timeout. No action required, just thought it was interesting that the same general issue came up elsewhere. -k
Re: parallel build issues
On Mon, 21 Jun 2021, Werner LEMBERG wrote: Now I want to do better :-) However, the problem persists: `ttfautohintGUI` gets built in the `frontend` directory, and there is a dependency on the binary in the `doc` directory. Is there a clean solution for that? A non-recursive build (a single Makefile for everything) is quite valuable and avoids such problems. However, it requires more care to construct it. It may feel more efficient as a developer to cd to a subdirectory in order to execute just part of a build, but in my experience this is usually a false savings due to a recursive build being more wasteful. It is possible to build just part of the software using a non-recursive build but it may be confusing unless you add dummy target names to make things easier. It is possible to insert additional dependency lines in Makefile.am so software is always built in the desired order, but this approach might only work if you always build using the top level Makefile. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Re: parallel build issues
> For example, perhaps you have the same "frontend" directory listed > also in SUBDIRS for some other unrelated makefile? That is probably > the simplest way this situation could happen. The problem is as follows. * In the `frontend` directory, the binary `ttfautohintGUI` is built. * In the `doc` directory, I need the `ttfautohintGUI` binary to create a snapshot image of its GUI. To do this, I have something like the following rule: ttfautohintGUI.png: $(top_srcdir)/frontend/maingui.cpp \ $(top_srcdir)/configure.ac \ $(srcdir)/ttfautohintGUI.stylesheet cd $(top_builddir)/frontend \ && $(MAKE) $(AM_MAKEFLAGS) ttfautohintGUI$(EXEEXT) $(SHELL) $(srcdir)/make-snapshot.sh \ $(top_builddir)/frontend/ttfautohintGUI$(EXEEXT) I must admit that I no longer remember why I did it this way. It works fine for non-parallel builds (and I have obviously never tested with make's `-j` option). Now I want to do better :-) However, the problem persists: `ttfautohintGUI` gets built in the `frontend` directory, and there is a dependency on the binary in the `doc` directory. Is there a clean solution for that? Werner
Re: parallel build issues
On 2021-06-21, Werner LEMBERG wrote: > >> The problem is not related to the snippet you posted. The >> concurrent recursive make invocations are being spawned from >> somewhere else in your build system. > > The `Makefile.am` file one level higher is as follows. > > ACLOCAL_AMFLAGS = -I gnulib/m4 -I m4 > > SUBDIRS = gnulib/src \ > lib \ > frontend \ > doc > EXTRA_DIST = bootstrap \ >bootstrap.conf \ >FTL.TXT \ >gnulib/m4/gnulib-cache.m4 \ >GPLv2.TXT \ >README \ >TODO \ >.version > > BUILT_SOURCES = .version > .version: > echo $(VERSION) > $@-t && mv $@-t $@ > > dist-hook: > echo $(VERSION) > $(distdir)/VERSION.TXT > > Looks pretty standard to me, but maybe I'm wrong. Nothing shown here is going to cause this problem. But with recursive build problems it is insufficient to just look at just one makefile: the problematic make invocations could be coming from anywhere in your project. For example, perhaps you have the same "frontend" directory listed also in SUBDIRS for some other unrelated makefile? That is probably the simplest way this situation could happen. Or perhaps the parent directory's makefile was itself being processed by concurrent recursive invocations, which then results in independent recursive invocations in the subdirectories. If you can't find anything by a code inspection GNU make has some debug features which may help visualize what is going on. Cheers, Nick
Re: parallel build issues
>> The problem is not related to the snippet you posted. The >> concurrent recursive make invocations are being spawned from >> somewhere else in your build system. Found the problem! I was indeed calling `make` concurrently (within the `doc` directory). Thanks a lot for your help! Werner
Re: parallel build issues
On 2021-06-21, Warren Young wrote: > On Jun 21, 2021, at 11:49 AM, Werner LEMBERG wrote: >> >> bin_PROGRAMS += ttfautohintGUI > > Is Automake smart enough to realize what you’ve done there? This is not a problem. Automake interprets this assignment syntax (and outputs a single assignment to bin_PROGRAMS in Makefile.in). Cheers, Nick
Re: parallel build issues
> The problem is not related to the snippet you posted. The > concurrent recursive make invocations are being spawned from > somewhere else in your build system. The `Makefile.am` file one level higher is as follows. ACLOCAL_AMFLAGS = -I gnulib/m4 -I m4 SUBDIRS = gnulib/src \ lib \ frontend \ doc EXTRA_DIST = bootstrap \ bootstrap.conf \ FTL.TXT \ gnulib/m4/gnulib-cache.m4 \ GPLv2.TXT \ README \ TODO \ .version BUILT_SOURCES = .version .version: echo $(VERSION) > $@-t && mv $@-t $@ dist-hook: echo $(VERSION) > $(distdir)/VERSION.TXT Looks pretty standard to me, but maybe I'm wrong. Werner
Re: parallel build issues
>> bin_PROGRAMS += ttfautohintGUI > > Is Automake smart enough to realize what you’ve done there? According to the documentation, `+=` is recognized. > Try listing both output programs on a single bin_PROGRAMS line. Alas, this doesn't work; it aborts with the same error. Werner
Re: parallel build issues
On 2021-06-21, Werner LEMBERG wrote: > I have a `Makefile.am` in a `frontend` subdirectory that looks like > the following (abridged). [...] > Running the generated `Makefile` with `make -j12`, I get this: > > ... > Making all in frontend > make[2]: Entering directory '.../frontend' > ... > make all-am > make[3]: Entering directory '.../frontend' > CXX info.o > CXX main.o > CXX ttfautohintGUI-ddlineedit.o > CXX ttfautohintGUI-info.o > CXX ttfautohintGUI-main.o > CXX ttfautohintGUI-maingui.o > CXX ttfautohintGUI-ttlineedit.o > make ttfautohint > CXX ttfautohintGUI-ddlineedit.moc.o > CXX ttfautohintGUI-maingui.moc.o > CXX ttfautohintGUI-static-plugins.o > CXX ttfautohintGUI-ttlineedit.moc.o > make[4]: Entering directory '.../frontend' Here one of your make rules has started a recursive build in the "frontend" directory. > CXX info.o > make ttfautohintGUI > make[4]: Entering directory '.../frontend' Here *another* one of your make rules has *also* started a recursive build in "frontend", concurrently with the one that was started previously. > CXXLDttfautohintGUI > CXX main.o > mv: cannot stat '.deps/info.Tpo': No such file or directory Those independent make invocations are simultaneously running the same compilation rules and have no knowledge of what the other make processes are doing. Failure is the inevitable outcome. > make[4]: *** [Makefile:1401: info.o] Error 1 > make[4]: *** Waiting for unfinished jobs > CXXLDttfautohint > make[4]: Leaving directory '.../frontend' > ... > > What am I doing wrong? I would be glad for any pointers. The problem is not related to the snippet you posted. The concurrent recursive make invocations are being spawned from somewhere else in your build system. Hope that helps, Nick
Re: parallel build issues
On Jun 21, 2021, at 11:49 AM, Werner LEMBERG wrote: > > bin_PROGRAMS += ttfautohintGUI Is Automake smart enough to realize what you’ve done there? It seems to me this does something down at the GNU make level that should be done up at the Automake level, preventing Automake from building its dependency tree properly. Try listing both output programs on a single bin_PROGRAMS line.
parallel build issues
[automake 1.16.3 autoconf 2.71] Folks, I have a `Makefile.am` in a `frontend` subdirectory that looks like the following (abridged). bin_PROGRAMS = ttfautohint ttfautohint_SOURCES = info.cpp \ info.h \ main.cpp bin_PROGRAMS += ttfautohintGUI ttfautohintGUI_SOURCES = ddlineedit.cpp \ ddlineedit.h \ info.cpp \ info.h \ main.cpp \ maingui.cpp \ maingui.h \ ttlineedit.cpp \ ttlineedit.h nodist_ttfautohintGUI_SOURCES = ddlineedit.moc.cpp \ maingui.moc.cpp \ static-plugins.cpp \ ttlineedit.moc.cpp ttfautohintGUI_CXXFLAGS = $(QT_CXXFLAGS) ttfautohintGUI_LDFLAGS = $(QT_LDFLAGS) ttfautohintGUI_CPPFLAGS = $(AM_CPPFLAGS) \ $(QT_CPPFLAGS) \ -DBUILD_GUI ttfautohintGUI_LDADD = $(LDADD) \ $(QT_LIBS) BUILT_SOURCES = ddlineedit.moc.cpp \ maingui.moc.cpp \ static-plugins.cpp \ ttlineedit.moc.cpp Running the generated `Makefile` with `make -j12`, I get this: ... Making all in frontend make[2]: Entering directory '.../frontend' ... make all-am make[3]: Entering directory '.../frontend' CXX info.o CXX main.o CXX ttfautohintGUI-ddlineedit.o CXX ttfautohintGUI-info.o CXX ttfautohintGUI-main.o CXX ttfautohintGUI-maingui.o CXX ttfautohintGUI-ttlineedit.o make ttfautohint CXX ttfautohintGUI-ddlineedit.moc.o CXX ttfautohintGUI-maingui.moc.o CXX ttfautohintGUI-static-plugins.o CXX ttfautohintGUI-ttlineedit.moc.o make[4]: Entering directory '.../frontend' CXX info.o make ttfautohintGUI make[4]: Entering directory '.../frontend' CXXLDttfautohintGUI CXX main.o mv: cannot stat '.deps/info.Tpo': No such file or directory make[4]: *** [Makefile:1401: info.o] Error 1 make[4]: *** Waiting for unfinished jobs CXXLDttfautohint make[4]: Leaving directory '.../frontend' ... What am I doing wrong? I would be glad for any pointers. Werner
