default,
> if there was to be one at all, should be conservative so nproc * 2 would be a
> bad choice.
I've always used nproc * 1.5. This factor was determined empirically by
observing idle time on
many builds using nproc * 1 and nproc * 2.
--
-- Howard Chu
CTO, Symas Corp. ht
ry will Just Work after that, and all of
the relevant OSs have support for named pipes.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
pable of
supporting gmake that don't already have a make utility.
Or that telling anyone so unlucky "first bootstrap with gmake 4.3" is a bad
answer.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
Paul Smith wrote:
> On Fri, 2021-08-13 at 14:14 +0100, Howard Chu via Bug reports and
> discussion for GNU make wrote:
>> I'm looking for a way to expose a job ID number to the individual
>> jobs, in a consecutive range from 1-[number of jobs]. Not just
>> unique numbers
use the range of 9000-9020. On a build/test machine we don't worry
about conflicts,
a developer has no business running conflicting software at the same time as
building/testing
our software. And of course, the baseport 9000 can be moved at will.
>
> Christian
>
> On Fri, Aug 13,
of client/server test scripts in parallel.
Is there anything that could do this now, exposed as a Make variable?
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
=
all: copy1 copy2 copy3 copy4
copy1: Makefile
install -c -m 644 Makefile $@
copy2: copy1
install -c -m 644 Makefile $@
copy3: copy2
install -c -m 644 Makefile $@
copy4: copy3
install -c -m 644 Makefile $@
======
and avoid introducing any regressions.
>
> For example, if -j is given without an argument and no -l option is provided,
> behave as if -j -l $(nproc) was specified.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsu
ctually a socket pair instead of a pipe, but then
> that jobserver needs to be created externally, as the GNU make
> creating the jobserver creates a pipe and not a socket pair. As a
> hack, a wrapper program could do it and then call GNU make.
>
> What are your opinions of this? I
-m 644 Makefile copy3
>> install -c -m 644 Makefile copy4
>
> I think the "representative" file should be copy4 here, because it's the one
> that gets created last.
That sort of thing is only true in serial make, you can't rely on it in
parallel make.
--
-- Howard Ch
in your Makefile.
Read this:
http://highlandsun.com/hyc/#Make
in particular this
http://highlandsun.com/hyc/GNUYou.htm
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLD
preparation step will remain serial no matter what -j arguments
are used, so it should be safe to just invoke make -j xx at the
beginning and let things run serially or in parallel as necessary.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun
.
___
Bug-make mailing list
Bug-make@gnu.org
https://lists.gnu.org/mailman/listinfo/bug-make
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http
to stdout. That has nothing to do with stdin. Looks to me like you
have no F'ing idea what you're talking about.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project
be able to do while waiting for a
build.
Eddy.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
___
Bug
the string up into words,
then operate on those, and that happens every time the function is
invoked.
If that happens a lot, then you should simply construct the lists once and
keep them...
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http
.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
___
Bug-make mailing list
Bug-make@gnu.org
https
another reason not to do things that way...
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
___
Bug-make mailing
/bar/.. != foo.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
___
Bug-make mailing list
Bug-make@gnu.org
https
. (Default pipe buffer is 4K which
sets -j4096 as the practical limit without mucking with kernel params.)
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project
to not mingle/mangle lines from different
processes together. But again, that's not gnu-make's problem, that's a Cygwin
issue.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org
would be useful. In fact, since load averages
are calculated so slowly, by the time your -l limit is reached the actual CPU
load will have blown past it and your machine will be thrashing. That's the
entire reason I came up with the -j implementation in the first place.
--
-- Howard Chu
CTO
since it's usually
nonbuffered, but again, that's not the common case.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project
) and by how long
2x is too much. 1.5x has been the best in my experience, any more than that
and you're losing too much CPU to scheduling overhead instead of real work.
Any less and you're giving up too much in idle or I/O time.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
thing to have
this ability though.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
___
Bug-make mailing list
Bug
Follow-up Comment #7, bug #22923 (project make):
Reaching back 25 years again, this is basically what Alliant Concentrix
parallel make did. It prefixed each output line with the job number |1|, |2|
and so on. (This is also why in my original proposal for make -j, I used
different numbered job
Follow-up Comment #3, bug #28134 (project make):
As a complete digression, and based on 26 years of doing parallel builds
(damn it's been a long time), I just wanted to point out that you can
generally use 50% more jobs than your number of CPUs/cores. E.g., for a
dual-core machine, use make -j3.
of their output was prefixed with |jobnum|. It was
quite nice, but I never felt it was important enough to duplicate that
behavior for gmake.
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http
with it doesn't really enter the equation.
Aside from that, happy to leave this for later.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sunhttp://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project
use maybe -j 10 (when I know 10 more calls to
make are made)
On some machines this might be unacceptable and bring them to their
knees.
On other machines it can work really well.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sunhttp
doesn't impose restrictions retroactively. (E.g., if you open a file and
have a valid descriptor on it, and someone else chmods the file,
removing your access permissions, you still have your original
privileges through the open descriptor.)
--
-- Howard Chu
Chief Architect, Symas Corp. http
are good, but stack-based allocation is still better
a lot of the time. Especially for temporary variables that are just
going to be discarded after a few computations.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director, Highland Sunhttp://highlandsun.com/hyc
Follow-up Comment #3, bug #17873 (project make):
Haven't been paying a lot of attention, but when is such a lame thing nas
.NOTPARALLEL needed? Why aren't people just writing proper dependencies so
that parallel/serial is irrelevant?
___
URL:
http://savannah.gnu.org/bugs/?17881
Summary: Better documentation of make rules
Project: make
Submitted by: hyc
Submitted on: Thursday 09/28/2006 at 20:26
Severity: 3 - Normal
Item Group: Enhancement
Bah. Pretty sure I selected Documentation for this submission, not
Enhancement.
Howard Chu wrote:
URL:
http://savannah.gnu.org/bugs/?17881
Summary: Better documentation of make rules
Project: make
Submitted by: hyc
Submitted
Follow-up Comment #1, bug #17881 (project make):
It might be worthwhile to add some kind of makefile validation flag, which
processes all prerequisites in random or right-to-left order, specifically to
expose occurrences of incorrect prerequisites.
that!?) I've noticed that SCons has this feature, but
it might come in handy for verifying that a makefile is properly
written.
Yes, I've suggested this a number of times in the past. Never formally
in a bug report, but #17881 which I just submitted lists that as well.
--
-- Howard Chu
Chief
that I couldn't live without, and it was worth my effort to push it
forward. Frankly this is trivial and personally uninteresting, and I can
live without it just fine, since I know how to write correct Makefiles.
--
-- Howard Chu
Chief Architect, Symas Corp. http://www.symas.com
Director
: ; @echo $@; sleep 1; echo [EMAIL PROTECTED] mailto:[EMAIL
PROTECTED]
--
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support
, there will be no difference
whether make is run serially or in parallel. If there is any difference,
then the Makefile rules didn't correctly capture all of the real
dependencies of the project.
--
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com
examples have nothing to do with each other, I don't see
how you can reach this conclusion.
--
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
Symas: Premier OpenSource Development and Support
I would omit the XARGS LEN. Presumably the local platform's xargs command is
already properly configured for the native system's line length limitations.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
problem mechanics were
understood...
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
-Original Message-
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]]On Behalf Of
Reid Madsen
Sent: Monday
Try "xargs command tmpfile" instead of "cat tmpfile | xargs command" ...
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
-Original Message-
From: [EMAIL PROTECTED] [mai
This is a known bug in 3.77. 3.79 is the current version, you really need to
update.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com http://highlandsun.com/hyc
-Original Message-
From: John Lewis 425-865-3510 [mailto
that a file does not exist, even
though it does. The big question is what is generating the signals -
presumably
it's SIGCHLD in this case, and there's probably not much you can do about
that.
-- Howard Chu
Chief Architect, Symas Corp. Director, Highland Sun
http://www.symas.com
46 matches
Mail list logo
