Re: make modules kicks the first module directory twice
On Tue, 14 Nov 2000, Warner Losh wrote:
make depend is already *REQUIRED* when you are updating a kernel from
an older version of the kernel. For config -r FOO kernels it isn't.
It is neither necessary nor sufficient for updating. config -r removes
conflicting garbage from previous builds. Removing the conflicting
garbage is sufficient. `make clean' normally suffices to remove it,
but there are cases where .depend itself is garbage and must be removed
to unbreak `make depend'.
Even a make clean after a make depend will require that make depend be
run again.
This was broken in rev.1.348 of sys/conf/files, etc. `make clean'
removes the generated bus interface headers, and there are no longer
enough dependencies for make(1) to know how to rebuild them. There
are enough dependencies to rebuild them for `make depend', and for
plain `make' when .depend doesn't exist (minor magic involving
${BEFORE_DEPEND}). .depend still doesn't contain all the dependencies
on headers, because at least the generated bus interface .c files are
handled too specially.
Bruce
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Mon, Nov 13, 2000 at 09:17:54PM -0700, Warner Losh wrote: The implications are that make obj isn't done unless you've run make depend first. If a new directory is added and a make depend isn't run, then the modules won't get built into the obj tree, but instead will be built into $S/modules. Having modules wind up in two trees is not acceptable IMHO. I'd rather take a major compile time hit and be deterministic than not. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: RQ review: [was: Re: make modules kicks the first module directory twice]
On Mon, Nov 13, 2000 at 08:02:47PM -0800, Marcel Moolenaar wrote:
Any objections?
Yes.
(patches follow for your convenience)
[its easier to read patches when they aren't quoted in their entirety ;-)]
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj
! env ${MKMODULESENV} ${MAKE} depend
This is broken for non -j case.
--
-- David ([EMAIL PROTECTED])
GNU is Not Unix / Linux Is Not UniX
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
In message [EMAIL PROTECTED] "David O'Brien" writes: : On Mon, Nov 13, 2000 at 09:17:54PM -0700, Warner Losh wrote: : The implications are that make obj isn't done unless you've run make : depend first. If a new directory is added and a make depend isn't : run, then the modules won't get built into the obj tree, but instead : will be built into $S/modules. : : Having modules wind up in two trees is not acceptable IMHO. But they are both in the $S tree. :-) : I'd rather take a major compile time hit and be deterministic than not. I'd rather not. We don't do an implicit make obj in the rest of the tree. If I go build the world, and then someone adds a new program to the tree, you are in the same boat. If you cd to that program and type make it will wind up in . rather than /usr/obj. Completely deterministic, the same thing will happen every time you do the scenario. make depend is already *REQUIRED* when you are updating a kernel from an older version of the kernel. For config -r FOO kernels it isn't. Even a make clean after a make depend will require that make depend be run again. But before making major changes to this, let's see Peter Wemm's new all singing all dancing config work does for us. I'd rather see what he's come up with than argue further on this. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
[stable dropped, this should have only been in a single list to start with!!] On Tue, Nov 14, 2000 at 01:27:32PM -0700, Warner Losh wrote: : I'd rather take a major compile time hit and be deterministic than not. I'd rather not. We don't do an implicit make obj in the rest of the tree. It is in `make world' if /usr/obj exists. Otherwise how does all the .o's get in /usr/obj/ ? If I go build the world, and then someone adds a new program to the tree, you are in the same boat. Nope, `make world' will DTRT. If you cd to that program and type make it will wind up in . rather than /usr/obj. Yes, and how many times do we have to tell people to run ``make cleandir make cleandir'' or ``rm -rf /usr/obj/* ; cd /usr/src ; make cleandir '' Completely deterministic, the same thing will happen every time you do the scenario. Ok, completely deterministic given enough detail -- detail which 99% of the time will not be provided in email to the lists saying this and that is broken. But before making major changes to this, let's see Peter Wemm's new all singing all dancing config work does for us. I'd rather see what he's come up with than argue further on this. Earlier today, I too I was wondering if his plan makes all this OBE. -- -- David ([EMAIL PROTECTED]) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
In message [EMAIL PROTECTED] "David O'Brien" writes: : [stable dropped, this should have only been in a single list to start with!!] Agreed. My summary: o We disagree about the support impact o Peter's stuff may OBE this whole thread o My change shouldn't be committed until we know it will have no support impact, or it can be fixed to be more robust. : On Tue, Nov 14, 2000 at 01:27:32PM -0700, Warner Losh wrote: : : I'd rather take a major compile time hit and be deterministic than not. : : I'd rather not. We don't do an implicit make obj in the rest of the : tree. : : It is in `make world' if /usr/obj exists. Otherwise how does all the : .o's get in /usr/obj/ ? I should have stated this more clearly. We don't have an implicit make obj anywhere else in the tree for the default "all" target. We do have it for other, special targets. : If I go build the world, and then someone adds a new program to : the tree, you are in the same boat. : : Nope, `make world' will DTRT. No. Not if you compile that one file. Also, make all won't do the right thing. That's my point. We have extra special targets that are all singing all dancing that do the right thing, but the plain vanilla ones act in a plain vanilla way. Again, make world isn't the default target. It is an extra special target that does special things. I've been burned by the example that I sighted. : If you cd to that program and type make it will wind up in . rather : than /usr/obj. : : Yes, and how many times do we have to tell people to run : ``make cleandir make cleandir'' or : ``rm -rf /usr/obj/* ; cd /usr/src ; make cleandir '' A few, but a lot less than I'd expect :-). : Completely deterministic, the same thing will happen every time you do : the scenario. : : Ok, completely deterministic given enough detail -- detail which 99% of : the time will not be provided in email to the lists saying this and that : is broken. True. But it would be rare enough that people would have two different kernels, with different revs of the sources in the same source tree and that those differences would cause problems. It would sure be hard to find it, I'll grant that. I think it would be rare enough that we won't get mail on it, but I could be wrong about that. : But before making major changes to this, let's see Peter Wemm's new : all singing all dancing config work does for us. I'd rather see what : he's come up with than argue further on this. : : Earlier today, I too I was wondering if his plan makes all this OBE. He sure has been silent on all of this. :-) I do think that we're all in agreement that make obj all should be changed to make obj\n make all to make the parallel case work. The rest shouldn't be committed until we have it more robust. I'll take alook to see if there's a robust way to know if we need to run make obj or not. To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: RQ review: [was: Re: make modules kicks the first module directory twice]
David O'Brien wrote:
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj
! env ${MKMODULESENV} ${MAKE} depend
This is broken for non -j case.
Yes, this was known. The right diff was given at the beginning of the
message including the comment that the original patch had this breakage
:-)
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Sun, 12 Nov 2000, Marcel Moolenaar wrote:
Makoto MATSUSHITA wrote:
% make -j 2 modules
cd ../../modules env MAKEOBJDIRPREFIX=/usr/src/sys/compile/GENERIC/modules
KMODDIR=/boot/kernel make obj all
=== 3dfx
=== 3dfx
Warning: Object directory not changed from original /usr/src/sys/modules/3dfx
(... ok, break it ...)
The problem is in the fact that the Makefile (the one in /sys/conf)
contains something like:
${MAKE} obj all
and
${MAKE} obj depend
The net effect is that these targets are built in parallel, which
obviously isn't right. The following solves the problem (i386 only):
That's just one of the problems :-). "make obj all" is usually an
error, but in Makefile.${MACHINE} it should be just a bad example,
since the `obj' and `all' targets should be built sequentially and then
the object directories will exist by the time make(1) recurses into
them for the `all' target. This doesn't work right for the -j case.
(In the above example, the targets are built concurrently and race
each other. This is bad when the `all' target wins the race. The
`obj' target runs faster, so it usually wins the race except in the
first directory (3dfx)). More .ORDER statements in *.mk are required.
Index: Makefile.i386
===
RCS file: /home/ncvs/src/sys/conf/Makefile.i386,v
retrieving revision 1.212
diff -u -r1.212 Makefile.i386
--- Makefile.i386 2000/10/29 09:47:50 1.212
+++ Makefile.i386 2000/11/13 07:49:00
@@ -271,11 +271,13 @@
modules:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj \
+ env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj $$ \
+ env ${MKMODULESENV} ${MAKE} depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
`' should never be used in shell commands in makefiles, although it
may be only a bad example. This is because multiple commands are
executed in the same shell in the -j case, and `' gives non-simple
commands which may defeat the shell's -e setting. E.g., the command:
cd /; set -e; cd /nonesuch false; rm -rf *
removes everything under "/", not everything under /nonesuch, despite
checking that the cd to /nonesuch worked.
Bruce
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
bde (In the above example, the targets are built concurrently and race
bde each other. This is bad when the `all' target wins the race. The
bde `obj' target runs faster, so it usually wins the race except in the
bde first directory (3dfx)). More .ORDER statements in *.mk are required.
Thank you for giving us a details of this problem. But can we fix this
problem with .ORDER statements? Putting ".ORDER: obj all" or alike to
Makefile doesn't fix to me (maybe I misunderstand the usage of .ORDER
statement)...
bde `' should never be used in shell commands in makefiles, although it
bde may be only a bad example. This is because multiple commands are
bde executed in the same shell in the -j case, and `' gives non-simple
bde commands which may defeat the shell's -e setting.
What should we do if we want to check the existence of a directory and
kick one (not two or more) command after chdir to that directory? Much
Makefiles in our FreeBSD repository employ "cd ${dir} command" to
do this... Should we say
if [ -d ${dir} ]; then \
(cd ${dir}; command) \
else
false
fi
or exist() directive of make(1) ?
***
Anyway, attached below is current sample patch to fix this problem.
"make obj" and "make all" are separated to two individual commands
(redundant 'cd' clause, but it should work).
-- -
Makoto `MAR' MATSUSHITA
Index: Makefile
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/release/Makefile,v
retrieving revision 1.585
diff -c -r1.585 Makefile
*** Makefile2000/11/12 11:04:11 1.585
--- Makefile2000/11/13 05:37:12
***
*** 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make kernel-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
--- 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make modules-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
Index: Makefile.alpha
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.alpha,v
retrieving revision 1.79
diff -c -r1.79 Makefile.alpha
*** Makefile.alpha 2000/10/29 09:47:50 1.79
--- Makefile.alpha 2000/11/13 10:09:34
***
*** 310,320
modules:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
--- 310,322
modules:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
Index: Makefile.i386
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.i386,v
retrieving revision 1.212
diff -c -r1.212 Makefile.i386
*** Makefile.i386 2000/10/29 09:47:50 1.212
--- Makefile.i386 2000/11/13 10:10:03
***
*** 271,281
modules:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
--- 271,283
modules:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
Index: Makefile.ia64
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.ia64,v
retrieving revision 1.4
diff -c -r1.4 Makefile.ia64
*** Makefile.ia64 2000/10/29 09:47:50 1.4
--- Makefile.ia64 2000/11/13 10:10:41
***
*** 275,285
modules:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
modules-depend:
Re: make modules kicks the first module directory twice
On Mon, 13 Nov 2000, Makoto MATSUSHITA wrote:
bde (In the above example, the targets are built concurrently and race
bde each other. This is bad when the `all' target wins the race. The
bde `obj' target runs faster, so it usually wins the race except in the
bde first directory (3dfx)). More .ORDER statements in *.mk are required.
Thank you for giving us a details of this problem. But can we fix this
problem with .ORDER statements? Putting ".ORDER: obj all" or alike to
Makefile doesn't fix to me (maybe I misunderstand the usage of .ORDER
statement)...
It didn't work for me either :-). I put it in bsd.obj.mk near the `obj'
target, with the idea that this would cover all uses of the obj target.
The problem seems to be that we both put it in the wrong place. I think
it needs to be in bsd.subdir.mk for this case and in bsd.obj.mk for most
cases.
bde `' should never be used in shell commands in makefiles, although it
bde may be only a bad example. This is because multiple commands are
bde executed in the same shell in the -j case, and `' gives non-simple
bde commands which may defeat the shell's -e setting.
What should we do if we want to check the existence of a directory and
kick one (not two or more) command after chdir to that directory? Much
Makefiles in our FreeBSD repository employ "cd ${dir} command" to
do this... Should we say
if [ -d ${dir} ]; then \
(cd ${dir}; command) \
else
false
fi
or exist() directive of make(1) ?
Just use a semicolon instead of "" ("cd foo; command"). This gives
multiple single commands, and make(1) execs sh(1) with -e, so the shell
exits if any of the simple commands fails. ("simple" here is a technical
term. See sh.1.)
Bruce
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Sun, Nov 12, 2000 at 11:58:39PM -0700, Warner Losh wrote: In message [EMAIL PROTECTED] Makoto MATSUSHITA writes: : It does not fix this problem. However, if we separate the execution of : "make obj" and "make all", we can avoid (again, not *fix*) the problem. : Maybe this change is reasonable; there is few meaning doing "make : kernel-depend" in "doMODULES" target. I think that make has no business doing an implicit make obj for the all target. Someone has to run `make obj' for the modules tree. How are you doing it locally? I have been running with patches in my tree that doesn't do the obj target if .depend exists. can you post them for review. Of course, you have to be more careful about running make depend in that case Now *that* sounds scarry to me. We'll have people all over the lists who forget to and would now get rather bitten. From the sounds of it, your patch doesn't create as robust a world, but maybe it does. -- -- David ([EMAIL PROTECTED]) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Mon, Nov 13, 2000 at 07:51:11PM +0900, Makoto MATSUSHITA wrote:
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
--- 310,322
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} all
I can certainly commit this type of fix. The combined make invocation
was an optimization someone recommended.
--
-- David ([EMAIL PROTECTED])
GNU is Not Unix / Linux Is Not UniX
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
Bruce Evans wrote:
Index: Makefile.i386
===
RCS file: /home/ncvs/src/sys/conf/Makefile.i386,v
retrieving revision 1.212
diff -u -r1.212 Makefile.i386
--- Makefile.i386 2000/10/29 09:47:50 1.212
+++ Makefile.i386 2000/11/13 07:49:00
@@ -271,11 +271,13 @@
modules:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj \
+ env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj $$ \
+ env ${MKMODULESENV} ${MAKE} depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
`' should never be used in shell commands in makefiles, although it
may be only a bad example.
Mostly that, yes. The first try for a "fix" was:
cd $S/modules env ${MKMODULESENV} ${MAKE} obj
cd $S/modules env ${MKMODULESENV} ${MAKE} all
But since $S expands to "../.." and -j runs a single shell, the first cd
will succeed, but not the second.
I simply copied the "construction" in the second and posted fix.
I'll commit a fix with just semi-colons today for all architectures if
someone hasn't done that already by that time.
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
David O'Brien wrote:
On Mon, Nov 13, 2000 at 07:51:11PM +0900, Makoto MATSUSHITA wrote:
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
--- 310,322
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} all
I can certainly commit this type of fix. The combined make invocation
was an optimization someone recommended.
No you can't. $S expands to "../.." which only works for the first cd in
the -jX case. The second cd will fail.
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
Marcel Moolenaar wrote:
David O'Brien wrote:
On Mon, Nov 13, 2000 at 07:51:11PM +0900, Makoto MATSUSHITA wrote:
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
--- 310,322
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj
! cd $S/modules env ${MKMODULESENV} ${MAKE} all
I can certainly commit this type of fix. The combined make invocation
was an optimization someone recommended.
No you can't.
Oops. A smiley was intended here. I'm not the authoritive kind of guy
:-)
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Mon, Nov 13, 2000 at 01:19:38PM -0500, Marcel Moolenaar wrote: I'll commit a fix with just semi-colons today for all architectures if someone hasn't done that already by that time. Can you post a patch first. There seems to be some subtleties here that might make a review useful. -- -- David ([EMAIL PROTECTED]) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
On Mon, Nov 13, 2000 at 12:23:08PM -0700, Warner Losh wrote: : I think that make has no business doing an implicit make obj for the : all target. : Someone has to run `make obj' for the modules tree. How are you doing it : locally? Right now we do it twice. Once in make dpeend and again in make all. My patch removes it from make all. ..snip.. I'm not sure what you're criteria for a robust world is here. If the directory doesn't exist, it will warn the user. It doesn't warn the user, it errors out (possibly a suttle distinction I'm making). Also in the past a `make depend' for the kernel was not required. Just highly suggested. Are we really prepared to make it a requirement now? To tell the truth, IMHO the modules should just build in the current directory (and thus could share some .o's with the kernel build). -- -- David ([EMAIL PROTECTED]) GNU is Not Unix / Linux Is Not UniX To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
In message [EMAIL PROTECTED] "David O'Brien" writes: : It doesn't warn the user, it errors out (possibly a suttle distinction : I'm making). Also in the past a `make depend' for the kernel was not : required. Just highly suggested. Are we really prepared to make it a : requirement now? I am, but others might not be. : To tell the truth, IMHO the modules should just build in the current : directory (and thus could share some .o's with the kernel build). Me too. I'm tired of building things twice :-) Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
David O'Brien wrote: On Mon, Nov 13, 2000 at 01:19:38PM -0500, Marcel Moolenaar wrote: I'll commit a fix with just semi-colons today for all architectures if someone hasn't done that already by that time. Can you post a patch first. There seems to be some subtleties here that might make a review useful. Sure. BTW: I'm also looking at Warner's patch. Maybe that's the better fix for it, but I have to dig into the Makefiles a bit more to get a better picture... -- Marcel Moolenaar mail: [EMAIL PROTECTED] / [EMAIL PROTECTED] tel: (408) 447-4222 To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
marcel No you can't. $S expands to "../.." which only works for the
marcel first cd in the -jX case. The second cd will fail.
Ouch... give me one more chance to submit a patch. Here's summary:
* src/release/Makefile should use 'module-depend' while
checking dependancy of modules (not kernel-depend).
* For parallel build: "command command" - "command; command"
* For parallel build: "make obj depend" - "make obj; make depend"
* Avoid to do run "obj" target again in "modules" target
(comes from an email of [EMAIL PROTECTED])
-- -
Makoto `MAR' MATSUSHITA
Index: Makefile
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/release/Makefile,v
retrieving revision 1.585
diff -c -r1.585 Makefile
*** Makefile2000/11/12 11:04:11 1.585
--- Makefile2000/11/13 05:37:12
***
*** 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make kernel-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
--- 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make modules-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
Index: Makefile.alpha
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.alpha,v
retrieving revision 1.79
diff -c -r1.79 Makefile.alpha
*** Makefile.alpha 2000/10/29 09:47:50 1.79
--- Makefile.alpha 2000/11/14 00:42:39
***
*** 252,258
echo ${CFILES} | tr -s ' ' '\12' | sed 's/\.c/.o/' | \
sort -u | comm -23 - dontlink | \
sed 's,../.*/\(.*.o\),rm -f \1;ln -s ../GENERIC/\1 \1,' makelinks
! sh makelinks rm -f dontlink
kernel-tags:
@[ -f .depend ] || { echo "you must make depend first"; exit 1; }
--- 252,258
echo ${CFILES} | tr -s ' ' '\12' | sed 's/\.c/.o/' | \
sort -u | comm -23 - dontlink | \
sed 's,../.*/\(.*.o\),rm -f \1;ln -s ../GENERIC/\1 \1,' makelinks
! sh makelinks; rm -f dontlink
kernel-tags:
@[ -f .depend ] || { echo "you must make depend first"; exit 1; }
***
*** 309,338
MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR}
modules:
! @mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
modules-clean:
! cd $S/modules env ${MKMODULESENV} ${MAKE} clean
modules-cleandepend:
! cd $S/modules env ${MKMODULESENV} ${MAKE} cleandepend
modules-cleandir:
! cd $S/modules env ${MKMODULESENV} ${MAKE} cleandir
modules-tags:
! cd $S/modules env ${MKMODULESENV} ${MAKE} tags
modules-install modules-install.debug:
! cd $S/modules env ${MKMODULESENV} ${MAKE} install
modules-reinstall modules-reinstall.debug:
! cd $S/modules env ${MKMODULESENV} ${MAKE} install
config.o:
${NORMAL_C}
--- 309,342
MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR}
modules:
! @if [ ! -d ${.OBJDIR}/modules ]; then \
! echo You must run make depend before building modules; \
! exit 1; \
! fi
! cd $S/modules; env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj
! env ${MKMODULESENV} ${MAKE} depend
modules-clean:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} clean
modules-cleandepend:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} cleandepend
modules-cleandir:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} cleandir
modules-tags:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} tags
modules-install modules-install.debug:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} install
modules-reinstall modules-reinstall.debug:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} install
config.o:
${NORMAL_C}
Index: Makefile.i386
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.i386,v
retrieving revision 1.212
diff -c -r1.212 Makefile.i386
*** Makefile.i386 2000/10/29 09:47:50 1.212
--- Makefile.i386 2000/11/14 00:43:16
***
*** 212,218
echo ${CFILES} | tr -s ' ' '\12' | sed 's/\.c/.o/' | \
sort -u | comm -23 - dontlink | \
sed 's,../.*/\(.*.o\),rm
Re: make modules kicks the first module directory twice
[-stable removed from the cc list]
Makoto MATSUSHITA wrote:
marcel No you can't. $S expands to "../.." which only works for the
marcel first cd in the -jX case. The second cd will fail.
Ouch...
Sorry :-)
give me one more chance to submit a patch. Here's summary:
You're not going to like this:
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj
! env ${MKMODULESENV} ${MAKE} depend
For the non parallel case this doesn't work, because the "${MAKE}
depend" is not run from the right directory (ie "$S/modules") because
the second make doesn't get started from the same shell. Example:
gauss% more ~/mf
all:
cd ../marcel; pwd
pwd
*** Non-parallel case
gauss% cd /xtra/ncvs
gauss% make -f ~/mf
cd ../marcel; pwd
/xtra/marcel
pwd
/xtra/ncvs
*** Parallel case
gauss% make -f ~/mf -j2
cd ../marcel; pwd
/xtra/marcel
pwd
/xtra/marcel
...other than that... :-)
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
RQ review: [was: Re: make modules kicks the first module directory twice]
Makoto MATSUSHITA wrote:
Ouch... give me one more chance to submit a patch. Here's summary:
I see no reason to not commit Makoto-san's patches with the fix he sent
me for the modules-depend target. The fix is (modulo indentation):
modules-depend:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj; \
! env ${MKMODULESENV} ${MAKE} depend
It contains everything mentioned so far, which is:
* src/release/Makefile should use 'module-depend' while
checking dependancy of modules (not kernel-depend).
* For parallel build: "command command" - "command; command"
* For parallel build: "make obj depend" - "make obj; make depend"
* Avoid to do run "obj" target again in "modules" target
(comes from an email of [EMAIL PROTECTED])
Any objections?
(patches follow for your convenience)
Index: Makefile
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/release/Makefile,v
retrieving revision 1.585
diff -c -r1.585 Makefile
*** Makefile2000/11/12 11:04:11 1.585
--- Makefile2000/11/13 05:37:12
***
*** 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make kernel-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
--- 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make modules-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
Index: Makefile.alpha
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.alpha,v
retrieving revision 1.79
diff -c -r1.79 Makefile.alpha
*** Makefile.alpha 2000/10/29 09:47:50 1.79
--- Makefile.alpha 2000/11/14 00:42:39
***
*** 252,258
echo ${CFILES} | tr -s ' ' '\12' | sed 's/\.c/.o/' | \
sort -u | comm -23 - dontlink | \
sed 's,../.*/\(.*.o\),rm -f \1;ln -s ../GENERIC/\1 \1,' makelinks
! sh makelinks rm -f dontlink
kernel-tags:
@[ -f .depend ] || { echo "you must make depend first"; exit 1; }
--- 252,258
echo ${CFILES} | tr -s ' ' '\12' | sed 's/\.c/.o/' | \
sort -u | comm -23 - dontlink | \
sed 's,../.*/\(.*.o\),rm -f \1;ln -s ../GENERIC/\1 \1,' makelinks
! sh makelinks; rm -f dontlink
kernel-tags:
@[ -f .depend ] || { echo "you must make depend first"; exit 1; }
***
*** 309,338
MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR}
modules:
! @mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
modules-clean:
! cd $S/modules env ${MKMODULESENV} ${MAKE} clean
modules-cleandepend:
! cd $S/modules env ${MKMODULESENV} ${MAKE} cleandepend
modules-cleandir:
! cd $S/modules env ${MKMODULESENV} ${MAKE} cleandir
modules-tags:
! cd $S/modules env ${MKMODULESENV} ${MAKE} tags
modules-install modules-install.debug:
! cd $S/modules env ${MKMODULESENV} ${MAKE} install
modules-reinstall modules-reinstall.debug:
! cd $S/modules env ${MKMODULESENV} ${MAKE} install
config.o:
${NORMAL_C}
--- 309,342
MKMODULESENV= MAKEOBJDIRPREFIX=${.OBJDIR}/modules KMODDIR=${KODIR}
modules:
! @if [ ! -d ${.OBJDIR}/modules ]; then \
! echo You must run make depend before building modules; \
! exit 1; \
! fi
! cd $S/modules; env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
! cd $S/modules; env ${MKMODULESENV} ${MAKE} obj
! env ${MKMODULESENV} ${MAKE} depend
modules-clean:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} clean
modules-cleandepend:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} cleandepend
modules-cleandir:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} cleandir
modules-tags:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} tags
modules-install modules-install.debug:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} install
modules-reinstall modules-reinstall.debug:
! cd $S/modules; env ${MKMODULESENV} ${MAKE} install
config.o:
${NORMAL_C}
Index: Makefile.i386
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/sys/conf/Makefile.i386,v
Re: make modules kicks the first module directory twice
In message [EMAIL PROTECTED] Marcel Moolenaar writes: : BTW: I'm also looking at Warner's patch. Maybe that's the better fix for : it, but I have to dig into the Makefiles a bit more to get a better : picture... The implications are that make obj isn't done unless you've run make depend first. If a new directory is added and a make depend isn't run, then the modules won't get built into the obj tree, but instead will be built into $S/modules. This isn't too bad because unless you have multiple kernels that are skewed in version that you'll be doing make installs from from the same tree (which isn't supported, officially, now, but does accidentally usually work). Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
make modules kicks the first module directory twice
(Strip down To: address, maybe it's only -current problem so I send
this email to [EMAIL PROTECTED] also. Set appropriate Subject:. -- MAR)
marcel Makoto-san mentioned there's a problem with 3dfx. Is this
marcel still the case or has it been resolved?
Unfortunately, no. It seems that nobody use -j option already :-)
***
Most audiences doesn't understand what's the problem (it's my fault).
I've checked again, and here is "how to reproduce the problem easily":
(Prepare fresh kernel source code tree; this example uses /usr/src/sys.)
% cd /usr/src/sys/conf/GENERIC; config GENERIC
(... stuff deleted ...)
% cd ../../compile/GENERIC
% make -j 2 modules
cd ../../modules env MAKEOBJDIRPREFIX=/usr/src/sys/compile/GENERIC/modules
KMODDIR=/boot/kernel make obj all
=== 3dfx
=== 3dfx
Warning: Object directory not changed from original /usr/src/sys/modules/3dfx
(... ok, break it ...)
'3dfx' directory is kicked twice. Apparantly, the one assumes that
/usr/src/sys/modules/3dfx is an object directory, and yet another one
assumes maybe /usr/src/sys/compile/GENERIC/modules is an object
directory (this is correct). Note that make(1) uses only one directory
for module building (I dunno why), so if the former one is used, the
latter one is simply ignored (again, I dunno why).
It's O.K. if /usr/src/sys/compile/GENERIC/modules is used correctly;
however, what will happen if only /usr/src/sys/modules/3dfx is used as
an object directory? -- that's problem.
So... you cannot supply the make variable, KERNEL_FLAG='-j 2', to your
"make release" since you've hit this problem. Don't blame '3dfx'
module. It's simply the first module directory listed in SUBDIRS
variable; you can see the same problem even if you change the order of
SUBDIRS directories.
I tried to fix, but sorry I cannot find an answer. However, here is an
escape of this problem:
Index: Makefile
===
RCS file: /lab/FreeBSD/FreeBSD.cvs/src/release/Makefile,v
retrieving revision 1.585
diff -c -r1.585 Makefile
*** Makefile2000/11/12 11:04:11 1.585
--- Makefile2000/11/13 05:37:12
***
*** 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make kernel-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
--- 831,837
@rm -f ${RD}/kernels/*.ko
@cd ${.CURDIR}/../sys/${MACHINE}/conf config ${KERNEL}
@cd ${.CURDIR}/../sys/compile/${KERNEL} \
! make modules-depend \
make ${KERNEL_FLAGS} modules \
make modules-reinstall DESTDIR=${RD}/kernels \
It does not fix this problem. However, if we separate the execution of
"make obj" and "make all", we can avoid (again, not *fix*) the problem.
Maybe this change is reasonable; there is few meaning doing "make
kernel-depend" in "doMODULES" target.
-- -
Makoto `MAR' MATSUSHITA
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
In message [EMAIL PROTECTED] Makoto MATSUSHITA writes: : It does not fix this problem. However, if we separate the execution of : "make obj" and "make all", we can avoid (again, not *fix*) the problem. : Maybe this change is reasonable; there is few meaning doing "make : kernel-depend" in "doMODULES" target. I think that make has no business doing an implicit make obj for the all target. I have been running with patches in my tree that doesn't do the obj target if .depend exists. Of course, you have to be more careful about running make depend in that case, but right now the make obj in the all target is hiding this problem. So I would agree with your patch and your proposal. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-current" in the body of the message
Re: make modules kicks the first module directory twice
Makoto MATSUSHITA wrote:
% make -j 2 modules
cd ../../modules env MAKEOBJDIRPREFIX=/usr/src/sys/compile/GENERIC/modules
KMODDIR=/boot/kernel make obj all
=== 3dfx
=== 3dfx
Warning: Object directory not changed from original /usr/src/sys/modules/3dfx
(... ok, break it ...)
The problem is in the fact that the Makefile (the one in /sys/conf)
contains something like:
${MAKE} obj all
and
${MAKE} obj depend
The net effect is that these targets are built in parallel, which
obviously isn't right. The following solves the problem (i386 only):
Index: Makefile.i386
===
RCS file: /home/ncvs/src/sys/conf/Makefile.i386,v
retrieving revision 1.212
diff -u -r1.212 Makefile.i386
--- Makefile.i386 2000/10/29 09:47:50 1.212
+++ Makefile.i386 2000/11/13 07:49:00
@@ -271,11 +271,13 @@
modules:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj all
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj \
+ env ${MKMODULESENV} ${MAKE} all
modules-depend:
@mkdir -p ${.OBJDIR}/modules
- cd $S/modules env ${MKMODULESENV} ${MAKE} obj depend
+ cd $S/modules env ${MKMODULESENV} ${MAKE} obj $$ \
+ env ${MKMODULESENV} ${MAKE} depend
modules-clean:
cd $S/modules env ${MKMODULESENV} ${MAKE} clean
--
Marcel Moolenaar
mail: [EMAIL PROTECTED] / [EMAIL PROTECTED]
tel: (408) 447-4222
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message
