Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 October 2012 22:10, Simon J. Gerraty s...@juniper.net wrote: On Sat, 27 Oct 2012 19:53:56 +0100, Chris Rees writes: I'm saying that it's unacceptable to expect people to change their systems just to make the ports tree work after we have broken it on a supposedly supported version. But there's no suggestion of that. The ports tree would take care of itself. The comment about fixing makefiles refered to the concern about things outside of base/ports. From your comment, I now understand that we aren't having the same conversation :) Please answer me these to check we're on the same page: Are we planning to replace /usr/bin/make with bmake in the near future? If yes, what changes are we going to make to the ports tree to ensure that -CURRENT can still use it? Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 28 October 2012 19:11, Simon J. Gerraty s...@juniper.net wrote: On Sun, 28 Oct 2012 14:06:41 +, Chris Rees writes: Are we planning to replace /usr/bin/make with bmake in the near future? That was what I heard, but any such move is dependent on dealing with ports. The ~sjg/ports2bmake.tar.gz on freefall is the plan I came up with after the above requirement was introduced at last BSDCan. If yes, what changes are we going to make to the ports tree to ensure that -CURRENT can still use it? If you mean -current (aka head); the plan is to convert ports to bmake syntax wrt to the 2 conflicting modifiers. At my last test there are just under 300 makefiles in ports that use the old modifiers. Now for head (ie. /usr/bin/make is an old version), the above ports tree detects that bmake is not being used, and invokes a shell script (bmake-sh) to do what was asked. That script will look for bmake and if necessary build/install it. To do that, it creates a temp copy of Mk/*.mk converted back to the old syntax so that the old make can build and install bmake, and then the old system is on par with -current. That's what I meant by ports will take care of itself. The main gap btw in the above, is if a user who does not have privs to install bmake, is the only person trying to do something with ports. The above plan needs to be approved by portmgr, and obviouslty a test run of building all ports is needed (possibly more than one). Does that help? Certainly, thank you. I didn't find it clear when inspecting the tarball (obviously I hadn't read the README clearly enough). I'm going to have to echo John's non-obvious comment however, and I think it would be very helpful to have a clear public writeup, perhaps QA style to allay any other such fears. Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sun, 28 Oct 2012 14:06:41 +, Chris Rees writes: Are we planning to replace /usr/bin/make with bmake in the near future? That was what I heard, but any such move is dependent on dealing with ports. The ~sjg/ports2bmake.tar.gz on freefall is the plan I came up with after the above requirement was introduced at last BSDCan. If yes, what changes are we going to make to the ports tree to ensure that -CURRENT can still use it? If you mean -current (aka head); the plan is to convert ports to bmake syntax wrt to the 2 conflicting modifiers. At my last test there are just under 300 makefiles in ports that use the old modifiers. Now for head (ie. /usr/bin/make is an old version), the above ports tree detects that bmake is not being used, and invokes a shell script (bmake-sh) to do what was asked. That script will look for bmake and if necessary build/install it. To do that, it creates a temp copy of Mk/*.mk converted back to the old syntax so that the old make can build and install bmake, and then the old system is on par with -current. That's what I meant by ports will take care of itself. The main gap btw in the above, is if a user who does not have privs to install bmake, is the only person trying to do something with ports. The above plan needs to be approved by portmgr, and obviouslty a test run of building all ports is needed (possibly more than one). Does that help? --sjg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote: On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes: In that case we have a switch time on the order of years, not weeks; 8.3 is supported until May '14, and unless we get a :tl etc MFC into 8, even longer. All this time the ports tree must work with pmake. I'm pretty sure I was told it is already in 8 and 7 Not in 8.3 at least: svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log I don't want to discourage you or belittle your excellent work here, but Marcel made me very nervous with his comment on the process being a few weeks. That was based on discussions at the last devsummit. These discussions need backing up with a real roadmap, including detail on exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree still works. I don't see where these considerations have been made. Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
BTW, would it be useful to put a devel/fmake into ports to make it easy for people with older systems to install an up to date version of freebsd make (which groks both sets of toupper/tolower modifiers)? Perhaps a knob to install it or put in a link as /usr/bin/make ? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
[trim CC list a little to stop people regretting replying to this thread] On 27 October 2012 10:15, Chris Rees utis...@gmail.com wrote: On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote: On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes: In that case we have a switch time on the order of years, not weeks; 8.3 is supported until May '14, and unless we get a :tl etc MFC into 8, even longer. All this time the ports tree must work with pmake. I'm pretty sure I was told it is already in 8 and 7 Not in 8.3 at least: svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log I don't want to discourage you or belittle your excellent work here, but Marcel made me very nervous with his comment on the process being a few weeks. That was based on discussions at the last devsummit. These discussions need backing up with a real roadmap, including detail on exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree still works. I don't see where these considerations have been made. OK, so how about this. We (ab)use the security update mechanism to merge the pmake changes (:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier releng branches such as 7.3, 8.2, 9.0). We could then send out a message on ports-announce, giving a few weeks' notice that the change to bsd.port.mk is going through and that users need the latest 'security' patches. When we change bsd.port.mk over, include a snippet such as the one at [1], which gives more informative error text and refers user to documentation. Although I still think this is less than ideal, it is the only way I can see that we can switch before May '14, if the urgency is there. Chris [1] http://www.bayofrum.net/~crees/patches/bmake-pmake.diff ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 October 2012 15:32, Bryan Drewery br...@shatow.net wrote: On 10/27/2012 8:23 AM, Chris Rees wrote: [trim CC list a little to stop people regretting replying to this thread] On 27 October 2012 10:15, Chris Rees utis...@gmail.com wrote: On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote: On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes: In that case we have a switch time on the order of years, not weeks; 8.3 is supported until May '14, and unless we get a :tl etc MFC into 8, even longer. All this time the ports tree must work with pmake. I'm pretty sure I was told it is already in 8 and 7 Not in 8.3 at least: svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log I don't want to discourage you or belittle your excellent work here, but Marcel made me very nervous with his comment on the process being a few weeks. That was based on discussions at the last devsummit. These discussions need backing up with a real roadmap, including detail on exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree still works. I don't see where these considerations have been made. OK, so how about this. We (ab)use the security update mechanism to merge the pmake changes (:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier releng branches such as 7.3, 8.2, 9.0). We could then send out a message on ports-announce, giving a few weeks' notice that the change to bsd.port.mk is going through and that users need the latest 'security' patches. This weeks is making a assumptions that users 1. reads ports@ or 2. Update to security/errata patches in a timely manner or 3. Read UPDATING Quite. This should be at least a few months, otherwise we're making unreasonable requests of our users, and yet again annoy them by breaking older versions-- this time with no real benefit for end-users. Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 10/27/2012 8:23 AM, Chris Rees wrote: [trim CC list a little to stop people regretting replying to this thread] On 27 October 2012 10:15, Chris Rees utis...@gmail.com wrote: On 27 Oct 2012 00:35, Simon J. Gerraty s...@juniper.net wrote: On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes: In that case we have a switch time on the order of years, not weeks; 8.3 is supported until May '14, and unless we get a :tl etc MFC into 8, even longer. All this time the ports tree must work with pmake. I'm pretty sure I was told it is already in 8 and 7 Not in 8.3 at least: svnweb.freebsd.org/base/releng/8.3/usr.bin/make/var.c?view=log I don't want to discourage you or belittle your excellent work here, but Marcel made me very nervous with his comment on the process being a few weeks. That was based on discussions at the last devsummit. These discussions need backing up with a real roadmap, including detail on exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree still works. I don't see where these considerations have been made. OK, so how about this. We (ab)use the security update mechanism to merge the pmake changes (:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier releng branches such as 7.3, 8.2, 9.0). We could then send out a message on ports-announce, giving a few weeks' notice that the change to bsd.port.mk is going through and that users need the latest 'security' patches. This weeks is making a assumptions that users 1. reads ports@ or 2. Update to security/errata patches in a timely manner or 3. Read UPDATING When we change bsd.port.mk over, include a snippet such as the one at [1], which gives more informative error text and refers user to documentation. Although I still think this is less than ideal, it is the only way I can see that we can switch before May '14, if the urgency is there. Chris [1] http://www.bayofrum.net/~crees/patches/bmake-pmake.diff ___ freebsd-a...@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-arch To unsubscribe, send any mail to freebsd-arch-unsubscr...@freebsd.org ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 October 2012 10:34, Chris Rees utis...@gmail.com wrote: This weeks is making a assumptions that users 1. reads ports@ or 2. Update to security/errata patches in a timely manner or 3. Read UPDATING Quite. This should be at least a few months, otherwise we're making unreasonable requests of our users, and yet again annoy them by breaking older versions-- this time with no real benefit for end-users. +1 I would venture to guess that most of our users don't even read -announce. In addition there are non-ports concerns here. Many people probably have custom Makefiles they use for their own projects which may rely on existing behavior. -- Eitan Adler ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 10/27/2012 9:40 AM, Eitan Adler wrote: On 27 October 2012 10:34, Chris Rees utis...@gmail.com wrote: This weeks is making a assumptions that users 1. reads ports@ or 2. Update to security/errata patches in a timely manner or 3. Read UPDATING Quite. This should be at least a few months, otherwise we're making unreasonable requests of our users, and yet again annoy them by breaking older versions-- this time with no real benefit for end-users. +1 I would venture to guess that most of our users don't even read -announce. In addition there are non-ports concerns here. Many people probably have custom Makefiles they use for their own projects which may rely on existing behavior. I apologize for not reading the full thread. Could there be a make.conf/env setting to make bmake run AS pmake in full compat mode? On by default until all older branches are EoL, then it can flip and be optional. Or even via a symlink, whatever it is invoked as is what mode it runs in. Bryan ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 October 2012 18:27, Simon J. Gerraty s...@juniper.net wrote:
These discussions need backing up with a real roadmap, including detail on
exactly what 8.3 and 7.4 users will have to do to ensure that the ports
tree still works.
I've tested the ports tree converted to bmake - per the patch I
mentioned on a 7.1 box. It worked for me. Once the ports tree has
found or installed bmake, the system version makes no further
difference.
Obviously not a conclusive result, but yes this issue has been given
consideration.
What about these?
[crees@pegasus]~% grep -n :\[LU] /usr/ports/Mk/bsd.port.mk | tee
/dev/tty | wc -l
1324:PORTVERSION=
${DISTVERSION:L:C/([a-z])[a-z]+/\1/g:C/([0-9])([a-z])/\1.\2/g:C/:(.)/\1/g:C/[^a-z0-9+]+/./g}
1451:.if (defined(USE_QT_VER) ${USE_QT_VER:L} == 3) ||
defined(USE_KDELIBS_VER) || defined(USE_KDEBASE_VER)
1455:.if defined(USE_QT_VER) ${USE_QT_VER:L} == 4 || defined(USE_QT4)
1674:.if ${USE_PKGCONFIG:L} == yes || ${USE_PKGCONFIG:L} == build
1677:.elif ${USE_PKGCONFIG:L} == both
1681:.elif ${USE_PKGCONFIG:L} == run
1696:${b}= ${LOCALBASE}/bin/${b:C/PP/++/:L}
1763:_USE_OPENAL+= ${_OPENAL_${_OPENAL_SYSTEM:U}}
1783:_USE_OPENAL+= ${_OPENAL_${component:U}}
1829:.if defined(FAM_SYSTEM_${FAM_SYSTEM:U})
1830:LIB_DEPENDS+= ${FAM_SYSTEM_${FAM_SYSTEM:U}}
1836:.if defined(USE_RC_SUBR) ${USE_RC_SUBR:U} != YES
1844:.if defined(USE_LDCONFIG) ${USE_LDCONFIG:L} == yes
1847:.if defined(USE_LDCONFIG32) ${USE_LDCONFIG32:L} == yes
1856:. if ${USE_GETTEXT:L} == build
1858:. elif ${USE_GETTEXT:L} == run
1860:. elif ${USE_GETTEXT:L} == yes
1888:. if ${USE_LINUX:L} == yes
1899:. if ${USE_LINUX:L} == yes
1977:. if ${USE_GL:L} == yes
1994:. if ${USE_BISON:L} == build
1996:. elif ${USE_BISON:L} == run
1998:. elif ${USE_BISON:L} == both
2044:.if defined(USE_QT_VER) ${USE_QT_VER:L} == 4 || defined(USE_QT4)
3038:_MANPAGES+=
${MAN${sect}:S%^%${MAN${sect}PREFIX}/${manlang}/man${sect:L}/%}
3043:.if defined(MAN${sect}_${manlang:S%^man/%%:U})
3044:_MANPAGES+=
${MAN${sect}_${manlang:S%^man/%%:U}:S%^%${MAN${sect}PREFIX}/${manlang}/man${sect:L}/%}
3056:_MANPAGES+=
${MAN${sect}_EN:S%^%${MAN${sect}PREFIX}/man/man${sect:L}/%}
3312: || defined(CONFIG_DONE_${UNIQUENAME:U}) || \
3600:.if ${USE_DOS2UNIX:U}==YES
4361:${target}: ${${target:U}_COOKIE}
4364: @cd ${.CURDIR} ${MAKE} CONFIG_DONE_${UNIQUENAME:U}=1
${${target:U}_COOKIE}
4368:.if !exists(${${target:U}_COOKIE})
4370:.if ${UID} != 0 defined(_${target:U}_SUSEQ) !defined(INSTALL_AS_USER)
4372:${${target:U}_COOKIE}: ${_${target:U}_DEP}
4373: @cd ${.CURDIR} ${MAKE} ${_${target:U}_SEQ}
4375:${${target:U}_COOKIE}: ${_${target:U}_DEP} ${_${target:U}_SEQ}
4379: ${SU_CMD} ${MAKE} ${_${target:U}_SUSEQ}
4383:${${target:U}_COOKIE}: ${_${target:U}_DEP}
4385: ${MAKE} ${_${target:U}_SEQ} ${_${target:U}_SUSEQ}
4388:${${target:U}_COOKIE}: ${_${target:U}_DEP} ${_${target:U}_SEQ}
${_${target:U}_SUSEQ}
4393:${${target:U}_COOKIE}::
4802: for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4825: for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4836: for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4850: for alg in ${CHECKSUM_ALGORITHMS:U}; do \
4904: for alg in ${CHECKSUM_ALGORITHMS:U}; do \
5032:${deptype:L}-depends:
5653:${i:S/-//:U}= ${WRKDIR}/${SUB_FILES:M${i}*}
5700:.if defined(PLIST_REINPLACE_${reinplace:U})
5701: @${SED} -i -e '${PLIST_REINPLACE_${reinplace:U}}' ${TMPPLIST}
5854:.if defined(USE_RCORDER) || defined(USE_RC_SUBR)
${USE_RC_SUBR:U} != YES
5864:.if defined(USE_RC_SUBR) ${USE_RC_SUBR:U} != YES
53
[crees@pegasus]~%
Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
These discussions need backing up with a real roadmap, including detail on exactly what 8.3 and 7.4 users will have to do to ensure that the ports tree still works. I've tested the ports tree converted to bmake - per the patch I mentioned on a 7.1 box. It worked for me. Once the ports tree has found or installed bmake, the system version makes no further difference. Obviously not a conclusive result, but yes this issue has been given consideration. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 27 October 2012 19:52, Simon J. Gerraty s...@juniper.net wrote:
On Sat, 27 Oct 2012 14:23:29 +0100, Chris Rees writes:
We (ab)use the security update mechanism to merge the pmake changes
(:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier
I originally provided the :tl and :tu patch for something like that
(not planning any abuse mind ;-)
But, if portmgr test my patch and find it works ok (for some value of
ok) for older releases, this probably isn't necessary?
It may still be useful though to provide an updated fmake via ports,
which could make it easier for folk to migrate other code bases.
The sed script to be applied to makefiles is trivial btw:
$ cat f2bmake.sed
/$.*:[UL][:)}]/ { s,:L,:tl,g;s,:U,:tu,g; }
$
I know the fix is trivial :)
I'm saying that it's unacceptable to expect people to change their
systems just to make the ports tree work after we have broken it on a
supposedly supported version.
Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, 27 Oct 2012 09:44:36 -0500, Bryan Drewery writes: Could there be a make.conf/env setting to make bmake run AS pmake in full compat mode? On by default until all older branches are EoL, then it can flip and be optional. This has been mentioned before. Firstly, I have changed bmake behavior in a number of ways to better fit FreeBSD, but in each case I could justify the change to the NetBSD folk as well (or at least most of them ;-) The above idea though would require doing more violence to bmake's internals than I think is desirable, plus it would be counter productive. Today, you can test for defined(.PARSEDIR) and *know* if you have bmake or not, and if you have, how it behaves. If we start hacking compat modes and such to avoid changing, it would be more trouble that it is worth to try and make use of bmake in any meaningful way. The simpler implementation of this idea is to simply leave the old make in place. Or even via a symlink, whatever it is invoked as is what mode it runs in. This is more practical I think. Making /usr/bin/make - [fb]make ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, 27 Oct 2012 18:32:56 +0100, Chris Rees writes:
On 27 October 2012 18:27, Simon J. Gerraty s...@juniper.net wrote:
I've tested the ports tree converted to bmake - per the patch I
mentioned on a 7.1 box. It worked for me. Once the ports tree has
What about these?
[crees@pegasus]~% grep -n :\[LU] /usr/ports/Mk/bsd.port.mk | tee
/dev/tty | wc -l
1324:PORTVERSION=
${DISTVERSION:L:C/([a-z])[a-z]+/\1/g:C/([0-9])([a-z])/\1.\2/g:C/:(.)/\1/g:C/[^
a-z0-9+]+/./g}
1451:.if (defined(USE_QT_VER) ${USE_QT_VER:L} == 3) ||
defined(USE_KDELIBS_VER) || defined(USE_KDEBASE_VER)
I'm not sure I follow, that tree has not been patched.
If it were:
$ grep -l '$.*:[UL][:)}]' Mk/*mk
$
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, 27 Oct 2012 14:23:29 +0100, Chris Rees writes:
We (ab)use the security update mechanism to merge the pmake changes
(:tl and :tu) into releng/7.4 and releng/8.3 (possibly the earlier
I originally provided the :tl and :tu patch for something like that
(not planning any abuse mind ;-)
But, if portmgr test my patch and find it works ok (for some value of
ok) for older releases, this probably isn't necessary?
It may still be useful though to provide an updated fmake via ports,
which could make it easier for folk to migrate other code bases.
The sed script to be applied to makefiles is trivial btw:
$ cat f2bmake.sed
/$.*:[UL][:)}]/ { s,:L,:tl,g;s,:U,:tu,g; }
$
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, 27 Oct 2012 19:53:56 +0100, Chris Rees writes: I'm saying that it's unacceptable to expect people to change their systems just to make the ports tree work after we have broken it on a supposedly supported version. But there's no suggestion of that. The ports tree would take care of itself. The comment about fixing makefiles refered to the concern about things outside of base/ports. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
Can someone please explain to me what the original reason is for causing such ridiculously large, far reaching issues? And why people seem to be in a really, really big rush for it? Adrian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 26 Oct 2012 06:01, Konstantin Belousov kostik...@gmail.com wrote: On Thu, Oct 25, 2012 at 03:53:53PM -0700, Simon J. Gerraty wrote: On Thu, 25 Oct 2012 23:01:27 +0100, Chris Rees writes: Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. There is no need/plan for two versions of bsd.port.mk, the patch I just mentioned, deals with older systems by detecting that bmake was not used, and using it (installing if need be). Have you discussed this on ports@? I have not at least. This was discussed at the last couple of BSDCan's and dev summits. The original plan discussed at BSDCan a couple of years ago, was to allow bmake and the old make to cooexist for some time so that ports could continue to use the old make. At the last BSDCan we were told that wasn't an option - hence the patch to ports that was mentioned. FWIW the changes to 99.9% of the ports tree are trivial (:L - :tl etc). The only interesting changes are to bsd.port.mk (the diff other than the above is 54 lines) they cover 2 things - dealing with old make as mentioned above, and man pages. The nested .for loops that deal with MLINKS are replaced with one line - this was safer that attempting to hack those .for loops to work with both makes. I am watching the serial for some time. Could please, someone, describe why bmake cannot grow the compat features to be a drop-in replacement for FreeBSD make, instead of patching all the trees ? In particular, why cannot the ':L' and ':U' support be added ? :U is already used by bmake for something else- I can't remember what, but I checked the man page last night :( Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
In particular, why cannot the ':L' and ':U' support be added ? :U is already used by bmake for something else- I can't remember what, but I checked the man page last night :( http://www.crufty.net/sjg/blog/freebsd-meta-mode.htm might provide some interesting background. It is a more FreeBSD focused (and up to date), coverage of material I presented at BSDCan a couple of years ago. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
In particular, why cannot the ':L' and ':U' support be added ? Because they already exist - with different meanings. They were added to NetBSD make over 10 years ago, from the OSF version of pmake. In several areas the behavior of bmake has been changed to make it a drop in replacement for FreeBSD, but the above (not used at all in the FreeBSD base) are easier dealt with the other way. The :tl and :tu equivalents were added to FreeBSD make a while back to ease the transition. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 10/25/12 23:23, Simon J. Gerraty wrote: In particular, why cannot the ':L' and ':U' support be added ? Because they already exist - with different meanings. They were added to NetBSD make over 10 years ago, from the OSF version of pmake. In several areas the behavior of bmake has been changed to make it a drop in replacement for FreeBSD, but the above (not used at all in the FreeBSD base) are easier dealt with the other way. The :tl and :tu equivalents were added to FreeBSD make a while back to ease the transition. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org At the risk of getting a ton of email telling me what an idiot I am, why not add a command line flag OR a make variable/value to set this behavior? You could put the flag/option in the /etc/make.conf file... and once you get all of the make files fixed up you could take it out... -- Patrick Powell Astart Technologies papow...@astart.com1530 Jamacha Road, Suite X, Network and System El Cajon, CA 92019 Consulting 858-874-6543 Web Site: www.astart.com ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 26, 2012, at 12:23 AM, Simon J. Gerraty wrote: In particular, why cannot the ':L' and ':U' support be added ? Because they already exist - with different meanings. They were added to NetBSD make over 10 years ago, from the OSF version of pmake. And we've had the :U and :L for a similar period of time as well. Arguing age here is an interesting historical footnote, but not a compelling argument to justify the pain to our users. In several areas the behavior of bmake has been changed to make it a drop in replacement for FreeBSD, but the above (not used at all in the FreeBSD base) are easier dealt with the other way. The :tl and :tu equivalents were added to FreeBSD make a while back to ease the transition. Why can't there be a make target that turns them on in FreeBSD compat mode. You could then just drop those into bsd.port.mk and be done with it? We already do this with the posix target, so there's precedent for it. I know you've objected to this as ugly, but as I pointed out when I mentioned it before, I think this is less ugly and less work and would offer a smoother transition than forcing all the scripts to change. Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 2012-10-26 at 08:27 -0600, Warner Losh wrote: On Oct 26, 2012, at 12:23 AM, Simon J. Gerraty wrote: In particular, why cannot the ':L' and ':U' support be added ? Because they already exist - with different meanings. They were added to NetBSD make over 10 years ago, from the OSF version of pmake. And we've had the :U and :L for a similar period of time as well. Arguing age here is an interesting historical footnote, but not a compelling argument to justify the pain to our users. In several areas the behavior of bmake has been changed to make it a drop in replacement for FreeBSD, but the above (not used at all in the FreeBSD base) are easier dealt with the other way. The :tl and :tu equivalents were added to FreeBSD make a while back to ease the transition. Why can't there be a make target that turns them on in FreeBSD compat mode. You could then just drop those into bsd.port.mk and be done with it? We already do this with the posix target, so there's precedent for it. I know you've objected to this as ugly, but as I pointed out when I mentioned it before, I think this is less ugly and less work and would offer a smoother transition than forcing all the scripts to change. Warner I second this concept. At work, we create dozens of products using literally hundreds of makefiles scattered throughout a huge source base. We have to be able to build the same source for multiple versions of freebsd, so even finding all the old :U and :L and any other incompatibilities and fixing them isn't an option because we'd just trade works in freebsd 10 for broken in every other environment. If there were some way to turn on a compatibility mode, we'd have a way to slowly transition to the newer stuff over the course of a couple OS versions. Eventually we'd reach the point where we no longer need to build products using an older version and we could update to the newer syntax and stop using compatibility mode. -- Ian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
On Thu, Oct 25, 2012 at 03:00:21PM -0700, Garrett Cooper wrote:
Here's an updated version of the workaround that works properly in all
cases and installs bmake as make and links make to pmake when
WITH_BMAKE=yes, and installs make as make when WITHOUT_BMAKE is
specified (this works better than the previous patch I sent to Simon).
Garrett,
I don't see how this could be committed -- it will make it difficult
for 10-CURRENT folks to build ports (and there are no pre-build packages
for 10-CURRENT).
Are you not able to use this instead (w/WANT_USRBIN_BMAKE= in /etc/src.conf)?
Index: usr.bin/Makefile
===
--- usr.bin/Makefile(revision 241927)
+++ usr.bin/Makefile(working copy)
@@ -281,6 +281,9 @@ SUBDIR+=msgs
.if ${MK_BMAKE} != no
SUBDIR+= bmake
.else
+.if defined(WANT_USRBIN_BMAKE)
+SUBDIR+= bmake
+.endif
SUBDIR+= make
.endif
.endif
--
-- David (obr...@freebsd.org)
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
On Thu, Oct 25, 2012 at 03:00:21PM -0700, Garrett Cooper wrote: Here's an updated version of the workaround that works properly in all cases and installs bmake as make and links make to pmake when WITH_BMAKE=yes, and installs make as make when WITHOUT_BMAKE is specified (this works better than the previous patch I sent to Simon). Also, please note we have a 'pmake' port that is the proper original pmake (before *BSD embellished it). Perhaps a different name than 'pmake' is appropriate. It would not surprise me for someone to end up adding a port of the current FreeBSD make in case there are folks that find bmake incompatible with their use of FreeBSD's make in their own projects. So picking a good name now would be helpful. -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tue, Oct 02, 2012 at 07:19:55AM -0700, Garrett Cooper wrote: Hmmm... that's one of the 3 approaches I provided, but it turned out ... 1. Test programs live with the sources (this was the requested approach), e.g. 2. Test programs live in subdirs: 3. Test programs completely decoupled from the source tree: Could someone please commit at least one working .c test and one .sh test? There is nothing to follow for others trying to write their own tests in the FreeBSD-way. I could not find a single consumer of ATF in HEAD. This makes it seem this is still a WIP that should be living in a branch and not in HEAD. But we're paying the price for checkout build times, etc... See the recent 9.1-R thread and Peter Wemm (and others) comments in this regard. (this is why I hadn't committed the WIP I had - it wasn't ready for HEAD) thanks, -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 26, 2012, at 11:21 AM, Simon J. Gerraty wrote:
On Fri, 26 Oct 2012 08:27:06 -0600, Warner Losh writes:
And we've had the :U and :L for a similar period of time as well. =
Sorry, I didn't mean to imply age has anything to do with it.
The doc I refered to makes it clear that the two sets of conflicting
modifers were introduced at about the same time.
Why can't there be a make target that turns them on in FreeBSD compat =
mode. You could then just drop those into bsd.port.mk and be done with =
Because then you would lose the functionality that the alternative
modifiers provide. Imagine throwing away the ability in /bin/sh to
do${foo:-bar}
Also it would perpetuate the divergence in syntax for little reason.
It's called a transition period for a reason. The historical use has permeated
itself into many places, not all of which are obvious.
For many years, sun had two shells so that old shell scripts would work until
they could be adapted to the new shell's syntax. So your argument rings a bit
hollow. Compatibility always has been about being compatible, not about
growing the feature set or purposely leaving features out.
BTW there are currently 300 makefiles in ports/ affected by the
transition to bmake, and there were an even smaller number in src/.
And there are many companies (I know of at least two) that have enough
infrastructure that depend on these modifiers that moving to 10 will be hard
for them. Stupid (in their view) incompatibilities like this are a
disincentive to upgrade or keep with FreeBSD. Easing the transition for them
will help keep them in the fold. It is no different than keeping old IOCTLs
around for a release or three to ease that burden.
Warner
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 02:23:06PM -0700, Marcel Moolenaar wrote: I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. I'm trying to create an ATF test for filemon, but I don't want to have to build make back and forth when I want to build a port. Likely that doesn't put me in the people working on ATF in your book. So let's work the bmake side and be able to get rid of the knob as soon as possible. Do we have any commitment as to when Portmgr will have bandwidth to for testing bmake (I expect it will be several iterations)? I suspect they're pretty busy with 9.1-RELEASE, so is this gated by 9.1-R? 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. What can I and others do to work on this? I'm not on Portmgr and most aren't either. In short: this isn't a 2-knob problem by any stretch of the imagination. I disagree. Before sending my mail, I ran this by sjg and his response was: I have absolutely no objection. -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 08:27:06 -0600, Warner Losh writes:
And we've had the :U and :L for a similar period of time as well. =
Sorry, I didn't mean to imply age has anything to do with it.
The doc I refered to makes it clear that the two sets of conflicting
modifers were introduced at about the same time.
Why can't there be a make target that turns them on in FreeBSD compat =
mode. You could then just drop those into bsd.port.mk and be done with =
Because then you would lose the functionality that the alternative
modifiers provide. Imagine throwing away the ability in /bin/sh to do
${foo:-bar}
Also it would perpetuate the divergence in syntax for little reason.
BTW there are currently 300 makefiles in ports/ affected by the
transition to bmake, and there were an even smaller number in src/.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
with their use of FreeBSD's make in their own projects. So picking a good name now would be helpful. FWIW I keep a copy in /usr/bin/fmake so I can compare behavior. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 09:41:36AM -0600, Ian Lepore wrote: We have to be able to build the same source for multiple versions of freebsd, so even finding all the old :U and :L and any other incompatibilities and fixing them isn't an option because we'd just trade works in freebsd 10 for broken in every other environment. Ian, If you're using FreeBSD 9 after 2012-06-14, or FreeBSD 8 or 7 after 2012-10-09 you can use the Bmake spelling of :U and :L (:tu/:tl). I am not aruging against you, just giving some information you may not be aware of. -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote: Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE. -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 11:12:53AM -0700, Simon J. Gerraty wrote: On Fri, 26 Oct 2012 11:41:46 -0600, Warner Losh writes: It's called a transition period for a reason. The historical use has = permeated itself into many places, not all of which are obvious. It would seem that leaving FreeBSD make as make, for the transition period and installing bmake as bmake, would cause the least disruption to everyone. This was the original proposal presented at BSDCan in 2011. FreeBSD make already grok's the :tl and :tu modifiers, so it is quite simple for the two to coexist for some period. The only reason we are talking about having to frob ports etc now, is a new requirement introduced this year (by yourself I think) that bmake replace make in base rather than allow coexistence. If we are all happy to go back to the original plan, we can ease the concerns of the folk you speak of? The only downside is we wait a few more years for major build improvments. Can system build, initiated by make, call bmake immediately ? I suppose it could be fine even to error out if make is typed instead of bmake for src/. pgp1H8SYjf8vh.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 11:41:46 -0600, Warner Losh writes: It's called a transition period for a reason. The historical use has = permeated itself into many places, not all of which are obvious. It would seem that leaving FreeBSD make as make, for the transition period and installing bmake as bmake, would cause the least disruption to everyone. This was the original proposal presented at BSDCan in 2011. FreeBSD make already grok's the :tl and :tu modifiers, so it is quite simple for the two to coexist for some period. The only reason we are talking about having to frob ports etc now, is a new requirement introduced this year (by yourself I think) that bmake replace make in base rather than allow coexistence. If we are all happy to go back to the original plan, we can ease the concerns of the folk you speak of? The only downside is we wait a few more years for major build improvments. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 2012-10-26 at 11:09 -0700, David O'Brien wrote: On Fri, Oct 26, 2012 at 09:41:36AM -0600, Ian Lepore wrote: We have to be able to build the same source for multiple versions of freebsd, so even finding all the old :U and :L and any other incompatibilities and fixing them isn't an option because we'd just trade works in freebsd 10 for broken in every other environment. Ian, If you're using FreeBSD 9 after 2012-06-14, or FreeBSD 8 or 7 after 2012-10-09 you can use the Bmake spelling of :U and :L (:tu/:tl). I am not aruging against you, just giving some information you may not be aware of. Yeah. And if I have to, I could modify all our makefiles to use the new syntax, then backport support for the new syntax to earlier freebsd make source in our local repos. But to give you some idea of what I've got to support... yesterday afternoon I was struggling with whether I can find the time in a release schedule to update an old product that needs a new feature from freebsd 6 to 8. The sad fact is that I can't, I'm going to have to do another freebsd 6-based release to meet the schedule. It's interesting having to work on a daily basis in everything between freebsd 6.2 and -current. -- Ian ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 26, 2012, at 12:11 PM, David O'Brien wrote: On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote: Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE. It isn't in 8 yet, so there's no good transition strategy there... Warner ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 10:55:59 -0700, David O'Brien writes: I'm trying to create an ATF test for filemon, but I don't want to have to build make back and forth when I want to build a port. Likely that doesn't put me in the people working on ATF in your book. What can I and others do to work on this? I'm not on Portmgr and most aren't either. Why not simply install bmake as bmake? You can even use devel/bmake in ports. I disagree. Before sending my mail, I ran this by sjg and his response was: I have absolutely no objection. I have no objection to bmake being installed as bmake. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 11:11:52AM -0700, David O'Brien wrote: On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote: Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE. -- -- David (obr...@freebsd.org) Perfect thanks, I wasn't sure Bapt pgpfjZZjJ4ItK.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 26 Oct 2012 19:12, David O'Brien obr...@freebsd.org wrote: On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote: Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. :tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE. Then we only have two supported stable branches you propose to break... Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
On Fri, Oct 26, 2012 at 9:34 AM, David O'Brien obr...@freebsd.org wrote: On Thu, Oct 25, 2012 at 03:00:21PM -0700, Garrett Cooper wrote: Here's an updated version of the workaround that works properly in all cases and installs bmake as make and links make to pmake when WITH_BMAKE=yes, and installs make as make when WITHOUT_BMAKE is specified (this works better than the previous patch I sent to Simon). Garrett, I don't see how this could be committed -- it will make it difficult for 10-CURRENT folks to build ports (and there are no pre-build packages for 10-CURRENT). I don't want it committed because Simon's move makes sense longterm: I wanted to offer an alternative as opposed to just being stuck in purgatory and figured that others might benefit from it. We're stuck at a point now that we need to make a break but we also need to ensure that we don't break things too badly for users with older versions of make. Installing our version of make as something other than `make` would at least allow us to use make as pmake in ports, but I realized it would requiring hacking around portmaster, portupgrade, and a number of other tools that expect FreeBSD make to be make and don't have a means of parameterizing make in the environment or on the command line. So looking back now my mitigation solution would not be ideal and would not fix any problems really. Are you not able to use this instead (w/WANT_USRBIN_BMAKE= in /etc/src.conf)? That would be interesting too (and is a lot less involved than my patch), and probably would have less fallout. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 9:54 AM, David O'Brien obr...@freebsd.org wrote: On Tue, Oct 02, 2012 at 07:19:55AM -0700, Garrett Cooper wrote: Hmmm... that's one of the 3 approaches I provided, but it turned out ... 1. Test programs live with the sources (this was the requested approach), e.g. 2. Test programs live in subdirs: 3. Test programs completely decoupled from the source tree: Could someone please commit at least one working .c test and one .sh test? There is nothing to follow for others trying to write their own tests in the FreeBSD-way. I could not find a single consumer of ATF in HEAD. This makes it seem this is still a WIP that should be living in a branch and not in HEAD. But we're paying the price for checkout build times, etc... See the recent 9.1-R thread and Peter Wemm (and others) comments in this regard. (this is why I hadn't committed the WIP I had - it wasn't ready for HEAD) There are some basic examples, but they're in my p4 branch and unfortunately they depend on atf.test.mk/bsd.test.mk/bsd.progs.mk existing before they can be built (please see the Examples section in http://wiki.freebsd.org/TestingFreeBSD ). I also have the tests integrated in my perforce branch and running, but it doesn't do a bit of good unless the build pieces are in. I've been trying to get these things into HEAD in proper order so they can be used effectively. Thanks, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
Minor disambiguation: On Fri, Oct 26, 2012 at 12:27 PM, Garrett Cooper yaneg...@gmail.com wrote: ... There are some basic examples, but they're in my p4 branch and unfortunately they depend on atf.test.mk/bsd.test.mk/bsd.progs.mk existing before they can be built (please see the Examples section in http://wiki.freebsd.org/TestingFreeBSD ). I also have the tests I also have the tests - I also have the ATF feature/integration tests integrated in my perforce branch and running, but it doesn't do a bit of good unless the build pieces are in. I've been trying to get these things into HEAD in proper order so they can be used effectively. Thanks, -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 26 Oct 2012 20:15, Chris Rees utis...@gmail.com wrote:
On 26 Oct 2012 19:12, David O'Brien obr...@freebsd.org wrote:
On Fri, Oct 26, 2012 at 12:12:44AM +0200, Baptiste Daroussin wrote:
Do be able to get the ports tree working with bmake asap, I also asked
him to MFC it to 9.1, from latest reply he got positive answer from
re@
about this, but was waiting for something I don't remember.
:tu/:tl is in releng/9.1, so it will also be in 9.1-RELEASE.
Then we only have two supported stable branches you propose to break...
OK, how about this:
:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 12:54 PM, Simon J. Gerraty s...@juniper.net wrote: On Fri, 26 Oct 2012 12:27:35 -0700, Garrett Cooper writes: There are some basic examples, but they're in my p4 branch and unfortunately they depend on atf.test.mk/bsd.test.mk/bsd.progs.mk Speaking of which. I notice there is now a bsd.progs.mk in head, which bears little relationship to the one I was talking about (from devel/bmake). Is this bsd.progs.mk essentially your port of netbsd's bsd.prog.mk it seems much more complex than should be necessary (my progs.mk is 75 lines vs 350 for the one in head). Yup. I was a bit surprised it was committed, but there might have been some confusion over what all was going to go in with the ATF port. Would you be awfully upset if I replace it with the simpler one? Nah. Feel free to nuke it and remove it from the build once you get progs.mk in proper shape; I'm more concerned about completing the atf.test.mk/bsd.test.mk snippets because once those are committed I can get the test example code and the integration test patches reviewed and committed. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 12:27:35 -0700, Garrett Cooper writes: There are some basic examples, but they're in my p4 branch and unfortunately they depend on atf.test.mk/bsd.test.mk/bsd.progs.mk Speaking of which. I notice there is now a bsd.progs.mk in head, which bears little relationship to the one I was talking about (from devel/bmake). Is this bsd.progs.mk essentially your port of netbsd's bsd.prog.mk it seems much more complex than should be necessary (my progs.mk is 75 lines vs 350 for the one in head). Would you be awfully upset if I replace it with the simpler one? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 26 Oct 2012 21:51, Simon J. Gerraty s...@juniper.net wrote:
On Fri, 26 Oct 2012 21:00:26 +0100, Chris Rees writes:
:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.
Acutally it is very useful.
The debugging facilities in dirdeps.mk rely on it.
The junos build uses it in many other places too.
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
No, please don't do that.
I'm trying to reduce the divergence b/w freebsd and netbsd.
In that case we have a switch time on the order of years, not weeks; 8.3 is
supported until May '14, and unless we get a :tl etc MFC into 8, even
longer. All this time the ports tree must work with pmake.
I don't want to discourage you or belittle your excellent work here, but
Marcel made me very nervous with his comment on the process being a few
weeks.
Chris
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 21:00:26 +0100, Chris Rees writes:
:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.
Acutally it is very useful.
The debugging facilities in dirdeps.mk rely on it.
The junos build uses it in many other places too.
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
No, please don't do that.
I'm trying to reduce the divergence b/w freebsd and netbsd.
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 09:00:26PM +0100, Chris Rees wrote:
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
${VAR:U} is useful for bmake as well. For example, .if conditionals can
avoid explicit checks for defined and/or quoting that way.
Joerg
___
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
On Fri, Oct 26, 2012 at 09:34:20AM -0700, David O'Brien wrote: (there are no pre-build packages for 10-CURRENT). Please see the first two entries on: http://pkgbeta.freebsd.org/ mcl ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, Oct 26, 2012 at 10:02:00PM +0100, Chris Rees wrote:
On 26 Oct 2012 21:51, Simon J. Gerraty s...@juniper.net wrote:
On Fri, 26 Oct 2012 21:00:26 +0100, Chris Rees writes:
:L -- seems that bmake's use for this is kinda pointless; returning the
name of the variable; we could swap that usage over directly.
Acutally it is very useful.
The debugging facilities in dirdeps.mk rely on it.
The junos build uses it in many other places too.
:U -- with bmake has non-optional arguments, so for example:
${VAR:U} - pmake behaviour
${VAR:Uval} - make behaviour.
Would that be acceptable? I can get a patch in if that's popular.
No, please don't do that.
I'm trying to reduce the divergence b/w freebsd and netbsd.
In that case we have a switch time on the order of years, not weeks; 8.3 is
supported until May '14, and unless we get a :tl etc MFC into 8, even
longer. All this time the ports tree must work with pmake.
:tl/:tu has already been MFCed to 8 iirc.
I don't want to discourage you or belittle your excellent work here, but
Marcel made me very nervous with his comment on the process being a few
weeks.
Chris
pgpnZ5uCglcDf.pgp
Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Fri, 26 Oct 2012 22:02:00 +0100, Chris Rees writes: In that case we have a switch time on the order of years, not weeks; 8.3 is supported until May '14, and unless we get a :tl etc MFC into 8, even longer. All this time the ports tree must work with pmake. I'm pretty sure I was told it is already in 8 and 7 I don't want to discourage you or belittle your excellent work here, but Marcel made me very nervous with his comment on the process being a few weeks. That was based on discussions at the last devsummit. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote: two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. It seems we do have the situation where folks feel there is a dependency between the two. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage It could be a while (many weeks) before we get to 4, so the question Given the time this will take, I feel we need to add another knob to the Bmake build so that 'make world' gives one both the FreeBSD make as /usr/bin/make and Bmake as /usr/bin/bmake. thoughts, -- -- David (obr...@freebsd.org) ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 25 October 2012 22:15, David O'Brien obr...@freebsd.org wrote: On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote: two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. It seems we do have the situation where folks feel there is a dependency between the two. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage It could be a while (many weeks) before we get to 4, so the question Given the time this will take, I feel we need to add another knob to the Bmake build so that 'make world' gives one both the FreeBSD make as /usr/bin/make and Bmake as /usr/bin/bmake. We really aren't going to have any luck yet... [crees@pegasus]/usr/ports% sudo make MAKE=/usr/bin/bmake index | head Generating INDEX-9 - please wait..bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here Looks like a few missing .if !target s, but the breakage is pretty big even for simple things :/ Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 25, 2012, at 2:15 PM, David O'Brien obr...@freebsd.org wrote: On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote: two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. It seems we do have the situation where folks feel there is a dependency between the two. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage It could be a while (many weeks) before we get to 4, so the question Given the time this will take, I feel we need to add another knob to the Bmake build so that 'make world' gives one both the FreeBSD make as /usr/bin/make and Bmake as /usr/bin/bmake. I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. So let's work the bmake side and be able to get rid of the knob as soon as possible. 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. In short: this isn't a 2-knob problem by any stretch of the imagination. -- Marcel Moolenaar mar...@xcllnt.net ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 2:23 PM, Marcel Moolenaar mar...@xcllnt.net wrote: ... I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. So let's work the bmake side and be able to get rid of the knob as soon as possible. It is annoying with the magnitude of build-related errors, but I have a workaround. 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. Yes, but not being able to update one's machine makes me sad panda. In short: this isn't a 2-knob problem by any stretch of the imagination. The real issue is that I need to take the patch Simon developed, run with it, and in parallel he needs to -- and hopefully already is -- engage portmgr to get it through a number of exp- runs to make sure bmake does what it's supposed to do with his patch. Backwards compatibility will need to be maintained for ports because ports has to work on multiple versions of FreeBSD [where bmake isn't yet available/present], so maybe a fork in the road for bsd.port.mk should be devised in order to make everything work. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Installing make as pmake when WITH_BMAKE specified (was Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program)
On Thu, Oct 25, 2012 at 2:32 PM, Garrett Cooper yaneg...@gmail.com wrote: ... The real issue is that I need to take the patch Simon developed, run with it, and in parallel he needs to -- and hopefully already is -- engage portmgr to get it through a number of exp- runs to make sure bmake does what it's supposed to do with his patch. Backwards compatibility will need to be maintained for ports because ports has to work on multiple versions of FreeBSD [where bmake isn't yet available/present], so maybe a fork in the road for bsd.port.mk should be devised in order to make everything work. Here's an updated version of the workaround that works properly in all cases and installs bmake as make and links make to pmake when WITH_BMAKE=yes, and installs make as make when WITHOUT_BMAKE is specified (this works better than the previous patch I sent to Simon). The point of the patch isn't to discourage bmake use; in fact this encourages bmake use more because I'm able to use bmake as my system make, but be able to fall back to pmake as needed. Thanks! -Garrett install-make-as-pmake-when-WITH_BMAKE-specified.patch Description: Binary data ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 25 October 2012 22:32, Garrett Cooper yaneg...@gmail.com wrote: On Thu, Oct 25, 2012 at 2:23 PM, Marcel Moolenaar mar...@xcllnt.net wrote: ... I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. So let's work the bmake side and be able to get rid of the knob as soon as possible. It is annoying with the magnitude of build-related errors, but I have a workaround. 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. Yes, but not being able to update one's machine makes me sad panda. In short: this isn't a 2-knob problem by any stretch of the imagination. The real issue is that I need to take the patch Simon developed, run with it, and in parallel he needs to -- and hopefully already is -- engage portmgr to get it through a number of exp- runs to make sure bmake does what it's supposed to do with his patch. Backwards compatibility will need to be maintained for ports because ports has to work on multiple versions of FreeBSD [where bmake isn't yet available/present], so maybe a fork in the road for bsd.port.mk should be devised in order to make everything work. Now you've terrified me, and probably most other ports people too. Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. Have you discussed this on ports@? Chris ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 11:01:27PM +0100, Chris Rees wrote: On 25 October 2012 22:32, Garrett Cooper yaneg...@gmail.com wrote: On Thu, Oct 25, 2012 at 2:23 PM, Marcel Moolenaar mar...@xcllnt.net wrote: ... I think there are 2 reasons why not to: 1. The people working on ATF have not raised this concern and have expressed that using the WITH_BMAKE knob is but a small price to pay. So let's work the bmake side and be able to get rid of the knob as soon as possible. It is annoying with the magnitude of build-related errors, but I have a workaround. 2. More knobs isn't better -- we must have none of the knobs in the end, so the more we create, the more work we have to get rid of them. That's just more work spent not focusing on the task at hand and thus more time wasted. Yes, but not being able to update one's machine makes me sad panda. In short: this isn't a 2-knob problem by any stretch of the imagination. The real issue is that I need to take the patch Simon developed, run with it, and in parallel he needs to -- and hopefully already is -- engage portmgr to get it through a number of exp- runs to make sure bmake does what it's supposed to do with his patch. Backwards compatibility will need to be maintained for ports because ports has to work on multiple versions of FreeBSD [where bmake isn't yet available/present], so maybe a fork in the road for bsd.port.mk should be devised in order to make everything work. Now you've terrified me, and probably most other ports people too. Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. Not much test has been done on the ports tree about it, from what I have tested so far, except from the :tu :tl difference the ports seems to work ootb with both bmake and make, I asked obrien to MFC the support for :tl :tu in make(1) to all available platform which he did. Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. regards, Bapt pgpp7TctkVsBS.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On 25 October 2012 18:12, Baptiste Daroussin b...@freebsd.org wrote: Not much test has been done on the ports tree about it, from what I have tested so far, except from the :tu :tl difference the ports seems to work ootb with both bmake and make, I asked obrien to MFC the support for :tl :tu in make(1) to all available platform which he did. Do be able to get the ports tree working with bmake asap, I also asked him to MFC it to 9.1, from latest reply he got positive answer from re@ about this, but was waiting for something I don't remember. The ports tree isn't the only concern. We also need to think about upstream users of bmake that relied on :U and the like working as it does now. We will either need to patch them, or implement a USE_OLD_MAKE flag. -- Eitan Adler ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 3:01 PM, Chris Rees cr...@freebsd.org wrote: ... Now you've terrified me, and probably most other ports people too. Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. I'm not the best advocate for bmake vs pmake; like bapt@ said in the followup email, most things work out of the box where people aren't trying to be clever, but I've found some interesting edgecases where bmake works and pmake doesn't, and vice versa because the old code depends upon incorrect behavior. I wasn't necessarily advocating having two bsd.*.mk files as the best idea -- it's just what came to mind first. Have you discussed this on ports@? I haven't, but I hope that someone else started this discussion... Thanks, -Garrett PS I am an optimist, but I'm a realist more than an optimist. I know that changing major/fundamental system components like make, the toolchain, etc requires a good deal of testing and there will be bugs/issues that need to be resolved. We just should make sure that things work as best possible for those looking back as well as those looking forward because it's considerably easier doing development on FreeBSD when I can just update a ports tree, build on 6.x/7.2/7.3, run some quick tests, then switch up to 10 and do other development. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, 25 Oct 2012 22:21:59 +0100, Chris Rees writes: We really aren't going to have any luck yet... [crees@pegasus]/usr/ports% sudo make MAKE=/usr/bin/bmake index | head If anyone is eager to play with this, I just have put a copy of ports2bmake.tar.gz in ~sjg/ on freefall. This contains a script that *should* convert ports to bmake syntax, while adding a hack to allow older systems to still work. It will generate a list of all the files it frobs so you can easily revert them - eg. before updating the tree. There's a README file in the tarball which hopefully explains all. Any issues - pls let me know. --sjg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 10:21:59PM +0100, Chris Rees wrote: On 25 October 2012 22:15, David O'Brien obr...@freebsd.org wrote: On Mon, Oct 08, 2012 at 09:11:29AM -0700, Marcel Moolenaar wrote: two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. It seems we do have the situation where folks feel there is a dependency between the two. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage It could be a while (many weeks) before we get to 4, so the question Given the time this will take, I feel we need to add another knob to the Bmake build so that 'make world' gives one both the FreeBSD make as /usr/bin/make and Bmake as /usr/bin/bmake. We really aren't going to have any luck yet... [crees@pegasus]/usr/ports% sudo make MAKE=/usr/bin/bmake index | head Generating INDEX-9 - please wait..bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5127: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: duplicate script for target -depends ignored bmake: /usr/ports/Mk/bsd.port.mk line 5124: warning: using previous script for -depends defined here Looks like a few missing .if !target s, but the breakage is pretty big even for simple things :/ Have you converted the :U to :tu and :L to :tl? regards, Bapt pgpGGmrYitgPo.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, 25 Oct 2012 23:01:27 +0100, Chris Rees writes: Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. There is no need/plan for two versions of bsd.port.mk, the patch I just mentioned, deals with older systems by detecting that bmake was not used, and using it (installing if need be). Have you discussed this on ports@? I have not at least. This was discussed at the last couple of BSDCan's and dev summits. The original plan discussed at BSDCan a couple of years ago, was to allow bmake and the old make to cooexist for some time so that ports could continue to use the old make. At the last BSDCan we were told that wasn't an option - hence the patch to ports that was mentioned. FWIW the changes to 99.9% of the ports tree are trivial (:L - :tl etc). The only interesting changes are to bsd.port.mk (the diff other than the above is 54 lines) they cover 2 things - dealing with old make as mentioned above, and man pages. The nested .for loops that deal with MLINKS are replaced with one line - this was safer that attempting to hack those .for loops to work with both makes. --sjg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 25, 2012 at 03:53:53PM -0700, Simon J. Gerraty wrote: On Thu, 25 Oct 2012 23:01:27 +0100, Chris Rees writes: Is there a Wiki page where the actual benefits of moving to bmake are made clear? This is a major, *major* upheaval, and having two versions of bsd.port.mk for years is simply not an option. There is no need/plan for two versions of bsd.port.mk, the patch I just mentioned, deals with older systems by detecting that bmake was not used, and using it (installing if need be). Have you discussed this on ports@? I have not at least. This was discussed at the last couple of BSDCan's and dev summits. The original plan discussed at BSDCan a couple of years ago, was to allow bmake and the old make to cooexist for some time so that ports could continue to use the old make. At the last BSDCan we were told that wasn't an option - hence the patch to ports that was mentioned. FWIW the changes to 99.9% of the ports tree are trivial (:L - :tl etc). The only interesting changes are to bsd.port.mk (the diff other than the above is 54 lines) they cover 2 things - dealing with old make as mentioned above, and man pages. The nested .for loops that deal with MLINKS are replaced with one line - this was safer that attempting to hack those .for loops to work with both makes. I am watching the serial for some time. Could please, someone, describe why bmake cannot grow the compat features to be a drop-in replacement for FreeBSD make, instead of patching all the trees ? In particular, why cannot the ':L' and ':U' support be added ? pgpZs4ID2dTbM.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 8, 2012, at 12:11 , Marcel Moolenaar mar...@xcllnt.net wrote: On Oct 4, 2012, at 9:42 AM, Garrett Cooper yaneg...@gmail.com wrote: Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). I'm in part to blame for this. There was some discussion -- but not at length; unfortunately no one from Juniper was present at the meet and greet; the information I got was second hand; I didn't follow up to figure out the exact details / clarify what I had in mind with the appropriate parties. Hang on. I want in on the blame part! :-) Seriously: no-one is really to blame as far as I can see. We just had two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. I just committed the bmake bits. It not only adds bmake to the build, but also includes the changes necessary to use bmake. With that in place it's easier to decide whether we want the dependency or not. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage Then: 4. Switch. It could be a while (many weeks) before we get to 4, so the question really is whether the people working on ATF are willing and able to build and install FreeBSD using WITH_BMAKE? I think that's a small price to pay for getting going with the ATF stuff now rather than in 4 weeks. What's the right way to do this now with HEAD? Best, George ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 13, 2012, at 12:15 PM, George Neville-Neil g...@neville-neil.com wrote: I think that's a small price to pay for getting going with the ATF stuff now rather than in 4 weeks. What's the right way to do this now with HEAD? Set WITH_BMAKE=yes in /etc/src.conf or /etc/make.conf and you're good to go. One caveat: manually rebuild and re-install usr.bin/bmake after the buildworld installworld with WITH_BMAKE=yes set. The one created as part of the buildworld has a bug due to being built by FreeBSD's make. A fix is known and will be committed soon, but until then, the manual step is needed. That's it... -- Marcel Moolenaar mar...@xcllnt.net ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, 13 Oct 2012 15:15:59 -0400, George Neville-Neil writes: It could be a while (many weeks) before we get to 4, so the question really is whether the people working on ATF are willing and able to build and install FreeBSD using WITH_BMAKE? =20 I think that's a small price to pay for getting going with the ATF stuff now rather than in 4 weeks. What's the right way to do this now with HEAD? We can add bsd.progs.mk (if you have devel/bmake port installed you have it as /usr/local/share/mk/progs.mk) and atf.test.mk and people can just go for it ? ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, Oct 13, 2012 at 1:13 PM, Simon J. Gerraty s...@juniper.net wrote: On Sat, 13 Oct 2012 15:15:59 -0400, George Neville-Neil writes: It could be a while (many weeks) before we get to 4, so the question really is whether the people working on ATF are willing and able to build and install FreeBSD using WITH_BMAKE? I think that's a small price to pay for getting going with the ATF stuff now rather than in 4 weeks. What's the right way to do this now with HEAD? We can add bsd.progs.mk (if you have devel/bmake port installed you have it as /usr/local/share/mk/progs.mk) and atf.test.mk and people can just go for it ? As long as it can function sanely in a NetBSD-like manner and I can start writing tests, I don't mind... Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 4, 2012, at 9:42 AM, Garrett Cooper yaneg...@gmail.com wrote: Both parties (Isilon/Juniper) are converging on the ATF porting work that Giorgos/myself have done after talking at the FreeBSD Foundation meet-n-greet. I have contributed all of the patches that I have other to marcel for feedback. This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). I'm in part to blame for this. There was some discussion -- but not at length; unfortunately no one from Juniper was present at the meet and greet; the information I got was second hand; I didn't follow up to figure out the exact details / clarify what I had in mind with the appropriate parties. Hang on. I want in on the blame part! :-) Seriously: no-one is really to blame as far as I can see. We just had two independent efforts (ATF bmake) and there was no indication that one would be greatly benefitted from the other. At least not to the point of creating a dependency. I just committed the bmake bits. It not only adds bmake to the build, but also includes the changes necessary to use bmake. With that in place it's easier to decide whether we want the dependency or not. Before we can switch permanently to bmake, we need to do the following first: 1. Request an EXP ports build with bmake as make(1). This should tell us the damage of switching to bmake for ports. 2. In parallel with 1: build www docs with bmake and assess the damage 3. Fix all the damage Then: 4. Switch. It could be a while (many weeks) before we get to 4, so the question really is whether the people working on ATF are willing and able to build and install FreeBSD using WITH_BMAKE? -- Marcel Moolenaar mar...@xcllnt.net ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Oct 2, 2012, at 10:37 , John Baldwin j...@freebsd.org wrote: This is very non-obvious to the public at large (e.g. there was no public response to one group's inquiry about the second ATF import for example). Also, given that you had no idea that sgf@ and obrien@ were working on importing NetBSD's bmake as a prerequisite for ATF, it seems that whatever discussions were held were not very detailed at best. I think it would be good to have the various folks working on ATF to at least summarize the current state of things and sketch out some sort of plan or roadmap for future work in a public forum (such as atf@, though a summary mail would be quite appropriate for arch@). I take partial responsibility for the privacy of the discussions hitherto. My apologies, it should have be moved out onto a public list sooner. But, I would like to drive this to a solution on arch@. We don't have an atf@, but we do have a test@ and testing@. We have too many mailing lists already, so let's finish this up here if we can and then continue talking on testing@. Best, George ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: ... But, I would like to drive this to a solution on arch@. We don't have an atf@, but we do have a test@ and testing@. We have too many mailing lists already, so let's finish this up here if we can and then continue talking on testing@. ... Before folks get too excited about test@ testing@: * test@ is for testing ability to send mail to FreeBSD.org (mailing lists). * testing@ *used* to exist, but was retired for lack of relevant traffic. * There's little difference in effort in resurrecting testing@ vs. creating atf@. Peace, david (current hat: part of postmas...@freebsd.org) -- David H. Wolfskill da...@catwhisker.org Depriving a girl or boy of an opportunity for education is evil. See http://www.catwhisker.org/~david/publickey.gpg for my public key. pgpqIh3blipIV.pgp Description: PGP signature
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Thu, Oct 4, 2012 at 9:29 AM, David Wolfskill da...@catwhisker.org wrote: On Thu, Oct 04, 2012 at 12:11:21PM -0400, George Neville-Neil wrote: ... But, I would like to drive this to a solution on arch@. We don't have an atf@, but we do have a test@ and testing@. We have too many mailing lists already, so let's finish this up here if we can and then continue talking on testing@. ... Before folks get too excited about test@ testing@: * test@ is for testing ability to send mail to FreeBSD.org (mailing lists). * testing@ *used* to exist, but was retired for lack of relevant traffic. * There's little difference in effort in resurrecting testing@ vs. creating atf@. I think that resurrecting testing@ makes more sense as creating an atf specific list seems a bit too focused on ATF, primarily because atf is being partially superseded by kyua (pronounced Q-A) in the near future. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Mon, Oct 1, 2012 at 5:00 PM, Simon J. Gerraty s...@juniper.net wrote: Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited = return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. Ok. Even though I originally thought that my changes were a bit hacky for not modifying bsd.man.mk, bsd.nls.mk, etc, looking back the way they were resolved was a little elegant in its simplicity (albeit not optimized). ... It order to do this, I need to be able to build multiple programs so = things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. 1. Test programs live with the sources (this was the requested approach), e.g. lib/libc/gen/... .../clock.c .../t_clock.c 2. Test programs live in subdirs: lib/libc/gen/... .../clock.c .../tests/... .../t_clock.c 3. Test programs completely decoupled from the source tree: lib/libc/gen/... .../clock.c tests/lib/libc/gen/... .../t_clock.c A hybrid approach (1.+3. / 2.+3.) should probably be used anyhow for stress tests and the like that really have no business living in one particular directory in the source tree (sort of achieving what tools/regression does today, but hopefully in a less messy manner as tools/regression appears to have grown organically minus any single sane order). Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to = function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Originally bsd.progs.mk was going to be a home grown effort that was going to be killed off and replaced with NetBSD's build infrastructure, but I didn't get a reply and had to put this (my work to replace bsd.prog.mk) plan in motion given requests I was provided above for resolving the source/unittest code consolidation effort. Ideally however, I would like doing this compared to running a custom = build infrastructure, but (as you probably know) this requires = rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. Yes for the most part, and I agree that bmake is definitely more advanced than FreeBSD pmake so I consider it a very welcome change. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? ... I know, but it is a very useful thing to be able to do. You can add knobs to control such things of course. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: 1. The it works for me on my machine! certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when testing their code (me included sometimes :)..); I've seen hardcoded paths, harcoded paths for named semaphores, things that just work because of someone's host environment, feature specific assumptions (developer X was doing testing on feature Y and things broke when he/she committed the testcase to head), etc where the it works for me on my machine! certification would be true more often than not. These same individuals would more likely than not execute things taking shortcuts, but I want to avoid creating a system where developers cut corners and commit too early because working within the confines of the system is not conducive to getting work done. 2. Failure
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Tue, 2 Oct 2012 07:19:55 -0700, Garrett Cooper writes: We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Hmmm... that's one of the 3 approaches I provided, but it turned out to be annoying to make this work at first inspection because of how things were done in our build system. Just for review (even though it's OT), the three approaches I presented were as follows... Note: in all 3 examples, clock.c is the source code and t_clock.c is the relevant unittest code. There is actually another: lib/libc/Makefile lib/libc/gen/... .../clock.c .../t_clock.c lib/libc/tests/Makefile that is the makefile for building/running the tests lives in the subdir, there are advantages to this, but the test code itself can be anywhere you like. Either next to the code that it tests, or in the tests/ subdir, another subdir, or whatever combination makes the most sense. Most of our ATF users put their test code in the subdir fwiw. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. This (consolidating everything under a single directory) is the way that was requested by the beforementioned parties. Sorry for not participating in that discussion ;-) John - perhaps we do need that ATF list? For example building a library and test apps in the one directory either requires turning bsd.lib.mk into an even more complex thing that a multi-prog bsd.pog.mk, or having the makefile behave completely differently depending on the target requested. That later is a bad idea. It precludes being able to seamlessly integrate unit-tests into the build, and would need to be fixed if/when attempting to introduce meta mode. Typically the unit-tests depend on the library they are testing, a separate subdir for the tests/Makefile makes capturing that dependency easy - otherwise you are back to the unspeakably complex bsd.lib.mk Far better to get these things right first up. I wish I had known that. I pinged marcel/obrien about this some weeks ago and didn't receive a response. Many appologies. What are the plans for importing bmake into FreeBSD (this should probably be a different thread)? The import has been done - just needs to be merged to contrib, and there's a small patch to be committed. We hope to get that done this week. I agree that it's a good thing (in theory) -- it'd just help to discuss this with Julio first because I know of a couple cases where this would be unusable given how bsd.test.mk is coded: I do talk to Julio about ATF (I'm s...@netbsd.org ;-) though not specifically this - beyond (I think) mentioning that I did it differently. 1. The it works for me on my machine! certification: It encourages environment tainting, which could be a really, really bad thing; I've seen developers take interesting shortcuts when True. In our (Junos) build we cleanse the environment rather carefully and have a very clear distinction between building for the host (which may just happend to be i386 based) and the i386 target for example. I can provide build smarts to do all that - but didn't want to shove it down anyone's throat ;-) What works for us doesn't necessarily work for everyone. 2. Failure in a subdirectory results in lost coverage: a/... .../b/... - tests live here .../c/d/... - tests live here If I execute make test from a/ (one of my goals), then it should iterate down a/b, then a/c/d and run the tests in a DFS style (BFS if This gets back to the bmake topic. We actually avoid walking the tree at all, eg. in the Junos build I have a target atf-tests which builds all ATF tests dirs regardless of where they are (and any pre-requisits they have). This is handy to include as a dependency of the build target we use for branch syncs etc, to ensure no unit-tests atrophy. But the most common case - and the one to optimize for, is the person making changes to libfoo, testing that he hasn't broken anything. could disguise other bugs. This would be fixed by changing bsd.test.mk to use atf-run properly, but that's work (not hard) that still needs to be done. Yes, atf.test.mk already does that. H... My goal was to make ATF a one ring to rule them all infrastructure so that way everything could be unified under ATF in ATF is a pretty good framework - which is why we (Juniper) use it. We want the flexibility to have more than one framework, but the number should be very small. So far I have persuaded teams that wanted to adopt alternate frameworks, that ATF could easily meet their needs. one way, shape or form (even if it's just reporting), because lord knows we don't need a lua, unittest/nose, etc wrapper for reporting results. All of the
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
Hi Simon! On Oct 1, 2012, at 3:31 PM, Simon J. Gerraty wrote: Hi Garrett, From: Garrett Cooper yaneg...@gmail.com Subject: [CFT/RFC]: refactor bsd.prog.mk to understand multiple = programs instead of a singular program Date: September 2, 2012 11:01:09 PM PDT To: freebsd-hackers@freebsd.org Cc: freebsd-a...@freebsd.org Arch freebsd-a...@freebsd.org =20 Hello, I've been a bit busy working on porting over ATF from NetBSD, and Thanks, we've been using ATF in Junos for a while and glad to see it being imported to FreeBSD. Thanks (and I appreciate the help marcel@ has given by taking the first step in importing it). one of the pieces that's currently not available in FreeBSD that's available in NetBSD is the ability to understand and compile multiple programs. In order to do this I had to refactor bsd.prog.mk (a lot). A change like this to bsd.prog.mk can have considerable fallout. Eg. any makefile that tweaks OBJS is suddenly out of luck. Well… any Makefiles attempting to be clever (e.g. crunchgen) are out of luck and would need to change. Most [include] Makefiles will see virtually no change. Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited return. I sort of tried to match bsd.prog.mk in NetBSD in the beginning, but things diverged from there… I'm sure things could be made more readable so any comments would be much appreciated :)! Apart from ATF, is there any huge demand for building multiple progs in the same directory? Well, I noticed that there are a couple ugly messes in the gnu/... directory that attempt to work around the fact that bsd.prog.mk doesn't support more than one program by being tricky and emulating bsd.prog.mk instead of solving the generic problem (and once I got over that compatibility issue I stopped tracking this class of consumer). Most consumers don't care (they split up programs into different directories or use hardlink hacks/basename detection to differentiate one program functionally from another). Getting back to the idea at hand, the enhancement goal was to get the testcases to live [and optionally be built/installed] with the source code to avoid bitrot; I know this isn't the current NetBSD design, but this is what was requested be done by gnn, marcel, and mdf, and I agree. It order to do this, I need to be able to build multiple programs so things are as transparent. On the flipside, bsd.prog[s].mk can live on its own, be pulled in automatically by bsd.test.mk, and that would be it. This encourages code duplication though and bugs can persist in either Makefile, when I'd rather work out all the kinks in whatever succeeds the legacy bsd.prog.mk file. FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to function the last I checked as bsd.prog.mk depends upon bmake directives in order to function properly 100% of the time (and there are external dependencies and assumptions one has to deal with when using bsd.prog.mk, etc from NetBSD that made that a bit of a no-go without refactoring/pulling in bits from NetBSD). I could be wrong though, so please let me know if I am :). Ideally however, I would like doing this compared to running a custom build infrastructure, but (as you probably know) this requires rototilling the current FreeBSD build system to a large degree (definitely not impossible… just needs time and effort). We have an atf.test.mk which leverages that. It also handles automatically running the tests if building for the host. This allows us to fail the build if any unit-tests do not pass. Hmm… that wasn't really the end-goal of bsd.test.mk based on my reading, but I'm sure Julio would be interested in it. I need to do adjusting in order to allow automatically executing testcases compatible to the host architecture, but that isn't hard to do (no more than a couple hours work). Note, atf.test.mk is loosely derrived from NetBSD's bsd.tests.mk, but named for what it is (ATF specific tests) since we wanted the flexibility to have more than one test framework. bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't entirely ATF centric either. I'm trying not to stray too far from NetBSD in this area because there really isn't any reason to do so. FWIW I'd like to see other test frameworks (lua, unittest//nose, etc) just snap into bsd.test.mk as easily as possible as it would make some groups lives easier, but that requires a bit more thought for another day. Thanks for the feedback! -Garrett___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
Not to mention the fact that bsd.prog.mk goes from being relatively simple, to unspeakably hard to read, and all for rather limited = return. This btw I think is the more important issue. I was looking at bsd.prog.mk in netbsd the other day. It has no business being that complex. Getting back to the idea at hand, the enhancement goal was to get the = testcases to live [and optionally be built/installed] with the source = code to avoid bitrot; I know this isn't the current NetBSD design, but = this is what was requested be done by gnn, marcel, and mdf, and I agree. = I agree, that's what we do too. It order to do this, I need to be able to build multiple programs so = things are as transparent. On the flipside, bsd.prog[s].mk can live on = We put the test cases in a subdir of the lib/prog This has multiple benefits, and eliminates any impact on the normal build of said libs/progs. Also a number of our teams find it necessary to create mock classes etc to adequately test their libs. Keeping all this in a tests/ subdir makes it easy, to keep things neat up to date, and ensure that the tests pass. Trying to do all that in the same dir as the lib would be ugly. FWIW we use progs.mk (as bsd.progs.mk) from ftp://ftp.netbsd.org/pub/NetBSD/misc/sjg/mk-*.tar.gz=20 It isn't ideal, but it certainly avoids a lot of churn and complexity for what is essentially a corner case. This requires pulling bmake into FreeBSD proper in order for things to = function the last I checked as bsd.prog.mk depends upon bmake directives = This is already happening ;-) Ideally however, I would like doing this compared to running a custom = build infrastructure, but (as you probably know) this requires = rototilling the current FreeBSD build system to a large degree = Actually building FreeBSD-current using bmake requires only modest changes. We have an atf.test.mk which leverages that. It also handles automatically running the tests if building for the host. This allows us to fail the build if any unit-tests do not pass. Hmm=85 that wasn't really the end-goal of bsd.test.mk based on my = reading, I know, but it is a very useful thing to be able to do. You can add knobs to controll such things of course. bsd.test.mk in NetBSD isn't completely test-agnostic, but it isn't = entirely ATF centric either. I'm trying not to stray too far from NetBSD = in this area because there really isn't any reason to do so. FWIW I'd = Sure - we imported ATF directly from NetBSD to minimize the churn and not had to change much at all. bsd.test.mk was a point worth diverging on though. like to see other test frameworks (lua, unittest//nose, etc) just snap = into bsd.test.mk as easily as possible as it would make some groups = Yes, but making one test.mk handle potentially lots of frameworks will get rather ugly quite quickly. Better to add per-framework .mk files, and perhaps have them include a bsd.test.mk which only handles high level logic rather than the details of the frameworks. That's laregly why we didn't use bsd.test.mk --sjg ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
In message cagh67wqty4krgwspa5jaht-4hqd4veykley-r3c6k9f5xaf...@mail.gmail.com , Garrett Cooper writes: No difference proven at 95.0% confidence This is the important bit of information... Thanks for the tip :)! You're welcome :-) -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 p...@freebsd.org | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sep 24, 2012, at 1:21 AM, Poul-Henning Kamp p...@phk.freebsd.dk wrote: In message cagh67wqty4krgwspa5jaht-4hqd4veykley-r3c6k9f5xaf...@mail.gmail.com , Garrett Cooper writes: No difference proven at 95.0% confidence This is the important bit of information... Yeah.. It's been a few moons since I've taken statistics. Some other pieces of data: This was done with... - testing was done multiuser on a lightly loaded system. - build was done with -j24 on a my workstation with 4 SMT enabled cores. - the scratch disk is ata enabled, whereas the sources were housed on an mfi backed disk. Thanks! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sat, Sep 22, 2012 at 11:30 PM, Poul-Henning Kamp p...@phk.freebsd.dk wrote: In message cagh67wsux7zjrtb5gehqwhkqykog-atwkinw5csjlrzftzk...@mail.gmail.com , Garrett Cooper writes: Without the change: $ python calc_runtime.py bw.*_without.log | ministat -w 72 [...] $ python calc_runtime.py bw.*.log | ministat -w 72 Try: python calc_runtime.py bw.*_without.log _without python calc_runtime.py bw.*.log _with ministat -w 72 _with _without :-) (PS: your two chosen glob patterns are not exclusive, but I suppose that was for illustration only) Forgot to mention that I ran the with results before the without results, so technically the files didn't exist yet and hence the results are separate. But, just for absolute clarity for archiving sake (and provided your suggestion on how to overlay both graphs), here are the results: $ for i in bw.*[0-9].log; do mv $i `echo $i | sed -e 's/\.log/_with\.log/g'`; done $ python calc_runtime.py bw.*_with.log _with $ python calc_runtime.py bw.*_without.log _without $ ministat -w 72 _with _without x _with + _without ++ | +x | |++ + xx + x xx+ ++ x x +xx +| | ||A_MA___M___|__| | ++ N Min MaxMedian AvgStddev x 10 940 1002 988 972.3 24.476973 + 10 919 1010 972 958.9 33.994934 No difference proven at 95.0% confidence Thanks for the tip :)! -Garrett ___ freebsd-hackers@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-hackers To unsubscribe, send any mail to freebsd-hackers-unsubscr...@freebsd.org
Re: [CFT/RFC]: refactor bsd.prog.mk to understand multiple programs instead of a singular program
On Sun, Sep 2, 2012 at 11:01 PM, Garrett Cooper yaneg...@gmail.com wrote:
Hello,
I've been a bit busy working on porting over ATF from NetBSD, and
one of the pieces that's currently not available in FreeBSD that's
available in NetBSD is the ability to understand and compile multiple
programs. In order to do this I had to refactor bsd.prog.mk (a lot).
The attached patch is the end result so far, and I was wondering if
anyone could please review it and/or test it (outside of me doing so).
I wrote over 40 tests, but it's not exercising everything, and I would
like for someone to please review/test this out who has an interest in
NLS support (ala bsd.nls.mk) in particular. AFAICT this is the only
gap that I couldn't resolve right away (there isn't a ton of recent
documentation on how to use bsd.nls.mk).
I'll run a micro benchmark and buildworld a few times (in
progress) with and without the change to measure the performance
effect.
Any assistance would be much appreciated.
I've attached an updated version of the patch (run through
buildworld successfully multiple times; make universe successfully;
going to submit for an -exp run). I needed to modify bsd.crunchgen.mk
to grok OBJS.${PROG}, but apart from that backwards compatibility has
been maintained -- sans-INSTALLFLAGS_EDIT (does anyone still use that
undocumented functionality?).
Performance wise, it's slightly slower on my W3520 with the
change, but not by much (~20 seconds).
As always, questions and comments are welcome, and if someone has
a chance, please review the proposed patch.
Thanks!
-Garrett
Without the change:
$ python calc_runtime.py bw.*_without.log | ministat -w 72
x stdin
++
| x |
|xx x x x xx x x|
| |_A_M| |
++
N Min MaxMedian AvgStddev
x 10 919 1010 972 958.9 33.994934
With the change:
$ python calc_runtime.py bw.*.log | ministat -w 72
x stdin
++
| x|
|xx xx x x x xx|
| |___A_M_| |
++
N Min MaxMedian AvgStddev
x 10 940 1002 988 972.3 24.476973
make universe results:
# make universe MAKE_JUST_WORLDS=y
--
make universe started on Tue Sep 18 09:30:04 PDT 2012
--
amd64 started on Tue Sep 18 09:30:04 PDT 2012
amd64.amd64 buildworld started on Tue Sep 18 09:30:04 PDT 2012
amd64.amd64 buildworld completed on Tue Sep 18 11:20:19 PDT 2012
amd64 completed on Tue Sep 18 11:20:19 PDT 2012
arm started on Tue Sep 18 11:20:19 PDT 2012
arm.arm buildworld started on Tue Sep 18 11:20:19 PDT 2012
arm.arm buildworld completed on Tue Sep 18 12:25:24 PDT 2012
arm.armeb buildworld started on Tue Sep 18 12:25:24 PDT 2012
arm.armeb buildworld completed on Tue Sep 18 13:30:25 PDT 2012
arm.armv6 buildworld started on Tue Sep 18 13:30:25 PDT 2012
arm.armv6 buildworld completed on Tue Sep 18 14:35:14 PDT 2012
arm.armv6eb buildworld started on Tue Sep 18 14:35:14 PDT 2012
arm.armv6eb buildworld completed on Tue Sep 18 15:40:05 PDT 2012
arm completed on Tue Sep 18 15:40:05 PDT 2012
i386 started on Tue Sep 18 15:40:05 PDT 2012
i386.i386 buildworld started on Tue Sep 18 15:40:05 PDT 2012
i386.i386 buildworld completed on Tue Sep 18 16:56:06 PDT 2012
i386 completed on Tue Sep 18 16:56:06 PDT 2012
ia64 started on Tue Sep 18 16:56:06 PDT 2012
ia64.ia64 buildworld started on Tue Sep 18 16:56:06 PDT 2012
ia64.ia64 buildworld completed on Tue Sep 18 18:27:49 PDT 2012
ia64 completed on Tue Sep 18 18:27:49 PDT 2012
mips started on Tue Sep 18 18:27:49 PDT 2012
mips.mipsel buildworld started on Tue Sep 18 18:27:49 PDT 2012
mips.mipsel buildworld completed on Tue Sep 18 19:34:50 PDT 2012
mips.mips buildworld started on Tue Sep 18 19:34:50 PDT 2012
mips.mips buildworld completed on Tue Sep 18 20:41:49 PDT 2012
mips.mips64el buildworld started on Tue Sep 18 20:41:49 PDT 2012
mips.mips64el buildworld completed on Tue Sep 18 21:49:21 PDT 2012
mips.mips64 buildworld started on Tue Sep 18 21:49:21 PDT 2012
mips.mips64 buildworld completed on Tue Sep 18 22:57:13 PDT 2012
mips.mipsn32 buildworld started on Tue Sep 18 22:57:13 PDT 2012
mips.mipsn32 buildworld
