Re: Cross-Development with NetBSD
On Thu, Nov 21, 2002 at 11:29:54AM -0800, Terry Lambert wrote: Ruslan Ermilov wrote: NetBSD builds a directory full of tools that you can later use to incrementally build, say, 'ls' or 'cat' because one can define USETOOLS to be 'yes' and have the make automatically pick them up when rebuilding. There are a few of the details I'm a little unclear on, but that's the jist of it. We also can, this just requires a few really tiny tweaks to Makefile.inc1, and I've posted them already some time ago -- basically, for each architecture you should build the subset of buildworld targets (WMAKE_TGTS), up to and including _libraries (if you want to build roughly any bit later), and them you can ``make {depend|all} SUBDIR_OVERRIDE=bin/cat'' for each of the desired TARGET_ARCH. Any ETA on when this will be committed? It's already available, in some form (please see the attached). I know that the Alpha and sparc64 binaries produced on i386 work. I thought that the Alpha boot blocks ended up too large in the cross-build case? They did, last time I tried it. Binaries in the usual sense -- libraries and programs. As for boot blocks, they seem to be fine nowadays -- but I lack the necessary h/w to actually test (and fix if needed the building of) them: ftp://ftp.sunbay.net/pub/FreeBSD/snapshots/alpha/ I know that cross-compiling i386 on either Alpha or sparc64 is broken (GCC sometimes produces different assembler output than the native compiler). I lack the necessary hardware to actually test/fix the issues with cross-releases. I don't think he was attacking you, personally, to ask you to fix the problem, I think he was just noting the problem exists. I don't think either -- please see my reply to Warner. :-) One thing that would help a lot -- and probably be helpful in general -- would be a binary compare tool that ignored date stamps in things like libraries, tar images, etc., so that you could compare where things differ, easily, allowing someone to track down differences. I use -save-temps option to cc(1) to do this, if I find any differences, and compare .s and .i files instead. It would be helpful in general to be able to compare what you built vs. a release version, to assemble binary only delta lists, for preparations for upgrade tools, etc.. I keep meaning to do this, but I really don't want to have to release the tool under the GPL, if I don't have to. Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age ---BeginMessage--- On Thu, Jul 25, 2002 at 07:46:50PM -0700, Mark Peek wrote: At 6:50 PM +0300 7/25/02, Ruslan Ermilov wrote: How about this simple script (tinder.sh)? : #!/bin/sh : cd /usr/src : ARCHES=i386 alpha sparc64 : if [ x$1 = x ]; then :for arch in ${ARCHES}; do :make TARGET_ARCH=${arch} -j4 -m/usr/src/share/mk \ :-fMakefile.inc1 -DNOCLEAN crossenv :done : else :for arch in ${ARCHES}; do :for target in cleandir obj depend all; do :make TARGET_ARCH=${arch} ${target} SUBDIR_OVERRIDE=$1 :done :done : fi You could then run it twice: sh ./tinder.sh (rebuild cross environment: compiler, binutils, headers, libs) sh ./tinder.sh bin/tcsh (test if bin/tcsh builds ok) ... with this small assorted patch for Makefile.inc1, just to avoid doing a whole buildworld; we need it for PicoBSD anyway: %%% Index: Makefile.inc1 === RCS file: /home/ncvs/src/Makefile.inc1,v retrieving revision 1.297 diff -u -p -u -r1.297 Makefile.inc1 --- Makefile.inc124 Jul 2002 23:08:57 - 1.297 +++ Makefile.inc125 Jul 2002 15:37:51 - @@ -329,6 +329,8 @@ WMAKE_TGTS+= _includes _libraries _depen buildworld: ${WMAKE_TGTS} .ORDER: ${WMAKE_TGTS} +crossenv: ${WMAKE_TGTS:N_depend:Neverything} + # # installcheck # %%% Very cool! I like it in principle! :-) The crossenv build worked fine but the compilation of bin/tcsh only resulted in building 3 copies of the i386 tcsh. I think it needs the WMAKE environment exported so it has the path to the cross compiler and points to the appropriate OBJDIR. Here's the new script that should work. Sorry for a late reply. Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age tinder.sh Description: Bourne shell script msg47175/pgp0.pgp Description: PGP signature ---End Message--- msg47175/pgp1.pgp Description: PGP signature
Re: Cross-Development with NetBSD
On Thu, Nov 21, 2002 at 02:31:19PM -0500, John Baldwin wrote: On 21-Nov-2002 Terry Lambert wrote: Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create : a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? You know, like changing one line in /usr/src/lib/libstand on a source tree on a x86 box, typing make release, and having only the things that need to be rebuilt being rebuilt, resulting in a working FreeBSD-Alpha or FreeBSD-SPARC64 release CDROM image. Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. What would you prefer, I didn't quite understand? Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age msg47177/pgp0.pgp Description: PGP signature
Re: Why isn't NOCLEAN the default? (was: Re: Cross-Development with NetBSD)
On Thu, Nov 21, 2002 at 02:48:01PM -0800, David Schultz wrote: Thus spake Kris Kennaway [EMAIL PROTECTED]: I have long wondered why NOCLEAN isn't the default. There seem to be a few cases where it doesn't DTRT for kernel builds, but it seems a bit conservative to make incremental world builds require that an undocumented variable be defined. Any ideas? It often causes problems during upgrades (but is usually fine when just rebuilding a non-updated tree) Sounds reasonable. Maybe it should be documented in build(7), though. Yet it's documented in make.conf(5). Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age msg47178/pgp0.pgp Description: PGP signature
Re: Cross-Development with NetBSD
On 22-Nov-2002 Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 02:31:19PM -0500, John Baldwin wrote: On 21-Nov-2002 Terry Lambert wrote: Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create : a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? You know, like changing one line in /usr/src/lib/libstand on a source tree on a x86 box, typing make release, and having only the things that need to be rebuilt being rebuilt, resulting in a working FreeBSD-Alpha or FreeBSD-SPARC64 release CDROM image. Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. What would you prefer, I didn't quite understand? If I could: % cd /usr/src/usr.bin/foo_binary % make TARGET_ARCH=alpha and end up with an alpha foo binary. Granted, I might have to do some extra steps to setup the build environment, and that would be fine. Would be nice if I could do: % cd /usr/src % make TARGET_ARCH=alpha buildenv or some such. However, I think that might involve a good bit of work to get right and the gains might not be worth the expense. -- John Baldwin [EMAIL PROTECTED]http://www.FreeBSD.org/~jhb/ Power Users Use the Power to Serve! - http://www.FreeBSD.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Cross-Development with NetBSD
On Fri, Nov 22, 2002 at 10:57:44AM -0500, John Baldwin wrote: On 22-Nov-2002 Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 02:31:19PM -0500, John Baldwin wrote: On 21-Nov-2002 Terry Lambert wrote: Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create : a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? You know, like changing one line in /usr/src/lib/libstand on a source tree on a x86 box, typing make release, and having only the things that need to be rebuilt being rebuilt, resulting in a working FreeBSD-Alpha or FreeBSD-SPARC64 release CDROM image. Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. What would you prefer, I didn't quite understand? If I could: % cd /usr/src/usr.bin/foo_binary % make TARGET_ARCH=alpha and end up with an alpha foo binary. Granted, I might have to do some extra steps to setup the build environment, and that would be fine. Would be nice if I could do: % cd /usr/src % make TARGET_ARCH=alpha buildenv or some such. However, I think that might involve a good bit of work to get right and the gains might not be worth the expense. See my other message in this thread [EMAIL PROTECTED] which includes the tinder.sh script that does exactly that. Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age msg47212/pgp0.pgp Description: PGP signature
Re: Cross-Development with NetBSD
On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age msg47070/pgp0.pgp Description: PGP signature
Re: Cross-Development with NetBSD
In message: [EMAIL PROTECTED] Ruslan Ermilov [EMAIL PROTECTED] writes: : On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: : In message: [EMAIL PROTECTED] : Wilkinson,Alex [EMAIL PROTECTED] writes: : : Is FreeBSD likely to follow the in footsteps of NetBSD and create a framework to :do crossbuilds ? : : : : http://ezine.daemonnews.org/200211/xdevnetbsd.html : : FreeBSD already has cross builds for a while, since before NetBSD's : cross build infrastructure. However, NetBSD's infrastructure is a : little more extensive because it is possible to do incremental builds : and build full releases that work in a cross build evironment. : : What do you mean by incremental builds and full releases that work ...? ruslan, First and foremost, I ment no disrespect in what I said. You have made the upgrade path for FreeBSD almost painless for a wide range of releases (less painful than NetBSD has been at times for me). You gave FreeBSD the ability to cross build world about a year or a year and a half ahead of NetBSD's similarly integrated ability. NetBSD builds a directory full of tools that you can later use to incrementally build, say, 'ls' or 'cat' because one can define USETOOLS to be 'yes' and have the make automatically pick them up when rebuilding. There are a few of the details I'm a little unclear on, but that's the jist of it. Second, the NetBSD have tested almost all of their platforms in a cross build release scenario. They know that the binaries that result from 'make release' build on i386 actually boot on the target platform. They have build 1.6.x for many of their platforms this way. So far, the FreeBSD has done all its builds natively. One of the cool things about the NetBSD approach, btw, is that they have a src/tools directory. This directory lists all the build tools for building in one place. I think that this is a little cleaner than what we have in the Makefile.inc1 right now. On the other hand, it is a lot more complicated than what we have now, and I'm not sure that added complexity is worth it. Warner To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Cross-Development with NetBSD
On Thu, Nov 21, 2002 at 06:28:09AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Ruslan Ermilov [EMAIL PROTECTED] writes: : On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: : In message: [EMAIL PROTECTED] : Wilkinson,Alex [EMAIL PROTECTED] writes: : : Is FreeBSD likely to follow the in footsteps of NetBSD and create a framework to do crossbuilds ? : : : : http://ezine.daemonnews.org/200211/xdevnetbsd.html : : FreeBSD already has cross builds for a while, since before NetBSD's : cross build infrastructure. However, NetBSD's infrastructure is a : little more extensive because it is possible to do incremental builds : and build full releases that work in a cross build evironment. : : What do you mean by incremental builds and full releases that work ...? ruslan, First and foremost, I ment no disrespect in what I said. You have made the upgrade path for FreeBSD almost painless for a wide range of releases (less painful than NetBSD has been at times for me). You gave FreeBSD the ability to cross build world about a year or a year and a half ahead of NetBSD's similarly integrated ability. Sorry if I sounded so, I didn't mean to. :-) NetBSD builds a directory full of tools that you can later use to incrementally build, say, 'ls' or 'cat' because one can define USETOOLS to be 'yes' and have the make automatically pick them up when rebuilding. There are a few of the details I'm a little unclear on, but that's the jist of it. We also can, this just requires a few really tiny tweaks to Makefile.inc1, and I've posted them already some time ago -- basically, for each architecture you should build the subset of buildworld targets (WMAKE_TGTS), up to and including _libraries (if you want to build roughly any bit later), and them you can ``make {depend|all} SUBDIR_OVERRIDE=bin/cat'' for each of the desired TARGET_ARCH. Second, the NetBSD have tested almost all of their platforms in a cross build release scenario. They know that the binaries that result from 'make release' build on i386 actually boot on the target platform. They have build 1.6.x for many of their platforms this way. So far, the FreeBSD has done all its builds natively. I know that the Alpha and sparc64 binaries produced on i386 work. I know that cross-compiling i386 on either Alpha or sparc64 is broken (GCC sometimes produces different assembler output than the native compiler). I lack the necessary hardware to actually test/fix the issues with cross-releases. One of the cool things about the NetBSD approach, btw, is that they have a src/tools directory. This directory lists all the build tools for building in one place. I think that this is a little cleaner than what we have in the Makefile.inc1 right now. On the other hand, it is a lot more complicated than what we have now, and I'm not sure that added complexity is worth it. When you say build tools, do you mean them in our build-tools sense, or something else? Cheers, -- Ruslan Ermilov Sysadmin and DBA, [EMAIL PROTECTED] Sunbay Software AG, [EMAIL PROTECTED] FreeBSD committer, +380.652.512.251Simferopol, Ukraine http://www.FreeBSD.org The Power To Serve http://www.oracle.com Enabling The Information Age msg47086/pgp0.pgp Description: PGP signature
Re: Cross-Development with NetBSD
Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create : a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? You know, like changing one line in /usr/src/lib/libstand on a source tree on a x86 box, typing make release, and having only the things that need to be rebuilt being rebuilt, resulting in a working FreeBSD-Alpha or FreeBSD-SPARC64 release CDROM image. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Cross-Development with NetBSD
On 21-Nov-2002 Terry Lambert wrote: Ruslan Ermilov wrote: On Thu, Nov 21, 2002 at 12:10:14AM -0700, M. Warner Losh wrote: In message: [EMAIL PROTECTED] Wilkinson,Alex [EMAIL PROTECTED] writes: : Is FreeBSD likely to follow the in footsteps of NetBSD and create : a framework to do crossbuilds ? : : http://ezine.daemonnews.org/200211/xdevnetbsd.html FreeBSD already has cross builds for a while, since before NetBSD's cross build infrastructure. However, NetBSD's infrastructure is a little more extensive because it is possible to do incremental builds and build full releases that work in a cross build evironment. What do you mean by incremental builds and full releases that work ...? You know, like changing one line in /usr/src/lib/libstand on a source tree on a x86 box, typing make release, and having only the things that need to be rebuilt being rebuilt, resulting in a working FreeBSD-Alpha or FreeBSD-SPARC64 release CDROM image. Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. -- John Baldwin [EMAIL PROTECTED]http://www.FreeBSD.org/~jhb/ Power Users Use the Power to Serve! - http://www.FreeBSD.org/ To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Cross-Development with NetBSD
Ruslan Ermilov wrote: NetBSD builds a directory full of tools that you can later use to incrementally build, say, 'ls' or 'cat' because one can define USETOOLS to be 'yes' and have the make automatically pick them up when rebuilding. There are a few of the details I'm a little unclear on, but that's the jist of it. We also can, this just requires a few really tiny tweaks to Makefile.inc1, and I've posted them already some time ago -- basically, for each architecture you should build the subset of buildworld targets (WMAKE_TGTS), up to and including _libraries (if you want to build roughly any bit later), and them you can ``make {depend|all} SUBDIR_OVERRIDE=bin/cat'' for each of the desired TARGET_ARCH. Any ETA on when this will be committed? I know that the Alpha and sparc64 binaries produced on i386 work. I thought that the Alpha boot blocks ended up too large in the cross-build case? They did, last time I tried it. I know that cross-compiling i386 on either Alpha or sparc64 is broken (GCC sometimes produces different assembler output than the native compiler). I lack the necessary hardware to actually test/fix the issues with cross-releases. I don't think he was attacking you, personally, to ask you to fix the problem, I think he was just noting the problem exists. One thing that would help a lot -- and probably be helpful in general -- would be a binary compare tool that ignored date stamps in things like libraries, tar images, etc., so that you could compare where things differ, easily, allowing someone to track down differences. It would be helpful in general to be able to compare what you built vs. a release version, to assemble binary only delta lists, for preparations for upgrade tools, etc.. I keep meaning to do this, but I really don't want to have to release the tool under the GPL, if I don't have to. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Why isn't NOCLEAN the default? (was: Re: Cross-Development with NetBSD)
Thus spake John Baldwin [EMAIL PROTECTED]: Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. I have long wondered why NOCLEAN isn't the default. There seem to be a few cases where it doesn't DTRT for kernel builds, but it seems a bit conservative to make incremental world builds require that an undocumented variable be defined. Any ideas? To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Re: Why isn't NOCLEAN the default? (was: Re: Cross-Development with NetBSD)
On Thu, Nov 21, 2002 at 02:02:20PM -0800, David Schultz wrote: Thus spake John Baldwin [EMAIL PROTECTED]: Make release is a very poor example b/c make release goes to great efforts to create a clean-room environment for a release. make rerelease is quite helpful though and does do what you want to restart a previous release. :) Also, make buildworld -DNOCLEAN isn't too shabby, though if I could do make TARGET_ARCH=alpha everything I would prefer that. I have long wondered why NOCLEAN isn't the default. There seem to be a few cases where it doesn't DTRT for kernel builds, but it seems a bit conservative to make incremental world builds require that an undocumented variable be defined. Any ideas? It often causes problems during upgrades (but is usually fine when just rebuilding a non-updated tree) Kris msg47132/pgp0.pgp Description: PGP signature
Re: Why isn't NOCLEAN the default? (was: Re: Cross-Development with NetBSD)
Thus spake Kris Kennaway [EMAIL PROTECTED]: I have long wondered why NOCLEAN isn't the default. There seem to be a few cases where it doesn't DTRT for kernel builds, but it seems a bit conservative to make incremental world builds require that an undocumented variable be defined. Any ideas? It often causes problems during upgrades (but is usually fine when just rebuilding a non-updated tree) Sounds reasonable. Maybe it should be documented in build(7), though. To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message
Cross-Development with NetBSD
Is FreeBSD likely to follow the in footsteps of NetBSD and create a framework to do crossbuilds ? http://ezine.daemonnews.org/200211/xdevnetbsd.html - aW To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-current in the body of the message