Re: virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Fri, 10 Oct 2014, Rich Freeman wrote: On Fri, Oct 10, 2014 at 5:31 PM, W. Trevor King wk...@tremily.us wrote: On Fri, Oct 10, 2014 at 09:22:18PM +, Robin H. Johnson wrote: In a similar vein, would releng be open to moving stage1/2/3 package sets to virtual packages or package sets? Presently they are inside catalyst, and I think this would clean things up a lot. They're already in the Portage tree (the profile's packages.build for stage1, scripts/bootstrap.sh for stage2, and the profile's packages for stage3) [1]. Obviously this entails work on somebody's part, but would it still make sense to make the stage build process more generic along the lines Robin suggested? That is, instead of having 3 specific places we use to generate a stage1/2/3, we instead just define a stage as a virtual of some kind that optionally depends on another stage (not necessarily using the standard DEPEND mechanism) and then pulls in a list of packages. The root stage would not depend on another stage, and therefore would be built from the host system. All of the above suggestions would require changes to catalyst. In any case, given the way we build a stage1 and bootstrap stage2, that isn't possible. For stage1 and stage2 the *order* we build packages is relevant. We rely on portage following that list in sequence. Furthermore, because of the implicit dependencies and issues with circular dependencies, it would likely be impossible for portage to resolve the list of packages to build for stage1 and stage2 from a virtual. The build system would just iteratively start at the bottom and output a tarball or whatever as each level is completed, then use that as the basis for bootstrapping the next. That's how catalyst already works. To address one point in your last sentence in the above quote: The root stage would not depend on another stage, and therefore would be built from the host system. You almost always don't want to build with the state of a live system, but of a clean seed - that's why releng tells catalyst to use a stage3 as the seed for the stage1. Such a design would also eliminate the need to have the same list of packages define the contents of @system and a stage3. It could also support any number of stages, with any names we wish. No, not really. You're over-thinking this. To be able to split the system set and the stages releng provides, all we need is to fix the bug that was already mentioned before and have releng provide stages built from a stage3 (while removing some packages from the system set) and a list of packages that we want to provide (the ones dropped from system and a select few others). I would also still have stage builds use a profile of some kind - that could be useful from a standpoint of setting USE flags and so on. However, if it made sense to build earlier stages with different flags they could still be specified as USE dependencies (maybe due to circular deps you have to build a package with different set of USE flags before you can build the desired USE flags in a later stage). We build stages using a profile. One of the variables set on stage specs is profile to list the profile to be used in the stage. -- Rich Regards, Jorge Manuel B. S. Vicetto Gentoo Developer PS - As I've warned before, I'm not following closely the dev ml. So you got this reply now, because I just happened to look at the emails in this ml. If you want me to comment further, your best option is to poke me about it.
Re: virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Thu, Oct 16, 2014 at 12:13:45AM +, Jorge Manuel B. S. Vicetto wrote: For stage1 and stage2 the *order* we build packages is relevant. Is this really true? The stage1 is being built with ROOT, so it's only using the seed stage3 packages. It's hard to have cyclic dependencies when you're using packages from one root (the seed stage3) but installing into another (/tmp/stage1root). Looking through a stage1 log I see: emerge --quiet --oneshot --nodeps --update sys-apps/portage … emerge --quiet --update --deep --newuse --complete-graph --rebuild-if-new-ver gcc … emerge --quiet --usepkg --buildpkg --newuse --oneshot --nodeps sys-apps/baselayout … emerge --quiet --usepkg --buildpkg --newuse --oneshot sys-devel/gnuconfig sys-devel/bison … The first two are just covering us against serious missmatches between the package tree and the seed stage3 (and are installed with ROOT=/). I expect we could handle shoving baselayout into the final emerge along with the other packages.build stuff. The logs for stage2 aren't as clear, but looking through the script I only see: * A Portage-updating emerge * The main GCC, binutils, … emerge * A possible 'emerge --prune sys-devel/gcc' I'm not sure this is needed at all. I'll try and find time to build a stage3 directly from a stage1, and we'll see if it blows up ;). Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Sun, Sep 28, 2014 at 12:31:16PM -0400, Richard Yao wrote: May I suggest an alternative? We could implement sys-virtual/posix and make it depend on all packages that are not necessary for @system, but are necessary for proper POSIX compliance. Then we can tell users who need/want an environment containing all tools specified by POSIX, such as those not using sys-kernel/*-sources, to `emerge virtual/posix`. I like this idea, for a virtual/posix; would let us trim a lot of other things from some environments. In a similar vein, would releng be open to moving stage1/2/3 package sets to virtual packages or package sets? Presently they are inside catalyst, and I think this would clean things up a lot. -- Robin Hugh Johnson Gentoo Linux: Developer, Infrastructure Lead E-Mail : robb...@gentoo.org GnuPG FP : 11ACBA4F 4778E3F6 E4EDF38E B27B944E 34884E85
Re: virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Fri, Oct 10, 2014 at 09:22:18PM +, Robin H. Johnson wrote: In a similar vein, would releng be open to moving stage1/2/3 package sets to virtual packages or package sets? Presently they are inside catalyst, and I think this would clean things up a lot. They're already in the Portage tree (the profile's packages.build for stage1, scripts/bootstrap.sh for stage2, and the profile's packages for stage3) [1]. Cheers, Trevor [1]: http://git.tremily.us/?p=catalyst.git;a=blob;f=doc/HOWTO.txt;h=cec22c35484d480fa127beb7be6bbdd9e942dc39;hb=HEAD#l139 Linking to my public repo, because the gitweb service on git.overlays.gentoo.org is still broken. -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
Re: virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Fri, Oct 10, 2014 at 5:31 PM, W. Trevor King wk...@tremily.us wrote: On Fri, Oct 10, 2014 at 09:22:18PM +, Robin H. Johnson wrote: In a similar vein, would releng be open to moving stage1/2/3 package sets to virtual packages or package sets? Presently they are inside catalyst, and I think this would clean things up a lot. They're already in the Portage tree (the profile's packages.build for stage1, scripts/bootstrap.sh for stage2, and the profile's packages for stage3) [1]. Obviously this entails work on somebody's part, but would it still make sense to make the stage build process more generic along the lines Robin suggested? That is, instead of having 3 specific places we use to generate a stage1/2/3, we instead just define a stage as a virtual of some kind that optionally depends on another stage (not necessarily using the standard DEPEND mechanism) and then pulls in a list of packages. The root stage would not depend on another stage, and therefore would be built from the host system. The build system would just iteratively start at the bottom and output a tarball or whatever as each level is completed, then use that as the basis for bootstrapping the next. Such a design would also eliminate the need to have the same list of packages define the contents of @system and a stage3. It could also support any number of stages, with any names we wish. My intent here isn't to get three cheers for making the releng team do more work. I'm actually more interested in feedback as to whether there are any obvious flaws in an approach like this that I'm missing, or whether something entirely different would be better. I would also still have stage builds use a profile of some kind - that could be useful from a standpoint of setting USE flags and so on. However, if it made sense to build earlier stages with different flags they could still be specified as USE dependencies (maybe due to circular deps you have to build a package with different set of USE flags before you can build the desired USE flags in a later stage). -- Rich
Re: virtual/{posix,stage1,2,3} Was: [gentoo-dev] Add bc back to the stage3
On Fri, Oct 10, 2014 at 09:45:37PM -0400, Rich Freeman wrote: Obviously this entails work on somebody's part, but would it still make sense to make the stage build process more generic along the lines Robin suggested? That is, instead of having 3 specific places we use to generate a stage1/2/3, we instead just define a stage as a virtual of some kind that optionally depends on another stage (not necessarily using the standard DEPEND mechanism) and then pulls in a list of packages. The root stage would not depend on another stage, and therefore would be built from the host system. Why bother with multiple layers and per-profile package lists if a straight emerge is all you need? My ideal long-term situation would be to explicitly list all a package's dependencies (no implicit @system dependencies). Then have the stage1 just emerge a self-hosting virtual/toolchain (which can have per-arch dependencies if it needs them) from an arbitrary seed, and stage2 rebuild -e using itself. Everything after that could be user-generated @world stuff, and you might want virtual/release with nano and whatever else you want to put up under releases/$ARCH/autobuilds/gentoo-$ARCH-$DATE.tar.bz. That's less magical, and Catalyst could be a bit simpler, but I don't have the energy to move things there ;). It's also pretty much what we have now, except @system is floating around between virtual/toolchain (our current stage1 packages) and virtual/release (our current stage3 packages). Cheers, Trevor -- This email may be signed or encrypted with GnuPG (http://www.gnupg.org). For more information, see http://en.wikipedia.org/wiki/Pretty_Good_Privacy signature.asc Description: OpenPGP digital signature
