I have posted for your reviewing pleasure, the changes to convert the X gate build to IPS, and in the process refactor our packages, in a series of webrevs to http://cr.opensolaris.org/~alanc/xnv-ips/
These should cover: 6941932 X should be able to build IPS packages natively 6964006 X package boundaries are all wrong 15612 nvidia packages should be split from X into their own consolidation incorporation My target for integration is build snv_144, which in theory closes on Monday, July 5. However, that's a holiday in the US but not for our QA team in Beijing, so I'm locking everyone else out of the gate at noon Friday, July 2 and the actual gate close for X will likely be at whatever point between Friday and Sunday July 4 I decide this integration is ready for me to set the cron jobs so the build is available for QA to pick up and start pre-integration testing while we're out watching things blow up (hopefully in the sky, not the build). http://cr.opensolaris.org/~alanc/xnv-ips/pkg-infrastructure/ - This is the set of changes that actually build packages from our proto area and publish them to an IPS repo. pkg/Makefile should look very familiar to those who've read ON's usr/src/pkg/Makefile. Major differences from ON in the X implementation: - X only supports building one repo, not redist + extra, since all our packages are fully open source/redistributable - X only supports building packages on the same platform as the binaries, no cross-platform packaging - X has no bfu legacy to deal with - X has, due to a historical mistake, exactly one kernel module (which I've been wanting to move to ON for a decade but never seem to get around to), so we have a lot fewer global-zone-only headaches to deal with. - ON has all manifests in one directory, X splits into current & legacy - Integrates with X consolidation makefile/build system instead of ON's - X incorporates a lot (really, a whole lot) of packaging metadata from files generated during the build of the gate, plus a bit more generated during the package build (font tags from pkg/generate_font_metadata.pl) - I know how to write perl but not python, so our scripts all end in .pl, not .py. (I know, this is a bug I should fix someday, but haven't had a chance yet.) make_release_packages is the script that we used to use to run pkgmk on the SVR4 prototypes to generate packages - it's been mostly gutted and replaced with running the appropriate make targets to do the IPS package build, so that developers and our build cron jobs can continue to just run that command either directly or via the -p option to our gate's buildit script. find-build-errors is a helper script that our developers use (as does our nightly cron jobs) to try to pick out errors faster from build logs - it's updated to read the output generated by the "make" and "make check" rules in the package gate. http://cr.opensolaris.org/~alanc/xnv-ips/pkg-data/ All 295 package manifests, including 194 current packages, and 101 "legacy" (renamed or obsolete) packages, plus 2 bonus files for the default copyright/license notice. Most of the manifests for current packages were initially generated via "make manifest" and then hand tweaked as needed. (This is a large part of why I wanted to do migration & refactoring at the same time.) There is still time to convince me I got any of the names or package breakdowns/mashups incorrect - but thanks to the magic of IPS rename tracking, that's true even after this is putback and the build is done, since we can simply change it again in a later build, so I'm not going to be super-anxious about getting it absolutely perfect the first time, just as long as we have more paint to cover up the bad spots on the bikeshed later. http://cr.opensolaris.org/~alanc/xnv-ips/build-changes/ The changes to the building side of the gate. The bulk of the changes are updating/adding metadata to the Makefiles for each upstream module, much of which gets passed onto the packaging system. The updated package names are also fed into our script to add the Solaris Attributes table to the upstream man pages. open-src/README should explain all the new variables that can be set in the Makefiles and how they get used. The most interesting bits are probably in open-src/common/Makefile.init and open-src/common/Makefile.inc, which most other makefiles include. (The new pkg/Makefile includes open-src/common/Makefile.init but not .inc) Additionally the previous data/smf-rbac module that had all our SMF & RBAC files for SUNWxwplr has been split up and moved into the modules that actually need the various files, including two new modules - data/ogl-select for the ogl-select SMF service to create the OpenGL links and xserver/xserver-common for the x11-server SMF service to store various configuration properties for the X server and the Xserver script that reads them when gdm/xdm/etc. start the server. Not shown in any webrev: hg rm packages* - the entire directory containing our SVR4 package prototypes was removed. If you want to see what was in it, you can browse http://src.opensolaris.org/source/xref/x-cons/xnv-clone/packages/ or check out the diffs in: http://cr.opensolaris.org/~alanc/xnv-ips/hg.diff [ 2.7 MB!! ] 934 files changed, 25654 insertions(+), 27328 deletions(-) A complete hg diff --git, which I think you should be able to apply to a hg clone of ssh://[email protected]/hg/x-cons/xnv-clone if you want to see this all in complete context or try it out. http://cr.opensolaris.org/~alanc/xnv-ips/pkg-gate/ The distro-import changes I think will be needed for this change. Unlike the X gate changes, I've not tested these in any way, shape or form, and have especially not verified that the subset of packages I listed to be included in slim_cluster lists all of their dependencies. (I've left the include lines in for the X package import files, since the ON ones are still there, though I don't know why they're there.) I have purposely not chosen to continue including in the LiveCD everything that was in the old X mega-packages. One of the main points of refactoring is to allow better minimization, and I think a lot of the stuff we used to include just because it was in SUNWxwplt or SUNWxwopt doesn't really need to be there. I expect David and/or David will refine these over time as they do for all changes to the set of packages on the LiveCD (probably even before they integrate for 144). Since the development, documentation and runtime components have all been merged into a single package, but tagged with the appropriate facets, it would be necessary to set facet.devel=false for slim_install to more closely emulate the old pattern of including the documentation but not the headers/lint libs/etc. For those who know their way around the X gate already, this will introduce some changes to the top-level layout of directories in the gate. Previously, it was: open-src/ All the source files/build area packages/ All the package prototypes/templates proto-sun4-svr4/ The SPARC proto area where files are installed during the build and the package build looks for them proto-i386-svr4/ The x86 proto area... proto-packages/ Where make_release_packages puts the packages it builds for whichever platform you last ran it for log/ Where buildit writes the build log Now it is: open-src/ All the source files/build area pkg/ All the package manifests/templates proto/ Parent directory for all the proto areas & package repositories. Can rm -r the whole thing when cleaning the gate and it will be rebuilt on the next build. proto/root_sparc The SPARC proto area where files are installed during the build and the package build looks for them proto/root_i386 The x86 proto area... proto/pkg_sparc The SPARC pkg repository built by make_release_packages proto/pkg_i386 The x86 pkg repository built by make_release_packages proto/metadata_sparc Metadata generated during the build, including the proto/metadata_i386 license files previously installed to proto*/licenses proto/tools_sparc Tools built and used during the build but not proto/tools_i386 packaged (previously proto-*-svr4/tools) log/ Where buildit & make_release packages write their logs Additionally you will find the proto/root_* directories much more closely match the layout of the delivered files than previously, as we no longer perform path rewriting in the packaging process. The build now includes validation of the packages against the proto area, to catch mistakes with files that get added in new upstream builds that we forget to add - if we determine a new file should not be added, either the Makefile can be updated to remove it from the proto area after the upstream "make install" step, or the exceptions_list/packaging file can be updated to tell the validation script to ignore it. -- -Alan Coopersmith- [email protected] Oracle Solaris Platform Engineering: X Window System _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
