On Tue, 2006-06-20 at 10:09 -0700, Keith M Wesolowski wrote: > Thanks a lot for writing this up; I think it will serve as a valuable > guide for future contributors. A few thoughts inline. > > > FWIW, I do believe that this system works great for ON and other > > in-house developed projects where developers can update the Makefiles > > and the pkgmaps when they introduce a new header or library or source > > file. However, for Open Source packages we have to do this for the > > community developers: lots of possible mistakes and in fairness, they > > know it better than us, so it's better to use their Makefiles. > > In general, that's logical, but unfortunately they also make a lot of > assumptions that aren't valid for an integrated offering like > OpenSolaris. They may use nonstandard (or differently-standardised) > directories, some of which may not be possible to change at build > time; they may deliver private headers that should not be delivered at > all;
Okay, so our approach in JDS is that these should be exceptions and probably just bugs and reporting them to the maintainers and providing patches is the best way of dealing with them. > they may overwrite existing files in the shared proto area, > making package construction difficult. Using a per-component proto > area is helpful in solving this last problem, but it has two major > drawbacks as well: it makes it effectively impossible to manage > intra-consolidation, inter-component flag days (because you can't > build dependents against their current dependencies), In JDS, we don't build, build, build, pkg, pkg, pkg, but: uninstall, uninstall, uninstall, build, pkg, install, build, pkg, install, build, pkg, install. so this is not an issue for us. > and it requires > additional work to deliver packages consisting of parts or all of > multiple components (such as SUNWsfwhea, unfortunate though that > package may be). Right. I do think that's an unfortunate pkg in the case of independent open source components. One possible way around this is to make install into a per-pkg proto area, then remove whatever you need to remove (move around, whatever), and then copy that into the shared proto area. Not very nice, but still nicer than cp'ing shared libs from .libs and things like that. > We've often suggested that make install (when available) is the better > route for the Companion, and perhaps even for SFW, but only once its > entire effects are thoroughly understood. For example, many makefiles > are broken and attempt (and succeed, if building as root) to install > files to /, ignoring DESTDIR or similar variables. Other more subtle > brokenness is common as well. Right. We never build JDS as root so any such attempt would break the build. > > maintainers intended to install are installed, and to the correct > > (relative) locations. Then we remove any files that we decided not to > > deliver, for example lib*.a, lib*.la. This is the opposite of SFW's > > philosophy, where we decide which files we wish to deliver and deal with > > them one by one. We are now ready to create packages. Pkgmaps are > > created dynamically from glob lists. > > And here's where we really come to the crux of the philosophical > differences between JDS and the rest of the Solaris organisation, > especially but not only ON. The procedure you're describing is one > which asserts that "upstream is assumed to be correct" So far I agree. When we ship external open source software, Sun has to become a member of that community and any changes that we think are necessary have to be discussed with the community maintainers and get them to accept them (except for local customizations and branding and such). If they don't accept them, there is a good chance that they are right and we need to find a better, mutually agreeable fix. Otherwise, if we make unilateral changes, we will end up with more and more local changes to maintain and more and more differences from other vendors that ship the same component. > and if in > doubt, the effects of using the autotools-based build system are > assumed to be desirable. No, I don't agree with that. Autotools have some very undesirable effects. And SFW cannot avoid them either. Otherwise samba would not have been broken by updating gnutls (can't find the bugid). > As a concrete point, why should the default be to include all new > files built by a component's makefiles? Again, if the makefiles install files that are unnecessary, then it's a bug and we need to work with the maintainers to fix the Makefiles. So you're going to ask: how do I know if unnecessary files are installed? Without reading the code, I don't. And without reading the code, the engineers working on SFW equally don't know which files need to be installed. Files known to cause problems (lib*.la) and files obviously not relevant to Solaris should be removed. For other files, understanding whether to remove them or whether to install them is the same task. > Shouldn't we at least know what we're shipping to customers? We should have a good understanding of what we're shipping to customers. > And if we know what we're shipping, > is it so much to ask that we explicitly specify it (in packaging > files, if not in install-sfw or equivalent) so that upon future > updates engineers are reminded that this is *their* product, and they > are expected to know what *they* are shipping to customers I do not believe that engineers working on SFW read the entire code and know their products. That's almost impossible for something like gcc. So no, I don't think we know any external open source component better than the community maintainers do. Beyond that, it's just a question of how you build and package the code. > It's bad enough that huge portions of > Solaris - a Sun product, with the Sun brand affixed to it - go out the > door without anyone except (possibly) the authors having read the > code, but to dispense with even the rudiments of change control in the > name of expediency is unconscionable. Exactly my point: Jim asked a couple of days ago how many lines of code each consolidation represents. I responded that JDS consist of ~19 million lines of code. There is no way we can assign engineers to reading and understanding that. We have to trust the originating community and work with them. We need to read their mailing lists, go to conferences and participate in discussions that determine the future of those projects. > A perfect example of this is 6434055 python 2.4 SSL support is > missing. Had JDS been using the SFW approach to delivering software, > the missing ssl.so would have been immediately noticed (your build > would fail), and the defective diffs would have been readily apparent. That may be right. But I do believe there are easier ways to find out if files are missing that should be there. > If the ON approach were in use, it would probably have been impossible > from the beginning, since you would have been maintaining makefiles > that you wrote yourself, forcing you to understand both the makefiles > themselves and at least the rough outline of the product you're > building. But this bug highlights an even worse problem: how do we > know there aren't other pieces of functionality missing? The simple > answer is that we don't, and can't. Even construction of a > comprehensive after-the-fact test suite is made impossible by the > assumption that the software itself is a moving target, changing in > ways we don't take the time to understand. If a test that passed with > the previous delivery fails with the new, do we write this off to > mistaken assumptions in the test, or do we assume the new code is > broken? Without understanding the intended changes to the code, it's > simply not possible to know. Right. Unfortunately Python is one of those components where we are not involved in originating the community and we should be. If we were part of the community, we would know when pieces are added or removed. > This all boils down to pride of ownership. ON is so good in large > part because the people who write it deliver it under their name and > their brand, and are held individually accountable for it by their > peers. > > That just doesn't happen with GNOME, because we're simply > accepting a block of code without considering its suitability for our > purposes, without understanding its technical characteristics, and > without taking in it any pride of ownership. Khmm... we do understand and (especially in the past when we were better resourced) contribute(ed) to large portions of the GNOME project and I have to say, my team is proud to bring a modern desktop environment to Solaris and we all love GNOME with all its problems (: > This isn't your fault; > it's the result of unfortunate business decisions, and it has little > or nothing to do with OpenSolaris (which will have the exact same > problems any time third-party code is incorporated without due > consideration for its correctness for *Open*Solaris). But it's still > true. I think if Sun needed to make the decision on whether we resource developing and maintaining our own desktop environment that will be a gem of design and interface stability but will take many engineers many years to implement during which time we're stuck with CDE then Sun made the right business decision. This is totally off topic, though. > > The above steps are basically the same for all packages and all versions, > > so updating to a newer version is often as simple as changing the > > version number in the spec file[4]. Once the packages are built, it's > > the same in both systems: we need to test the new packages. > > But how do you test them? You've already said that you just assume > whatever files the makefiles install are the right ones, less any you > already know we don't want. How can you possibly test new > functionality, since you don't know what it is? And if you do know > what it is, why is it so onerous a burden to list the files that > provide it? Again, this comes back to being part of the community, knowing what changes are happening. Also, knowing and testing new functionality is independent from listing the files one by one. > > I realise we have long-standing traditions at Sun for doing things > > a certain way and it's difficult to change. I'm just offering my > > experiences and recommend that the SFW and CCD builds be made more > > automatic and more convenient. In it's present form, SFW/CCD isn't > > encouraging or inviting contributions, because of the tedious work > > involved.[5] > > Inviting contributions is not an end of itself but a means to an end. > The goal, ultimately, is to make the software delivered as OpenSolaris > (and, for Sun employees, as Solaris) the best it can possibly be for > the people who use it. Inviting more people, with more knowledge and > skill and talent, to contribute to the software they use, and by doing > so to improve it, can be - and, I believe, is and will remain for > OpenSolaris - a huge win for quality. But it is not itself the goal; > if a tradeoff exists between making work easier (so that more people > will want to do it) and ensuring that work is of acceptable quality, > quality must always take precedence. I wasn't for a moment suggesting to give up on quality. Being able to stay on the edge and building and testing the community code 2 days after released is very important for the success of our project. It allows us to influence the product (GNOME) before the code base for the next release is frozen. > I'm in no way suggesting that > the SFW strategy is ideal, or even that it's the right one; it's, as > you've noted, time-consuming, obscure, and in some cases even > error-prone. But the JDS solution is worse - it represents a belief > that saving engineers time and effort is more important than > delivering a well-understood, high-quality product. I don't think these 2 are related. I was suggesting to automate error-prone manual tasks so that engineers can concentrate on the product and not lists of files or writing scripts that duplicate what existing Makefiles already do. > That philosophy I > can never accept, and will for as long as I can type use every means > within my power to contain and extinguish. Again, the JDS philosophy is that of being part of the originating community and trusting it. I do believe that's the best way of delivering external open source components. > Instead, we need to look for ways to invite contributions to SFW (and > the Companion) within a framework that supports the delivery of > quality, well-understood software, and those which would improve that > framework. Well said, this is my attempt of doing that. (: Laca
