On Tue, Sep 16, 2008 at 10:03 PM, Stephen Hahn <[EMAIL PROTECTED]> wrote: >> The client does not seem to be that lightweight. It has to do a >> bunch of processing checking package revisions, processing metadata >> and generating package plans which are non-trivial computations. >> I'd tend to think that the processing load is being balanced 50-50 >> between server and client. It should be possible to have the >> server-side component being done by the client as well without >> functionality loss. > > The client is not going to become lightweight; your CPU processing > estimate is not accurate (nor realistic, if one thinks about scale). > In fact, the client is doing the bulk of the work--the server is only > handling our HTTP/1.0 workaround and the search operation.
I believe that Moinak's thoughts are : if the search operation is pushed to the client (like, say, with rpm/apt), then the server would become http, file, ftp, etc without needing a custom server behind those services. >> This is fine and removes one big problem of the sparse file. However >> rsync is still not straightforward. When rsync-ing from server_a to >> server_b >> the depotd on the server_b will have to be stopped for the duration of the >> rsync. Alternatively one has to maintain a duplicate directory structure >> on server_b, rsync to that and then cpio it to the actual depot to reduce >> downtime. In any case this some amount of round-about activity and >> does not fit into the straight zero-complexity distribution of content >> used >> all over the place today. > > On a ZFS-based system, it's easier than that, but two trees and a > symbolic link is sufficient for systems without snapshots. If one is > only mirroring content, then you can simply rsync on top; since that's > the bulk of the install traffic, it's the most likely to have value > when mirroring. As many other community distributions, the Belenix team too intends to have mirrors around the world. These public mirroring sites (say, ibiblio, planetmirror, sarovar, JAIST), would not be willing to run custom servers. They'd rather have just rsync,http, ftp. Now if a depot server were to become mandatory, this would become a problem when seeking mirroring hosts Yes, one may say that convincing mirror hosts to run custom servers would be Belenix's headache, but given that we've always given to and taken from the various opensolaris technologies, we see it to everyone's advantage if we had a discussion first before going elsewhere :) >> pkg image-update today doe not give an easy way to upgrade my base OS >> without upgrading all the bundled applications. > > Yes, the inherited dependencies have problems, which we recognize. We > have a plan to constrain dependencies, but we think it's more precise > than the single interval approach. > It is good to know that there are plans in place (and I'm sure these will be acted upon). >> >> However the approach of naming files in the repo as hashes instead of the >> actual filenames is confusing. One cannot figure out what is what without >> cross-checking with the manifest. > > I suppose we'll have to disagree on the importance of this point. I am a sysadmin, and this is a vital functionality for me. If, for whatever reason, the manifests get corrupt, then I'd like to have a mechanism to repair the manifests, as well as to skip repairing the repairing and directly go pick up files that I need. Here's a scenario that happened to me recently: Due to some reasons, the server side repository manifests were incomplete. rsyncing in the correct format from my mirror servers in other countries was not possible since large sections of the ATT network were down in the US. However, I needed to have the platform specific gcc and ruby installables. All I needed to do was to scp over the rpms from my server side repository. Given the other responses on this thread which have useful information, I feel that it may be possible to repair repository side manifests. Could someone confirm this please ? > >> >> 11. IPS operations are somewhat opaque from the observability point of >> >> view. It is rather difficult for developers. >> > >> > Vague; please expand. >> >> I will point you to an example: >> http://www.thewrittenword.com/www/projects/pkgutils/pkgadd/ >> >> Excruciating verbosity yes but I will expect it if I am providing a ' -v ' >> argument. It is clear as daylight was the utility is doing underneath. In >> contrast pkg image-update -v for eg. is excruciatingly silent. What if >> fetching pkg.opensolaris.org/catalog/0 is slow due to a network problem >> ... the user won't have a clue. > > Sure, that's a bug that the team has acknowledged and is working on. Again, good to know. >> Ability too see filenames makes it clear what is there in the cache. One >> cannot predict what kind of crooked emergency situation might arise >> requiring hackery beyond one's dreams just to get a something critical >> to work. Pkg itsef screwed up, hand copying of files and so on. I cannot >> articulate examples right now but everyone including myself have faced >> situations in the past. I remember one case where I desperately needed >> to apply a patch and a bug in patchadd caused me to hand-edit pkginfo >> files of 30 packages to get the patch to install. > > I expect we would see snapshots come into play in any recovery > process. I don't agree that the cache is a general solution to such > problems--if anything, the cache is probably as untrustworthy as the > image's contents. Ok. This needs to be explained to a large section of people who're used to trusting the cache contents on other distros, though. I don't know what the case is in the US, but at least in India, we have a growing number of Linux sysadmins who're taking up an interest in OpenSolaris. > >> While in this thread, I will dare to make a few more comments which I >> was able to recollect yesterday: >> >> *) Adopting an existing FOSS packaging framework and working with that >> community would have gone a long way to boost SUN's perception among >> FOSS communities. > > And forking from one of those frameworks might have done just as much > to doom how OpenSolaris is perceived. If well discussed in an open and amicable manner, upstream contributions could actually happen in a nice manner. Sun has a good track record on several projects on this front, after all. > >> *) How does IPS compare to something like Smart (http://labix.org/smart). >> I'd guess IPS still has ways to go to match those features. By that time >> those solutions will have moved further forward. > > Perhaps, perhaps we'll catch up. > Reminder: Knowing Moinak, I can tell you that was a point about how adding Solaris awareness to an existing tool may have helped us all, and not any criticism of IPS progress. >> *) Why tie every package version into an ON build number. What sense >> it makes to refer to an ON build number for say Thunderbird ? It is >> understandable that one may require tagging as releases are synced to >> ON builds but a separate taglist property should have been more useful. >> This will also allow flexibility in tagging packages for multiple >> different >> kinds of deliverables like say a Network Appliance focussed distro. > > As others have noted, this is an artifact of how we're currently > importing the historical operating system. At some point, I would > expect each major package to have its own revision and branch history, > and those to be assembled by successors to the current "entire" > package. > Nice to know. Is this planned or something that you feel from experience would happen later ? >> Imagine a small town college student in India sitting with a 128Kbps link >> trying to install OpenOffice, SunStudio and blah, blah, blah on his >> freshly >> installed OpenSolaris 2008.xx(in his home PC) that he got from a recent >> Bangalore OpenSolaris user group meet. Unfortunately he can't even ask >> someone in Bangalore with say a 2Mbps link to download the packages >> and provide those to him on a DVD! > > And yet we've already seen published a repository on a DVD as an > experiment for C`T Magazine--thanks, Detlef--and are planning to > refine that process further. > Ok, this partly addresses some of the Belenix and various bandwidth-challenged groups' concerns. Could someone point me to documentation on how the above was achieved ? >> *) One final point from my observation, enterprises today have heterogenous >> environments having Windows, Linux, Solaris and possibly other legacy >> OS-es like say AIX. Leaving aside Windows and legacy, there are significant >> frameworks setup for controlled delivery of software to hundreds and >> thousands >> of boxes typically involving a package repository. The management of all >> these >> can become hell of a lot easier if it is possible to use a uniform >> repository >> across platforms. So the repository needs to be modular and extensible to >> different native packaging systems. Unfortunately IPS tightly couples >> packaging and network repository making this use-case impossible. If IPS >> had defined an independent stable on-disk format, had worked with an >> existing >> community repository project rather than re-doing from scratch, it would >> have >> made possible a common repository deployment for both Linux and Solaris, >> reduced administrative and maintenance cost and reduced one small barrier >> to entry for OpenSolaris. > > I'm sorry we aren't doing the project you wanted, in the order you > wanted. I hope you'll do us the courtesy of letting us do the project > we believe we need to do, in the order we believe is open to us. > Er... that is not Moinak's personal set of requirements :) A number of sysadmins in various companies have the exact same set of constraints that he's listed there. I happen to be one of them. At work, my other sysadmin colleages have stopped poking fun at Slowlaris, and are coming around to appreciate that it has features to offer which make it better than other unixes. Please also visit the Belenix discuss and BOSUG mailing lists sometime and see the kind of problems that people face. These are sometimes related to packaging, installation, and the like, and often, these people are bandwidth constrained. Some people who install Belenix/OpenSolaris (we publicize both), install the OS on company supplied computers. They go through great pains to ensure that OpenSolaris will co-exist with other operating systems, will be easy to maintain. Others share the same computer with their room mates at college hostels, and have negotiate to have opensolaris installed. A friend of mine works with a company where he tests Virtualization. He cannot afford a computer at home. When he was in this city, he used to imagine in his mind how opensolaris would work, call me and ask me questions, and come meet me on weekends just to see opensolaris. These are the kinds of people whom OpenSolaris is reaching. They are people who use borrowed equipment, have little or no bandwidth, but who are starting to see that OpenSolaris is something that can make a difference. Also, winning over a various developers from other communities is done by people like us who may not be OpenSolaris developers, but who are OpenSolaris evangelists. The Bangalore OpenSolaris User Group sometimes receives members from other communities who ask smart questions. Some of them are custom kernel developers, others are sysadmins, yet others are system integrators. We receive questions from all, receive inputs from all. If we are here to say something, it's not to take pot shots at the IPS project. The Belenix group has worked with and intends to continue to work with the various OpenSolaris projects. Like many other evangelists, we are busy spreading the good word about OpenSolaris. We talk to various CTOs, receive inputs on what works for them, what's needed, and intend to use all that information to come up with a better version of Belenix. More importantly, we talk to the end users who may lack the skills to repair their systems. We're bringing to this list our concerns of how they may be affected by IPS. As with everything else in Belenix, we're here to share these concerns and thoughts with the opensolaris community. > - Stephen > > -- > [EMAIL PROTECTED] http://blogs.sun.com/sch/ > _______________________________________________ > pkg-discuss mailing list > [email protected] > http://mail.opensolaris.org/mailman/listinfo/pkg-discuss > _______________________________________________ pkg-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/pkg-discuss
