Who am I? I think it is very important to introduce myself and I wish everybody else will follow my example. For some reason there are many experts in install around. They sounds authoritative but do not have experience with technology - less they know more authoritative they sounds. So I like to show my "credentials", I do not like to be seen as someone who do not know what is he talking about.
In general I worked as emergency developer on different critical Core Install Projects and was Operation System Instances I-Team leader (this team does not exist any more). In many cases I worked alone, sometimes with Gary as a lead: 1. Live Upgrade 2. Flash 3. Zone Install 4. Diskless (s8) 5. Zone Patching 6. New Solaris Upgrade 7. Packaging/Patching Area which I covered is Solaris Core Installation technology - backbone of Solaris software delivery machine which is far far away from GUI Installer and things of this nature. Many people do not even know that Solaris Installation is not only installer but much much more... Actually big customers do not really use Installer but instead Flash, Live Upgrade, Jumpstart and of course most what IT doing in big companies - continuously patching their computer base... Live Upgrade Me and Gary were hired into Install Group in 2000 to fix Live Upgrade and we fixed it and improve it significantly in four months or so. I was proud to introduce new filesystem handling which allows complete Boot Environment cloning (it was only Solaris filesystems not user areas before) and as a result it turn LU from simple upgrade utility to system copies (BE) managing software which one of our customer was very excited to use in production as fast backup solution. They have model where three copy of the system were in use on each production machine. Two was used for frequent software upgrades. When new software came they install it on currently inactive copy and reboot to it - normal Live Upgrade usecase. But third copy was used as a "hot-swap", nightly they repopulates it and if some failure happen they just switched to it. This is different animal then backup-restore - it allows fast recovery, you just reboot from "hot-swap" system copy. And this is different then disk solution like RAID because it works even if software failure happend not hardware, including data messed up etc. And I make Live Upgrade work with flash and with 86 boot partitions. I also introduced methodological improvement path to step by step reimplement LU from kshell script to C-program by using simple utility ldo where I reimplement parts of the shell-script in c-code and then call this utility from shell script removing old inefficient script algorithm. This '?'do.c methodic was then used widely all around install technology on which I and Gary works later. Flash Then I was assigned to Flash (I worked on it alone, nobody lead me or helped me) which was in terrible shape. I make it useful by introducing file selection mechanism which make flash useful - now customers able to select flash archive content safely - I checked that no any files which belongs to any package will be excluded. It was major improvement in Flash making it product useful for customers. For you to understand how important was that I should say that previous "solutions" offered to customers were - unarchive flash in separate directory and then manually remove files which not needed and then archive flash back and the other one was to list all system files in file (find / -print) and then manually remove filenames from this list and then set this list to flarcreate as parameter.. It is ridiculous and very inconvenient for customers as well as very damaging for Install reputation. I introduced fast selection mechanism based on exclusion/inclusion marks for file-tree knots (-x/-y parameters). In general it is fast search for filename prefixes - not quite classical hashtable but similar (fast search here is based on string length), flarcreate deals with all files on the system and here performance of this algorithm is extremely important. Next what I did for Flash was differential Flash. It was full case (previous my projects were fasttracks) which I went through alone and in result learned process very well - from one-pager to inception (and yes design review of course). Differential flash was pronounced very important part of IChange. In Flash I introduced fdo, in general I copy that utility schema and put there new code which allows to call C from flarcreate script to implement selection and differential flash handling. Initial it was flu - Flash Utility, which is fine by me, but my English speaking coworkers insists to change the name and at this point line of '*'do.c started. Zone Install Then I was thrown into Zone Installation support. I work on major design for this support and push for idea to have separate software installations on each zones - Initially was some plans for local zones to be all similar exact copy of Global zone - which make it kind of useless. Other my contribution is idea to have locally repository instead of came up with some global contents file for all zones located on global zone. And in terms of installing zones - I suggested to do entire copy of global zone to local using cpio with filtering - similar what I done for flash and then just reconfigure packages using pkgadd but without copy. At this point I introduced hashtable for package names to be used for package files filtering. It was of course zdo.c and this small tings was the only part I implemented for zone installation - everything else was done by Gary, he was leading developer and designer in this project. My biggest contribution was design, which I am really proud of and this hashtable based filtering engine. Diskless S8 Then I was removed from Zone Installation and thrown to support diskless for Solaris 8 by special request from big customer. I fix dcpatch and turn overnight patching for diskless into 3 minutes operation. Dcpatch has gorgeous relational database SQL engine implemented in kshell script - all operations and dependency checks were done using this SQL engine. It was very quite, but on my opinion this is slowest possible implementation, I think to make something slower then this I have to insert delays. And of course reimplementing this in C make it faster 100 times. So at this point I first face patches and patch dependencies and the fact that it is all implemented using Kshell scripts. At this point prehistoric pdo.c emerges on the face of Earth. It was different then one which now is in place, but it was good experience good introduction into patch world. Zone Patching Next what happen - I was pulled back into Zone Installation support to make zone patching. Zone Installation support as we planned from the beginning has three stages: Zone Installation - first and most important, Zone Packaging, Zone Patching and Zone Upgrade. I was asked to do Zone Patching - this happened August and in December it mast be shipped. Gary was covering Zone Packaging. So I had 2-3 months for development alone. However I was already familiar with patching and applied my diskless patching experience to the problem. With great effort, working overnight and during Weekends we both me and Gary delivered in time me - Zone Patching and Gary - Zone Packaging. And as a side effect I greatly improve dependency check performance for patchadd as well as convenience for multipatching. Gary as well greatly improved pkgadd - improve performance more then twice on real packaging and get rid of useless and heavy database solution. After this I introduce partially installed patches support for pdo.c and finally resolved dcpatch problem for diskless clients, based on new pdo.c dependency check - SQL engine from dcpatch I had to ignore. Until I was told to stop doing anything I was doing including patching support for diskless in Solaris_8 (which anyway was done at that point I just give away my workspace to produce IDR etc) and focus on Zone Upgrade. New Solaris Upgrade Engine I focused on Zone Upgrade and again as it was for Zone Install work together with Gary. At that point it came clear to me that to do zone upgrade we need to have new Solaris Upgrade algorithm this is most critical part and I focused on new Solaris Upgrade. It was on schedule and I worked on new Solaris Upgrade for last half of 2005. It was done as part of pdo. I introduced new datastructures for Software Clusters, HistoryFile, Locales etc. and implement similar dependency check algorithm to analyze current system and how to evolve it to new Release or Update package set. New Solaris Upgrade analyzer was successfully implemented it works faster than current, also it works with locales right. While testing this code I found out that between upgrades about 60% of packages that are reinstalled in fact do not have any changes! So by ignoring this packages we can easy and right away save 60% of upgrade time... Patching - Packaging I start working with Bart last febrary on packagin-patching improvements and right now I am working on this. As part of this we are delivering small improvement like "One Step Circular dependency for Patches" delivered first (which was intended to eliminate need of merging patchas but instead makes them mutually dependent - to fight monsters like KU patch etc) and now it is "Recursive Patching" in two week readiness (until we have too much fun with design review), then it will be Recursive Packaging and many more like Contents File Bottleneck, Package by Patch, Live Patching, Distributed Install and many, many more... vassun This message posted from opensolaris.org
