Re: [oi-dev] Desktop Illumos Still Matters
Jim Klimov jimkli...@cos.ru wrote: I've recently redone this on my laptop with no problems, following my own logs on wiki and bugtracker; the only substantial blocker was and is the /sbin/sh being a symlink to ../usr/bin/ksh or somesuch. System fails to boot itself when /usr is separate. Replacing this symlink with a hard copy of /usr/bin/bash (as /sbin/sh) does not break the system as much as I can see (several machines doing this for several months now) and allows to split /usr off into its own compressed dataset. I thought thet IPS is too dumb to support this. Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in SchilliX-ON since a longer time. Before I introduce my fix, I will first replace the unmaintained Bourne Shell from Sun sources by the current maintained one which you can find in: ftp://ftp.berlios.de/pub/schily/ This Bourne Shell - fixes all Bourne Shell bugs that have been documented since the late 1980 but never fixed by ATT or Sun. - Was converted from using sbrk(2) to malloc(3) to achieve portability and allows to use libc routines that depend on malloc(). - fixes some previouly unknown bugs in string handling that have been a result from removing a dependency on SIGSEV to auto-expand memory for automatic string management as implemented in the original version from 1977. - Adds the Commandline History editor I developed in 1982 and implemented in 1984 for my bsh. - Adds the advanced alias mechanism from my bsh that is better than the alias implementation from ksh. - Adds a new builtin dosh that allows to write intrinsic shell scripts via aliases. - Adds pushd/popd/dirs builtins - Adds a mapper for e.g. cursor keys that works reliably in contrast to what is known from ksh and bash that sometimes do not map strings and thus insert raw key strings. The combination of editable history, aliases and pushd/popd makes it nice for interactive work. I plan to install this shell (that is still _much_ smaller than ksh) as /sbin/sh and to fix the 6 SMF scripts that have been modified to only work with ksh. As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to no longer depending on sbrk(2)) and if you use it to interpret autoconf scripts, this is 3x faster than bash. I hope that other OpenSolaris distros will follow SchilliX-ON and SchilliX with this enhancement. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 5 September 2012 11:14, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in SchilliX-ON since a longer time. Before I introduce my fix, I will first replace the unmaintained Bourne Shell from Sun sources by the current maintained one which you can find in: ftp://ftp.berlios.de/pub/schily/ do you have a patch/diffs to source supplied elsewhere? Is this project stored in a git repository, or even in an SCCS tar ball separate to the Schillix-ON project? Are you offering these changes as an update to the Illumos bash project to be upstreamed, will you be working with these changes in the community? is it possible to exclude your added functionality that was not in the original shell so that we can stay compatible with all older POSIX compliant systems that don't have this additional functionality? Jon ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Jonathan Adams t12nsloo...@gmail.com wrote: On 5 September 2012 11:14, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Linking /sbin/sh to ksh definitely was a mistake and I plan to fix this in SchilliX-ON since a longer time. Before I introduce my fix, I will first replace the unmaintained Bourne Shell from Sun sources by the current maintained one which you can find in: ftp://ftp.berlios.de/pub/schily/ do you have a patch/diffs to source supplied elsewhere? Is this project stored in a git repository, or even in an SCCS tar ball separate to the Schillix-ON project? The Bourne Shell enhancement project started in November 2006 and the first action was to transform the unmaintained code into modern code that makes use of C prototypes to allow for better code review. These changes alone will prevent you from being able to do anything useful by comparing older versions of the source from Sun with recent ones as half of the code did change even though I did not yet re-indent the code to match cstyle. The project itself is implemented using SCCS, you may know that I am also enhancing SCCS towards a network aware distributed suystem that handles changesets. This SCCS will soon appear in SchillIX-ON as I believe that everything that was traditionally inside the UNIX sources should become part of ONNV again. Are you offering these changes as an update to the Illumos bash project to be upstreamed, will you be working with these changes in the community? What is the Illumos bsh project? I tried to get interaction with the OpenSolaris community before to no avail. Later, Illumos signalled not to be interested in collaboration with non-Nexenta people. The discussions I had, have been made with the Bourne Shell expert Sven Maschek, see http://www.in-ulm.de/~mascheck/bourne/ for a list of features and Bugs I fixed. I plan to rename usr/src/cmd/sh into usr/src/cmd/osh and to add the current state of the Bourne Shell into SchilliX-ON as usr/src/cmd/sh. is it possible to exclude your added functionality that was not in the original shell so that we can stay compatible with all older POSIX compliant systems that don't have this additional functionality? There are #ifdefs for the editor, but not for the rest of the enhancements. Please note that the Bourne Shell is not in POSIX and it is questuonable whether a reduced functionality will make sense. Switching off new builtins would be easy, but things like support for set -o is spread over the code. On the other side, there is a #ifdef ARGS_RIGHT_TO_LEFT to switch back to the old incorrect macro evaluation order that is a result from the fact that Steve Bourne may not yet have known how to effiviently add to the end of a list in 1977. There is also no #ifdef to switch on the security bugs introduced by Sun when implementing a kernel based ofexec(). Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 01:30, Alasdair Lumsden wrote: I've actually realised it would be really useful if there was a single document explaining all this stuff, a kind of In the beginning there was... style walk through of how things came to be. I'll try to write one over the next few weeks and put it on the wiki, as it would probably help new developers get their head around things. That would be very helpful. When I attended the userland hackathon I was quite confused about all of the different build systems and their history. Thanks for your patience and support there :-). I should dig up my notes and scripts for creating a build environment. Henk ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Tue, Sep 4, 2012 at 5:39 PM, Alasdair Lumsden alasdai...@gmail.com wrote: Hi All, On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, Andrew M. Hettinger ahettin...@prominic.net wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as JDS (Java Desktop System, renamed to just desktop on Solaris 11), SFW (Sun Freeware, replaced with userland in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a giant leap forward replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide lockstep upgrades between known working version sets of software. entire is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called foo. Foo depends on version 2.0 of bar. oi_148 delivers bar version 1.0. Without incorporations, if you pkg install foo it will upgrade bar no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop you installing new versions of things. The way to get around this is to have empty incorporations on your development system, that way you are free to shaft your own install if you want to. It's like taking your seatbelt off. Well, incorporations sound like a great feature, imho. I guess the only problem is when two packages have mutually exclusive dependencies (foo depends on libbar.so.1, and baz depends on libbar.so.2). But even then, one can probably avoid this by using NGZ's, if the foo package isn't updated to use libbar.so.2, or if that software is no longer maintained by the original author. Incorporations map to consolidations, so SFW, JDS, etc each have their own incorporation. This means the incorporations have to be updated when you move software from one consolidation to another, eg from JDS to oi-build. Hope this makes sense. Alasdair ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
[oi-dev] Distro Constructor in NG Zone?
Has anyone tried getting the DC to work in an NGZ? Does anyone know if this feasible? Thanks. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 4:18 AM, Jim Klimov jimkli...@cos.ru wrote: 2012-09-04 22:25, Andrew M. Hettinger пишет: was kept in /bin and /sbin that did not depend on anything. This was done to allow you to NFS mount everything else. IIRC the decision was made to go ahead and make them dynamicly linked because noone NFS mounts them anymore anyway, and it meant we had to keep both a simplified and full version of the programs. I think this will encounter many similar issues as that. If we are going back down this road, we could consider just delivering a /bin and /sbin we can use as you propose. It would have the advantage of bringing back this lost (albit rarely used) functionality. Well, as a little offtopic from desktopness, I have long ago posted an illumos RFE 829 to (again) support separatable /usr datasets, allowing one to compress much of the rootfs among other benefits of hierarchical environment (quotas, different storage and stuff). I've recently redone this on my laptop with no problems, following my own logs on wiki and bugtracker; the only substantial blocker was and is the /sbin/sh being a symlink to ../usr/bin/ksh or somesuch. System fails to boot itself when /usr is separate. Replacing this symlink with a hard copy of /usr/bin/bash (as /sbin/sh) does not break the system as much as I can see (several machines doing this for several months now) and allows to split /usr off into its own compressed dataset. There were some other nuances discussed in the RFE, like additions to the SMF service which mounts minimal root environment, and problems with beadm/zfsclone not setting compression attributes on clones, but I won't bother you here with those ;) I just wanted to stress that this is not a feature only useful for diskless NFS boots, but also on a PC or a laptop or a VM, especially with limited HDD or SSD space and/or IOPS/bandwidth (less reads = faster boots). Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms of their purpose. For example we have /usr/X11. According to `man filesystem` /opt is meant to hold add-on/third-party software. If that's the case shouldn't X11 be in /opt/X11? Or should the convention be updated, so that we store the bundles or consolidation in /usr as is already being done? If sub-directories of /usr are separate datasets (like /usr/X11 is rpool/X11), that should make migration easier. Basically, I'm asking if it is better to have one convention (everything in /usr/$consolidation) instead of two (some things in /usr/$consolidation and others in /opt/$consolidation)? Also, for the SMF nits you ran into, _I think_ we can probably update the manifests so that SMF doesn't try to start, for example, gdm/X11 until it mounts rpool/X11 onto /usr/X11. Thanks. //Jim Klimov ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Distro Constructor in NG Zone?
Last time I checked that it failed because of it's lofi usage. -- Piotr Jasiukajtis On Sep 5, 2012, at 3:58 PM, Nick Zivkovic zivkovic.n...@gmail.com wrote: Has anyone tried getting the DC to work in an NGZ? Does anyone know if this feasible? Thanks. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
[oi-dev] is there a vector for donating to OI?
I've received email from someone who wants to make a donation (of unspecified magnitude) to OpenIndiana. They contacted me as manager of the parent illumos community, but I think it right to turn the question over to the OI community. Is there any mechanism in place for taking donations? If (as I suspect) not, the best alternative I can think of is to accept the donation to the illumos Foundation (which is now a legal entity with a bank account and a board, working on by-laws), and earmark it for OI. If large enough, perhaps it could be used to fund travel for an OI representative at illumos/ZFS Days in October: http://zfsday.com/ Your thoughts? -- best regards, Deirdré Straughan Community Architect, SmartOS illumos Community Manager cell 720 371 4107 ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356(int'l) Alasdair Lumsden alasdai...@gmail.com wrote on 09/04/2012 05:39:58 PM: On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, Andrew M. Hettinger ahettin...@prominic.net wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as JDS (Java Desktop System, renamed to just desktop on Solaris 11), SFW (Sun Freeware, replaced with userland in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a giant leap forward replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide lockstep upgrades between known working version sets of software. entire is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called foo. Foo depends on version 2.0 of bar. oi_148 delivers bar version 1.0. Without incorporations, if you pkg install foo it will upgrade bar no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop you installing new versions of things. The way to get around this is to have empty incorporations on your development system, that way you are free to shaft your own install if you want to. It's like taking your seatbelt off. Incorporations map to consolidations, so SFW, JDS, etc each have their own incorporation. This means the incorporations have to be updated when you move software from one consolidation to another, eg from JDS to oi-build. Hope this makes sense. Alasdair I used terminology sloppily, thank you for clarifying for everyone. Source Juicer used the same RPM style spec file that SFE uses, with a web frontend to automatically handle submitting and building packages. What it lacked as a simple way to promote a package once it had been testing for a while. And the process for getting that done that was always a thorn in all of our sides. As I recall, for someone on the outside, it was badger the right people in sun until you where annoying enough that they'll do promotions just to get you out of their hair. Even for all it's problems, it was a really good system, which (atleast for those of us who knew about it) strongly encouraged contribution. I feel that as long as there are so many differing build systems, people will tend to confine themselves to one of them, and not be as productive as they can be.___
Re: [oi-dev] is there a vector for donating to OI?
On 5 September 2012 16:47, Deirdre Straughan deirdre.straug...@joyent.com wrote: snip I know that it's really none of my business, since I have yet to actually give anything except support/bugs to the community ... I think that the money being placed with the Illumos Foundation, with potential ring-fencing for OI is a sane idea. If large enough, perhaps it could be used to fund travel for an OI representative at illumos/ZFS Days in October: http://zfsday.com/ Your thoughts? I personally would prefer it to be set aside for OI specific bounties ... but anything that enables OI to have a more visible/vocal voice in the Illumos community could only be seen as a good thing. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] is there a vector for donating to OI?
Originally Alasdair indicated that he wanted OI to be part of the illumos foundation, so this approach of earmarking the donation makes sense. Note that in the future I hope that illumos will reserve a percentage of earmark donations for its general fund but since we have not set up the administration to deal with that yet, it seems like right now is a great time to donate and ensure that 100% of your donation goes exactly to OI. - Garrett On Sep 5, 2012, at 8:08 PM, Jonathan Adams t12nsloo...@gmail.com wrote: On 5 September 2012 16:47, Deirdre Straughan deirdre.straug...@joyent.com wrote: snip I know that it's really none of my business, since I have yet to actually give anything except support/bugs to the community ... I think that the money being placed with the Illumos Foundation, with potential ring-fencing for OI is a sane idea. If large enough, perhaps it could be used to fund travel for an OI representative at illumos/ZFS Days in October: http://zfsday.com/ Your thoughts? I personally would prefer it to be set aside for OI specific bounties ... but anything that enables OI to have a more visible/vocal voice in the Illumos community could only be seen as a good thing. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
I think that Andrew want to use a unified build system, instead of the loose confederation of radically different systems that's currently in use. I agree. A unified build system is necessary. The only question is: what should it be? Makefile-based, like ports/pkgsrc/oi-build? specfile-based? tcl-base like macports? shell-based like Gentoo's and Exherbo's? python-based like conary? I'm fine with any of the above (as well as any that I've not mentioned). As long as we end up with a standardized build system so that contributors can cross-fertilize consolidations instead of confining themselves to just one. What do existing OI-contributors think? Anyone know what the most technically-superior or technically-advanced build system is? On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger ahettin...@prominic.net wrote: Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) Alasdair Lumsden alasdai...@gmail.com wrote on 09/04/2012 05:39:58 PM: On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, Andrew M. Hettinger ahettin...@prominic.net wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as JDS (Java Desktop System, renamed to just desktop on Solaris 11), SFW (Sun Freeware, replaced with userland in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a giant leap forward replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide lockstep upgrades between known working version sets of software. entire is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called foo. Foo depends on version 2.0 of bar. oi_148 delivers bar version 1.0. Without incorporations, if you pkg install foo it will upgrade bar no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop you installing new versions of things. The way to get around this is to have empty incorporations on your development system, that way you are free to shaft your own install if you want to. It's like taking your seatbelt off. Incorporations map to consolidations, so SFW, JDS, etc each have their own incorporation. This means the incorporations have to be updated when you move software from one consolidation to another, eg from JDS to oi-build. Hope this makes sense. Alasdair I used terminology sloppily, thank you for clarifying for everyone. Source
Re: [oi-dev] Desktop Illumos Still Matters
On 5 Sep 2012, at 18:04, Nick Zivkovic zivkovic.n...@gmail.com wrote: I think that Andrew want to use a unified build system, instead of the loose confederation of radically different systems that's currently in use. I agree. A unified build system is necessary. The only question is: what should it be? Makefile-based, like ports/pkgsrc/oi-build? specfile-based? tcl-base like macports? shell-based like Gentoo's and Exherbo's? python-based like conary? Userland already has a perfectly good build system. I don't understand what you're trying to accomplish here. Andrew S. I'm fine with any of the above (as well as any that I've not mentioned). As long as we end up with a standardized build system so that contributors can cross-fertilize consolidations instead of confining themselves to just one. What do existing OI-contributors think? Anyone know what the most technically-superior or technically-advanced build system is? On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger ahettin...@prominic.net wrote: Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) Alasdair Lumsden alasdai...@gmail.com wrote on 09/04/2012 05:39:58 PM: On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, Andrew M. Hettinger ahettin...@prominic.net wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as JDS (Java Desktop System, renamed to just desktop on Solaris 11), SFW (Sun Freeware, replaced with userland in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a giant leap forward replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide lockstep upgrades between known working version sets of software. entire is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called foo. Foo depends on version 2.0 of bar. oi_148 delivers bar version 1.0. Without incorporations, if you pkg install foo it will upgrade bar no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop you installing new versions of things. The way to get around this is to have empty incorporations on your development system, that way you are free to shaft your own install if you want to. It's like taking your seatbelt off. Incorporations map to consolidations, so SFW, JDS, etc each have their own incorporation. This means the
Re: [oi-dev] Desktop Illumos Still Matters
Someone thought it would be a good idea to have a unified build system across consolidations. I think it's a pretty good idea to standardize on one build system. I'm merely asking which one would be preferred by the community (assuming the community would be willing to standardize). On Wed, Sep 5, 2012 at 12:31 PM, Andrew Stormont andyjstorm...@gmail.com wrote: On 5 Sep 2012, at 18:04, Nick Zivkovic zivkovic.n...@gmail.com wrote: I think that Andrew want to use a unified build system, instead of the loose confederation of radically different systems that's currently in use. I agree. A unified build system is necessary. The only question is: what should it be? Makefile-based, like ports/pkgsrc/oi-build? specfile-based? tcl-base like macports? shell-based like Gentoo's and Exherbo's? python-based like conary? Userland already has a perfectly good build system. I don't understand what you're trying to accomplish here. Andrew S. I'm fine with any of the above (as well as any that I've not mentioned). As long as we end up with a standardized build system so that contributors can cross-fertilize consolidations instead of confining themselves to just one. What do existing OI-contributors think? Anyone know what the most technically-superior or technically-advanced build system is? On Wed, Sep 5, 2012 at 11:05 AM, Andrew M. Hettinger ahettin...@prominic.net wrote: Andrew Hettinger http://Prominic.NET || ahettin...@prominic.net Tel: 866.339.3169 (toll free) -or- +1.217.356.2888 x.110 (int'l) Fax: 866.372.3356 (toll free) -or- +1.217.356.3356 (int'l) Alasdair Lumsden alasdai...@gmail.com wrote on 09/04/2012 05:39:58 PM: On 04/09/2012 22:42, mag...@yonderway.com wrote: On Tue, 4 Sep 2012 13:25:39 -0500, Andrew M. Hettinger ahettin...@prominic.net wrote: One of the biggest issues here isn't that packages are particularly HARD to make with IPS (they aren't). It's that there are about three different approaches to it, and we need to get that standardized. Many of the packages are tied up in the consolidations, which DO seem to have a high barrier to entry. So what are the cliff's notes to the three different approaches, and is one of those approaches going to have a lower barrier to entry with still high-quality results? I'm thinking along the lines of a third party repo. I think there's a bit of confusion surrounding the terms involved. A consolidation is just a logical grouping of packages, such as JDS (Java Desktop System, renamed to just desktop on Solaris 11), SFW (Sun Freeware, replaced with userland in Solaris 11), etc. The big issue is that all the consolidations use different build systems. JDS uses RPM style spec-files similar to SFE (Spec files extra, a collection of 3rd party package recipes). SFW used a horrible Makefile based system. Userland is an excellent replacement for SFW, and uses Makefiles but in a way very similar to the FreeBSD ports tree or pkgsrc. I was attempting (with some others) to get OI updated in a giant leap forward replacing SFW with userland-gate (renamed to oi-build, and later illumos-userland after Nexenta started meddling). The idea was that we would try to focus on one single build system, the userland-gate style, which is the best of the lot. New software would go in there, and if we needed to update something in another consolidation, we would instead re-implement the recipe for it in userland-gate format. Unfortunately my approach with /experimental was quite ambitious and didn't quite work how we wanted. Jon Tibble is continuing to release new OpenIndiana builds (such as prestable 6, released *today* - http://wiki.openindiana.org/oi/oi_151a_prestable6+Release+Notes) and is advocating we do move to a single build system based on userland-gate, but more slowly and in a more controlled fashion. oi_151a actually already has a mini userland-gate/oi-build, which you can see here: https://hg.openindiana.org/sustaining/oi_151a/oi-build/ It's used to deliver some additional software and some bits from other consolidations have been moved there. It is probably where people should focus their efforts moving forward. Incorporations are probably what people are bitching about, which are there to provide lockstep upgrades between known working version sets of software. entire is the best known incorporation, which with each release locks all system software at a particular version. Incorporations are needed to prevent your system getting shafted. Lets say you are on oi_148, and in oi_151a we introduced some new software called foo. Foo depends on version 2.0 of bar. oi_148 delivers bar version 1.0. Without incorporations, if you pkg install foo it will upgrade bar no questions asked. Any software linked against bar probably just stopped working with libbar.so.1 changed to libbar.so.2. Incorporations present a challenge when you're developing software, because they stop
Re: [oi-dev] Desktop Illumos Still Matters
Hi Nick, On 05/09/2012 18:49, Nick Zivkovic wrote: Someone thought it would be a good idea to have a unified build system across consolidations. I think it's a pretty good idea to standardize on one build system. I'm merely asking which one would be preferred by the community (assuming the community would be willing to standardize). The decision was already made by the core OI devs to use the userland-gate format. That's the future unified build system. So the choice doesn't really have to be made again - it's why oi-build is in userland-gate format. Cheers, Alasdair ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 12:56 PM, Alasdair Lumsden alasdai...@gmail.com wrote: Hi Nick, On 05/09/2012 18:49, Nick Zivkovic wrote: Someone thought it would be a good idea to have a unified build system across consolidations. I think it's a pretty good idea to standardize on one build system. I'm merely asking which one would be preferred by the community (assuming the community would be willing to standardize). The decision was already made by the core OI devs to use the userland-gate format. That's the future unified build system. So the choice doesn't really have to be made again - it's why oi-build is in userland-gate format. Cheers, Alasdair Oh, ok. I'm still catching up on what's been going on here, while I was away in my cave. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, 5 Sep 2012, Joerg Schilling wrote: As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to no longer depending on sbrk(2)) and if you use it to interpret autoconf scripts, this is 3x faster than bash. This sounds great. How does its performance compare with 'dash'? Are the various issues described in the GNU Autoconf portability notes (http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Portable-Shell.html#Portable-Shell) avoided/fixed by this shell? SunOS's stagnant /bin/sh has been a continuing issue with POSIX shell script portability. As a result, Autoconf configure scripts typically elect to re-run themselves with bash (the World's Slowest Shell) on Solaris sytems. If this shell is indeed good enough for Autoconf configure scripts, then it would be good to submit an Autoconf patch so that future configure scripts know how to detect and use it. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Stormont andyjstorm...@gmail.com wrote: On 5 Sep 2012, at 18:04, Nick Zivkovic zivkovic.n...@gmail.com wrote: I think that Andrew want to use a unified build system, instead of the loose confederation of radically different systems that's currently in use. I agree. A unified build system is necessary. The only question is: what should it be? Makefile-based, like ports/pkgsrc/oi-build? specfile-based? tcl-base like macports? shell-based like Gentoo's and Exherbo's? python-based like conary? Userland already has a perfectly good build system. I don't understand what you're trying to accomplish here. I asume that what you call userland is the successor for sfw. The buildsystem for sfw is a nightmare: - It only works if the whole set of tools has already been installed in /usr on the compiling system before with exactly the same version as the one that is going to be compiled. This causes that you need an unknown number of iteration to compile and install on the build machine before you are able to compile everything at all. You need at least one additional install/compile cycle before you can be sure that the compile/link results will no longer change. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Are these problems still in effect? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Andrew Gabriel illu...@cucumber.demon.co.uk wrote: Nick Zivkovic wrote: Agreed. Also, I see that /opt and /usr/$consolidation overlap in terms of their purpose. For example we have /usr/X11. According to `man filesystem` /opt is meant to hold add-on/third-party software. /opt was meant for unbundled software. Ideally, it should be empty immediately following a full install of a distro, as everything is by definition bundled. I don't think Solaris ever quite got that right, but it was almost there. Everything you install after that (which isn't part of the distro) should be in /opt (and /etc/opt and /var/opt), but a lot of 3rd-party software developers got that wrong too. There was a nice talk from Steve Bourne at the Sun User Group meeting in December 1990. He explained that first /usr/bin was hijacked by the system and people started with /usr/local. Then external sources hijacked /usr/local and as a result a FHS summit with most UNIX vendors decided to usr /opt. We are now nearly 25 years after that /opt decision and still not everybody got it. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Wed, 5 Sep 2012, Joerg Schilling wrote: As a nice hint: The new Bourne Shell compiles and runs on Cygwin (thanks to no longer depending on sbrk(2)) and if you use it to interpret autoconf scripts, this is 3x faster than bash. This sounds great. How does its performance compare with 'dash'? Are the various issues described in the GNU Autoconf portability notes (http://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/html_node/Portable-Shell.html#Portable-Shell) avoided/fixed by this shell? SunOS's stagnant /bin/sh has been a continuing issue with POSIX shell script portability. As a result, Autoconf configure scripts typically elect to re-run themselves with bash (the World's Slowest Shell) on Solaris sytems. If this shell is indeed good enough for Autoconf configure scripts, then it would be good to submit an Autoconf patch so that future configure scripts know how to detect and use it. Given the fact that GNU autoconf has been more or less destroyed after release 2.13, so I personally base my work on an extremely enhanced gnu autoconf 2.13. The timing tests I did run, have been run with my enhanced autoconf-2.13 Autoconf 2.13 works with all known shells - I have no idea why the FSF stopped to support this. I suspect that this is just a bash marketing action. The problem why newer GNU autoconf versions are so slow may be that they call a bew bash for each single test unless /bin/sh is bash - what you don't like to have on a POSIX compliant system. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 5/12 11:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: I asume that what you call userland is the successor for sfw. Yes. The buildsystem for sfw is a nightmare Which is why it was completely thrown out and Userland started with a new design from scratch. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
[oi-dev] Schily shell as /bin/sh
On Wed, 5 Sep 2012, Joerg Schilling wrote: Given the fact that GNU autoconf has been more or less destroyed after release 2.13, so I personally base my work on an extremely enhanced gnu autoconf 2.13. The timing tests I did run, have been run with my enhanced autoconf-2.13 It was temporarily destroyed but then it was undestroyed so things are ok again (while you were sleeping). However, work is required in order to catch up. Autoconf 2.13 works with all known shells - I have no idea why the FSF stopped to support this. I suspect that this is just a bash marketing action. The problem why newer GNU autoconf versions are so slow may be that they call a bew bash for each single test unless /bin/sh is bash - what you don't like to have on a POSIX compliant system. Try invoking like it like CONFIG_SHELL=/usr/bin/dash /usr/bin/dash ./configure If the starting shell and the official configure shell are the same, then it should avoid the paranoid behavior that you describe. This is totally off topic for this discussion thread. A new discussion thread should be started for the possibility of replacing /bin/sh with your Schily shell. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 09/ 5/12 10:55 AM, Andrew Gabriel wrote: Nick Zivkovic wrote: Basically, I'm asking if it is better to have one convention (everything in /usr/$consolidation) instead of two (some things in /usr/$consolidation and others in /opt/$consolidation)? There's never been any rule about consolidations being funneled into specific directories. It may be that it makes sense in a few specific cases because of functional groupings, but not universally. In fact, we've been going the other way for years, moving away from /usr/$subsystem directories that impose meaningless boundaries in the way of users. In the last OpenSolaris builds released (snv_130 later), OpenIndiana, and Solaris 11 you should find that /usr/X11 is simply a bunch of backwards compatibility symlinks. Most of the X11 libraries were simply reached via /usr/lib since Solaris 2.6, and the rest of the X11 files moved to /usr/bin, /usr/share, /usr/lib, etc. in those Nevada builds. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, Sep 5, 2012 at 2:43 PM, Alan Coopersmith alan.coopersm...@oracle.com wrote: On 09/ 5/12 10:55 AM, Andrew Gabriel wrote: Nick Zivkovic wrote: Basically, I'm asking if it is better to have one convention (everything in /usr/$consolidation) instead of two (some things in /usr/$consolidation and others in /opt/$consolidation)? There's never been any rule about consolidations being funneled into specific directories. It may be that it makes sense in a few specific cases because of functional groupings, but not universally. In fact, we've been going the other way for years, moving away from /usr/$subsystem directories that impose meaningless boundaries in the way of users. In the last OpenSolaris builds released (snv_130 later), OpenIndiana, and Solaris 11 you should find that /usr/X11 is simply a bunch of backwards compatibility symlinks. Most of the X11 libraries were simply reached via /usr/lib since Solaris 2.6, and the rest of the X11 files moved to /usr/bin, /usr/share, /usr/lib, etc. in those Nevada builds. Thanks for clearing this up, Alan. Besides, these boundaries are better enforced through NG zones than through the filesystem heirarchy. -- -Alan Coopersmith- alan.coopersm...@oracle.com Oracle Solaris Engineering - http://blogs.oracle.com/alanc ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 19:49, Joerg Schilling wrote: The buildsystem for sfw is a nightmare: - It only works if the whole set of tools has already been installed in /usr on the compiling system before with exactly the same version as the one that is going to be compiled. This causes that you need an unknown number of iteration to compile and install on the build machine before you are able to compile everything at all. You need at least one additional install/compile cycle before you can be sure that the compile/link results will no longer change. It sounds like what you want is a completely self contained build system, like pkgsrc, which bootstraps itself, requires only a compiler, knows how to build things in the correct order, and installs everything to a custom destination prefix. That approach is fine for building 3rd party software, but not for maintaining system software which ships to /usr. Why? Because even in a minimal zone, bootstrapping involves overwriting things in /usr that are already maintained by the packaging system. You could build software and upgrade to those packages as you go, but that's very hard to do especially when refactoring packages. If you instead tried to install everything to a custom destination prefix, you couldn't then just package it up and install it to /usr, because lots of software embeds their build prefix in the binary. If you built stuff with /foo as your prefix, then packaged it and delivered it to /usr, /usr/bin/someprogram would be looking for /foo/etc/someconfigfile.conf It's far easier just to use a build zone and install the dependencies you need, and ensure your build zone is running the latest bits from the package repository. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... That is not an issue with userland-gate or oi-build. You do have to install gmake, bash and a bunch of dependencies, but they're all available in the package repo. - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Can you supply an example? I'm not sure I understand this complaint. I do find it highly annoying that a lot of software jots down the compiler flags it was built with and stores them in a [softwarename]_config file, such as mysql_config, which is used by extensions to get the CFLAGS/LDFLAGS/etc. On OSOL/OI these are often Sun Studio flags that aren't compatible with gcc. You then end up in a situation where if you're doing gem install somelibrary or pecl install somelibrary with gcc as your compiler, it chokes when linking against a system program that supplies Sun Studio CFLAGS/LDFLAGS. SFW was a nightmare for many reasons, but not the reasons you listed. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alan Coopersmith alan.coopersm...@oracle.com wrote: On 09/ 5/12 11:49 AM, joerg.schill...@fokus.fraunhofer.de wrote: I asume that what you call userland is the successor for sfw. Yes. The buildsystem for sfw is a nightmare Which is why it was completely thrown out and Userland started with a new design from scratch. But as this did not exist before Spring 2010, I asume that the new system is not able to create native Svr4 packages. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:12, Alan Coopersmith wrote: Correct. Userland was designed from the ground up for IPS, since that was the only packaging system in use when it was created. Nexenta enhanced their fork of userland to support generating .deb packages, so adding SVR4 probably wouldn't be too hard. Why you'd want to is another matter. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:22, Bob Friesenhahn wrote: What do you suggest as a better replacement for this? Oh it's easy - you strip most of them out after the file is generated. Very easy to do with a post-install sed rule in the build recipe. The bulk of them are pointless optimisations that aren't really relevant when compiling an extension. The main LDFLAGS to preserve are -L/-R and -l, and for CFLAGS its -D and -I. As an example, mysql_config spits out this for CFLAGS: -I/usr/mysql/5.1/include/mysql -xprefetch=auto -xprefetch_level=3 -mt -fns=no -fsimple=1 -xbuiltin=%all -xlibmil -xlibmopt -xnorunpath -DHAVE_RWLOCK_T -DUNIV_SOLARIS The only thing you really need for extensions to build is the -I bit. The rest is Sun Studio pedantry. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Schily shell as /bin/sh
Bob Friesenhahn bfrie...@simple.dallas.tx.us wrote: On Wed, 5 Sep 2012, Joerg Schilling wrote: Given the fact that GNU autoconf has been more or less destroyed after release 2.13, so I personally base my work on an extremely enhanced gnu autoconf 2.13. The timing tests I did run, have been run with my enhanced autoconf-2.13 It was temporarily destroyed but then it was undestroyed so things are ok again (while you were sleeping). However, work is required in order to catch up. Maybe, but this does not help me. My software compiles on more platforms than the ones that are supported by GNU autoconf. In order to achieve this, I was forced to add massive enhancements into config.guess and config.sub. Unfortunately GNU autoconf did enhance their versions of these files in a way that does not allow a code review, so I am not able to verify whether the new GNU versions support every platform I support. My conclusion is that a recent GNU autoconf is worth less than my version and the FSF slept while I was working. Try invoking like it like CONFIG_SHELL=/usr/bin/dash /usr/bin/dash ./configure If the starting shell and the official configure shell are the same, then it should avoid the paranoid behavior that you describe. I know that this works as expected for my autoconf. I did never check with recent GNU autoconf and to me it does not make sense to switch to a less usable system from the FSF - I stay with my enhanced version. This is totally off topic for this discussion thread. A new discussion thread should be started for the possibility of replacing /bin/sh with your Schily shell. This is a new proposal from you. I will definitely replace /sbin/sh back to the Bourne Shell in order to permit again to split root and /usr. I am not shure whether I will do more later. Given the fact that POSIX intentionally does not deal with path names like /bin/sh and the fact that even ksh93 is not fully POSIX compliant, see e.g. the output from the times command, there is no need to have /bin/sh == ksh93. We had a really POSIX compliant shell on Solaris, which was the enhanced ksh88 but that was closed source and cannot be used in the future. Note e.g. that we (the POSIX commitee) recently changed the POSIX standard to match users expectations for an orthogonal behavior and now require that for i; do works. David Korn changed this for ksh93 and I changed this for the Bourne Shell, but for the closed source ksh88, we have no chance to get it to the current standard requirements. POSIX requires that you get a POSIX compliant shell after you call: PATH=`getconf PATH` sh This PATH is /usr/xpg4/bin:/usr/ccs/bin:/usr/bin:/opt/SUNWspro/bin on Solaris, so we just _need_ to have a POSIX shell in /usr/xpg4/bin/sh. As I mentioned before, there are 6 scripts in ONNV that have become non-portable after Sun switched /bin/sh to ksh93. Therse scripts are easy to fix. I however cannot speak for other scripts that are part of additional software. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alasdair Lumsden alasdai...@gmail.com wrote: On 05/09/2012 19:49, Joerg Schilling wrote: The buildsystem for sfw is a nightmare: - It only works if the whole set of tools has already been installed in /usr on the compiling system before with exactly the same version as the one that is going to be compiled. This causes that you need an unknown number of iteration to compile and install on the build machine before you are able to compile everything at all. You need at least one additional install/compile cycle before you can be sure that the compile/link results will no longer change. It sounds like what you want is a completely self contained build system, like pkgsrc, which bootstraps itself, requires only a compiler, knows how to build things in the correct order, and installs everything to a custom destination prefix. That approach is fine for building 3rd party software, but not for maintaining system software which ships to /usr. It seems that you missunderstand the problem. The main issue is that the build system linked against /usr instead of linking against something like: /home/user/proto/usr - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... That is not an issue with userland-gate or oi-build. You do have to install gmake, bash and a bunch of dependencies, but they're all available in the package repo. It is not a real issue with SchilliX either but for a looking at a self-contained system it would. - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Can you supply an example? I'm not sure I understand this complaint. Check e.g. the notes about sfw in: ftp://ftp.berlios.de/pub/schillix/AN-0.8 Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On Wed, 5 Sep 2012, Andrew Stormont wrote: The only thing you really need for extensions to build is the -I bit. The rest is Sun Studio pedantry. These sorts of scripts are just broken. What it really should do is check the value of CC before adding any compiler specific flags. Patching it to do that would be my preferred way of solving the problem. Things get complicated because it might not even be possible to combine code compiled by two different C compilers. Some compiler-specific options might be needed in order to obtain special compilation modes and secret library dependencies (especially for threads and OpenMP). There are also explicit library dependencies to get right. Bob -- Bob Friesenhahn bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer,http://www.GraphicsMagick.org/ ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:36, Andrew Stormont wrote: These sorts of scripts are just broken. What it really should do is check the value of CC before adding any compiler specific flags. Patching it to do that would be my preferred way of solving the problem. That works too. The thing is they're pretty dumb in operation - they pick up the compiler environment, which in the case of mysql was optimised for the database server. Client libraries really won't benefit from the optimisations. We could store the Sun Studio optimisations, and expose them with CC is detected as Studio, but gcc users miss out on them. So for equality it seems easier just to strip them. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
Alan Coopersmith alan.coopersm...@oracle.com wrote: Which is why it was completely thrown out and Userland started with a new design from scratch. But as this did not exist before Spring 2010, I asume that the new system is not able to create native Svr4 packages. Correct. Userland was designed from the ground up for IPS, since that was the only packaging system in use when it was created. So there would be a need to add the missing code. The important question here would be whether this is an open project that accepts enhancements to support the native packaging system and that would support to then change the related meta data the same way the IPS meta data is changed. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
Re: [oi-dev] Desktop Illumos Still Matters
On 05/09/2012 21:58, Joerg Schilling wrote: Alasdair Lumsden alasdai...@gmail.com wrote: It seems that you missunderstand the problem. The main issue is that the build system linked against /usr instead of linking against something like: /home/user/proto/usr userland-gate still links against /usr - it has a per-package proto area rather than a build-system wide proto area. - It may be that you would need to manually install at least older versions of strategic tools before you may start to compile at all. The programs in question are gmake, bash, gm4, ... That is not an issue with userland-gate or oi-build. You do have to install gmake, bash and a bunch of dependencies, but they're all available in the package repo. It is not a real issue with SchilliX either but for a looking at a self-contained system it would. - It installs unmodified autoconf results in /usr/include with the effect that you cannot compile depending other software using an older version of the compilers than the one you used to compile sfw. Can you supply an example? I'm not sure I understand this complaint. Check e.g. the notes about sfw in: ftp://ftp.berlios.de/pub/schillix/AN-0.8 You need to comment out line 71 of the file /usr/include/net-snmp/net-snmp-config.h do that it then looks this way: /*#define HAVE_CPP_UNDERBAR_FUNCTION_DEFINED 1*/ This is needed as the sunstudio-12 compiler and gcc-3.4.3 do not support __FUNCTION__ That's an autoconf problem, not a problem with the build system. If you build software with a new compiler, autoconf will detect its new features. To work around it, the net-snmp build recipe can modify the generated header prior to packaging it. However, typically a distribution will ship with a particular supported compiler version, and the headers of the software on the system will match that version. It is unreasonable to expect an older compiler to link against all software delivered by the system when the system uses a newer compiler. ___ oi-dev mailing list oi-dev@openindiana.org http://openindiana.org/mailman/listinfo/oi-dev
[oi-dev] VNIC Creation failure
Nick,
Sorry to break the message threading, I was not joined to the list
when you asked the question.
Yes it's possible to use an etherstub with your per-zone vnics hanging
off of it. I have the details of it documented at work, but basically
you
1. Create the etherstub
2. Create the VNICs, possibly with one for the host if it needs access
to the subnet the zones are on
3. Configure each zone with a static exclusive interface assigned to
the given vnic.
4. Set up NAT by configuring /etc/ipf/ipnat.conf with something like:
map bge0 192.168.190.0/24 - 0/32 portmap tcp/udp auto
map bge0 192.168.190.0/24 - 0/32
where the network specified is your zone subnet.
5. Enable IP forwarding and update the config
routeadm -e ipv4-forwarding
routeadm -u
6. Enable the IP filter service:
svcadm -e svc:/network/ipfilter:default
Well that's off the top of my head, if you need more details I can
give them when I get to work in ~9 hours. Otherwise I just did a quick
google and it looks like this is documented much better than when I
pieced it together for my self. You may want to check
http://wiki.openindiana.org/oi/Using+host-only+networking+to+get+from+build+zones+and+test+VMs+to+the+Internet.
--
Ron Parker
Don't type things you find on the Internet into your computer!
:(){ :|:};:
___
oi-dev mailing list
oi-dev@openindiana.org
http://openindiana.org/mailman/listinfo/oi-dev
