Dan Price wrote:
>
> On Thu 24 Aug 2006 at 06:46AM, Roland Mainz wrote:
> > >
> > > I think Alan, Jim and I have clearly articulated at least two clean
> > > ways to do this with expediency and minimal suffering.
> >
> > But both proposals do not gurantee the full binary compatibility... ;-/
>
> Binary compatibilty for whom? We've already had that discussion: unless
> I am mistaken, there is no externally facing interface here, and hence
> there is nothing to be binary compatible with:
>
> # pvs /usr/lib/libcmd.so.1
> libc.so.1 (SUNW_0.7, SUNWprivate_1.1);
> libcmd.so.1;
> SUNWprivate_1.1;
The initial putback of ksh93 into OS/Net is the first step of MANY MANY
others that will follow. We're just trying to create the foundation for
all the followups which will come in the future - see examples below.
The idea is to make the initial putback "minimal invasive", which means
we want to add "ksh93" (and it's aliases "rksh93", "pfksh93" etc.), the
required libraries (libshell, libdll, libcmd, libast), some user
configuration (mainly the /etc/ksh.kshrc script to satisfy the requests
of the community for "working cursor keys" :-) ) and some other minor
stuff. That's all.
We're not intending or trying to do ANYTHING else and really like to
keep a very low profile (and we've sacrified lots of features of the
original ksh93/AST distribution (like "shcomp"&co) to avoid possible
(and even theoretical) harm to others) - and that includes that we
should not create a "pompoes" flag day just to satisfy the needs of the
buerocray and get the attention of everyone at Sun that we just finished
our putback.
[snip]
> > The last point isn't completely correct - both Solaris libcmd and
> > ksh93's libcmd target commands as comsumers and at least the native
> > Solaris command versions of the ksh93 builtin commands provided by
> > libcmd have intersections - which quicky leads to the problem that
> > future changes in this area may force us to make libcmd.so depend on
> > liboldsolariscmdwithdeffunctions.so.666 ... ;-(
>
> I had trouble working out which libcmd you meant in each part of the
> sentence, so I am confused. You are saying that some of the
> functionality x() in old-libcmd is also implemented as x() in new-libcmd
> in a compatible way. Is that right? And...? I don't follow the next
> part, or to which code "future changes" applies.
Erm... lets rework my original sentence (and simply the example): If we
have a comamnd "xyz" which wants to read/parse the commands in
/etc/default/ and we make that a builtin command in ksh93's libcmd.so
later we're ending-up with a construct which would require is to link
libcmd.so against liboldsolariscmdwithdeffunctions.so.666 - and that is
not funny.
> > > correct me I don't know of a specific policy which dictates that
> > > cross-consolidation dependencies are inherently bad.
> >
> > Umpf... I've been instructed otherwise and IMO OS/Net should remain
> > self-contained. It shouldn't really end like the DLL dependicy hell of
> > "Gnome" or "X.org after the great modularisation" ... ;-(
>
> By whom? I'd love to understand this more completely.
>
> I'd note that 'make' is not part of OS/Net. Nor are the compilers.
Yes, but I do not want to have an endless list of build dependicies - I
would strongly prefer that OS/Net remains self-contained as it is now.
Just two examples: Both X.org and Gnome are now in a hell where you have
to install zillions of other packages and applicatioons just to build
these projects, something which is IMO a pain since these dependicies
change from time to time which means you can only build it when you have
exactly version 781.871,revision_234 of build tool "foobar" installed
(that is just an example). At least for X.org a couple of contributors
have simply left the boat since the build requirements increased to an
"insane" level and usually require some surgery (including "root"
permissions) to get things setup and running.
[snip]
> As for inclusion into OS/Net, I'm only arguing that the bar be *higher*
> for technology which does not originate in OS/Net. Why? By bringing it
> into OS/Net, we force every OS/Net developer to compile it. It swells
> the size of the OS/Net consolidation, and it binds together the version
> of the component to the version of OS/Net when no such binding is
> warranted. (I have thoughts about how this might impact OpenSolaris
> distro builders, but I'm going to ponder them some more).
>
> Now perhaps in the case of ksh93 (especially if it is /usr/bin/ksh),
> this higher bar *is* reached: and that is what I want to understand. I
> think Mike's comments about reimplenting wc, printf, etc. could form the
> basis of a more firm rationale, and I'd like to hear more.
Ok, I can list some examples, with the note that these are plans for the
(near and far) future (please don't throw stones after those items -
this stuff will not be implemented for the first putback (and we're only
talking about that right now)):
- We'd like to see some of the commonly used (and smaller) commands
(that includes "wc", "sleep", "printf", "test", "cat", "paste", "join",
"grep", "egrep", "fgrep", "tee", "mkdir", "rmdir", "rm", "mv", "chown",
"chmod", "basename", "dirname", "expr" etc. etc.) to combined into one
library (that is libcmd.so) to lower the footprint and improve the
performace (this will even boost the performace (and lower the
footprint) of normal bourne shell/csh/bash/tcsh/etc. scripts (and with
minor modifications to /sbin/sh and csh we could simply avoid wastefull
|fork()|s for such commands (like ksh93 does) in some cases)).
- Performance-sensitive applications in OS/Net may want to trade "stdio"
for "sfio" (which is provided by libast) which is MUCH MUCH faster and
much more flexible. Possible examples include "perl" (the SuSE people
are currently working on that which means at some point we only have to
turn the matching build switch "on" for that... :-) ), TCL/TK, "mysql",
"apache" etc. etc.
- We'd like to a kernel module (similar to /kernel/exec/intpexec) which
can execute "compiled ksh shell scripts" and the matching compiler
("shcomp") which generates this compiled shell code (ksh93 compiles the
shell script code internally into an intermediate format and then
executes it (this works similar to JAVA's bytecode), delivering better
performance)
- ksh93 may be highly valuable within OS/Net and other consoludations -
lots of existing ksh scripts could be simplified (for example much of
the complex code in "webrev" could be reworked to be less complex (and
faster) with ksh93 constructs (for example the usage of some of the
"egrep" and "awk" constructs could be replaced)) and the performance
improved by using the new features introduced by the new version of the
korn shell
- The libc function |wordexp()| could be rewritten (already done, see
our prototype tree) in a much simpler and more secure form (and the new
version fixes bugs like that the old code isn't thread-safe and uses a
secret switch in Solaris's /usr/bin/ksh to do this work) - whcih means
the matching libc will depend on ksh93 (and later on libshell when the
2nd cleanup patch lands (which targets at the complete elimination of
the |fork()| for each |libc::wordexp()| call))
- libshell may be usefull for the various "*adm" tools - instead of
building their own syntax and command line user interface we could get
one customizeable and easy to use interface for those applications
(which would greatly reduce the engineering work at Sun's side, would
reduce the training required at the customers side and would make all
those applications fully scriptable for "free")
- Once libshell has a API which is stable enougth for Sun-internal use
we can fix "dtksh" (which is based on "ksh93d alpha" and could easily be
updated to use libshell (which would fix many many bugs for "free")) and
then develop a Gnome-equivalent (e.g. "gtksh") for it (note that this is
not another "lanugage binding" as many people think - if you take a
close look at "dtksh" you will see that it contains many features to
make the creation of GUI application via shell scripts (well, we're
using "dtksh" in an production environment (at some point I regret the
idea of teaching our staff how to write shell scripts and dtksh usage
since our lab staff bloaded the matching dtksh-based applications to a
12MB monstrosity (and still continue to work on that))) much easier).
- At some point we'd like to replace /usr/xpg4/bin/sh with ksh93 once we
killed the (minor) issues with the Sun test suites
> To ask a related informational question: will ksh93 be ARC'd as
> External, or Committed? (or something else?)
Uhm... good question... you better ask April or Mike for that - I
usually mix-up the different terminologies for that... ;-/
[snip]
> > > I am just trying to make sure that the plan here makes sense. If you
> > > convince me, I can hopefully be a helpful ally in helping you finish.
> >
> > Yes, but you're giving me a hard time... total time to answer all emails
> > in this matter today: More than six hours... ;-((
>
> I don't know what to tell you. It sounds like you are saying: this
> discussion has insufficient value to continue.
Erm... no. That wasn't my intention, however if you look at the time
when I wrote the email you may realise that it was written after a long
day (and night) which included much frustration... I didn't intend to
offend you or anyone else. Sorry... ;-(
> If you're saying that
> email is too slow, and would like me to set up a conference call, I will
> be happy to try to work out how to do that (I don't know how to do an
> international one).
My intention was to say that it took almost all my free time that day to
get your emails answered (I am a slow email writer and usually prefer
smaller emails that I can priorise the answers). My proposal would be to
do a (physical) presentation somewhere, for example at the Sun campus at
SFBay or something like that. A phone call would also be possible but
that cannot really replace a normal presentation... ;-(
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
