Dan Price wrote:
> On Thu 24 Aug 2006 at 12:35AM, Roland Mainz wrote:
> > > I have a couple of questions regarding the shape of the
> > > project:
> > >
> > > - What is the nature of the replacing of libcmd.so? If I'm
> > > not mistaken, I noticed that the existing libcmd.so is
> > > being overwritten by the tarball which was recently
> > > published. I wondered why that would be.
> >
> > Both Solaris and ksh93/AST have a library called libcmd - for Solaris it
> > is an old "relict" from ancient times which currently only carries the
> > |def*()|-functions to read+parse files from /etc/default/. ksh93/AST has
> > a libcmd which contains some ksh93 builtin commands (those commands
> > which are not mandatory to get libshell up and running - those commands
> > live in libshell (David/Glenn may be able to explain that better)).
> >
> > We've looked at the problem and realised that renaming libcmd.so.1 will
> > be a HORRIBLE pain since many other consolidations depend on it, making
>
> ``HORRIBLE'' appears to be a rhetorical overstatement
It was not thought as "rhetorical overstatement" - it is what I think
how it will end. I've logged a few hundred hours getting various
scenarious tested and the two prototypes set up ("prototype001" was a
complete failure, partially because it tried to bypass the libcmd
problem) and I think I can at least outline the scenario of such a
implementation.
But my first question would be: Who actually benefits from moving the
functions in the current Solaris libcmd.so to another library ? AFAIK
noone will benefit - but many [at Sun] will suffer. And the result will
not even be "more clean" than the current design.
> there is a
> problem here, and I feel that we should solve in an architecturally
> clean way-- I don't see why merging two completely unrelated libraries
> makes any architectural sense
I'd suggest to dismantle libraries like "libnspr" or "libc", too - they
are merging various functionality and fusing them together in one
library. Or think about "librt" which was consumed by "libc" - that's
another example of such fusions.
I make it clear: I consider the CURRENT design of a merged libcmd as a
clean way to deal with ALL issues we're facing, it solves all issues and
even avoids (as side-effect) the MASSIVE overhead of a "flag day".
> Note also that libcmd can be renamed
> prior to the ksh93 putback as a separable entity-- this could be done by
> you, or by any other community member, or by an engineer who works for
> Sun. While it will require some cross-consolidation coordination, many
> projects do. While your project may require some cleanup to the source
> base-- guess what, that's what happens...
Why should this be neccesary ? The current libcmd addresses all problems
in a clean way and we do not need a cross-consolidation coordination.
Doing such stuff is just extra work without getting anything usefull
back.
> > even the process of one or multiple flag days a serious pain. The pain
> > will even be MUCH harder because I am external contriutor which means
> > getting one or more flag days syncronised is almost impossible (for me).
>
> I agree that you'd need some assistance. C'est la vie.
Who should do that work ? And why should we actually add this extra
burden ? We do not need to rename libcmd or move the functionality to
another library.
> > Another option would be to rename ksh93's libcmd.so - but that would
>
> It's clearly not an option, and I have not (and won't) suggest it.
Ok...
> > "closed") and /sbin/sh now lives. libshell (which is ksh93 made
> > available as shared library (which means that /usr/bin/ksh93 is just a
> > 5k-8k wrapper which directly calls into libshell code)) may be usefull
> > for other applications like zone/ufs/SMF (adminstration) tools
> > (supplementing or replacing libtecla) which make it mandatory that ksh93
> > lives in OS/net directly.
>
> Your statement is inaccurate. Things in OS/Net can depend upon things
> not in OS/Net. For example, Zones depends upon libxml2, which is in
> SFW.
Erm... the last thing I heared is that libxml2 is moved into the OS/Net
codebase... or do I mix things up ? AFAIK the general policy is that
such extra dependicies should be avoided, right ?
Please take a look at the old "ksh" - it was integrated into OS/Net many
moons ago. ksh93 is it's successor and at some poing it should be
possible to completely replease the old ksh with the new one. Note we
have two themes in the project description: "Integration" and
"Migration". The integration into OS/Net is a requirement for the
"migration".
> > > understand that ksh93 has its own build system, and
> > > I was wondering which consolidation would best accomodate
> > > that uniqueness.
> >
> > We do not use the original AST build system for ksh93 in OS/Net - I
>
> Why not? Doesn't that just increase the maintenance burden when
> ksh93 revs to a new version? With the SFW build system, you just
> drop in the new .tar.gz, update some build metadata, and be done.
You've totally wrong at this point.
The current ksh93 integration into OS/Net is done in a way that you can
simply make an "unfied diff" (e.g. /usr/bin/diff -u ...) between the old
and the new source version of ksh93, adjust the source paths in the
patch file, apply that patch and regenerate the include files. That's
all. I've done that for each single alpha release of ksh93r+_alpha and
this update scheme works fine. For comparisation - building ksh93 with
the native build system can be VERY tricky as the built system in it's
default configuration probes the underlying build machine (and not any
proto area etc.) for information which makes it very tricky as even
minor changes in the build machine can turn features on/off - and that
would be a perfect "call generator". You can fix that - but then you're
getting more or less to that point where we are currently with our
integration into OS/Net... :-)
> > wrote Makefiles which works like all other OS/Net stuff. Everything is
> > done, see
> > http://polaris.blastwave.org/browser/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported/usr/src
>
> It's pretty hard to work out what specifically this represents: how
> it is different from Dave Korn's ksh93, and what the delta is from the
> existing O/N gate; I would be aided greatly by a webrev, if you could
> point me to one.
I don't have a Webrev around right now, maybe Mike Kupfer can help here
because he has a script which can geneate webrevs from Subversion
repositories... in the meantime youmay use the "Trac" browser - it's
less powerfull than webrev or Bonsai but still allows you to track
things - see
http://polaris.blastwave.org/log/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported/
(this is the commit log for that Subversin subtree). Another option may
be to pull a diff... AFAIK % svn diff -r 178:401
http://svn.genunix.org/repos/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported
# should do the trick...
> Setting that aside, the fact that something is finished doesn't make it
> correct. I don't see why we wouldn't just drop this into SFW, using its
> much simpler build system. Integration into ON means that it will be
> that much harder to rev ksh93, and swells up the consolidation. Again,
> what's the architectural reason for including this piece of free
> software into the OS/Net consolidation, when we already have a
> consolidation designed to accept pieces of free software? SFW is where
> bash, tcsh, zsh, apache, etc. live.
What about the "migration" part of our project ? The old ksh lives in
OS/Net since it is used there - very often.
Question back: Why does "perl" actually live in OS/Net ? If I apply the
same pattern as your're using for ksh93 "perl" would be the last thing
needed in OS/Net... it could - in theory - happily live in /usr/sfw/hin/
... =:-)
> In my mind, unless we're doing some
> really deep changes to enhance ksh and solaris-ify it (like with e.g.
> ipfilter), it doesn't really belong in ON-- and in fact putting in SFW
> should help us to resist that temptation. I'm less clear whether there
> are formal rules about this at the product level; but there probably
> should be. If you can articulate ways in which ksh is inextricably
> intertwined into the system in ways the 'bash' is not, then I'm
> interested to learn about them.
See above... there are items like the future "migration" of the old ksh
to ksh93 or (for example) Solaris-specific goodies that we're going to
add a 64bit ksh93 - for the first time Solaris would have a scripting
language which is capable to deal with large datasets (and ksh93 is NOT
slow :-) ), something which even "perl" didn't archive yet. And
applications in OS/Net may be intestested to use libshell.so which would
make the creation of all the "*adm" tools MUCH easier as they could be
build on top of libshell's customizeable framework (if you ever worked
with "dbx" - it's based on another bariant of "ksh" - and in the long
term it may be nice to switch "dbx" to use libshell to avoid the massive
code duplication the various Sun products).
> To prioritize, my deepest concerns here are about
> (a) not reestablishing the practice of having large piles of
> asynchronously changing free software in OS/Net (i.e. I
> believe that 'sendmail' is grandfathered, not a precedent).
Noone wants to let that happen for ksh93. I even designed the source
layout in the OS/Net tree in a way to make updates as painless as
possible... usually less than on hour of work. There is even the
ksh93/AST test suite integrated into OS/Net which should catch dumb
errors during such updates.
> (b) The practice of re-engineering something to fit
> into our build process when there is not a compelling case
> to do so.
See above. The "re-engineering" only consisted of a steep learning curve
for me how the original AST build system works - beyond that point the
integration into OS/Net was very easy. Just plain, dumb integration work
and no magic involved.
> (c) The merging of libcmd with libcmd.
>
> It is possible that libcmd could be a forcing factor for inclusion
> within ON, but you've not articulated any architectural reason that
> solaris:libcmd.so can't be transformed into solaris:libcmd_private.so,
> have overstated the difficulty of such a change, and anyway, it seems
> like the tail wagging the dog.
Did "libc" have any architectural reasons for consuming "librt" ? What
about "libnspr" and other libraries which are a combination of various
sub-libraries ?
> To reiterate, I'm interested mostly in the architectural cleanliness
> of the project, and that things are done with appropriate rationale--
> number of commits, number of customers requesting, and even
> backportability are in my mind at best secondary concerns. You have
> done an admirable job blazing a trail to accomplish a large and
> complex project. I'm part of the system of checks and balances
> which helps to make sure that we're doing things "right" not just
> "with inertia." And making sure it is clean architecturally and
> in terms of maintainability sets a really strong precedent for
> future work.
Just for the log: I've spend much time in figuring out what's the "best
path" for maintaining the code. Currently our codebase is IDENTICAL to
the AT&T ksh93 codebase with the sole exception of
http://polaris.blastwave.org/browser/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported/usr/src/lib/libshell/misc/ksh93_solaris_builtin_patch.diff
which is described in the matching readme
(http://polaris.blastwave.org/browser/on/branches/ksh93/gisburn/prototype002/m1_ast_ast_imported/usr/src/lib/libshell/misc/ksh93_solaris_builtin_patch.readme).
The full update procedure is outlined above. The only thing which may be
slightly less easy are updates between major ksh93 versions (e.g.
between ksh93r+ and ksh93s) - those may require changes in the
Makefiles. But that is easy to do, too.
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
