Hi!
----
The mailing list `ksh93-integration-discuss' has just been created
(thanks to Eric Boutilier and the other people at opensoalris.org).
* About the "ksh93 integration/migration" project:
The project should investigate and execute the integration of ksh93 and
related features, including:
- introduction of ksh93 (/usr/bin/nksh and /usr/bin/ksh93 were proposed
in the private discussion)
- introduction of libshell.so
- investigate the migration of /usr/bin/ksh to ksh93 standard
- discussion/bugfixing of Solaris-specific ksh93 bugs+fixes
- enhancement of Solaris tools to use libshell instead of homegrown
commandline parsers (like mdb, zfs, xauth etc.)
- replacement of duplicate ksh versions in various Sun products with
libshell.so (which is ksh93 as a shared library), including dbx and
dtksh
+ More items need to be discussed...
* About the list:
- To post to this list, send your email to:
[EMAIL PROTECTED]
NOTE: Please SUBSCRIBE _before_ posting messages. Postings by
non-members will be put in "hold" for moderation (to avoid that this
list becomes a SPAM-redistributor) ...
- Subscription/unsubscription and membership options can be handled via
two ways:
1. Using the web interface:
http://mail.opensolaris.org/mailman/listinfo/ksh93-integration-discuss
2. All mailman user options/functions can be controlled via email. For a
detailed list please send an email to
[EMAIL PROTECTED]
with the word `help' in the subject or body (don't include the quotes),
and you will get back a message with instructions.
More options are described in the mailman user guide which can be found
under http://www.gnu.org/software/mailman/users.html
- List archive:
The list is being archived under
http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/
* Notes:
- Attached is the original migration plan (2006 version) which was more
or less the basis for this project.
- I've subscribed all people which were involved in the original 2006
discussion
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) [EMAIL PROTECTED]
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)--- Begin Message ---
Mike Kupfer wrote:
> Hi Roland, Felix Schulte mentioned recently that you had written up a
> plan for how Solaris could migrate from ksh88 to ksh93. I would like to
> take a look at it, as would the Sun engineer who is currently planning
> on doing ksh work, but we don't know where to get a copy of it.
Actually the migration itself is quite easy - but only if you see it
from the technical viewpoint... I did some reading through the
OpenSolaris mailing lists, talked to a couple of Sun people and Felix
this week and it seems that this issue is primarily a political (e.g.
Sun's policy of handling backwards compatibility) and "religions" one
(bourne shell vs. current Solaris ksh vs. ksh88 vs. ksh93 vs. bash vs.
"never touch a running system").
And I fear that nothing has changed in Sun's position about getting
/usr/bin/ksh updated to ksh93 status - it seems to be impossible like it
was impossible (to even think about (slaughtering the holy cow called
"backwards-compatibillty-must-never-ever-be-broken")) in the 2001/2002
time frame.
Anyway... I've cut the original plan (which was written for Solaris 8)
down to the main items (for two reasons: 1. the exact details need to be
reinvestigated for Solaris 10 and 2. I do not have much faith that Sun
will ever be willing to do the full migration (which means: Getting
/usr/bin/ksh updated to ksh93)):
* The plan now looks like this:
-- snip --
- Move current /usr/bin/ksh to /usr/bin/oksh (now called "old" korn
shell)
- Edit all existing scripts in the Solaris code base which use
/usr/bin/ksh to use /usr/bin/oksh instead until they have been tested
with ksh93.
A test in 2001 revealed that from 598 ksh scripts in a Solaris 8
installation 29 needed changes (unfortunately very prominent stuff like
"patchadd" / "patchrm" are on that list... ;-/ ) - that's AFAIK less
than 5%... :-)
And the changes required were usually minor AND made the scripts
"portable", e.g. they were able to run both with Solaris ksh and
ksh93...
- Put ksh93 into /usr/bin/nksh ("new" korn shell. ksh93 MUST NOT be
placed to /usr/bin/ksh93 - arguments can be listed here but that is
AFAIK outside the scope of this mail))
An alternative may be to just stuff ksh93 to /usr/xpg4/bin/sh (instead
of creating /usr/bin/nksh (on the other side the creation of
/usr/bin/nksh may be better... it indicates that there is a "new" korn
shell... :-) ))
- /usr/bin/ksh then gets a link (hard-/softlink - whatever you prefer)
to the following positions:
Solaris 9: /usr/bin/ksh --> /usr/bin/oksh
Solaris 10: /usr/bin/ksh --> /usr/bin/oksh
OpenSolaris: /usr/bin/ksh --> /usr/bin/nksh
Solaris 11: /usr/bin/ksh --> /usr/bin/nksh
- /etc/shells, the defaults for getusershell(3C) and the matching manual
pages needs to be adjusted
- libshell.so.1 needs to be introduced (both 32bit and 64bit versions)
- libc's wordexp(3) needs to be adjusted to use /usr/bin/oksh for now
(ignoring the fact that it is a horrible hack to abuse ksh (any version)
for such a purpose - but fixing that is IMO beyond the scope of this
project)
- /usr/xpg4/bin/sh can simply link to /usr/bin/nksh (except for the
wordexp(3) hack)
- Document /usr/bin/oksh as being depreciated, stop working on it and
let it silently die (and do not forget to move it into a seperate
package that people do not have to install it anymore unless it is
really required) ... :-)
* Note: I am listing Solaris 9/10 here too because products such as
"dbx" could be much simplified - instead of using their own
(unmaintained ?!) copy of ksh it would simply use libshell.so. Same
applies to "mdb" and the new **adm" tools which are very limited due
lack of expression&&error handling. Unfortunately this may require to
backport "nksh" and libshell.so to older releases to support the tools
depending on the new features (and that's the reason why I invented the
"oksh" and "nksh" naming scheme).
* Known problems/issues/questions:
- How should the old "rksh" and the old profile shells be named ?
Possible solution may be to stuff a "o" in front of the old ones and a
"n" in front of the new ones...
- Scripts need to be developed (unless David Korn and the AST community
already did that) to aid the migration from ksh88 to ksh93, ranging from
simple scanners which search for ksh88-specific syntax (such as
incomplete brackets, usage of undocumented ksh88 features) usage to
hook-ins which trigger shell traps into a debugger for ksh88-specific
stuff
- Solaris has some nice "easter-eggs" such as wordexp(3) which require
minor extra care (but please not via hacking ksh93 to fit the need)
- Should APIs such as libast.so be made available as public API ?
- Do we need a 64bit binary of ksh93 ? AFAIK the answer is "yes" since
we may want to support 64bit loadable add-ons for ksh93 (this may be
easy for SPARC since Solaris no longer supports 32bit SPARC machines so
only a 64bit SPARCV9 binary is required (except we want a backport (of
the "nksh" / "oksh" naming scheme) to Solaris 9 and/or make sure that
the 32bit version of libshell.so has a user... :-) ) ... but on Intel we
may need both (to support IA-32 and AMD64... ;-/ ))
- Does any other Unix/Linux version use "oksh" and/or "nksh" for
anything else (e.g. avoid "name space" collision) ?
- Could "pdksh" be used instead ? AFAIK it cannot be used for at least
three reasons:
1. a quick look at the README shows the date "Jul '99" which seems to
indicate that the project is no longer alive (pls correct me anyone if I
am wrong...) ...
2. "pdksh" is horribly behind ksh93 (looking at the number of supported
features and stability)
3. "ksh93" is the STANDARD (even die-hard opensource companies such as
RedHat and SuSE/Novell realised that... :-) )
- Is the ksh93 license compatible with Sun's CDDL used in Solaris (AFAIK
this can only be answered by Sun's lawyers... ;-/ ) ?
- Should "kshdbx" be shipped with Solaris, too (may be usefull) ?
-- snip --
* Arguments "Pro" introduction of ksh93 (more can be found in the
OpenSolaris discussion triggered by Felix):
- Ksh93 provides libshell.so which is basically the ksh93 shell
interpreter made available as an API. This could lead to a drastic
simplification in some of Sun's products - AFAIK Sun currently has five
copies of the old ksh in various (rotting) states, ranging from
/usr/bin/ksh, the profile shells, "dbx", "dtksh" and so on...
Instead of maintaining five copies there would be only one, central copy
of ksh for all consumers, leading to a much lower maintaince burden on
Sun's side.
- ksh93 provides much improved syntax and new features such as
associative arrays (which are very very usefull when dealing with large
datasets - sometimes it's even mandatory to use this feature for more
complex applications without getting mad to find workarounds), direct
TCP/IP support (required to create "networked" shell scripts), better
support for traps etc.
Additionally quite interesting is the ability to enhance the shell via
loadable add-ons (shared libraries). For example that's how we stuffed
lots of bioinformatics-related functionality (such as "pregap4" etc.)
into a ksh93-based shell toolkit (and even created a giant dtksh-based
application the same way) ... :-)
- ksh93 is under active development - there is an "upstream" (David
Korn, the AST community and others) who are working on it, doing
bugfixing, enhancements, patch integration etc.
Solaris ksh is Solaris-only - it has no "upstream" and Sun engineers
have all the burden of maintaing it. I've heared lots of complains in
the past about this maintaince nightmare called /usr/bin/ksh from Sun
engineers... ;-(
And Sun's bug database is still full of unfixed bugs which no longer
exist in ksh93...
The introduction of ksh93 (and depreciation of /usr/bin/oksh (= current
Solaris /usr/bin/ksh)) would be another good opportuniny for Sun to save
lots of engineering time in the long term.
- Usage of tbe new features in ksh93 could make existing shell scripts
shipped with Solaris much more faster+efficient (looking at the
patchadd/patchrm horror show... ;-/ )
- Better performance and lower footprint: ksh93 does not only perform
better than current Solaris ksh - the concept of sharing most of the
code between various versions of ksh (/usr/bin/ksh, profile shells,
dtksh, dbx etc.) via libshell.so should also reduce the overall resource
(memory) usage a lot...
- Greater interoperability between operating systems: Currently Linux
and some of the other Unix derivates (AFAIK AIX) ship ksh93 as
/usr/bin/ksh - and Solaris is one of the last remaining OSes which still
uses some kind of ksh88-derivate (note: Solaris /usr/bin/ksh is not even
fully ksh88-compatible anymore... it's stuck somewhere between ksh88,
ksh93 and POSIX sh, making it even a greater interoperbility nightmare
than any other shell (well, maybe excluding "csh"... =:-) )) for that
job. And (sales) people would no longer be able to refer to Solaris as
an "outdated" operating system ("... Solaris is stuck in the 80' of the
last century - just check the version of ksh they're shipping..." etc.).
- There would be no need to make the old ksh88 OpenSource (which also
means: Less work for the lawyers... :-) ). Instead OpenSolaris could
simply start "cleanly" with "ksh93" as /usr/bin/ksh and leave the "dark
age" of korn shell (88) hacking behind
* Arguments "Contra" introduction of ksh93 (again, more rants can be
found in the OpenSolaris mailinglist):
- ksh93 is not fully backwards-compatible to current Solaris ksh. This
was the main concern of Sun in 2001/2002 and is likely even currently
the problem which even prevents the change indefinately (unless all
opposing managers get fed to hungry komodo warans... :-) or can be
convinced somehow else...). This is more a political problem with Sun's
style of "backwards-compatibility-must-never-be-sacrified" then a
technical issue (actually it is also the reason why I kept this email so
short - I still do not have much faith that Solaris's /usr/bin/ksh will
ever be updated to ksh93. The WORST thing which could happen was the
idea that ksh93 gets installed into /usr/bin/ksh93 - giving Sun the
perfect opportunity to ignore further requests for proper ksh93
integration
(sorry for this harsh comment - but I've seen to many "transitions"
stuck like this in Solaris (CDE2.x being one of the more prominent
ones) half-done (blocked by various mythical/possible/theoretical
incompatibilities which may occur (at 0:61am on February the 30th or
something like that)))).
- Initially much work is required, including the integration of ksh93
into the Solaris build system, script testing+conversion, libshell.so
integration (and conversion of tools such as "dbx" to libshell.so) etc.
So far... comments/suggestions/etc. welcome... :-)
----
Bye,
Roland
P.S.: Just for the log: I would be willing to work on the /usr/bin/ksh
migration to ksh93 (as intern or whatever status is needed for that)
_IF_ (and that's the only requirement from my side) Sun would gurantee
that the transition is completely done (and doesn't get stuck somewhere
between heaven, hell, limbo and some kind of excuse called
"/usr/bin/ksh93") for Solaris and OpenSolaris.
P.P.S: Sorry for all the typos but it's 7:28h AM here right now... ;-/
--
__ . . __
(o.\ \/ /.o) [EMAIL PROTECTED]
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
--- End Message ---
_______________________________________________
opensolaris-discuss mailing list
opensolaris-discuss@opensolaris.org
