David.Comay at Sun.COM wrote:
> > In the case of the Mamfiles it's IMO "documentation". We check-in these
> > files and keep them updated to have a reference point for the matching
> > OS/Net Makfiles. I know that this is not perfect but it's at least
> > economical for those who work on the current tree (e.g. April and me).
> > Currently we can deploy updates within less than an hour (excluding
> > testing and a full OS/Net build) and randomly removing files will only
> > cost more time since the more or less automated procedure will no longer
> > work and then patches need to be applied by hand which takes far longer.
>
> But for many others, it's just cruft in the gate. I definitely could
> understand it if the source directory was being dropped in situ into
> the source base but that's not the case here.
Uhm... that's not exactly correct. The ast-ksh/ast-open upstream package
is just split into the matching locations, e.g.
src/lib/libast/ ---> usr/src/lib/libast/common/
src/lib/libdll/ ---> usr/src/lib/libdll/common/
src/lib/libcmd/ ---> usr/src/lib/libcmd/common/
etc.
... which allows to create a more or less automated patch process in
this case since we "only" have to create a diff between the old and new
upstream sources, adjust the paths in that diff, backout any local
modifications (that's the purpose of the files in
usr/src/lib/libshell/misc/*.diff), apply the diff and then re-apply the
patches from usr/src/lib/libshell/misc/*.diff (and update these patches
if they did not apply cleanly) etc. etc.
> For good reasons, you're
> putting the libraries with the other libraries and at some point, it's
> not clear if keeping the files which don't really serve a purpose as
> documentation (at least for OpenSolaris.) Anyway, this is something
> for you to work with the C-team on.
Erm... I thought the reviewers have to decide that... or not ?
[snip]
> >> That certainly is a good argument if bourne shell implementation is
> >> unreadable. But there seem to be some Makefiles where I didn't see any
> >> ksh93 constructs at all, or minimal ones at that. For example, why the
> >> changes to libcmd's Makefiles to use /bin/ksh?
> >
> > Because libcmd's Makefile "include"s things like "Makefile.libastl10n"
> > and "Makefile.astinclude". I know that this functionality could be
> > re-done using SHELL=/bin/sh which then calls scripts which use
> > "#!/usr/bin/ksh" and do the same job - the question is whether this is
> > really neccesary when the same could be done in the Makefile's itself
> > with less effort.
>
> Thanks for the pointers to the included Makefiles. Although I didn't
> really see any ksh'isms in Makefile.libastl10n,
Uhm... small example:
-- snip --
msgs/%.mso: ../common/%.c
@mkdir -p "$$(dirname "$@")"
@print "# Processing file $< to $@" >>msgcc.out
@$(ASTMSGCC) -M-set=ast $(ASTMSGCCFLAGS) $(CFLAGS) $(CPPFLAGS) -c
"$<" -o "$@"
-- snip --
This uses:
- $( command ) for subprocesses (instead of ` command `)
- ksh "print" builtin
- In the case of ksh93 "print", "dirname" and "mkdir" are builtin
commands
[snip]
> >> Put another
> >> way, should this package me an internal-only package right now because
> >> the end-developer cannot use it given the commitment level that was
> >> ARCed?
> >
> > What do you mean with "internal-only package" ? And see my question
>
> By internal, it would be built as part of the ON build process but it
> would not be included at this time in the WOS. When you install
> Nevada, you wouldn't see the package installed on the system. However,
> it would be available for those with ON build machines.
How would normal developers get that package ? I'd like to avoid that we
force gatekeepers to bundle it with "SUNWonbld" and then change that
again after a few months. That only generates unneccesary work...
> > above about the future target of the package (like the other ksh93 and
> > AST components we'd like to put them into their correct places from the
> > beginning... anything else (like moving files etc. around) will just
> > generate more paperwork which will consume much time (at least I am a
> > volunteer and (unfortunately) have to work for food on other stuff
> > (which means I don't like to spend time on paperwork unless it's really
> > mandatory))).
>
> When the commitment level of the required interfaces in libshell, etc
> are raised, you wouldn't need to move any components but rather add in
> the *.so links and the lint libraries.
>
> The amount of so-called paperwork is minimal - besides a PSARC
> fast-track, there is a short ASCII form called a "package RTI" that
> tells the Solaris Release Engineering group to include this package and
> which meta-cluster is appears in.
Even the "minimal" paperwork needs time. It may be easier for people
within Sun but for people outside Sun we're talking about weeks up to
months of waiting. And it seems (based on your request to get
/usr/bin/sparcv9/ksh93 removed because "isaexec&&magical builds" isn't
done yet (yes, I know.. you wrote personally vs. delivering what was
specified in the two ARC cases)) that getting things implemented/putback
in the wrong order may affect the project progress by adding even more
paperwork or forcing a serialisation of project operations to make sure
that one step creates all building blocks for the next case (instead of
doing the work in parallel and even build on top of things which may
only be available after the putback). Yes... I am aware that engineering
may prefer to have paperwork and all things in place (as someone else
said here... it may always happen that the engineer working on this
stuff gets run over by a truck etc.) but I am not sure whether this
should be mandatory for a "feature putback" like ksh93 or the
"SUNWastdev" package.
[snip]
> >>> See may reply to James Carlson's email
> >>> (http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2007-February/002225.html).
> >>> Basically the upstream code uses this as a platform identifer and we use
> >>> the same algorithm to provide the same value in this case (the code
> >>> doesn't build without a value and I don't want to pass a hardcoded
> >>> value).
> >>
> >> I saw the response but it still sems short-sighted on including part of
> >> the `uname -r` value but not the rest.
> >
> > Erm, what should I do in this case ? We can't really add a random value
> > here. I am just emulating the behaviour of the upstream build (without
> > hardcoding values).
>
> I guess I'm wondering though what sort of release values are used with
> other systems? Could it instead be based on something like (major # *
> 100) + (minor #?)
In any case this would AFAIK require major changes in the upstream
sources (and we try to not fork() the code, remember ?) because AST is a
large codebase (ksh93 is only a tiny piece). It's possible to change but
it may require some time (the problem is that this value comes from the
base architecture directory (e.g. if you build the upstream package the
generated binaries and includes are stored in
arch/<value>/(bin|include/ast|libs|etc)/ where <value> matches the value
we're currently talking about) and I am not sure which code depend on
this (we're talking of a codebase which matches OS/Net in size) ...) ...
[snip]
> >> I didn't explain my question well. Are you planning on moving
> >> usr/src/lib/libcmd/common/mapfile-vers to
> >> usr/src/lib/libcmd/mapfile-vers and updating the file? Perhaps this is
> >> a question for April since it's a Teamware operation.
> >
> > Erm, yes. I am following the suggestion of Roger Faulkner (if I recall
> > the name correctly) who suggested to put the files in the base
> > directory. At least the following libraries do it the same way (this is
> > from our B51 tree):
>
> Well if Roger has reviewed this and signed off on it, then I'm happy.
> :-)
See
http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2006-August/000782.html
Is that enougth ?
> >>>> usr/src/lib/libshell/misc/buildksh93.ksh
> >>>> usr/src/lib/libshell/misc/buildksh93.readme
>
> >> Or is the script that's used to generate the files which are eventually
> >> *checked in* to ON?
> >
> > Right. Please read the script and look at the original AST build system
> > now it works - "buildksh93.ksh" does some "modifications" to the build
> > setup to make sure we find some features in libraries which are normally
> > not detected automatically and enforce things like XPG6/C99 to avoid the
> > limitations of a normal build (which both dramatically affects
> > performance and other details).
> > BTW: "perl" in OS/Net does the same - see
> > http://mail.opensolaris.org/pipermail/ksh93-integration-discuss/2007-February/002244.html
>
> Yes, and I'm not sure if for that reason Perl really belongs in ON
> either :-) but I understand you're following that precedent.
Ok... does that mean you're no longer demanding to see the head of
"buildksh93.ksh" on a stick in front of your office (instead of being
part of this putback) ? :-)
> >>>> usr/src/pkgdefs/SUNWcsu/prototype_sparc
> >>>>
> >>>> Lines 50-51 - What is the reason for shipping a 32-bit version
> >>>> on SPARC? Can ksh93 be used to read 32-bit core files or
> >>>> processes? :-)
> >>>
> >>> There are several reasons including:
> >>> - ksh93 supports (loadable) binary plugins which may itself rely on
> >>> other shared libraries which may not be available as 64bit versions. In
> >>> those cases a 32bit ksh93 is needed. I am waiting for a sponsor for
> >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6474270
> >>> ("isaexec and magical builds") (see request-sponsor queue) to make
> >>> "isaexec" adjustable for applications which need such a functionality
> >>> (via restricting the list of ISAs based on accept and/or reject filter
> >>> environment variables).
> >>
> >> So how does a SPARC user use ksh93 in that case? Do they code their
> >> scripts to use /usr/bin/sparcv7/ksh93 directly?
> >
> > No, the idea was to implement Sun-Bug #6474270 ("isaexec and magical
> > builds") and set an environment variable which tells isaexec which ISA
> > should be used. I was hoping to get this work done in parallel but
> > somehow the request is still waiting in the request-sponsor list (see
> > http://opensolaris.org/os/bug_reports/request_sponsor/) ... ;-(
>
> But that hasn't been implemented yet so it seems including the sparcv7
> version at this time is premature.
Umpf... what can I do in this case ? in a perfect world where
"isaexec&&magical builds" would already be "done" we would not have this
discussion, right ?
[snip]
> > Well, we heavily stripped the original ARC case to make the first
> > putback self-contained, e.g. it should not affect anything else except
> > adding ksh93+libraries (we even stripped "shcomp" since it would deliver
> > a new kernel module (to recognize compiled shell code (similar to
> > javaexec))) - but ARC 2006/550 specifies these files and the usage of
> > isaexec (from
> > http://www.opensolaris.org/os/community/arc/testbed/caselog/2006/550/onepager/):
> > -- snip --
> > Interface Description
> > --------- -----------
> > /usr/bin/ksh93 the korn shell
> > /usr/bin/pfksh93 profile shell
> > /usr/bin/rksh93 restricted shell
> >
> > which will be hard links to /usr/lib/isaexec; isaexec will execute
> > the corresponding 32-bit binary in /usr/bin/{sparcv7,i86} or
> > the 64-bit binary/usr/bin/{sparcv9,amd64}, depending on the
> > architecture.
> > The isaexec links will allow the 64-bit version of ksh93
> > to be executed by default on 64-bit platforms.
> > -- snip --
> > As I said this is done intentional, even on SPARC (Garret bickered
> > already about the use of isaexec... ;-( ) ...
> > ... you're not coming up with the argumentation that we have to remove
> > parts which are not in use yet and re-add them later with the part which
> > starts using it, right ?
>
> Actually, I am arguing for delivering what was specified in the two
> PSARC cases and nothing else at this time. It's not a huge thing
> delivering the 32-bit version on SPARC but it's unnecessary at this
> time. Personally, I would rather see just the 64-bit version delivered
> at this time (yes, using isaexec just as we do with many 64-bit only
> SPARC deliverables) and then deliver the 32-bit version when the new
> plugins arrive.
Umpf... is this really neccesary ? The 32bit libs are needed soon by
"shcomp", "sleep" and all the things which currently use "alias.sh". And
the 32bit executable which sits in /usr/bin/sparcv7/ksh93 is just a
wrapper which calls directly into libshell.so using something more or
less like |int main(int ac, char *av) { return sh_main(ac, av); }| - the
stripped binary is around 7k - which is smaller than this email.
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.mainz at nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
