I am sponsoring the following fasttrack for myself.
I propose a new mapfile language for the Solaris link-editor (ld).
-----
Release Binding: Minor
New mapfile syntax: Committed
Although I believe this work qualifies for Patch/Micro binding,
there is no desire or intent to take this work back to any Solaris
release before Nevada/OpenSolaris.
Although a new mapfile language may seem a large step, I believe that a
fasttrack is appropriate: The link-editor maintains full backward
compatibility with existing mapfiles, and the new language is rather
evolutionary in nature, as it is based upon the existing syntax for symbol
scope/version in the original language. However, given the time of
year, the bulk of materials included, and the fact that we are heading
into a series of restricted builds during which this work would not integrate,
I am setting a longer than usual timeout for this case of three weeks.
-----
Within the linker group, we have long believed that it would become
necessary to replace the mapfile language used by the Solaris link-editor
(ld). The existing mapfile language was not designed to be extended, and
yet, has had various features grafted on for 20+ years. It is inconsistent,
difficult to document, and has become an impediment to future progress.
We believe that it is time to make a fresh start with a syntax that is
simple, consistent, and designed to simplify additions.
In doing this work, I had the following high level goals:
- Full support for existing mapfiles written in the original language
(which we now refer to as version 1). Existing mapfiles will
"just work", without requiring any end user adjustment.
- Human Readable: The original language is extremely terse,
and uses "magic characters" such as '=', ':', '|', and '@', to
differentiate the various statements in the language, with
widespread overloading of characters such as '$' which have
different meanings in different contexts. It is generally not
possible to read a Solaris mapfile without a copy of the Linker
and Libraries manual at hand to decode it. A more readable keyword
based syntax is desired.
- Extensible: Primarily due to the previous point, the original mapfile
has become considerably more difficult to extend and maintain as
time has gone by, to the point where few people claim to understand
the language, and adding new features has become problematic.
The new syntax will use keywords to represent the different
available options, which is considerably more mnemonic and which
offers very large namespace for additions. It will use a standard
underlying abstract syntax for all features, instead of the ad hoc
approach taken in the original language, which will also make
the system easy to extend.
- Simple conditional input: I have observed that multiple mapfiles
are sometimes used in the ON code base to express minor per-platform
variations (e.g. a symbol may only exist on a specific platform).
These extra mapfiles are clutter, and a potential vector for versioning
errors to go undetected. A simple mechanism to allow conditional
input of mapfile lines would be of value.
- Familiarity: I believe that the problems with the original mapfile
language are primarily matters of syntax, and that the semantic level
of the language is reasonable, and appropriate. Furthermore, the vast
majority of extant mapfiles are restricted to the scope/version subset,
which is newer than the rest of the language (Solaris 2.5), has fewer
problems, and is generally well liked. I therefore seek to clean up
the mapfile language without losing the benefit of the familiarity
people have with the original, by creating a new syntax based upon
the scope/version subset of the old version.
I have placed several files in the case materials:
[mapfile.html]
A replacement chapter for the Solaris Linker and Libraries
mapfiles chapter. The proposed language is defined by
this document.
[sysv_mapfiles.html]
Rough draft of future article to be posted to my Sun
blog after this case is complete, detailing the flaws
of the original mapfile language, in support of the
need to replace it. I do not consider this document to be
part of this case, but it does provide useful background
information for interested parties.
[new_mapfiles.html]
Rough draft of a followup article for my Sun blog, intended
to follow sysv_mapfiles.html once this work has been put back.
This document describes the design requirements applied to the
new language, covers the same abstract syntax details documented
in mapfile.html, and then provides a large number of examples showing
how you express various concepts in both the old and new syntax.
The mapfile language described in these documents has been fully
implemented and code reviewed, and I believe it is ready for integration.
Within SWAN, the machines rtld.central (x86) and ldd.central (sparc)
have linkers containing this work installed. I invite any interested parties
to log on to those machines and experiment.
-----
A good way to evaluate the new syntax is to compare mapfiles written in
both the old and new languages. The new_mapfiles.html document contains
several such examples. In addition, as a proof of concept, and foreshadowing
a future project, I have converted every mapfile in ON from the old syntax to
the new, and using the new linker, have built and BFU'd the results on
both sparc and x86 systems, currently running on rtld.central and ldd.central.
The webrev for this workspace can be viewed at:
http://linkers.central/webrev/mapfile_osnet
within SWAN, and at
http://cr.opensolaris.org/~alib/mapfile_osnet
outside. Please note that I do not intend to integrate this ON workspace
as part of the putback for this new mapfile syntax. Any such change must
wait for a period after the linker support integrates. For the purposes of
this case, this webrev is provided solely as illustration of the new
language, and proof of suitability for its intended purpose. You can look
at how the parts of the system you're familiar with might change using
this new functionality.
99% of the mapfiles in ON do nothing other than symbol versioning.
You'll find that these have only changed in minor ways:
http://linkers.central/webrev/mapfile_osnet/usr/src/lib/README.mapfiles.sdiff.html
The most substantial difference is that conditional input directives
can be used to eliminate the ISA-specific mapfiles in the
{amd64,i386,sparc,sparcv9} subdirectories, which is almost always
a simplification. Libc is probably the most complex library in ON in
this regard, and the mapfile changes for it are worth examining as a
demonstration of how things work in a large history-laden example:
http://linkers.central/webrev/mapfile_osnet/usr/src/lib/libc/port/mapfile-vers.sdiff.html
Kernel developers may be interested in how the more traditional segment
related parts of the language have evolved:
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86pc/conf/Mapfile.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86pc/conf/Mapfile.amd64.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86pc/conf/Mapfile.bios.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86pc/conf/Mapfile.fb_swtch.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86pc/unix/dboot/Mapfile.dboot.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86xpv/conf/Mapfile.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86xpv/conf/Mapfile.amd64.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/i86xpv/unix/dboot/Mapfile.dboot.sdiff.html
http://linkers.central/webrev/mapfile_osnet/usr/src/uts/sun4/conf/Mapfile.sdiff.html
