Thank you for your instructional description of the C++ language with
respect to templates.
There are a few significant points, that I think you keep missing, or
are glossing over:
When I said C++ doesn't separate *binary* interface from implementation,
I was *not* talking about the nice separation done for the benefit of
the programmer. I was talking about separation done across the boundary
created by the linker ... AFAIK, C++ does not recognize that there is
such an interface boundary.
In C++, if an inline function is expanded by the compiler, but makes use
of *private* members of the class, then the *binary implementation* of
the resulting code is *tied* to the implementation, and if the
implementation changes (e.g. by changing the dynamically linked library)
then there can be unfortunate consequences.
I'm not a template expert, so I'll demonstrate with a simple inline
function using C++ circa 1990.
class box {
int width;
int height;
/* put more private details here... */
public:
void draw(); // private implementation elsewhere
public inline int getwidth() { return width; }
};
If this is in header, then the implementation parts (the width, the
height and their offsets within the class) wind up getting encoded into
the binary that includes this header. If the width or the height ever
move (e..g by moving another member in front of them in the class), then
the binary is busted. Likewise, if the private members change type
(e.g. from a uint16_t to a uint32_t), breakage ensues.
It is possible to have implementations that don't suffer from this
problem, but it requires that the implementation refrain from inline
functions, and take care that the headers *only* expose portions of the
classes that will never ever change (or change in a way that breaks the
binary implementation of the class.) I won't endeavor to describe how
this problem may or may not affect templates, as such is out of my area
of expertise.
I've not kept up with the C++ standards admittedly, but I believe that
this aspect of binary compatibility is not one that has been addressed
in the standards, and likely not in the specification of the libraries.
It is possible that the Apache product has taken great care to avoid
this kind of potential breakage, but my first reaction is doubt.
Now, that's only *one* aspect of the "compatibility problem."
There is a grave and other serious problem, which relates to the fact
that this proposal effectively creates a new baseline (i.e. a new ABI)
for C++ programs, where programs linked (directly or indirectly) against
this library are incompatible with our one and only bless (at this time)
ABI, based on the libC that ships with Solaris today.
And, the project recognizes that future breakage is not just possible,
but *likely*, when either the compiler team ships another project, or
when the library needs an update due to changes in the standard.
Imagine the plight of the poor user who downloads a program
(WhizzyDownloadTracker) which uses two different class libraries. On
the one hand, it uses the NiftyNetworking library for networking
functionality, and it uses the KDE libraries for user interface work.
Well, good thing, the IPS repo contains both NiftyNetworking and KDE.
Download both, install, and then download, compile (via ./configure) and
install the WhizzyDownloadTracker. All's good, right?
Well, a few hours later, when the compile for WhizzyDownloadTracker
finally completes, we find out life isn't so good.
Because you see, the NiftyNetworking set was not built with the Apache
libstdc++, but instead with the "default" libraries that we have been
recommending people use for years. The user tries to run the program,
and if he's lucky, gets a meaningful link error message. In reality,
the error message, if any, that results is probably completely cryptic
to the end user, who was just trying to install software. In a worse
scenario, the program just dumps core mysteriously.
The worst part of the above scenario, apart from the confusion that the
poor non-C++-developing victim gets to experience, is that he's
downloaded the foundation libraries from a single source (Sun's IPS
repo), so they *should* work together, shouldn't they?!?
Too bad.
So now the user complains to the FOSS authors for WhizzyDownloadTracker,
who are unfamiliar with Solaris (they did their development on Linux,
see?), and don't know that Solaris has multiple incompatible C++ ABIs,
nor how to tell which components were built against which ABI.
So, the FOSS author probably gives up, and tells the poor user that he
needs to recompile the base libraries (either KDE or NiftyNetworking),
so that they are built with compatible libraries.
At this point, the FOSS author is laughing at Sun and Solaris, and
probably also cheerfully writes a blog somewhere admonishing folks
against running Solaris.
And, the original user? He's probably given up. Most likely he either
he chooses not to use WhizzyDownloadTracker, or winds up giving up on
Solaris and switches back to Linux or Windows. Either way he's probably
pretty ticked that "Sun" is delivering crapware that just doesn't work
together.
I don't know if the above illustration clearly enough paints the picture
of my features, but I think it certainly demonstrates that we cannot
just blithely ignore the issue and pretend it won't affect anyone.
-- Garrett
Stefan Teleman wrote:
>
>
> Garrett D'Amore wrote:
>> John Plocher wrote:
>>> Garrett D'Amore wrote:
>>>> One of the implications of such a binding (Volatile), is that
>>>> projects which build other C++ shared libraries upon this one
>>>> cannot have a commitment level higher than Volatile either.
>>>
>>> Braap. Bad Architecture Alert. The whole reason we provide
>>> abstractions
>>> like consolidations and components is precisely so that we can provide
>>> "higher than Volatile" expectations for things that theselves may
>>> exhibit
>>> "less than Volatile" stability.
>>>
>>> There is no reason this couldn't be made a part of the KDE
>>> consolidation,
>>> and maintained by them as Committed interfaces for use by any KDE
>>> consumers
>>> who need it. Volatile means "can change", not "will change", and
>>> both the
>>> Apache C++ Lib and the KDE projects certainly seem to meet the basic
>>> ARC
>>> expectations of managing the compatible evolution of their component.
>>>
>>> If the C++ basis that KDE builds upon were to change incompatibly, I'd
>>> expect KDE to react by producing a major release - again, just like the
>>> ARC would expect.
>>>
>>> Nothing here requires KDE to be Volatile.
>>
>> We're not talking about something that is delivered with
>> Consolidation Private binding... we're talking about (assuming
>> Volatile binding) something that could change underneath KDE. Such a
>> change would be devastating to binary compatibility for applications
>> linking against KDE C++ libraries.
>>
>> If KDE has a way to shield applications underneath from such a binary
>> breakage, then its a different story altogether. However, my
>> understanding is that in this case (unlike more simple cases
>> involving only C), there isn't a way for KDE to do that.
>
> This ARC Case is not about KDE, but about the Standard C++ Library.
> What KDE may or may not expose in its header files, and how it handles
> implementation delegation design patterns is for a different ARC Case.
>
>> I.e. with C++ code, if application #include's a KDE header, which
>> itself #include's a libstdc++ header, the binary bits of the
>> application *very* likely contain details of the underlying libstdc++
>> implementation encoded in them.
>
> It is the responsibility of the implementation to shield any private
> and potentially incompatible implementation details from the publicly
> exported interfaces. For the purposes of this statement, "interfaces"
> refers to both source and binary.
>
> The principle of separating interface from implementation is one of
> the Design Principles of the C++ Programming Language, and has been a
> C++ software design principle ever since the creation of the Language.
> It has been widely discussed and documented in relevant literature,
> and it has also been put in practice by many C++ software systems,
> including, but not limited to, the Apache Standard C++ Library, and/or
> the existing libCstd.so.1.
>
>> C++ is reasonably good at providing good programmatic boundaries
>> between interface and implementation at the *source* code level.
>> However, it falls down completely at the *binary* level. (Which
>> isn't to say that libraries simply *can't* prevent this sort of
>> problem -- merely that the expectation should *not* be that they do,
>> because it will probably require some rather grotesque contortions on
>> the part of the library to do so.)
>
> The private implementation details of any particular C++ software
> system are just that: private. The blanket statement that C++ falls
> down completely at the binary compatibility level is false, and it is
> invalidated by existing software practice. It is indeed possible to
> maintain binary compatibility for a C++ software system, and no
> grotesque contortions are required to achieve this goal.
>
>> Now, if the library (KDE) never #include's "standard" C++ headers
>> (provided by this library) in its own headers (that it exports to
>> applications), but only uses them in .cxx (or .cpp or .C or whatever)
>> implementation files, then I agree that there is no problem.
>> (Although the consuming application may itself still wind up needing
>> to have its own dependency upon the libstdc++, but that issue is
>> largely orthogonal as far as something like KDE is concerned.)
>
> The dependency constraint has already been clearly stated in the ARC
> Case.
>
> Although it is generally considered a poor software implementation
> choice to #include Standard C++ Library header files in application
> header files exporting public interfaces, the implementation of the
> Apache Standard C++ Library allows for this inclusion, without
> breaking ABI [ pursuant to the compatibility constraints described in
> the ARC Case Materials ]. However, the Language allows for the
> implicit inclusion of interfaces from the Standard C++ Library [ or
> for that matter, any other library ], in header files, without the
> need for explicit #include directives.
>
> The Library incompatibility constraint is still in effect: the Apache
> Standard C++ Library is not compatible with:
>
> - any implementation of the Apache Standard C++ Library, which is not
> at Major Release 4 level
> - any _other_ implementation of the Standard C++ Library, including,
> but not limited to: libCstd.so, the GNU Standard C++ Library, STLport,
> etc.
>
>> To put this in comparison, imagine if almost all of the standard C
>> functions were simply *macros* rather than functions, and the macros
>> made references to volatile innards of the C library. While the
>> *API* might be "clean" and safe, the *ABI* would most certainly not
>> be. This is the situation that we're in with C++, I think.
>
> Intentionally, and by Language Design, C++ inline functions, or
> templates (which is what you are referring to) are *NOT* C macros. C++
> inline functions are functions. C++ templates are templates. Their
> symbols are mangled, they obey all the language overloading rules,
> they are assigned the implicit "this" pointer by the compiler (if they
> are class members), and they behave exactly in the same way as any
> other C++ function. They [ classes ] also implement a distinct type.
>
> The complexity of the implementation of C++ functions increases in the
> case where the class, or function in question, is a template (which is
> the case with the majority of the classes in the Standard C++
> Library). If the class is a template, the compiler instantiates a
> distinct type, based on the type defined by the class template itself,
> and on the underlying type upon which the template class is
> instantiated. This instantiation mechanism alone is fundamentally
> different than that of C macros.
>
> The type instantiation mechanism applies to non-template functions and
> classes as well.
>
> The compiler may or may not decide to eliminate the function call
> altogether, by inlining in the resulting object file, and this is a
> private decision of the compiler. The compiler may or may not decide
> to eliminate the instantiation of the template [ class or function ]
> altogether, based on internal compiler heuristic rules, and/or based
> on whether or not the actual template function or template class is
> actually referenced in the translation unit [ the default compiler
> decision can be overridden with specific compiler flags, but that
> facility is besides the point for this discussion ]. This decision,
> again, is a private decision of the compiler.
>
> At this point, your comparison with C macros has fundamentally broken
> down: there is no guarantee whatsoever that any of the template
> classes or functions, whose interfaces have been imported by the
> translation unit via header files, would have actually created the
> required type and its corresponding instance, and/or that the compiler
> has, in fact, generated the corresponding symbol(s) in the binary object.
>
> The means by which the implementation achieves the asserted ABI
> compatibility goal is private to the implementation itself. In this
> particular case, under discussion, this compatibility is enforced by
> function calls to private, internal implementations of the facilities
> required by the Language Standard (hence the presence of the shared
> library object).
>
> Attempting to invalidate the ABI compatibility assertion of the
> implementation by drawing comparisons with C Language macros, or with
> the C Programming Language in general, is based on incorrect
> assumptions about the C++ Programming Language, and is bound to fail
> scrutiny.
>
> I am hereby requesting that the PSARC member who has derailed this
> case provides concrete proof of ABI breakage in the Apache Standard
> C++ Library, to the PSARC Committee, for review. Concrete proof means:
> source code, accompanied by an explanation of the breakage.
>
> For The Record: KDE has no intention whatsoever to modify the Standard
> C++ Library in an incompatible way.
>
> Thank you.
>
> --Stefan
>
