On Mon, Dec 28, 2015 at 7:47 AM, Nick Wellnhofer <[email protected]> wrote:
> On 22/12/2015 01:31, Marvin Humphrey wrote:
>>
>> Yeah, I know what you mean! The complexity arises, though, when there are
>> multiple consumer libraries which require conflicting versions of a
>> dependency AND which expect to exchange types defined by the dependency.
>> What if the types defined by different versions of the dependency are not
>> compatible?
>
> In this case, direct interoperability is simply impossible. But this is a
> problem that affects any language environment that allows to use multiple
> versions of a library in a single program.
>
> Nevertheless, it should be possible to support two isolated libraries that
> require different versions of a prerequisite like the Clownfish runtime. For
> example:
>
> Lucy 1.0 requires Clownfish 1.0
> ProjectX 2.0 requires Clownfish 2.0
>
> Even if Lucy 1.0 and ProjectX 2.0 don't exchange any Clownfish objects, the
> current implementation doesn't allow to run both libraries under Perl. This
> is a serious limitation that should be addressed at some point.
This problem has vexed me for a long time, too -- in a 2013 conversation of
ours I made a proposal intended to address it:
http://markmail.org/message/qim5rd2pwlhbg3wf
I'd like to propose that major version numbers be considered part of the
parcel identifier and that we use a name mangling scheme to embed them in
exported symbols, while providing macro aliases for programmer
convenience.
// For Lucy version 0.x:
#define lucy_Indexer_new
org_apache_lucy_0_Indexer_new
// For Lucy version 1.x:
#define lucy_Indexer_new
org_apache_lucy_1_Indexer_new
Unless I've missed something, I believe that scheme will allow multiple
major versions of a library to coexist within the same process...
However, I've since come to believe that other priorities should take
precedence.
Versioning and package naming are closely related. One way to introduce
incompatible changes without breaking backwards comptibility is to move the
project to a new namespace. Alternately, the namespace can include the
version number. These are two sides of the same coin.
There are a number of issues with package naming in Clownfish which we we have
yet to sort out.
* We agreed to replace the double colon namespace separator with a dot, but
have not yet done so.
* Clownfish does not support nested package names, e.g. `foo.bar` (or better
yet `Foo.Bar`).
* We do not have solid plans for mapping Clownfish package names to a wide
variety of host language namespaces.
* Clownfish's lower_snake_case inert function names and variable names do
not always map cleanly. For instance, Go uses lowerCamelCase for
variables and UpperCamelCase for exported functions. Arguably, Clownfish
should use Upper_Snake_Case for everything, since it maps more cleanly to
most options.
* Clownfish appears to support intermediate names, e.g. the "Index::" in
"Lucy::Index::Indexer", but what really matters is the package name and
the last component of the class name[1].
Then there are so many issues with versioning that are important to get
right[2].
* One-to-one correspondence between version numbers and installable units.
* Sane semantics for version numbers in terms of allowable names,
comparisons, etc.
* Version-numbers-to-installation-locations.
* Mapping version numbers from Clownfish to all host packaging systems.
* Registering feature sets by name (without a version number).
* ...
It seems to me that all of those naming and versioning problems can be solved
reasonably well, and that they are exactly the kind of problems that Clownfish
ought to be solving since a primary goal of Clownfish is to be as compatible
as possible with as many hosts as possible.
In contrast, I think that providing support for multiple versions of a library
within the same process is outside of Clownfish's scope, as well as poorly
supported by many hosts. I'm concerned about the complexity and compromises
we would have to accept to achieve any amount of progress on that problem, and
that the increased complexity would impede our ability to solve the other
problems which are more central to Clownfish's mission.
And so I've changed my mind since 2013. I think that support for multiple
versions of a library within the same process is the kind of thing that should
be addressed at the level of the individual host language environment (e.g.
it's possible with Java and OSGI), rather than at the level of a symbiotic
object system which strives for compatibility across multiple hosts.
>> I can live with compatibilityVersion. I think we should still endorse
>> SemVer, though -- even if we extend it with compatibilityVersion.
>
> I just realized that Darwin's "compatibility version" is a bit different
> than what we need for Perl. It's used for minor version compatibility, but
> we need something to ensure major version compatibility (something
> equivalent to "soname" for ELF or "install name" for Darwin libraries). So
> let's use a different name.
Any name that is not too common will do. Maybe "compatVersion"?
Whatever we do, it's important that it not impede our ability to support
SemVer.
Marvin Humphrey
[1] I've been thinking that the package name should be `Org.Apache.Lucy`, the
class name should be `Indexer`, and that there should be a customizable
mapping of package name e.g. `Org.Apache.Lucy.` to `lucy.` for Python,
`Lucy::` for Perl, `org.apache.lucy` for Java, `github.com/apache/lucy`
for Go, `lucy_` for C, and so on. Then, there should only be one level
inside that namespace, resulting in e.g. `lucy.Indexer` for Python and
Go, `Lucy::Indexer` for Perl and Ruby, `org.apache.lucy.Indexer` for
Java, `lucy_Indexer` for C, etc.
[2] The discussion about this proposal for versioning in Go touches on many of
the complexities:
https://github.com/golang/go/issues/12302
