We have hit an issue that we can't seem to find a workaround for. Our only
working workaround is no longer supported by Julia 0.5.
The issue
========
We implement determinant for various kinds of matrix types in Nemo and
Hecke (two computer algebra/number theory packages). To do this, we extend
Base.det in Nemo.
Hecke depends on Nemo and tries to extend Nemo.det (or Base.det).
Hecke wants to define det for one of its own types, let's call it
SpecialMat. Now Hecke's SpecialMat belongs to Nemo's abstract type MatElem,
i.e. SpecialMat <: MatElem.
Nothing unusual so far.
The problem is: Hecke would like to call the det function provided by Nemo
in case the algorithm they provide is going to be slower (a decision that
is made at runtime).
What we try
=========
So what we naturally try in Hecke is something like the following
(obviously this code doesn't actually work):
module Hecke
using Nemo
type SpecialMat <: MatElem ## Nemo has a MatElem type class and a
"generic" det algorithm for any type that belongs to it
# some data
end
function det(m::SpecialMat)
# do different things depending on properties of the matrix m
if check_some_properties_of_a_matrix(m)
# implementation of special determinant algorithm that only works
for Hecke SpecialMat's
else
Nemo.det(m) # fallback to the Nemo implementation of det (which is
extensive)
end
end
export det
end # module
Here are some potential solutions we tried which didn't work:
1) Only import the functions from Nemo that Hecke doesn't need to overload,
i.e. don't import det from Nemo (or Base)
This causes Julia to tell the user that det could refer to Base.det or
Hecke.det, even though Base.det doesn't provide a function for the
specified type. We certainly can't expect the user to have to look up all
the documentation for Base, Nemo and Hecke every time they call a function
so they know how to qualify it. So this isn't a workable solution,
obviously. It's also far too verbose.
2) Try to qualify the function name with the name of the module, i.e. call
Nemo.det in the Hecke definition of the function, as above.
This doesn't work, since it is Nemo.det that currently has being
overloaded. So the Hecke det function just segfaults when called.
3) Look up the method table for the function we require and call the
specific method.
This works in Julia 0.4, but the ability to call methods has been
removed in 0.5.
This is an exceedingly frustrating problem. In fact it also occurs within
Nemo itself, since every time we want to implement a specialised version of
a generic function for a specific type, and have it fall back to the
generic version in certain cases determined at runtime, we can't do it,
without first renaming the generic implementation to something else with a
different name.
This sort of thing is making it very difficult to build large systems. It's
not fair to the developers of Hecke to ask them to duplicate all the code
in Nemo just so they can make this work, or alternatively force Nemo to
define every function twice so that there is a callable version with a
different name.
Does anyone know a workaround (any hack that works will do) for this issue,
that works in Julia 0.4 and Julia 0.5?
And is there a plan to fix this sort of issue in Julia in the future? The
module system currently makes it quite hard to work with multiple modules.
We are often encouraged to split our large systems into smaller
modules/packages to get around certain issues, and then when we do that,
the module system actually gets in the way.
Bill.
