Hi David,Thanks - this is useful. At the very least it will point out missing dependencies at link time rather than run time. That is, provided it is used correctly - see the issue below.
I'm glad to see that Poly/ML does not include symbols in the object file that are referenced from ML but not required, e.g. because they occur in code that is never reached. That's a useful property because it allows a library of ML bindings to support more functions than actually provided by the installed version of a binary library and required by the application. Hopefully it will be possible to keep that property.
I have run into a general issue. After redefining a local version of the getSymbol function to use externalFunctionSymbol internally, I found that programs compiled and linked fine but seg faulted before entering the main ML function. This was due to the scope in which a symbol was declared using externalFunctionSymbol. The attached example call_c_test_13.tar.gz gives a simple example of the problem. The external function symbol isn't being declared until run time so this code is inadvertently depending on a feature of dynamic linking. I can see that use of externalFunctionSymbol is inherently more restricted that getSymbol: externalFunctionSymbol must be applied to a symbol name at compile time but the symbol cannot be used until run time, post-linking. Are there any sort of checks that could be done to warn users if they get this wrong? I can see it's not straightforward because run time is itself compile time for an exported function that is subsequently linked. At the very least, could an exception be raised rather than seg faulting if there is an attempt to use an unresolved symbol?
Finally, is there a reason you chose the name "externalFunctionSymbol" rather than, say, "externalSymbol"? It seems to work fine for non-functions too - see attached example call_c_test_14.tar.gz .
Phil On 12/09/18 21:18, David Matthews wrote:
I've been thinking about adding direct external references as an alternative to dynamic loading of libraries. The mechanism is all there now because the Poly/ML run-time library is linked to the exported code through external references.I've added Foreign.externalFunctionSymbol : string -> symbolThis creates an external reference which is filled in when the code is exported. The actual value is resolved by linking with the appropriate library. For your example that means changing it to replace references to getSymbol to externalFunctionSymbol and removing the calls to loadLibrary. i.e.open Foreign; type gsl_matrix_t = Memory.voidStar; (* for demonstration only *) localval call1 = buildCall2((externalFunctionSymbol "gsl_matrix_calloc"),(cInt,cInt), cPointer) val call2 = buildCall4((externalFunctionSymbol "gsl_matrix_set"), (cPointer,cInt,cInt,cDouble), cVoid) val call3 = buildCall3((externalFunctionSymbol "gsl_linalg_exponential_ss"), (cPointer, cPointer, cDouble), cVoid) in fun gsl_matrix_alloc (size1,size2) = call1(size1,size2)It seems to handle your example fine when the code is exported using "polyc" and then linked with the libraries.I've committed this to a separate branch "ExternalBranch" on GitHub rather than "master". I'd like some feedback on how useful this is and whether there are any problems before merging it.David On 09/09/2018 21:15, Mark Clements wrote:I am enjoying using polyml's Foreign structure. However, in using the GNU Scientific Library (GSL), I have run into a problem with linkage. The GSL library (libgsl.so) calls standard cBLAS matrix functions with unknown symbols to allow the user to link with different cBLAS implementations (NB: GSL provides an unoptimised cBLAS implementation named libgslcblas.so). When I call a GSL function that depends on the cBLAS functions, I get a symbol not defined error. I have tried unsuccessfully (a) loading the libgslcblas library either before or after loading libgsl, (b) changing the loadLibrary implementation to use dlopen with RTLD_LAZY | RTLD_GLOBAL, and (c) compiling the SML code to an object file and then linking against the libraries. How could this be made to work? Sincerely, Mark. PS test code for (a) is: open Foreign; val gsl = loadLibrary "/usr/lib/x86_64-linux-gnu/libgsl.so"; val gslcblas = loadLibrary "/usr/lib/x86_64-linux-gnu/libgslcblas.so"; type gsl_matrix_t = Memory.voidStar; (* for demonstration only *) local val call1 = buildCall2((getSymbol gsl "gsl_matrix_calloc"), (cInt,cInt), cPointer) val call2 = buildCall4((getSymbol gsl "gsl_matrix_set"), (cPointer,cInt,cInt,cDouble), cVoid) val call3 = buildCall3((getSymbol gsl "gsl_linalg_exponential_ss"), (cPointer, cPointer, cDouble), cVoid) in fun gsl_matrix_alloc (size1,size2) = call1(size1,size2) fun gsl_matrix_set (ptr : gsl_matrix_t, i, j, x) = call2(ptr, i, j, x) fun gsl_matrix (arr : real Array2.array) = let val size1 = Array2.nRows arr val size2 = Array2.nCols arr val ptr = gsl_matrix_alloc(size1, size2) val _ = Array2.tabulate Array2.RowMajor (size1, size2, fn (i,j) => gsl_matrix_set(ptr, i, j, Array2.sub(arr, i, j))) in ptr end fun gsl_mexp (arr : real Array2.array) = let val size1 = Array2.nRows arr val size2 = Array2.nCols arr val ptr = gsl_matrix arr val ptr2 = gsl_matrix_alloc(size1, size2) val _ = call3(ptr, ptr2, 0.01) in ptr2 end end; fun main() = let val a = Array2.fromList [[1.0,2.0],[3.0,4.0]]; val ptr = gsl_matrix(a); val m = gsl_mexp(a) (* fails *) in () end; (* symbol lookup error: /usr/lib/x86_64-linux-gnu/libgsl.so: undefined symbol: cblas_dgemm *) For (c): polyc -c test.sml g++ test.o -L/usr/lib/x86_64-linux-gnu -lgsl -lgslcblas `pkg-config --libs polyml` ./a.out # same error _______________________________________________ polyml mailing list polyml@inf.ed.ac.uk http://lists.inf.ed.ac.uk/mailman/listinfo/polyml_______________________________________________ polyml mailing list polyml@inf.ed.ac.uk http://lists.inf.ed.ac.uk/mailman/listinfo/polyml
call_c_test_13.tar.gz
Description: application/gzip
call_c_test_14.tar.gz
Description: application/gzip
_______________________________________________ polyml mailing list polyml@inf.ed.ac.uk http://lists.inf.ed.ac.uk/mailman/listinfo/polyml
