Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi Simon and Vladimir, >> On Oct 19, 2021, at 4:13 PM, Pariksheet Nanda wrote: >> The trouble is, R's installation process will only copy compiled files from ./libs/ that have exactly the extension ".so" and files ending with ".so.1" are ignored. --snip-- >> library(tsshmm) >> ... >> Error: package or namespace load failed for 'tsshmm' indyn.load(file, DLLpath = DLLpath, ...): >> unable to load shared object '/home/omsai/R/x86_64-pc-linux-gnu-library/4.1/tsshmm/libs/tsshmm.so': >> libghmm.so.1: cannot open shared object file: No such file or directory > Pariksheet On 10/19/21 5:00 AM, Simon Urbanek wrote: dynamic linking won't work, compile a static version with PIC enabled. If the subproject is autoconf-compatible this means using --disable-shared --with-pic. Then you only need to add libfoo.a to your PKG_LIBS. > > Simon On 10/19/21 6:39 AM, Vladimir Dergachev wrote: > > The simplest thing to try is to compile the library statically and link it > into your package. No extra files - no trouble. > > You can also try renaming the file from *.so.1 to *.so. > > Vladimir Dergachev Thank you both for your suggestions! I will link the code statically with PIC per your consensus. I found when linking the R package library, one also has to link the dependencies of the static library; in this case libghmm depends on libxml-2.0 > 2.6 and so I have to link libxml2 to my R package library after finding libxml2 with pkg-config. Thanks for the quick replies, Pariksheet __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi folks, On 10/18/21 11:13 PM, Pariksheet Nanda wrote: The trouble is, R's installation process will only copy compiled files from ./libs/ that have exactly the extension ".so" and files ending with ".so.1" are ignored. --snip-- So is there some mechanism to copy arbitrary files or symlinks to the final install location? I prefer not to patch upstreams Makefile.am to remove their -version-info, but currently that's the only option I can think of. It turns out removing -version-info or setting it to 0.0.0 will still try to link against libghmm.so.0 which is still problematic. I don't see how to disable libtool's versioning. So after playing around, the only way I can think of doing is is eliminating the dependency file by compiling it statically and linking it with the dynamic library, because when I try merging the 2 dynamic libraries with libtool it gives the same error of not finding "libghmm.so.1". I have a patch that works on my Debian machines, but not yet on the Ubuntu CI Image: https://gitlab.com/coregenomics/tsshmm/-/commit/e9608f01deb7baa13684d2bd65fe11e93f6c2e08 Also pasting the short diff below for search-ability. Pariksheet Pariksheet $ GIT_PAGER=cat git log -1 --patch commit e9608f01deb7baa13684d2bd65fe11e93f6c2e08 (HEAD -> master, origin/master, origin/HEAD) Author: Pariksheet Nanda Date: Tue Oct 19 01:43:09 2021 -0400 BLD: Link bundled dependency statically to workaround load errors diff --git a/configure.ac b/configure.ac index 87b4d31..4d0be6e 100644 --- a/configure.ac +++ b/configure.ac @@ -135,8 +135,6 @@ AS_IF([test x$with_ghmm_strategy = x], ) # AS_IF ) # AS_IF -AC_SUBST(GHMM_LIBS, -lghmm) - # If GHMM_ROOT was provided, set the header and library paths. # # Check for the existance of include/ and lib/ sub-directories and if both are @@ -180,7 +178,10 @@ AS_IF(test x$found_ghmm_system != xyes && AM_CONDITIONAL(BUNDLED_GHMM, true) [AX_SUBDIRS_CONFIGURE([src/ghmm-0.9-rc3], [[CFLAGS=$CFLAGS], - [--enable-gsl=no], + [--enable-static], + [--disable-shared], + [--with-pic], + [--enable-gsl=no], [--disable-gsltest], [--with-rng=mt], [--with-python=no], @@ -191,8 +192,14 @@ AS_IF(test x$found_ghmm_system != xyes && [AS_IF([test -d $GHMM_ROOT], [], AC_MSG_FAILURE(Directory of bundled GHMM not found.))] [AC_SUBST(GHMM_CPPFLAGS, ["-I$GHMM_ROOT/.."])] - # Using -rpath=. prefers the bundled over any system installation. - [AC_SUBST(GHMM_LDFLAGS, ["-Wl,-rpath=. -L$GHMM_ROOT/.libs"])] + # We don't need GMM_LIBS or GHMM_LDFLAGS because we can directly merge + # libraries using tsshmm_la_LIBADD per + # https://stackoverflow.com/a/13978856 and + # https://www.gnu.org/software/automake/manual/html_node/Libtool-Convenience-Libraries.html + # + # However we now need to link against libghmm's libxml2 dependency + # because we're merging libraries. + [PKG_CHECK_MODULES([LIBXML2], [libxml-2.0 >= 2.6])] AC_MSG_NOTICE(Applying patches to GHMM to fix errors and warnings from "R CMD check") # Patch bug in upstream's configure bug: # @@ -239,7 +246,9 @@ AS_IF(test x$found_ghmm_system != xyes && #include ' src/ghmm-0.9-rc3/tests/mcmc.c [touch -r src/ghmm-0.9-rc3/tests/mcmc.c{.bak,}] [diff -u src/ghmm-0.9-rc3/tests/mcmc.c{.bak,}] - AC_MSG_NOTICE(Finished patching GHMM) + AC_MSG_NOTICE(Finished patching GHMM), + # Only link if we're not using the static bundled dependency. + [AC_SUBST(GHMM_LIBS, -lghmm)] ) # AS_IF # Variables for Doxygen. diff --git a/src/Makefile.am b/src/Makefile.am index 617a4e7..0e38b4a 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -9,18 +9,19 @@ endif lib_LTLIBRARIES= tsshmm.la tsshmm_la_CFLAGS = $(PKG_CFLAGS) tsshmm_la_CPPFLAGS = $(PKG_CPPFLAGS) +if BUNDLED_GHMM +tsshmm_la_LIBADD = @GHMM_ROOT@/libghmm.la +tsshmm_la_LDFLAGS = -module $(PKG_LIBS) @LIBXML2_LIBS@ +else tsshmm_la_LDFLAGS = -module $(PKG_LIBS) +endif tsshmm_la_SOURCES = R_init_tsshmm.c R_wrap_tsshmm.c models.c \ simulate.c train.c tss.c viterbi.c ACLOCAL_AMFLAGS = -I tools # Hook that runs after the default "all" rule. -if BUNDLED_GHMM -all-local : tsshmm.so libghmm.so -else all-local : tsshmm.so -endif # One of the limitations with POSIX-compliant `make` is not being able to # specify multiple outputs from a single rule. Therefore, even though libtool @@ -30,14 +31,8 @@ tsshmm.so : tsshmm.la cp -av .libs/tsshmm.so.0.0.0 $@ chmod -x $@ -if BUNDLED_GHMM -libghm
[R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi folks, My package [1] depends on a C library libghmm-dev that's available in many GNU/Linux package managers. However, it's not available on all platforms and if this dependency is not installed, my autoconf generated configure script defaults to falling back to compiling and installing the dependency from my bundled copy of upstream's pristine source tarball [2]. Now, because upstream uses automake which in turn uses libtool, I also use automake and libtool in my build process to hook into their build artifacts using SUBDIRS and *-local automake rules [3]. As you may know libtool appends `-version-info` to its generated shared libraries in the form "libfoo.so.1.2.3". I'm linking against the bundled library which only sets the first value, namely libghmm.so.1. The trouble is, R's installation process will only copy compiled files from ./libs/ that have exactly the extension ".so" and files ending with ".so.1" are ignored. My current workaround is to set -Wl,-rpath to the location of the generated ".so.1" file. This allows the installation process to complete and sneakily pass the 2 canonical tests: ** testing if installed package can be loaded from temporary location ---snip--- ** testing if installed package can be loaded from final location However, not surprisingly, when I try to load the library from the final location after the temporary directory has been deleted it fails with: library(tsshmm) ... Error: package or namespace load failed for 'tsshmm' indyn.load(file, DLLpath = DLLpath, ...): unable to load shared object '/home/omsai/R/x86_64-pc-linux-gnu-library/4.1/tsshmm/libs/tsshmm.so': libghmm.so.1: cannot open shared object file: No such file or directory I can rename the dependency from ".so.1" to ".so" to also get the dependent library to the final location. But it still fails with the above error because the library links against the ".so.1" file and I would need an accompanying symlink. I tried creating a symlink but can't think of how to get the symlink to the final location. If my Makefile writes the symlink into ./inst/libs/libghmm.so.1 during compile time it is not actually installed; perhaps because the ./inst/ sub-directories are only copied earlier on when staging and are ignored later? If I were to create that dangling symlink inside ./inst/libs/ instead of generating it later during compile time, devtools::install() complains about the broken symlink with: cp: cannot stat 'tsshmm/inst/libs/libghmm.so.1': No such file or directory So is there some mechanism to copy arbitrary files or symlinks to the final install location? I prefer not to patch upstreams Makefile.am to remove their -version-info, but currently that's the only option I can think of. I can't find helpful discussion surrounding this in the mailing list archives. Last week when I've posted for help with my package on another issue on the Bioconductor mailing list, one adventurous soul tried installing the package using `remotes::install_gitlab("coregenomics/tsshmm")`. This won't work because I haven't committed the generated autotools files; if anyone wants to play with it, you'll have to follow the 2 additional steps run by the continuous integration script, namely, unpacking ./src/ghmm-0.9-rc3.tar.gz into ./src/ and running `autoreconf -ivf` in the package's top-level directory where configure.ac is located. Any help appreciated, Pariksheet [1] https://gitlab.com/coregenomics/tsshmm [2] The only patches I apply to the dependency are to fix 2 bugs for compiling, and to remedy a warning severe enough to be flagged by `R CMD check`. [3] You can see my Makefile.am here: https://gitlab.com/coregenomics/tsshmm/-/blob/master/src/Makefile.am __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel