Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Pariksheet,
dynamic linking won't work, compile a static version with PIC enabled. If the
subproject is autoconf-compatible this means using --disable-shared --with-pic.
Then you only need to add libfoo.a to your PKG_LIBS.
Cheers,
Simon
> On Oct 19, 2021, at 4:13 PM, Pariksheet Nanda
> wrote:
>
> Hi folks,
>
> My package [1] depends on a C library libghmm-dev that's available in many
> GNU/Linux package managers. However, it's not available on all platforms and
> if this dependency is not installed, my autoconf generated configure script
> defaults to falling back to compiling and installing the dependency from my
> bundled copy of upstream's pristine source tarball [2]. Now, because
> upstream uses automake which in turn uses libtool, I also use automake and
> libtool in my build process to hook into their build artifacts using SUBDIRS
> and *-local automake rules [3].
>
> As you may know libtool appends `-version-info` to its generated shared
> libraries in the form "libfoo.so.1.2.3". I'm linking against the bundled
> library which only sets the first value, namely libghmm.so.1.
>
> The trouble is, R's installation process will only copy compiled files from
> ./libs/ that have exactly the extension ".so" and files ending with ".so.1"
> are ignored.
>
> My current workaround is to set -Wl,-rpath to the location of the generated
> ".so.1" file. This allows the installation process to complete and sneakily
> pass the 2 canonical tests:
>
>
> ** testing if installed package can be loaded from temporary location
> ---snip---
> ** testing if installed package can be loaded from final location
>
>
> However, not surprisingly, when I try to load the library from the final
> location after the temporary directory has been deleted it fails with:
>
>
> library(tsshmm)
> ...
> Error: package or namespace load failed for 'tsshmm' indyn.load(file, DLLpath
> = DLLpath, ...):
> unable to load shared object
> '/home/omsai/R/x86_64-pc-linux-gnu-library/4.1/tsshmm/libs/tsshmm.so':
> libghmm.so.1: cannot open shared object file: No such file or directory
>
>
> I can rename the dependency from ".so.1" to ".so" to also get the dependent
> library to the final location. But it still fails with the above error
> because the library links against the ".so.1" file and I would need an
> accompanying symlink. I tried creating a symlink but can't think of how to
> get the symlink to the final location. If my Makefile writes the symlink
> into ./inst/libs/libghmm.so.1 during compile time it is not actually
> installed; perhaps because the ./inst/ sub-directories are only copied
> earlier on when staging and are ignored later? If I were to create that
> dangling symlink inside ./inst/libs/ instead of generating it later during
> compile time, devtools::install() complains about the broken symlink with:
>
>
> cp: cannot stat 'tsshmm/inst/libs/libghmm.so.1': No such file or directory
>
>
> So is there some mechanism to copy arbitrary files or symlinks to the final
> install location? I prefer not to patch upstreams Makefile.am to remove
> their -version-info, but currently that's the only option I can think of. I
> can't find helpful discussion surrounding this in the mailing list archives.
>
> Last week when I've posted for help with my package on another issue on the
> Bioconductor mailing list, one adventurous soul tried installing the package
> using `remotes::install_gitlab("coregenomics/tsshmm")`. This won't work
> because I haven't committed the generated autotools files; if anyone wants to
> play with it, you'll have to follow the 2 additional steps run by the
> continuous integration script, namely, unpacking ./src/ghmm-0.9-rc3.tar.gz
> into ./src/ and running `autoreconf -ivf` in the package's top-level
> directory where configure.ac is located.
>
> Any help appreciated,
>
> Pariksheet
>
>
> [1] https://gitlab.com/coregenomics/tsshmm
>
> [2] The only patches I apply to the dependency are to fix 2 bugs for
> compiling, and to remedy a warning severe enough to be flagged by `R CMD
> check`.
>
> [3] You can see my Makefile.am here:
> https://gitlab.com/coregenomics/tsshmm/-/blob/master/src/Makefile.am
>
> __
> R-package-devel@r-project.org mailing list
> https://stat.ethz.ch/mailman/listinfo/r-package-devel
>
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi Simon and Vladimir, >> On Oct 19, 2021, at 4:13 PM, Pariksheet Nanda wrote: >> The trouble is, R's installation process will only copy compiled files from ./libs/ that have exactly the extension ".so" and files ending with ".so.1" are ignored. --snip-- >> library(tsshmm) >> ... >> Error: package or namespace load failed for 'tsshmm' indyn.load(file, DLLpath = DLLpath, ...): >> unable to load shared object '/home/omsai/R/x86_64-pc-linux-gnu-library/4.1/tsshmm/libs/tsshmm.so': >> libghmm.so.1: cannot open shared object file: No such file or directory > Pariksheet On 10/19/21 5:00 AM, Simon Urbanek wrote: dynamic linking won't work, compile a static version with PIC enabled. If the subproject is autoconf-compatible this means using --disable-shared --with-pic. Then you only need to add libfoo.a to your PKG_LIBS. > > Simon On 10/19/21 6:39 AM, Vladimir Dergachev wrote: > > The simplest thing to try is to compile the library statically and link it > into your package. No extra files - no trouble. > > You can also try renaming the file from *.so.1 to *.so. > > Vladimir Dergachev Thank you both for your suggestions! I will link the code statically with PIC per your consensus. I found when linking the R package library, one also has to link the dependencies of the static library; in this case libghmm depends on libxml-2.0 > 2.6 and so I have to link libxml2 to my R package library after finding libxml2 with pkg-config. Thanks for the quick replies, Pariksheet __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
The simplest thing to try is to compile the library statically and link it into your package. No extra files - no trouble. You can also try renaming the file from *.so.1 to *.so. best Vladimir Dergachev __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi folks,
On 10/18/21 11:13 PM, Pariksheet Nanda wrote:
The trouble is, R's installation process will only copy compiled files
from ./libs/ that have exactly the extension ".so" and files ending with
".so.1" are ignored.
--snip--
So is there some mechanism to copy arbitrary files or symlinks to the
final install location? I prefer not to patch upstreams Makefile.am to
remove their -version-info, but currently that's the only option I can
think of.
It turns out removing -version-info or setting it to 0.0.0 will still
try to link against libghmm.so.0 which is still problematic. I don't
see how to disable libtool's versioning.
So after playing around, the only way I can think of doing is is
eliminating the dependency file by compiling it statically and linking
it with the dynamic library, because when I try merging the 2 dynamic
libraries with libtool it gives the same error of not finding
"libghmm.so.1". I have a patch that works on my Debian machines, but
not yet on the Ubuntu CI Image:
https://gitlab.com/coregenomics/tsshmm/-/commit/e9608f01deb7baa13684d2bd65fe11e93f6c2e08
Also pasting the short diff below for search-ability.
Pariksheet
Pariksheet
$ GIT_PAGER=cat git log -1 --patch
commit e9608f01deb7baa13684d2bd65fe11e93f6c2e08 (HEAD -> master,
origin/master, origin/HEAD)
Author: Pariksheet Nanda
Date: Tue Oct 19 01:43:09 2021 -0400
BLD: Link bundled dependency statically to workaround load errors
diff --git a/configure.ac b/configure.ac
index 87b4d31..4d0be6e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -135,8 +135,6 @@ AS_IF([test x$with_ghmm_strategy = x],
) # AS_IF
) # AS_IF
-AC_SUBST(GHMM_LIBS, -lghmm)
-
# If GHMM_ROOT was provided, set the header and library paths.
#
# Check for the existance of include/ and lib/ sub-directories and if
both are
@@ -180,7 +178,10 @@ AS_IF(test x$found_ghmm_system != xyes &&
AM_CONDITIONAL(BUNDLED_GHMM, true)
[AX_SUBDIRS_CONFIGURE([src/ghmm-0.9-rc3],
[[CFLAGS=$CFLAGS],
- [--enable-gsl=no],
+ [--enable-static],
+ [--disable-shared],
+ [--with-pic],
+ [--enable-gsl=no],
[--disable-gsltest],
[--with-rng=mt],
[--with-python=no],
@@ -191,8 +192,14 @@ AS_IF(test x$found_ghmm_system != xyes &&
[AS_IF([test -d $GHMM_ROOT], [],
AC_MSG_FAILURE(Directory of bundled GHMM not found.))]
[AC_SUBST(GHMM_CPPFLAGS, ["-I$GHMM_ROOT/.."])]
- # Using -rpath=. prefers the bundled over any system installation.
- [AC_SUBST(GHMM_LDFLAGS, ["-Wl,-rpath=. -L$GHMM_ROOT/.libs"])]
+ # We don't need GMM_LIBS or GHMM_LDFLAGS because we can directly
merge
+ # libraries using tsshmm_la_LIBADD per
+ # https://stackoverflow.com/a/13978856 and
+ #
https://www.gnu.org/software/automake/manual/html_node/Libtool-Convenience-Libraries.html
+ #
+ # However we now need to link against libghmm's libxml2 dependency
+ # because we're merging libraries.
+ [PKG_CHECK_MODULES([LIBXML2], [libxml-2.0 >= 2.6])]
AC_MSG_NOTICE(Applying patches to GHMM to fix errors and
warnings from "R CMD check")
# Patch bug in upstream's configure bug:
#
@@ -239,7 +246,9 @@ AS_IF(test x$found_ghmm_system != xyes &&
#include ' src/ghmm-0.9-rc3/tests/mcmc.c
[touch -r src/ghmm-0.9-rc3/tests/mcmc.c{.bak,}]
[diff -u src/ghmm-0.9-rc3/tests/mcmc.c{.bak,}]
- AC_MSG_NOTICE(Finished patching GHMM)
+ AC_MSG_NOTICE(Finished patching GHMM),
+ # Only link if we're not using the static bundled dependency.
+ [AC_SUBST(GHMM_LIBS, -lghmm)]
) # AS_IF
# Variables for Doxygen.
diff --git a/src/Makefile.am b/src/Makefile.am
index 617a4e7..0e38b4a 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -9,18 +9,19 @@ endif
lib_LTLIBRARIES= tsshmm.la
tsshmm_la_CFLAGS = $(PKG_CFLAGS)
tsshmm_la_CPPFLAGS = $(PKG_CPPFLAGS)
+if BUNDLED_GHMM
+tsshmm_la_LIBADD = @GHMM_ROOT@/libghmm.la
+tsshmm_la_LDFLAGS = -module $(PKG_LIBS) @LIBXML2_LIBS@
+else
tsshmm_la_LDFLAGS = -module $(PKG_LIBS)
+endif
tsshmm_la_SOURCES = R_init_tsshmm.c R_wrap_tsshmm.c models.c \
simulate.c train.c tss.c viterbi.c
ACLOCAL_AMFLAGS = -I tools
# Hook that runs after the default "all" rule.
-if BUNDLED_GHMM
-all-local : tsshmm.so libghmm.so
-else
all-local : tsshmm.so
-endif
# One of the limitations with POSIX-compliant `make` is not being able to
# specify multiple outputs from a single rule. Therefore, even though
libtool
@@ -30,14 +31,8 @@ tsshmm.so : tsshmm.la
cp -av .libs/tsshmm.so.0.0.0 $@
chmod -x $@
-if BUNDLED_GHMM
-libghmm.so : @GHMM_ROOT@/libghmm.la
- cp -av @GHMM_ROOT@/.libs/libghmm.so.1.0.0 $@
- chmod -x $@
-endif
-
clean-local :
- rm -f tsshmm.so
[R-pkg-devel] How does one install a libtool generated libfoo.so.1 file into ./libs/?
Hi folks,
My package [1] depends on a C library libghmm-dev that's available in
many GNU/Linux package managers. However, it's not available on all
platforms and if this dependency is not installed, my autoconf generated
configure script defaults to falling back to compiling and installing
the dependency from my bundled copy of upstream's pristine source
tarball [2]. Now, because upstream uses automake which in turn uses
libtool, I also use automake and libtool in my build process to hook
into their build artifacts using SUBDIRS and *-local automake rules [3].
As you may know libtool appends `-version-info` to its generated shared
libraries in the form "libfoo.so.1.2.3". I'm linking against the
bundled library which only sets the first value, namely libghmm.so.1.
The trouble is, R's installation process will only copy compiled files
from ./libs/ that have exactly the extension ".so" and files ending with
".so.1" are ignored.
My current workaround is to set -Wl,-rpath to the location of the
generated ".so.1" file. This allows the installation process to
complete and sneakily pass the 2 canonical tests:
** testing if installed package can be loaded from temporary location
---snip---
** testing if installed package can be loaded from final location
However, not surprisingly, when I try to load the library from the final
location after the temporary directory has been deleted it fails with:
library(tsshmm)
...
Error: package or namespace load failed for 'tsshmm' indyn.load(file,
DLLpath = DLLpath, ...):
unable to load shared object
'/home/omsai/R/x86_64-pc-linux-gnu-library/4.1/tsshmm/libs/tsshmm.so':
libghmm.so.1: cannot open shared object file: No such file or directory
I can rename the dependency from ".so.1" to ".so" to also get the
dependent library to the final location. But it still fails with the
above error because the library links against the ".so.1" file and I
would need an accompanying symlink. I tried creating a symlink but
can't think of how to get the symlink to the final location. If my
Makefile writes the symlink into ./inst/libs/libghmm.so.1 during compile
time it is not actually installed; perhaps because the ./inst/
sub-directories are only copied earlier on when staging and are ignored
later? If I were to create that dangling symlink inside ./inst/libs/
instead of generating it later during compile time, devtools::install()
complains about the broken symlink with:
cp: cannot stat 'tsshmm/inst/libs/libghmm.so.1': No such file or directory
So is there some mechanism to copy arbitrary files or symlinks to the
final install location? I prefer not to patch upstreams Makefile.am to
remove their -version-info, but currently that's the only option I can
think of. I can't find helpful discussion surrounding this in the
mailing list archives.
Last week when I've posted for help with my package on another issue on
the Bioconductor mailing list, one adventurous soul tried installing the
package using `remotes::install_gitlab("coregenomics/tsshmm")`. This
won't work because I haven't committed the generated autotools files; if
anyone wants to play with it, you'll have to follow the 2 additional
steps run by the continuous integration script, namely, unpacking
./src/ghmm-0.9-rc3.tar.gz into ./src/ and running `autoreconf -ivf` in
the package's top-level directory where configure.ac is located.
Any help appreciated,
Pariksheet
[1] https://gitlab.com/coregenomics/tsshmm
[2] The only patches I apply to the dependency are to fix 2 bugs for
compiling, and to remedy a warning severe enough to be flagged by `R CMD
check`.
[3] You can see my Makefile.am here:
https://gitlab.com/coregenomics/tsshmm/-/blob/master/src/Makefile.am
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
