Re: library not found in build on macos
On Wed, Nov 20, 2024 at 06:57:11PM +, Gavin Smith wrote: > Also, providing an extra flag when compiling adds to the RUNPATH in > the .so file: > > cc -shared -fPIC -DPIC -o .libs/Parsetexi.so > parsetexi/.libs/Parsetexi_la-Parsetexi.o > -L/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs -L./.libs -ltexinfoxs > -L/usr/local/lib -ltexinfo -lintl -lunistring -liconv > -fstack-protector-strong -O2 -Wl,-R/usr/libdata/perl5/mips64-openbsd/CORE > -Wl,-R /home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs > > I added the final -Wl,-R option from what was output by automake/libtool. Further to this, "ldd" works with the resulting file: $ ldd .libs/Parsetexi.soc .libs/Parsetexi.so: StartEnd Type Open Ref GrpRef Name 0010902b8000 0010902e4000 dlib 10 0 /home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs/Parsetexi.so 00108c2e 00108c36 rlib 01 0 /home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs/libtexinfoxs.so.0.0 00106f17c000 00106f2b4000 rlib 02 0 /home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs/libtexinfo.so.0.0 0010ecce8000 0010ecd3 rlib 03 0 /usr/local/lib/libintl.so.8.0 0010ce7dc000 0010ce9cc000 rlib 03 0 /usr/local/lib/libunistring.so.0.1 0010fdf98000 0010fe0e8000 rlib 05 0 /usr/local/lib/libiconv.so.7.1
Re: library not found in build on macos
On Wed, Nov 20, 2024 at 11:00:32AM +0100, Patrice Dumas wrote:
> On Tue, Nov 19, 2024 at 09:04:19PM +, Gavin Smith wrote:
> > Dynamic section at offset 0x390 contains 27 entries:
> > TagType Name/Value
> > 0x0001 (NEEDED) Shared library:
> > [libtexinfoxs.so.0.0]
> > 0x0001 (NEEDED) Shared library: [libtexinfo.so.0.0]
> > 0x0001 (NEEDED) Shared library: [libintl.so.8.0]
> > 0x0001 (NEEDED) Shared library:
> > [libunistring.so.0.1]
> > 0x0001 (NEEDED) Shared library: [libiconv.so.7.1]
> > 0x000f (RPATH) Library rpath:
> > [/usr/libdata/perl5/mips64-openbsd/CORE]
> > 0x001d (RUNPATH)Library runpath:
> > [/usr/libdata/perl5/mips64-openbsd/CORE]
> >
> > There is no mention of any .libs directory anywhere.
>
> Indeed, and because of that I can't see how it could work to find the
> libraries in the build.
> ...
> > No mention of any "/usr/local/lib/texi2any" directory.
>
> Did you try launching texi2any with TEXINFO_XS=debug? Since there is no
> RUNPATH/RPATH to /usr/local/lib/texi2any I would assume that the XS
> modules could not be loaded.
Yes, but only for the uninstalled version. On the system I'm using,
I don't have permission to install under /usr/local (and I can't change
the --prefix because the iconv libraries that are needed for most of
the XS modules are installed under /usr/local). The XS modules without
library dependencies load successfully:
checking ../../tp/maintain/lib/Text-Unidecode/lib/MiscXS.la
checking ../../tp/maintain/lib/Unicode-EastAsianWidth/lib/MiscXS.la
checking ../../tp/maintain/lib/libintl-perl/lib/MiscXS.la
checking ../tp/Texinfo/XS/MiscXS.la
found ../tp/Texinfo/XS/MiscXS.la
../tp/Texinfo/XS/.libs/MiscXS.so loaded
looking for boot_Texinfo__MiscXS
trying to call boot_Texinfo__MiscXS...
...succeeded
- and later:
checking ../../tp/maintain/lib/Text-Unidecode/lib/XSParagraph.la
checking ../../tp/maintain/lib/Unicode-EastAsianWidth/lib/XSParagraph.la
checking ../../tp/maintain/lib/libintl-perl/lib/XSParagraph.la
checking ../tp/Texinfo/XS/XSParagraph.la
found ../tp/Texinfo/XS/XSParagraph.la
../tp/Texinfo/XS/.libs/XSParagraph.so loaded
looking for boot_Texinfo__Convert__Paragraph
trying to call boot_Texinfo__Convert__Paragraph...
...succeeded
attempting to override Texinfo::Convert::Plaintext::_process_text_internal with
Texinfo::MiscXS::process_text...
...succeeded
The other modules fail.
However, I got them to work by setting LD_LIBRARY_PATH. This is an
out-of-source build:
export top_srcdir=../../ top_builddir=.. TEXINFO_DEV_SOURCE=1
LD_LIBRARY_PATH=Texinfo/XS/.libs TEXINFO_XS=debug ./texi2any
../../doc/info-stnd.texi
All the modules load successfully then.
Setting $ENV{LD_LIBRARY_PATH} inside texi2any doesn't work, though (as
confirmed by this perlmonks post from 2001:
https://www.perlmonks.org/?node_id=126587). There are various hacks and
workarounds to get it to work; the question is how easy it would be
to do and would it be worth it.
Here's what appears to be the relevant documentation from "man ld.so":
ld.so searches for shared objects in the following lists of paths:
1. LD_LIBRARY_PATH environment variable if set
2. DT_RUNPATH of the calling object if set, otherwise:
1. DT_RPATH of the calling object if set
2. DT_RPATH of the program if set
(The use of DT_RPATH is deprecated as its behaviour varies
across operating systems.)
3. default search path set by ldconfig(8)
4. /usr/lib
I'm not completely sure what the "calling object" is in the case of
library dependencies of an XS module: whether it is the Perl executable
itself, or the XS module library file. It's possible that DynaLoader
doesn't, or can't, control any of this.
It's possible this problem wouldn't arise for an uninstalled program binary
linking against uninstalled Libtool libraries, as DT_RUNPATH would be
set in the program binary (I haven't checked, though). This could be
a more common case than what we are doing with texi2any.
Also, providing an extra flag when compiling adds to the RUNPATH in
the .so file:
cc -shared -fPIC -DPIC -o .libs/Parsetexi.so
parsetexi/.libs/Parsetexi_la-Parsetexi.o
-L/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs -L./.libs -ltexinfoxs
-L/usr/local/lib -ltexinfo -lintl -lunistring -liconv -fstack-protector-strong
-O2 -Wl,-R/usr/libdata/perl5/mips64-openbsd/CORE -Wl,-R
/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs
I added the final -Wl,-R option from what was output by automake/libtool.
Then readelf shows:
0x000f (RPATH) Library rpath: [/usr/libdata/perl5/mips
64-openbsd/CORE:/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs]
0x001d (RUNPATH)Library runpa
Re: library not found in build on macos
On Mon, Nov 18, 2024 at 01:57:15PM +0200, Eli Zaretskii wrote: > > Date: Sun, 17 Nov 2024 21:21:05 +0100 > > From: Patrice Dumas > > > > % otool -L .libs/Parsetexi.so > > .libs/Parsetexi.so: > > /usr/local/lib/texi2any/libtexinfoxs.0.dylib (compatibility version > > 1.0.0, current version 1.0.0) > > /usr/local/lib/texi2any/libtexinfo.0.dylib (compatibility version > > 1.0.0, current version 1.0.0) > > /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version > > 7.0.0) > > /usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version > > 13.0.0, current version 13.0.0) > > /usr/local/opt/libunistring/lib/libunistring.5.dylib (compatibility > > version 8.0.0, current version 8.0.0) > > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current > > version 1319.0.0) > > > > Note that the /usr/local/lib/texi2any/ directory does not exist. > > Isn't the reason to this what libtool wrote in the .la file, i.e.: > > > # Directory that this library needs to be installed in: > > libdir='/usr/local/lib/texi2any' Exactly. But before the installation, libtool should also add the in build .libs directory in order for libraries to be found in build. > If you use -L on the link command line to mention the .libs directory, > does the problem go away? It is not possible to use -L on the command line, as the libraries are found through dlopen (or similar) call from within Perl code, using the Dynaloader facility, and it is not possible to add libraries directly at least on some platforms, they need to be found through the dlopen (or similar) call using information in XS objects or libraries. -- Pat
Re: library not found in build on macos
On Tue, Nov 19, 2024 at 09:04:19PM +, Gavin Smith wrote: > Dynamic section at offset 0x390 contains 27 entries: > TagType Name/Value > 0x0001 (NEEDED) Shared library: [libtexinfoxs.so.0.0] > 0x0001 (NEEDED) Shared library: [libtexinfo.so.0.0] > 0x0001 (NEEDED) Shared library: [libintl.so.8.0] > 0x0001 (NEEDED) Shared library: [libunistring.so.0.1] > 0x0001 (NEEDED) Shared library: [libiconv.so.7.1] > 0x000f (RPATH) Library rpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > 0x001d (RUNPATH)Library runpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > > There is no mention of any .libs directory anywhere. Indeed, and because of that I can't see how it could work to find the libraries in the build. > I thought I would try with "make install" to see what libtool did with > the .so files: > Then the lines in the output of readelf are the same: > > Dynamic section at offset 0x390 contains 27 entries: > TagType Name/Value > 0x0001 (NEEDED) Shared library: [libtexinfoxs.so.0.0] > 0x0001 (NEEDED) Shared library: [libtexinfo.so.0.0] > 0x0001 (NEEDED) Shared library: [libintl.so.8.0] > 0x0001 (NEEDED) Shared library: [libunistring.so.0.1] > 0x0001 (NEEDED) Shared library: [libiconv.so.7.1] > 0x000f (RPATH) Library rpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > 0x001d (RUNPATH)Library runpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > > No mention of any "/usr/local/lib/texi2any" directory. Did you try launching texi2any with TEXINFO_XS=debug? Since there is no RUNPATH/RPATH to /usr/local/lib/texi2any I would assume that the XS modules could not be loaded. > I expect that @dl_library_path in the Perl DynaLoader module could easily be > irrelevant for dependencies of libraries it loads, as this is only used > to find the library itself, not the library dependencies. I tried to add to @DynaLoader::dl_resolve_using the libtexinfo and libtexinfoxs libraries files, but it does not seems to work on GNU/Linux, so I do not think it will work better on any other platform. I test with ./configure and "make install DESTDIR=/home/dumas/tmp/dest/". I go to /home/dumas/tmp/dest/usr/bin. In that case, the runpath in ../lib/texi2any/Parsetexi.so is, as expected, set to the installation directory known by configure: 0x001d (RUNPATH)Library runpath: [/usr/local/lib/texi2any] In /home/dumas/tmp/dest/usr/bin, I run, with TEXINFO_XS set to debug: ./texi2any --html ~/src/texinfo/tp/t/input_files/simplest.texi -o - 2>&1 I get: checking ../lib/texi2any/Parsetexi.la found ../lib/texi2any/Parsetexi.la additional libraries: -ltexinfoxs|-ltexinfo found additional: ../lib/texi2any/libtexinfoxs.so|../lib/texi2any/libtexinfo.so Parsetexi: couldn't load file ../lib/texi2any/Parsetexi.so: libtexinfoxs.so.0: cannot open shared object file: No such file or directory at ../share/texi2any/Texinfo/XSLoader.pm That means that ../lib/texi2any/libtexinfo.so (which is a link to libtexinfoxs.so.0) was added to @DynaLoader::dl_resolve_using, but it did not help to find libtexinfoxs.so.0, presumably libtexinfoxs.so.0 is only searched for in /usr/local/lib/texi2any when loaded by dlopen. To me, that means that it is the dlopen or similar system that must find the libraries, we cannot use @DynaLoader::dl_resolve_using to supply other libraries. Then it means that for XS code that requires libtexinfo/libtexinfoxs, we need libtool to manage to setup things such that libtexinfo/libtexinfoxs are found by simply dlopen'ing Parsetexi.so and other similar XS objects, be it for in build testing or after installation in a directly when libraries are not searched for in the default case. So, I'd say that we cannot do anything else than filling bugs with libtool. And also that we should not wait for those issues to be fixed for the release. > > There is one thing that looks strange, a ^@ appears right after the > > texi2any error message for all the cases that fail to load, including > > macos. > > That's a null byte. Off-by-one error in the number of bytes to print? It comes from DynaLoader::dl_error(). Not a problem, actually. -- Pat
Re: library not found in build on macos
On Tue, Nov 19, 2024 at 09:04:23PM +, Gavin Smith wrote: > In the output of "readelf -a .libs/Parsetexi.so" there is: > > Dynamic section at offset 0x390 contains 27 entries: > TagType Name/Value > 0x0001 (NEEDED) Shared library: [libtexinfoxs.so.0.0] > 0x0001 (NEEDED) Shared library: [libtexinfo.so.0.0] > 0x0001 (NEEDED) Shared library: [libintl.so.8.0] > 0x0001 (NEEDED) Shared library: [libunistring.so.0.1] > 0x0001 (NEEDED) Shared library: [libiconv.so.7.1] > 0x000f (RPATH) Library rpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > 0x001d (RUNPATH)Library runpath: > [/usr/libdata/perl5/mips64-openbsd/CORE] > > There is no mention of any .libs directory anywhere. For reference, here is the corresponding output on my GNU/Linux system: Dynamic section at offset 0x5dd8 contains 28 entries: TagType Name/Value 0x0001 (NEEDED) Shared library: [libtexinfoxs.so.0] 0x0001 (NEEDED) Shared library: [libtexinfo.so.0] 0x0001 (NEEDED) Shared library: [libc.so.6] 0x000e (SONAME) Library soname: [Parsetexi.so] 0x001d (RUNPATH)Library runpath: [/home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs:/usr/local/lib/texi2any] When installed, this becomes: Dynamic section at offset 0x5dd8 contains 28 entries: TagType Name/Value 0x0001 (NEEDED) Shared library: [libtexinfoxs.so.0] 0x0001 (NEEDED) Shared library: [libtexinfo.so.0] 0x0001 (NEEDED) Shared library: [libc.so.6] 0x000e (SONAME) Library soname: [Parsetexi.so] 0x001d (RUNPATH)Library runpath: [/usr/local/lib/texi2any] i.e. the build directory is removed from RUNPATH.
Re: library not found in build on macos
On Tue, Nov 19, 2024 at 09:04:19PM +, Gavin Smith wrote: > While testing texi2any, I accidentally found a bug: > > $ export top_srcdir=../../ top_builddir=.. TEXINFO_DEV_SOURCE=1 > $ ./texi2any ../doc/info-stnd.info > texi2any: warning: input file info-stnd.info; did you mean info-stnd.texi? > You found a bug: Should be at end of line but have [inde] > ' > > Additional information: > last location: info-stnd.info:2106 > context_stack: (ct_base) > current: (before_node_section)[C1078] at ../../tp/Texinfo/ParserNonXS.pm line > 1144. > Died at ../../tp/Texinfo/ParserNonXS.pm line 7647. I could not reproduce. At info-stnd.info:2106 there is: ^@^H[index^@^H] There is therefore a NULL byte at that point, I got something similar by running texi2any on a binary file, this time rigth after ^T. I could't reproduce with a simple input file. With another binary file it is after ^Q. I only reproduce with TEXINFO_XS=omit. So I guess there is something unexpected going on with a regexp in Perl. I could investigate more in a few days. -- Pat
Re: library not found in build on macos
On Sun, Nov 17, 2024 at 10:14:37PM +0100, Patrice Dumas wrote:
> On Sun, Nov 17, 2024 at 09:37:23PM +0100, Patrice Dumas wrote:
> > I have added a report for all the platforms tested in the CI. It
> > appears that cygwin, both 32 and 64 bits and OpenBSD unexpectedly cannot
> > load Parsetexi.so/dll (in addition to macos 12 and macos 13). MiscXS is
> > loaded in all the platform, so it is not a basic issue with XS, but
> > something else.
>
> I couldn't find what was going on. The directories may not be the right
> ones, but otherwise I can't see what's wrong. For openbsd, for example,
> the link commands looks good:
>
> libtool: link: cc -shared -fPIC -DPIC -o .libs/Parsetexi.so
> parsetexi/.libs/Parsetexi_la-Parsetexi.o
> -L/home/runner/work/ci-check/ci-check/texinfo-7.1.91.20241117/build/tp/Texinfo/XS/.libs
> -L/usr/local/lib -L./.libs -ltexinfoxs -ltexinfo -fstack-protector-strong
> -O2 -Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE
>
Here are some notes from my attempts to investigate the issue this
evening, although I did not solve the issue and I feel like it may be
beyond us to fix without asking for help from experts on libtool and/or
experts on the operating systems in question.
As you said, MiscXS is loaded, as is TestXS when the configure script
runs (otherwise the XS modules would be disabled). So it appears to be
an issue with the library dependencies of the XS modules themselves.
On my GNU/Linux system (Linux Mint 21.2 Xfce, Linux 5.19.0 x86_64), the
corresponding command is:
libtool: link: x86_64-linux-gnu-gcc -shared -fPIC -DPIC
parsetexi/.libs/Parsetexi_la-Parsetexi.o -Wl,-rpath
-Wl,/home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs -Wl,-rpath
-Wl,/usr/local/lib/texi2any -L/home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs
./.libs/libtexinfoxs.so
/home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs/libtexinfo.so ./.libs/libtexinfo.so
-O2 -g -Wl,-E -Wl,-soname -Wl,Parsetexi.so -o .libs/Parsetexi.so
libtool: link: ( cd ".libs" && rm -f "Parsetexi.la" && ln -s "../Parsetexi.la"
"Parsetexi.la" )
What may explain the difference is the "-ltexinfoxs -ltexinfo" flags in
the macos case. This does not appear in the GNU/Linux output: instead,
the libraries are listed by name ("./.libs/libtexinfoxs.so"). (Or this
could just be an expected difference due to a different compiler being used.)
I got access to an OpenBSD system (OpenBSD 7.5, Cavium OCTEON (rev 0.2)) as
you said it was also an issue on OpenBSD. Here the output from building
Parsetexi.la is as follows:
$ rm Parsetexi.la
$ make Parsetexi.la
/bin/sh ./libtool --tag=CC--mode=link cc -DNO_LOCALE_NUMERIC
-DNO_LOCALE_COLLATE -fno-strict-aliasing -fno-delete-null-pointer-checks -pipe
-fstack-protector-strong -I/usr/local/include -O2 -DPIC -fpic-avoid-version
-module -Wl,-R/usr/libdata/perl5/mips64-openbsd/CORE -o Parsetexi.la -rpath
/usr/local/lib/texi2any parsetexi/Parsetexi_la-Parsetexi.lo libtexinfoxs.la
libtexinfo.la
libtool: link: rm -fr .libs/Parsetexi.la .libs/Parsetexi.lai .libs/Parsetexi.so
libtool: link: cc -shared -fPIC -DPIC -o .libs/Parsetexi.so
parsetexi/.libs/Parsetexi_la-Parsetexi.o
-L/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/.libs -L./.libs -ltexinfoxs
-L/usr/local/lib -ltexinfo -lintl -lunistring -liconv -fstack-protector-strong
-O2 -Wl,-R/usr/libdata/perl5/mips64-openbsd/CORE
libtool: link: ( cd ".libs" && rm -f "Parsetexi.la" && ln -s "../Parsetexi.la"
"Parsetexi.la" )
On OpenBSD, in Parsetexi.la, relink_command is given:
relink_command="(cd /home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS; /bin/sh
\"/home/gavin/texinfo-7.1.91/build/tp/Texinfo/XS/libtool\" --tag CC
--mode=relink cc -DNO_LOCALE_NUMERIC -DNO_LOCALE_COLLATE -fno-strict-aliasing
-fno-delete-null-pointer-checks -pipe -fstack-protector-strong
-I/usr/local/include -O2 -DPIC -fpic -avoid-version -module
-Wl,-R/usr/libdata/perl5/mips64-openbsd/CORE -o Parsetexi.la -rpath
/usr/local/lib/texi2any parsetexi/Parsetexi_la-Parsetexi.lo libtexinfoxs.la
libtexinfo.la @inst_prefix_dir@)"
While testing texi2any, I accidentally found a bug:
$ export top_srcdir=../../ top_builddir=.. TEXINFO_DEV_SOURCE=1
$ ./texi2any ../doc/info-stnd.info
texi2any: warning: input file info-stnd.info; did you mean info-stnd.texi?
You found a bug: Should be at end of line but have [inde]
'
Additional information:
last location: info-stnd.info:2106
context_stack: (ct_base)
current: (before_node_section)[C1078] at ../../tp/Texinfo/ParserNonXS.pm line
1144.
Died at ../../tp/Texinfo/ParserNonXS.pm line 7647.
TEXINFO_XS=debug reveals similar problems to the one you had with
macOS:
checking ../../tp/maintain/lib/Text-Unidecode/lib/Parsetexi.la
checking ../../tp/maintain/lib/Unicode-EastAsianWidth/lib/Parsetexi.la
checking ../../tp/maintain/lib/libintl-perl/lib/Parsetexi.la
checking ../tp/Texinfo/XS/Parsetexi.la
found ../tp/Texinfo/XS/Parsetexi.la
additional libraries: -lt
Re: library not found in build on macos
On Sun, Nov 17, 2024 at 07:41:37PM +0100, Patrice Dumas wrote: > Hello, > > I noticed that the macos parser runs the tests with the Pure Perl parser > in the CI, although for macos 12 and macos 13 it seemed that everything > was going on well regarding compilation. I could reproduce this issue > with an access to this platform. > > Launching texi2any.pl with TEXINFO_XS=debug shows the issue: > checking ../tp/maintain/lib/Text-Unidecode/lib/Parsetexi.la > checking ../tp/maintain/lib/Unicode-EastAsianWidth/lib/Parsetexi.la > checking ../tp/maintain/lib/libintl-perl/lib/Parsetexi.la > checking ../tp/Texinfo/XS/Parsetexi.la > found ../tp/Texinfo/XS/Parsetexi.la > Parsetexi: couldn't load file ../tp/Texinfo/XS/.libs/Parsetexi.so: > dlopen(../tp/Texinfo/XS/.libs/Parsetexi.so, 0x0001): Library not loaded: > '/usr/local/lib/texi2any/libtexinfoxs.0.dylib' > Referenced from: '/texinfo/tp/Texinfo/XS/.libs/Parsetexi.so' > Reason: tried: '/usr/local/lib/texi2any/libtexinfoxs.0.dylib' (no such > file), '/usr/local/lib/libtexinfoxs.0.dylib' (no such file), > '/usr/lib/libtexinfoxs.0.dylib' (no such file) at ../tp/Texinfo/XSLoader.pm > line 183, <$fh> line 8. > > The problem is that libtexinfoxs.0.dylib is only searched for in the > directory where it will be installed, not in the build directory. My > feeling is that it is wrong. My intuition would be that it is an issue > with libtool but I am not sure. I have checked that > ../tp/Texinfo/XS/.libs/ is in @DynaLoader::dl_library_path and that > libtexinfoxs.0.dylib is in ../tp/Texinfo/XS/.libs/. It does look like a bug somewhere. It could be an issue with how libtool treats this platform. I know that it is frequent for libtool libraries to be "re-linked" at the time of installation to work in their installed locations. Here it appears that the library is in its final, installed form before being installed. On GNU/Linux I can run a command on .libs/Parsetexi.so: ldd Parsetexi.so linux-vdso.so.1 (0x7ffe2e98c000) libtexinfoxs.so.0 => /home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs/libtexinfoxs.so.0 (0x7f1940f28000) libtexinfo.so.0 => /home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs/libtexinfo.so.0 (0x7f1940e4f000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f1940c0) /lib64/ld-linux-x86-64.so.2 (0x7f1940f82000) If you could do something similar on macos it could help to investigate the problem. Parsetexi.la has the line: relink_command="(cd /home/g/src/texinfo/GIT/tp/Texinfo/XS; /bin/bash \"/home/g/src/texinfo/GIT/tp/Texinfo/XS/libtool\" --tag CC --mode=relink x86_64-linux-gnu-gcc -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g -fPIC -avoid-version -module -Wl,-E -o Parsetexi.la -rpath /usr/local/lib/texi2any parsetexi/Parsetexi_la-Parsetexi.lo libtexinfoxs.la libtexinfo.la @inst_prefix_dir@)" This shows it needs to be relinked. The contents of libtexinfoxs.la could also be relevant and might give some hints why the installed locations are being used. > > It is not so problematic, as the pure Perl can be used and I do not > believe that there are that many users on macos, so I do not think that > it should delay a release, but it should probably be fixed some day. > Also it could be possible that the BSD have the same issue, I'll try to > investigate a bit using the CI. > > -- > Pat >
Re: library not found in build on macos
> Date: Sun, 17 Nov 2024 22:14:37 +0100 > From: Patrice Dumas > > On Sun, Nov 17, 2024 at 09:37:23PM +0100, Patrice Dumas wrote: > > I have added a report for all the platforms tested in the CI. It > > appears that cygwin, both 32 and 64 bits and OpenBSD unexpectedly cannot > > load Parsetexi.so/dll (in addition to macos 12 and macos 13). MiscXS is > > loaded in all the platform, so it is not a basic issue with XS, but > > something else. > > I couldn't find what was going on. The directories may not be the right > ones, but otherwise I can't see what's wrong. For openbsd, for example, > the link commands looks good: > > libtool: link: cc -shared -fPIC -DPIC -o .libs/Parsetexi.so > parsetexi/.libs/Parsetexi_la-Parsetexi.o > -L/home/runner/work/ci-check/ci-check/texinfo-7.1.91.20241117/build/tp/Texinfo/XS/.libs > -L/usr/local/lib -L./.libs -ltexinfoxs -ltexinfo -fstack-protector-strong > -O2 -Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE > > But the library is not found by texi2any: > ./tp/texi2any --html --no-split -o - ..//tp/t/input_files/simplest.texi > ... > checking tp/Texinfo/XS/Parsetexi.la > found tp/Texinfo/XS/Parsetexi.la > Parsetexi: couldn't load file tp/Texinfo/XS/.libs/Parsetexi.so: Cannot load > specified object at ../tp/Texinfo/XSLoader.pm line 182, <$fh> line 8. > > > Same for cygwin, here for example cygwin 64, but cygwin 32 looks very > similar: > > libtool: link: gcc -shared parsetexi/.libs/Parsetexi_la-Parsetexi.o > -L/cygdrive/d/a/ci-check/ci-check/texinfo-7.1.91.20241117/build/tp/Texinfo/XS/.libs > ./.libs/libtexinfoxs.dll.a -L/usr/lib/perl5/5.40/x86_64-cygwin-threads/CORE > ./.libs/libtexinfo.dll.a -lperl -ggdb -O2 -fstack-protector-strong -O3 -o > .libs/Parsetexi.dll -Wl,--enable-auto-image-base -Xlinker --out-implib > -Xlinker .libs/Parsetexi.dll.a > > ./tp/texi2any --html --no-split -o - ..//tp/t/input_files/simplest.texi > ... > checking tp/Texinfo/XS/Parsetexi.la > found tp/Texinfo/XS/Parsetexi.la > Parsetexi: couldn't load file tp/Texinfo/XS/.libs/Parsetexi.dll: No such file > or directory at ../tp/Texinfo/XSLoader.pm line 182, <$fh> line 8. tp/Texinfo/XS/.libs/Parsetexi.dll is a relative file name. But judging by the other file names on the command line, I'm guessing the link command runs in tp/Texinfo/XS/, is that right? Then either adding -L../../.. to the command line or changing the library name tp/Texinfo/XS/.libs/Parsetexi.dll to remove tp/Texinfo/XS/, should perhaps solve the problem?
Re: library not found in build on macos
> Date: Sun, 17 Nov 2024 21:21:05 +0100 > From: Patrice Dumas > > % otool -L .libs/Parsetexi.so > .libs/Parsetexi.so: > /usr/local/lib/texi2any/libtexinfoxs.0.dylib (compatibility version > 1.0.0, current version 1.0.0) > /usr/local/lib/texi2any/libtexinfo.0.dylib (compatibility version 1.0.0, > current version 1.0.0) > /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version > 7.0.0) > /usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 13.0.0, > current version 13.0.0) > /usr/local/opt/libunistring/lib/libunistring.5.dylib (compatibility > version 8.0.0, current version 8.0.0) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version > 1319.0.0) > > Note that the /usr/local/lib/texi2any/ directory does not exist. Isn't the reason to this what libtool wrote in the .la file, i.e.: > # Directory that this library needs to be installed in: > libdir='/usr/local/lib/texi2any' If you use -L on the link command line to mention the .libs directory, does the problem go away?
Re: library not found in build on macos
On Sun, Nov 17, 2024 at 07:41:37PM +0100, Patrice Dumas wrote: > > Also it could be possible that the BSD have the same issue, I'll try to > investigate a bit using the CI. I have added a report for all the platforms tested in the CI. It appears that cygwin, both 32 and 64 bits and OpenBSD unexpectedly cannot load Parsetexi.so/dll (in addition to macos 12 and macos 13). MiscXS is loaded in all the platform, so it is not a basic issue with XS, but something else. I'll have a deeper look at the logs to see if there is anything hinting at the issues. -- Pat
Re: library not found in build on macos
On Sun, Nov 17, 2024 at 09:37:23PM +0100, Patrice Dumas wrote: > I have added a report for all the platforms tested in the CI. It > appears that cygwin, both 32 and 64 bits and OpenBSD unexpectedly cannot > load Parsetexi.so/dll (in addition to macos 12 and macos 13). MiscXS is > loaded in all the platform, so it is not a basic issue with XS, but > something else. I couldn't find what was going on. The directories may not be the right ones, but otherwise I can't see what's wrong. For openbsd, for example, the link commands looks good: libtool: link: cc -shared -fPIC -DPIC -o .libs/Parsetexi.so parsetexi/.libs/Parsetexi_la-Parsetexi.o -L/home/runner/work/ci-check/ci-check/texinfo-7.1.91.20241117/build/tp/Texinfo/XS/.libs -L/usr/local/lib -L./.libs -ltexinfoxs -ltexinfo -fstack-protector-strong -O2 -Wl,-R/usr/libdata/perl5/amd64-openbsd/CORE But the library is not found by texi2any: ./tp/texi2any --html --no-split -o - ..//tp/t/input_files/simplest.texi ... checking tp/Texinfo/XS/Parsetexi.la found tp/Texinfo/XS/Parsetexi.la Parsetexi: couldn't load file tp/Texinfo/XS/.libs/Parsetexi.so: Cannot load specified object at ../tp/Texinfo/XSLoader.pm line 182, <$fh> line 8. Same for cygwin, here for example cygwin 64, but cygwin 32 looks very similar: libtool: link: gcc -shared parsetexi/.libs/Parsetexi_la-Parsetexi.o -L/cygdrive/d/a/ci-check/ci-check/texinfo-7.1.91.20241117/build/tp/Texinfo/XS/.libs ./.libs/libtexinfoxs.dll.a -L/usr/lib/perl5/5.40/x86_64-cygwin-threads/CORE ./.libs/libtexinfo.dll.a -lperl -ggdb -O2 -fstack-protector-strong -O3 -o .libs/Parsetexi.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/Parsetexi.dll.a ./tp/texi2any --html --no-split -o - ..//tp/t/input_files/simplest.texi ... checking tp/Texinfo/XS/Parsetexi.la found tp/Texinfo/XS/Parsetexi.la Parsetexi: couldn't load file tp/Texinfo/XS/.libs/Parsetexi.dll: No such file or directory at ../tp/Texinfo/XSLoader.pm line 182, <$fh> line 8. There is one thing that looks strange, a ^@ appears right after the texi2any error message for all the cases that fail to load, including macos. -- Pat
Re: library not found in build on macos
On Sun, Nov 17, 2024 at 07:29:18PM +, Gavin Smith wrote: > On Sun, Nov 17, 2024 at 07:41:37PM +0100, Patrice Dumas wrote: > > It does look like a bug somewhere. It could be an issue with how > libtool treats this platform. I know that it is frequent for libtool > libraries to be "re-linked" at the time of installation to work in their > installed locations. Here it appears that the library is in its final, > installed form before being installed. > > On GNU/Linux I can run a command on .libs/Parsetexi.so: > > ldd Parsetexi.so > linux-vdso.so.1 (0x7ffe2e98c000) > libtexinfoxs.so.0 => > /home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs/libtexinfoxs.so.0 > (0x7f1940f28000) > libtexinfo.so.0 => > /home/g/src/texinfo/GIT/tp/Texinfo/XS/.libs/libtexinfo.so.0 > (0x7f1940e4f000) > libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x7f1940c0) > /lib64/ld-linux-x86-64.so.2 (0x7f1940f82000) > > If you could do something similar on macos it could help to investigate > the problem. Here it is, it is consistent with the error: % otool -L .libs/Parsetexi.so .libs/Parsetexi.so: /usr/local/lib/texi2any/libtexinfoxs.0.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/local/lib/texi2any/libtexinfo.0.dylib (compatibility version 1.0.0, current version 1.0.0) /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0) /usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 13.0.0, current version 13.0.0) /usr/local/opt/libunistring/lib/libunistring.5.dylib (compatibility version 8.0.0, current version 8.0.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0) Note that the /usr/local/lib/texi2any/ directory does not exist. As a side note, file reports for Parsetexi.so Mach-0 64-bit bundle x86_64 and for libtexinfoxs.dylib Mach-0 64-bit dynamically linked shared library x86_64 > Parsetexi.la has the line: > > relink_command="(cd /home/g/src/texinfo/GIT/tp/Texinfo/XS; /bin/bash > \"/home/g/src/texinfo/GIT/tp/Texinfo/XS/libtool\" --tag CC --mode=relink > x86_64-linux-gnu-gcc -D_REENTRANT -D_GNU_SOURCE -DDEBIAN -fwrapv > -fno-strict-aliasing -pipe -I/usr/local/include -D_LARGEFILE_SOURCE > -D_FILE_OFFSET_BITS=64 -O2 -g -fPIC -avoid-version -module -Wl,-E -o > Parsetexi.la -rpath /usr/local/lib/texi2any > parsetexi/Parsetexi_la-Parsetexi.lo libtexinfoxs.la libtexinfo.la > @inst_prefix_dir@)" > > This shows it needs to be relinked. I attach Parsetexi.la and libtexinfoxs.la. The relink_command is "". -- Pat # Parsetexi.la - a libtool library file # Generated by libtool (GNU libtool) 2.5.3 # # Please DO NOT delete this file! # It is necessary for linking the library. # The name that we can dlopen(3). dlname='Parsetexi.so' # Names of this library. library_names='Parsetexi.so Parsetexi.so' # The name of the static archive. old_library='' # Linker flags that cannot go in dependency_libs. inherited_linker_flags=' ' # Libraries that this one depends upon. dependency_libs=' -R/usr/local/lib /Users/swiltgen/work/texinfo/tp/Texinfo/XS/libtexinfoxs.la -L/usr/local/lib /Users/swiltgen/work/texinfo/tp/Texinfo/XS/libtexinfo.la -liconv -lintl -lunistring' # Names of additional weak libraries provided by this library weak_library_names='' # Version information for Parsetexi. current=0 age=0 revision=0 # Is this an already installed library? installed=no # Should we warn about portability when linking against -modules? shouldnotlink=yes # Files to dlopen/dlpreopen dlopen='' dlpreopen='' # Directory that this library needs to be installed in: libdir='/usr/local/lib/texi2any' relink_command="" # libtexinfoxs.la - a libtool library file # Generated by libtool (GNU libtool) 2.5.3 # # Please DO NOT delete this file! # It is necessary for linking the library. # The name that we can dlopen(3). dlname='libtexinfoxs.0.dylib' # Names of this library. library_names='libtexinfoxs.0.dylib libtexinfoxs.dylib' # The name of the static archive. old_library='' # Linker flags that cannot go in dependency_libs. inherited_linker_flags=' ' # Libraries that this one depends upon. dependency_libs=' -R/usr/local/lib /Users/swiltgen/work/texinfo/tp/Texinfo/XS/libtexinfo.la -L/usr/local/lib -liconv -lintl -lunistring' # Names of additional weak libraries provided by this library weak_library_names='' # Version information for libtexinfoxs. current=0 age=0 revision=0 # Is this an already installed library? installed=no # Should we warn about portability when linking against -modules? shouldnotlink=no # Files to dlopen/dlpreopen dlopen='' dlpreopen='' # Directory that this library needs to be installed in: libdir='/usr/local/lib/texi2any' relink_command=""
