Re: [Pharo-users] How to set library path for UFFI on Linux?

[Dan Wilczak]

One obstacle to fixing these problems properly is that handling of library
search paths is not centralized, but it really should be. A Pharo package
for sqlite or cairo or whatever should provide only the library name; UFFI
or some associated package should deal with the search for it. Otherwise
there will just be more ad hoc code and fixes in the future.



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] How to set library path for UFFI on Linux?

[herby]

On October 1, 2017 12:33:04 AM GMT+02:00, "p...@highoctane.be" 
 wrote:
>Usually there are a number of names to try but on Linux, there are less
>users and more distros, so people are not adding more libs to try.
>
>So, yes, symlink in the plugin directory works usually.
>
>Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do
>this cleanly (not done at this point).

I did use LD_LIBRARY_PATH. But it did not work (it looks like Pharo vm us using 
it, as it is used and decorated in pharo runner script). Even link in the 
plugin dir did not work unless I specified simplified name. The library is 
libsqlite3.so.0. Using that name, it wasn't found ('lib' is probably tried, so 
it is likely '.0' part that somehow blocked Pharo).

Shouldn't this actually be solved somehow? So linux vm loads libs using linux 
conventions?

Or is it preinstalled wrongly in Ubuntu images on Digital Ocean when it does 
not have symlink named libsqlite3.so ?

Herby

>Phil
>
>On Sat, Sep 30, 2017 at 11:19 PM, Herby Vojčík 
>wrote:
>
>> Dan Wilczak wrote:
>>
>>> Hernan -
>>>
>>> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>>>
>>> About continuing the search - I only mean continuing the search of
>the
>>> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes
>would
>>> be
>>> needed to accomplish this:
>>>
>>
>> FWIW, I had problem with loading 'sqlite3' module. Crossposting the
>> solution:
>>
>> It seems FFI for some reason struggles with 'lib' and/or '.so.0'
>things in
>> linux (even if LD_LIBRARY_PATH is properly set).
>>
>> I had to do this:
>>
>> TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0
>dirname`
>> ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3`
>> ${TARGETDIR}/sqlite3.so
>>
>> (so, link in plugin directory, and the name is plain 'sqlite3.so').
>With
>> that, things work. Maybe, libsqlite.so would do the trick as well,
>but I
>> got no nerve to play with it more.
>>
>> But, frankly, do not tell me this is what ppl need to do to load
>external
>> libs in linux. :-/
>>
>> Herby
>>
>>
>>> 1) Athens-Cairo>>  CairoLibrary would have to return a list of paths
>to
>>> all
>>> the matching libraries rather than just the first one that it finds.
>This
>>> part seems easy.
>>>
>>> 2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have
>to
>>> take
>>> the list of paths (rather than just one), check them for being
>32-bit or
>>> 64-bit, and load the first correct one. I can't find any way to
>perform
>>> that
>>> check in Pharo directly. How would you fork or exec the "file"
>command
>>> from
>>> inside Pharo?
>>>
>>> Dan
>>>
>>>
>>>
>>>
>>> --
>>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>>
>>>
>>
>>
>>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[p...@highoctane.be]

Usually there are a number of names to try but on Linux, there are less
users and more distros, so people are not adding more libs to try.

So, yes, symlink in the plugin directory works usually.

Alternative would be to use LD_LIBRARY_PATH or pkg_config support to do
this cleanly (not done at this point).

Phil

On Sat, Sep 30, 2017 at 11:19 PM, Herby Vojčík  wrote:

> Dan Wilczak wrote:
>
>> Hernan -
>>
>> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>>
>> About continuing the search - I only mean continuing the search of the
>> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would
>> be
>> needed to accomplish this:
>>
>
> FWIW, I had problem with loading 'sqlite3' module. Crossposting the
> solution:
>
> It seems FFI for some reason struggles with 'lib' and/or '.so.0' things in
> linux (even if LD_LIBRARY_PATH is properly set).
>
> I had to do this:
>
> TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname`
> ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3`
> ${TARGETDIR}/sqlite3.so
>
> (so, link in plugin directory, and the name is plain 'sqlite3.so'). With
> that, things work. Maybe, libsqlite.so would do the trick as well, but I
> got no nerve to play with it more.
>
> But, frankly, do not tell me this is what ppl need to do to load external
> libs in linux. :-/
>
> Herby
>
>
>> 1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to
>> all
>> the matching libraries rather than just the first one that it finds. This
>> part seems easy.
>>
>> 2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to
>> take
>> the list of paths (rather than just one), check them for being 32-bit or
>> 64-bit, and load the first correct one. I can't find any way to perform
>> that
>> check in Pharo directly. How would you fork or exec the "file" command
>> from
>> inside Pharo?
>>
>> Dan
>>
>>
>>
>>
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>>
>>
>
>
>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Herby Vojčík]

Dan Wilczak wrote:

Hernan -

I haven't opened an issue - how do I do it? (I'm very new to Pharo.)

About continuing the search - I only mean continuing the search of the
LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
needed to accomplish this:


FWIW, I had problem with loading 'sqlite3' module. Crossposting the 
solution:


It seems FFI for some reason struggles with 'lib' and/or '.so.0' things 
in linux (even if LD_LIBRARY_PATH is properly set).


I had to do this:

TARGETDIR=`find . -type f -name SqueakSSL.so -print0 | xargs -0 dirname`
ln -s `/sbin/ldconfig -p | sed -e 's|[^/]*||' | grep sqlite3` 
${TARGETDIR}/sqlite3.so


(so, link in plugin directory, and the name is plain 'sqlite3.so'). With 
that, things work. Maybe, libsqlite.so would do the trick as well, but I 
got no nerve to play with it more.


But, frankly, do not tell me this is what ppl need to do to load 
external libs in linux. :-/


Herby



1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to all
the matching libraries rather than just the first one that it finds. This
part seems easy.

2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to take
the list of paths (rather than just one), check them for being 32-bit or
64-bit, and load the first correct one. I can't find any way to perform that
check in Pharo directly. How would you fork or exec the "file" command from
inside Pharo?

Dan




--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Herby Vojčík]

Stephane Ducasse wrote:

Would you pls care to look at the sibling thread "How to make pharo find 
SQLite", it seems that it struggles with similar problem, is it supposed 
to created symlinks on linux, or it should be able to fund the library 
but somehow fails?


Thanks, Herby


https://pharo.fogbugz.com

On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak  wrote:

Hernan -

I haven't opened an issue - how do I do it? (I'm very new to Pharo.)

About continuing the search - I only mean continuing the search of the
LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
needed to accomplish this:

1) Athens-Cairo>>  CairoLibrary would have to return a list of paths to all
the matching libraries rather than just the first one that it finds. This
part seems easy.

2) UFFI Libraries UnixDynamicLoader>>  loadLibrary:flag: would have to take
the list of paths (rather than just one), check them for being 32-bit or
64-bit, and load the first correct one. I can't find any way to perform that
check in Pharo directly. How would you fork or exec the "file" command from
inside Pharo?

Dan




--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Stephane Ducasse]

https://pharo.fogbugz.com

On Sat, Sep 30, 2017 at 6:53 PM, Dan Wilczak  wrote:
> Hernan -
>
> I haven't opened an issue - how do I do it? (I'm very new to Pharo.)
>
> About continuing the search - I only mean continuing the search of the
> LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
> needed to accomplish this:
>
> 1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all
> the matching libraries rather than just the first one that it finds. This
> part seems easy.
>
> 2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take
> the list of paths (rather than just one), check them for being 32-bit or
> 64-bit, and load the first correct one. I can't find any way to perform that
> check in Pharo directly. How would you fork or exec the "file" command from
> inside Pharo?
>
> Dan
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Dan Wilczak]

Hernan -

I haven't opened an issue - how do I do it? (I'm very new to Pharo.)

About continuing the search - I only mean continuing the search of the
LD_LIBRARY_PATH directories, not the whole filesystem. Two changes would be
needed to accomplish this:

1) Athens-Cairo >> CairoLibrary would have to return a list of paths to all
the matching libraries rather than just the first one that it finds. This
part seems easy.

2) UFFI Libraries UnixDynamicLoader >> loadLibrary:flag: would have to take
the list of paths (rather than just one), check them for being 32-bit or
64-bit, and load the first correct one. I can't find any way to perform that
check in Pharo directly. How would you fork or exec the "file" command from
inside Pharo?

Dan
 



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Hernán Morales Durand]

Hi Dan

2017-09-23 13:24 GMT-03:00 Dan Wilczak :
> It does search on LD_LIBRARY_PATH now, it just fails because it finds the
> 32-bit libcairo before the 64-bit one.
>
> The traceback displayed inside Pharo says "Error: External module not found"
> which is quite misleading.  A minimal fix would be to just provide a better
> error message.
>

Yes, better error messages saves a lot of time.
Have you opened an issue for this?

> There is a traceback that prints to the console if you launched pharo from
> it, and it does reveal the true failure, but only to someone who knows what
> ELFCLASS32 means:
>
> ioLoadModule(//usr/lib/libcairo.so.2):
>   //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32
> Error: External module not found
>
> This shows that the information for a better error message is available.
>
> A better fix would be to skip the 32-bit lib and keep searching. This looks
> like it would be messy because CairoLibrary provides the path but
> U
> FFI has to deal with it.

You mean keep searching the whole file system?

> As far as I can see, there's no way to just check whether a library is 32-
> or 64-bit, other than trying to call something in it.
>

One can use a fork() or exec() with the "file" command in Linux based
OS, to find library type. It will answer something like:

ELF 32-bit LSB shared object, Intel80386, version 1, not stripped

or

ELF 64-bit LSB shared object, version 1, not stripped

> I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several
> years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being
> handled in Athens-Cairo rather than UFFI itself?
>
> Dan
>
>
>
>
>
>
> On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano 
> wrote:
>>
>> hi,
>>
>> this is not a bug, UFFI cannot know where a library is (because it can be
>> anywhere).
>> but, not being a bug, this remains a problem :)
>> One solution is to search on LD_LIBRARY_PATH but this is not generalised
>> (it should) for the moment. And of course, this is a solution that works for
>> linux and not other platforms, so not perfect. Also we need to consider
>> architecture, etc. etc. etc.
>>
>> Esteban
>>
>>
>> > On 23 Sep 2017, at 09:16, Stephane Ducasse 
>> > wrote:
>> >
>> > Can you open a bug entry and propose your solution?
>> >
>> > Stef
>> >
>> > On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak 
>> > wrote:
>> >> That pointed me in the right direction, thanks. I've got it working
>> >> now.
>> >>
>> >> I think there's a bug there which may bite other people ---
>> >>
>> >> If you have both 32-bit and 64-bit versions of the library installed,
>> >> Unix64ModuleName can find the
>> >> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that
>> >> as the
>> >> lib to use. Then when
>> >> UFFI loads it, it recognizes it as 32-bit and gives up rather than
>> >> looking
>> >> in any more directories.
>> >>
>> >> Dan
>> >>
>> >>
>> >>
>> >> --
>> >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>> >>
>> >
>>
>>
>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Dan Wilczak]

It does search on LD_LIBRARY_PATH now, it just fails because it finds the
32-bit libcairo before the 64-bit one.

The traceback displayed inside Pharo says "Error: External module not
found" which is quite misleading.  A minimal fix would be to just provide a
better error message.

There is a traceback that prints to the console if you launched pharo from
it, and it does reveal the true failure, but only to someone who knows what
ELFCLASS32 means:

ioLoadModule(//usr/lib/libcairo.so.2):
  //usr/lib/libcairo.so.2: wrong ELF class: ELFCLASS32
Error: External module not found

This shows that the information for a better error message is available.

A better fix would be to skip the 32-bit lib and keep searching. This looks
like it would be messy because CairoLibrary provides the path but
​U​
FFI has to deal with it.
​ As far as I can see, there's no way to just check whether a library is
32- or 64-bit, other than trying to call something in it.

I'm a total Pharo novice (I wrote maybe 300 lines of Squeak code several
years ago) but this design seems odd to me. Why is LD_LIBRARY_PATH being
handled in​ Athens-Cairo rather than UFFI itself?

Dan






On Sat, Sep 23, 2017 at 4:03 AM, Esteban Lorenzano 
wrote:

> hi,
>
> this is not a bug, UFFI cannot know where a library is (because it can be
> anywhere).
> but, not being a bug, this remains a problem :)
> One solution is to search on LD_LIBRARY_PATH but this is not generalised
> (it should) for the moment. And of course, this is a solution that works
> for linux and not other platforms, so not perfect. Also we need to consider
> architecture, etc. etc. etc.
>
> Esteban
>
>
> > On 23 Sep 2017, at 09:16, Stephane Ducasse 
> wrote:
> >
> > Can you open a bug entry and propose your solution?
> >
> > Stef
> >
> > On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak 
> wrote:
> >> That pointed me in the right direction, thanks. I've got it working now.
> >>
> >> I think there's a bug there which may bite other people ---
> >>
> >> If you have both 32-bit and 64-bit versions of the library installed,
> >> Unix64ModuleName can find the
> >> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that
> as the
> >> lib to use. Then when
> >> UFFI loads it, it recognizes it as 32-bit and gives up rather than
> looking
> >> in any more directories.
> >>
> >> Dan
> >>
> >>
> >>
> >> --
> >> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
> >>
> >
>
>
>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Hilaire]

What about shipping the dedicated libcairo with the Linux Virtual Image?

Hilaire


Le 23/09/2017 à 10:03, Esteban Lorenzano a écrit :

hi,

this is not a bug, UFFI cannot know where a library is (because it can be 
anywhere).
but, not being a bug, this remains a problem:)
One solution is to search on LD_LIBRARY_PATH but this is not generalised (it 
should) for the moment. And of course, this is a solution that works for linux 
and not other platforms, so not perfect. Also we need to consider architecture, 
etc. etc. etc.

Esteban


--
Dr. Geo
http://drgeo.eu

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Esteban Lorenzano]

hi,

this is not a bug, UFFI cannot know where a library is (because it can be 
anywhere). 
but, not being a bug, this remains a problem :)
One solution is to search on LD_LIBRARY_PATH but this is not generalised (it 
should) for the moment. And of course, this is a solution that works for linux 
and not other platforms, so not perfect. Also we need to consider architecture, 
etc. etc. etc.

Esteban


> On 23 Sep 2017, at 09:16, Stephane Ducasse  wrote:
> 
> Can you open a bug entry and propose your solution?
> 
> Stef
> 
> On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak  wrote:
>> That pointed me in the right direction, thanks. I've got it working now.
>> 
>> I think there's a bug there which may bite other people ---
>> 
>> If you have both 32-bit and 64-bit versions of the library installed,
>> Unix64ModuleName can find the
>> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
>> lib to use. Then when
>> UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
>> in any more directories.
>> 
>> Dan
>> 
>> 
>> 
>> --
>> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>> 
> 

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Stephane Ducasse]

Can you open a bug entry and propose your solution?

Stef

On Fri, Sep 22, 2017 at 9:56 PM, Dan Wilczak  wrote:
> That pointed me in the right direction, thanks. I've got it working now.
>
> I think there's a bug there which may bite other people ---
>
> If you have both 32-bit and 64-bit versions of the library installed,
> Unix64ModuleName can find the
> 32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
> lib to use. Then when
> UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
> in any more directories.
>
> Dan
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Dan Wilczak]

That pointed me in the right direction, thanks. I've got it working now.

I think there's a bug there which may bite other people ---

If you have both 32-bit and 64-bit versions of the library installed,
Unix64ModuleName can find the 
32-bit version first (depending on your LD_LIBRARY_PATH) and set that as the
lib to use. Then when 
UFFI loads it, it recognizes it as 32-bit and gives up rather than looking
in any more directories. 

Dan



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] How to set library path for UFFI on Linux?

[Stephane Ducasse]

Hi dan

We are sorry that you encounter a problem.

Not sure that this is the solution but check

ffiLibraryName
^ CairoLibrary


CairoLibrary >> unix32ModuleName
"On different flavors of linux the path to library may differ
depending on OS distro or whether system is 32 or 64 bit.
Packages such as snapcraft can also change the path.
Look in a number of known locations, or LD_LIBRARY_PATH."

#('/usr/lib/i386-linux-gnu' '/usr/lib32' '/usr/lib')
, (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ])
substrings: ':')
do: [ :path |
| libraryPath |
libraryPath := path asFileReference / 'libcairo.so.2'.
libraryPath exists
ifTrue: [ ^ libraryPath fullName ] ].
self error: 'Cannot locate cairo library. Please check if it installed
on your system'

Better


CairoLibrary >> unix64ModuleName
"On different flavors of linux the path to library may differ
depending on OS distro or whether system is 32 or 64 bit.
Packages such as snapcraft can also change the path.
Look in a number of known locations, or LD_LIBRARY_PATH."

#('/lib/x86_64-linux-gnu' '/usr/lib')
, (((OSEnvironment current getEnv: 'LD_LIBRARY_PATH') ifNil: [ '' ])
substrings: ':')
do: [ :path |
| libraryPath |
libraryPath := path asFileReference / 'libcairo.so.2'.
libraryPath exists
ifTrue: [ ^ libraryPath fullName ] ].
self error: 'Cannot locate cairo library. Please check if it installed
on your system'

On Fri, Sep 22, 2017 at 6:37 PM, Dan Wilczak  wrote:
> I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64
> bit). The Athens demos fail because
> UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting
> LD_LIBRARY_PATH but that has no effect.
> How can I force it to use lib64?
>
> Dan
>
>
>
>
> --
> Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html
>

[Pharo-users] How to set library path for UFFI on Linux?

[Dan Wilczak]

I've just installed the 64-bit version of Pharo 6.1 on OpenSuse linux (64
bit). The Athens demos fail because 
UFFI looks for libcairo in /usr/lib rather the /usr/lib64. I tried setting
LD_LIBRARY_PATH but that has no effect. 
How can I force it to use lib64?

Dan




--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html