Re: [OE-core] wanting to clarify some shared lib install and packaging issues

2021-09-03 Thread Khem Raj



On 9/3/21 3:57 AM, Robert P. J. Day wrote:


   some really basic observations and questions about shared libs from
BB recipes as i'm trying to resolve some issues related to a weird
hybrid build system that overrides the standard do_install() task by
manually populating the "image" directory for each recipe, then hands
control over to bitbake to take it from there.

   first, quick review as i am going to have to explain all this to
some folks, so let's make sure i don't sound like an idiot. from the
explanation here:

   https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

when a BB recipe creates a shared lib, i typically expect to see the
following under image/usr/lib/ (example from "nettle" directory
building for aarch64):

   libhogweed.so -> libhogweed.so.6.4  [linker name]
   libhogweed.so.6 -> libhogweed.so.6.4[soname]
   libhogweed.so.6.4   [real name, actual lib]

based on default packaging defined in bitbake.conf, i then expect to
see the "base" package "nettle" install the following on the target:

   libhogweed.so.6 -> libhogweed.so.6.4
   libhogweed.so.6.4

while the development package nettle-dev will contain support the
linker name linking to, well, one of the above:

   libhogweed.so -> libhogweed.so.6.4

so far, nothing interesting, but this packaging means that, on the
target, if something needs to link against the libhogweed library, it
would normally link using the soname, "libhogweed.so.6", so a file by
that name *must* exist on the target and, normally, it does (even if
it's a symlink).

   here's what i just ran into -- this hybrid system that bypasses the
normal BB install step and manually populates the image directory
makes a bit of a mess with setting the symlinks and names properly,
but this was "fixed" by totally redefining the standard recipe
packaging, so that *all* shared lib files (all three forms) were
stuffed into the base package. and as long as that weird repackaging
was in place, everything "worked".

   along comes me, who really hates this re-definition of what goes
into each package as it packages far more for the target than is
necessary, and i revert the packaging back to the standard ... and
things break, and here's why.


you have to be a bit careful while undoing it. one usecase which can 
break subtly is when some program is trying to dlopen a .so since it 
maybe just calling it by libfoo.so




   the end result of the weird hybrid installation and my reverting the
packaging is that what ended up on the target in terms of the shared
libs was simply:

   libfubar.so

that is, the actual shared lib (not a symlink), but with the linker
name, not the soname, which would appear to be the cause of all sorts
of things suddenly not running anymore, as a quick inspection showed
systemd services trying to start and complaining:

   ... error while loading shared libraries: libfubar.so.1 ...

so i just want to confirm that all of that is due to the shared lib
having been packaged and installed on the target via its linker name,
not its proper soname, correct?


yes without looking at the code and what amends you made, it seems to be 
the reason.




   it *seems* like that's all i have to fix, but am i overlooking
anything? as long as the shared lib is findable via its soname, should
that be the general solution?


yeah, let OE installer do the right thing perhaps and fix the odd ones 
manually in do_install might be a middle way and reduce future 
maintenance, however watchout for shared libs whose builds might not be 
using soname versioning strategy. but I guess build time QA should warn 
about them loudly.




rday






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155682): 
https://lists.openembedded.org/g/openembedded-core/message/155682
Mute This Topic: https://lists.openembedded.org/mt/85348808/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-



[OE-core] wanting to clarify some shared lib install and packaging issues

2021-09-03 Thread Robert P. J. Day

  some really basic observations and questions about shared libs from
BB recipes as i'm trying to resolve some issues related to a weird
hybrid build system that overrides the standard do_install() task by
manually populating the "image" directory for each recipe, then hands
control over to bitbake to take it from there.

  first, quick review as i am going to have to explain all this to
some folks, so let's make sure i don't sound like an idiot. from the
explanation here:

  https://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

when a BB recipe creates a shared lib, i typically expect to see the
following under image/usr/lib/ (example from "nettle" directory
building for aarch64):

  libhogweed.so -> libhogweed.so.6.4  [linker name]
  libhogweed.so.6 -> libhogweed.so.6.4[soname]
  libhogweed.so.6.4   [real name, actual lib]

based on default packaging defined in bitbake.conf, i then expect to
see the "base" package "nettle" install the following on the target:

  libhogweed.so.6 -> libhogweed.so.6.4
  libhogweed.so.6.4

while the development package nettle-dev will contain support the
linker name linking to, well, one of the above:

  libhogweed.so -> libhogweed.so.6.4

so far, nothing interesting, but this packaging means that, on the
target, if something needs to link against the libhogweed library, it
would normally link using the soname, "libhogweed.so.6", so a file by
that name *must* exist on the target and, normally, it does (even if
it's a symlink).

  here's what i just ran into -- this hybrid system that bypasses the
normal BB install step and manually populates the image directory
makes a bit of a mess with setting the symlinks and names properly,
but this was "fixed" by totally redefining the standard recipe
packaging, so that *all* shared lib files (all three forms) were
stuffed into the base package. and as long as that weird repackaging
was in place, everything "worked".

  along comes me, who really hates this re-definition of what goes
into each package as it packages far more for the target than is
necessary, and i revert the packaging back to the standard ... and
things break, and here's why.

  the end result of the weird hybrid installation and my reverting the
packaging is that what ended up on the target in terms of the shared
libs was simply:

  libfubar.so

that is, the actual shared lib (not a symlink), but with the linker
name, not the soname, which would appear to be the cause of all sorts
of things suddenly not running anymore, as a quick inspection showed
systemd services trying to start and complaining:

  ... error while loading shared libraries: libfubar.so.1 ...

so i just want to confirm that all of that is due to the shared lib
having been packaged and installed on the target via its linker name,
not its proper soname, correct?

  it *seems* like that's all i have to fix, but am i overlooking
anything? as long as the shared lib is findable via its soname, should
that be the general solution?

rday

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#155648): 
https://lists.openembedded.org/g/openembedded-core/message/155648
Mute This Topic: https://lists.openembedded.org/mt/85348808/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-