The key problem is that the 'ldd' output does not match what the protobuf's pc says. The automatic library dependency adding thus does not give us a correct result in protobuf's case. As the list is written by protobuf's maintainers, I think we'd better trust them more than the ldd output.
Let's look at a regular package's behavior. Say packageB (libB) rdepends on packageA (libA). And we want to develop an application linking to libB inside SDK. The generated application will just run fine on target. Because all actual runtime dependencies of libB are on target. Unfortunately, this is not the case for protobuf and aseil-cpp. Protobuf does NOT have all the runtime dependency it needs. Why they are special? Part of the reason is that they are both kind of development packages, I guess. Regarding aseil-cpp, as I explained in previous email, its auto splitting is not reasonable. So I think what we really should do is: 1) split aseil-cpp based on .pc files (parsing its Requires field) 2) parse the protobuf.pc file to get the actual runtime dependencies and add them to RDEPEDNS automatically. If anyone has concern for the above solution, please let me know. Thanks. Before the above is done, we need to use RDEPENDS manually. Regards, Qi -----Original Message----- From: Christian Eggers <cegg...@arri.de> Sent: Monday, December 2, 2024 9:52 PM To: peter.kjellerst...@axis.com; Zhao, Yi <yi.z...@windriver.com>; Zhao, Yi <yi.z...@windriver.com>; openembedded-devel@lists.openembedded.org; Marko, Peter <peter.ma...@siemens.com> Cc: openembedded-devel@lists.openembedded.org; Chen, Qi <qi.c...@windriver.com> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to RDEPENDS Hi Peter, On Monday, 2 December 2024, 14:48:43 CET, Marko, Peter wrote: > Hello, > > > -----Original Message----- > > From: Christian Eggers <cegg...@arri.de> > > Sent: Monday, December 2, 2024 14:39 > > To: Marko, Peter (FT D EU SK BFS1) <peter.ma...@siemens.com>; > > peter.kjellerst...@axis.com; Yi Zhao <yi.z...@windriver.com>; Yi > > Zhao <yi.z...@eng.windriver.com>; openembedded- > > de...@lists.openembedded.org > > Cc: openembedded-devel@lists.openembedded.org; qi.c...@windriver.com > > Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp to > > RDEPENDS > > > > Hi Qi, > > > > On Monday, 2 December 2024, 06:35:22 CET, Chen Qi via > > lists.openembedded.org wrote: > > > Before we rework the abseil-cpp and protobuf recipes, a possible > > > improvement for this patch is that we list the needed libraries in > > > RDEPENDS manually. > > > > my main concern was (as you already may have guessed) not having the > > full set of absl libraries in my rootfs, as only some of them are > > required. But I cannot tell which is the best option to achieve > > this. > > If you want full abseil-cpp in your rootfs, please use IMAGE_INSTALL for that. > Packages should pull only their real dependencies, not something you need for > other reasons. > You use-case seems to be to develop application with abseil-cpp library but > not installing it explicitly. > SDK images need to install (or depend on) additional packages compared to > release images. sorry, but I meant it the other way around: "I do not like having the full set of abseil-cpp libraries in my rootfs only because I use protobuf in my application". Sorry for the confusion. regards, Christian > > Peter > > > > > regards, > > Christian > > > > > > > > Regards, > > > Qi > > > > > > On 12/2/24 13:21, Chen Qi via lists.openembedded.org wrote: > > > > I'll explain in more details what the problem is and why this > > > > patch is correct, given the current status of abseil-cpp recipe. > > > > > > > > In protobuf's abseil-cpp.cmake file, there's a hardcoded list of > > > > absl targets used by protobuf: > > > > https://github.com/protocolbuffers/protobuf/blob/main/cmake/abse > > > > il- > > cpp.cmake#L56 > > > > > > > > > > > > This value in turn is parsed and used to replace the > > > > @_protobuf_PC_REQUIRES@ in protobuf.pc.cmake. > > > > > > > > Here's the generated probobuf.pc file's contents: > > > > """ > > > > $ cat ../build/protobuf.pc > > > > prefix=/usr > > > > exec_prefix=/usr > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: Protocol Buffers > > > > Description: Google's Data Interchange Format > > > > Version: 25.3.0 > > > > Requires: absl_absl_check absl_absl_log absl_algorithm absl_base > > > > absl_bind_front absl_bits absl_btree absl_cleanup absl_cord > > > > absl_core_headers absl_debugging absl_die_if_null > > > > absl_dynamic_annotations absl_flags absl_flat_hash_map > > > > absl_flat_hash_set absl_function_ref absl_hash absl_layout > > > > absl_log_initialize absl_log_severity absl_memory > > > > absl_node_hash_map absl_node_hash_set absl_optional absl_span > > > > absl_status absl_statusor absl_strings absl_synchronization > > > > absl_time absl_type_traits absl_utility absl_variant utf8_range > > > > Libs: -L${libdir} -lprotobuf > > > > Cflags: -I${includedir} -DPROTOBUF_USE_DLLS > > > > Conflicts: protobuf-lite > > > > """ > > > > > > > > Let me use "absl_hash" and "absl_absl_log" in the above list as > > > > examples. > > > > > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_hash.so.*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl- > > hash/usr/lib64/libabsl_hash.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_hash.so.2401.0.0 > > > > > > > > $ find ../../../abseil-cpp/20240116.2/packa > > > > ges-split/ -name "libabsl_absl_log.so.*" > > > > $ > > > > """ > > > > > > > > See? The current package splitting of abseil-cpp does not match > > > > what protobuf assumes. And I thinks protobuf's assumption is > > > > correct, because there's really a absl_absl_log.pc file generated by > > > > abseil-cpp. > > > > > > > > And there's the contents and location of absl_absl_log.pc: > > > > """ > > > > $ cat > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_absl_log.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_absl_log > > > > Description: Abseil absl_log library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_log_impl = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX """ > > > > > > > > And there's the contents for the absl_log_internal_log_impl: > > > > """ > > > > $ cat ../../../abseil-cpp/20240116.2/packag > > > > es-split/abseil-cpp-dev/usr/lib64/pkgconfig/absl_log_internal_lo > > > > g_impl.pc > > > > prefix=/usr > > > > exec_prefix=${prefix} > > > > libdir=/usr/lib64 > > > > includedir=/usr/include > > > > > > > > Name: absl_log_internal_log_impl > > > > Description: Abseil log_internal_log_impl library > > > > URL: https://abseil.io/ > > > > Version: 20240116 > > > > Requires: absl_log_internal_conditions = 20240116, > > > > absl_log_internal_message = 20240116, absl_log_internal_strip = > > > > 20240116, absl_absl_vlog_is_on = 20240116 > > > > Libs: -L${libdir} > > > > Cflags: -I${includedir} -DNOMINMAX """ > > > > > > > > We can then find the absl_log_internal_conditions library: > > > > """ > > > > $ find ../../../abseil-cpp/20240116.2/packages-split/ -name > > > > "*absl_log_internal_message*" > > > > ../../../abseil-cpp/20240116.2/packages-split/libabsl-log-intern > > > > al- > > message/usr/lib64/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/libabsl_log_internal_message.so > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dev/usr/lib64/pkgconfig/absl_log_internal_message.pc > > > > > > > > ../../../abseil-cpp/20240116.2/packages-split/abseil-cpp- > > dbg/usr/lib64/.debug/libabsl_log_internal_message.so.2401.0.0 > > > > > > > > """ > > > > > > > > What does all above mean? It means that abseil-cpp is not > > > > splitted reasonably. This could be proven by the > > > > absl_absl_log.pc file as described above. > > > > > > > > So, given the current situation of abseil-cpp, Yi's solution is > > > > correct. Before abseil-cpp is re-splitted according to its pc > > > > files, which then could possibly lead to automatic parsing of > > > > the protobuf.pc file to add extra RDEPENDS automatically in > > > > protobuf recipe, we need to have Yi's solution. And we need to > > > > use RDEEPNDS instead of RRECOMMENDS (see the 'Requires' directive in > > > > corresponding .pc files). > > > > > > > > Regards, > > > > Qi > > > > > > > > On 10/1/24 20:42, Peter Marko via lists.openembedded.org wrote: > > > >>> -----Original Message----- > > > >>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> de...@lists.openembedded.org> On Behalf Of Peter Kjellerstedt > > > >>> via lists.openembedded.org > > > >>> Sent: Tuesday, October 1, 2024 14:00 > > > >>> To: Yi Zhao <yi.z...@windriver.com>; Yi Zhao > > > >>> <yi.z...@eng.windriver.com> > > > >>> Cc: openembedded-devel@lists.openembedded.org > > > >>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add > > > >>> abseil-cpp to RDEPENDS > > > >>> > > > >>>> -----Original Message----- > > > >>>> From: Yi Zhao <yi.z...@windriver.com> > > > >>>> Sent: den 25 september 2024 04:17 > > > >>>> To: Peter Kjellerstedt <peter.kjellerst...@axis.com>; Yi Zhao > > > >>> <yi.z...@eng.windriver.com> > > > >>>> Cc: openembedded-devel@lists.openembedded.org > > > >>>> Subject: Re: [oe] [meta-oe][PATCH 1/2] protobuf: add > > > >>>> abseil-cpp to > > > >>> RDEPENDS > > > >>>> > > > >>>> On 9/24/24 03:02, Peter Kjellerstedt wrote: > > > >>>>>> -----Original Message----- > > > >>>>>> From: openembedded-devel@lists.openembedded.org > > <openembedded- > > > >>> de...@lists.openembedded.org> On Behalf Of Yi Zhao > > > >>>>>> Sent: den 22 september 2024 09:36 > > > >>>>>> To: openembedded-devel@lists.openembedded.org > > > >>>>>> Subject: [oe] [meta-oe][PATCH 1/2] protobuf: add abseil-cpp > > > >>>>>> to > > > >>> RDEPENDS > > > >>>>>> Since commit[1], the libraries of abseil-cpp have been > > > >>>>>> splitted into separate packages. When protobuf is installed > > > >>>>>> into the image, only 48 abseil libraries are installed as > > > >>>>>> runtime dependencies. But the output of 'pkg-config --libs > > > >>>>>> protobuf' lists 66 abseil-cpp libraries are required. Add > > > >>>>>> abseil-cpp to RDEPENDS to ensure that all required > > > >>>>>> abseil-cpp libraries are installed. > > > >>>>>> > > > >>>>>> [1] https://git.openembedded.org/meta- > > > >>> > > openembedded/commit/?id=dd6421e65eb75c35a904f4b487ba022075035 > > > >>> ecc > > > >>>>>> Signed-off-by: Yi Zhao <yi.z...@windriver.com> > > > >>>>>> --- > > > >>>>>> meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb | 2 ++ > > > >>>>>> 1 file changed, 2 insertions(+) > > > >>>>>> > > > >>>>>> diff --git > > > >>>>>> a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>> b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> index cc5204de4..354b3213a 100644 > > > >>>>>> --- a/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> +++ b/meta-oe/recipes-devtools/protobuf/protobuf_4.25.4.bb > > > >>>>>> @@ -106,6 +106,8 @@ FILES:${PN}-lite = > > > >>>>>> "${libdir}/libprotobuf- > > > >>> lite${SOLIBS}" > > > >>>>>> # CMake requires protoc binary to exist in sysroot, even > > > >>>>>> if it has wrong > > > >>> architecture. > > > >>>>>> SYSROOT_DIRS += "${bindir}" > > > >>>>>> > > > >>>>>> +RDEPENDS:${PN} = "abseil-cpp" > > > >>>>>> +RDEPENDS:${PN}-lite = "abseil-cpp" > > > >>>>> This defeats the whole purpose of splitting up the abseil > > > >>>>> libraries. What triggered this change? Was it an actual > > > >>>>> runtime error, or were you only looking at the output from > > > >>>>> pkg-config --libs protobuf? > > > >>>>> > > > >>>>> Since the .pc files are in the protobuf-dev package, which > > > >>>>> depends on the abseil-cpp-dev package, all abseil libraries > > > >>>>> should be available if you are actually building on target > > > >>>>> or with an SDK. > > > >>>> Our customer reported this issue. Here is his description: > > > >>>> The abseil libraries will be included in the SDK, but not on > > > >>>> the target. > > > >>>> The application is building with SDK. The yocto build is > > > >>>> adding protobuf to the image (no explicit abseil libraries > > > >>>> are added). If an application then are built using the SDK it > > > >>>> can not be executed on target as libraries are missing. > > > >>> Can we at least change the dependency to a recommendation so > > > >>> that it can be avoided via BAD_RECOMMENDATIONS? Because as it > > > >>> is, it penalizes everyone that use protobuf for the case where > > > >>> one provides an SDK with protobuf. > > > >>> > > > >>> My suggestion would be to change it to: > > > >>> > > > >>> # Recommend abseil-cpp so that all its libraries are installed > > > >>> in case # protobuf is provided as part of an SDK. Everyone > > > >>> else should add # BAD_RECOMMENDATIONS += "abseil-cpp" to their > > > >>> configuration. > > > >>> RRECOMMENDS:${PN} = "abseil-cpp" > > > >>> RRECOMMENDS:${PN}-lite = "abseil-cpp" > > > >>> > > > >> I think that's also not the right way to do this. > > > >> Adding unnecessary stuff to release images to satisfy SDK needs > > > >> is wrong. > > > >> > > > >> I also wonder if the problem is not at the SDK level. > > > >> If you build an sdk, there is explicitly installed package base > > > >> and additional required dependencies of those packages. > > > >> The dependencies may not be complete and fully usable, only > > > >> their parts sufficient for the explicitly installed stuff is present. > > > >> Are the sdk users assuming that full abseil-cpp is installed > > > >> and thus using also parts not needed for protobuf? > > > >> The proper solution would be probably to include abseil-cpp > > > >> explicitly in SDK used in that way. > > > >> > > > >> Peter > > > >> > > > >>>> //Yi > > > >>>> > > > >>>>>> RDEPENDS:${PN}-compiler = "${PN}" > > > >>>>>> RDEPENDS:${PN}-dev += "${PN}-compiler" > > > >>>>>> RDEPENDS:${PN}-ptest = "bash > > > >>> ${@bb.utils.contains('PACKAGECONFIG', 'python', > > > >>> 'python3-protobuf', '', d)}" > > > >>>>> //Peter > > > >>> //Peter > > > >> > > > >> > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
<<attachment: winmail.dat>>
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#114177): https://lists.openembedded.org/g/openembedded-devel/message/114177 Mute This Topic: https://lists.openembedded.org/mt/108587631/21656 Group Owner: openembedded-devel+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
