[gentoo-dev] [RFC] News item: Most stable hppa keywords removed
Title: Most stable hppa keywords removed Author: Matt Turner Posted: 2020-12-26 Revision: 1 News-Item-Format: 2.0 Display-If-Keyword: hppa The Gentoo/HPPA team no longer thinks that the time invested in package stabilization is warranted for the small number of users on HPPA. As a result, we will drop most "hppa" keywords to "~hppa" on 2020-12-31. Around 850 packages will retain their stable keywords. Users need not make any changes to their systems—other than perhaps adding some entries to their package.accept_keywords file. The Gentoo/HPPA team has no plans to remove support for the architecture. Here's the GitHub PR I've made for stable masking a bunch of flags and dropping keywords: https://github.com/gentoo/gentoo/pull/18798
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
>> >> > > My intention with the suggestion was that the actual library be stored > in /usr/lib64/liblua52.so (or whatever the appropriate name is), but a > symlink used for linking be stored in /usr/lib64/lua5.2/liblua.so. When > you pass "-L /usr/lib64/lua5.2 -llua" to the compiler, it will find the > file /usr/lib64/lua5.2/liblua.so and read the DT_SONAME entry in it. > That will cause the linker to store "liblua52.so" as a DT_NEEDED entry > in the executable. At runtime, the runtime linker ld.so > (/lib64/ld-linux-x86-64.so.2) will read the DT_NEEDED entry, and try to > find the library liblua52.so, which is in /usr/lib64, so it will be > found without any ld.so.conf tricks. This requires no special runtime > support, as the actual name the compiler/linker will end up storing in > the output contains the proper version. This means that if > /usr/bin/binA linked to liblua51.so (via the linker finding > /usr/lib64/lua5.1/liblua.so) and /usr/bin/binB linked to liblua52.so > (likewise), they would both be able to find the correct libraries, as > the DT_SONAMEs are different. > > Jonathan Callen > Yes, this sounds doable and should work! Only problem is that if there is an actual liblua.so with the proper SONAME available in /usr/lib64 (I dunno if Gentoo has any provider of liblua.so with that SONAME, IIRC SLOT=0 currently does that) then it will ignore the wrong SONAMEd even with the -L/usr/lib64/lua5.2/ Aisha
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
>> >> > > My intention with the suggestion was that the actual library be stored > in /usr/lib64/liblua52.so (or whatever the appropriate name is), but a > symlink used for linking be stored in /usr/lib64/lua5.2/liblua.so. When > you pass "-L /usr/lib64/lua5.2 -llua" to the compiler, it will find the > file /usr/lib64/lua5.2/liblua.so and read the DT_SONAME entry in it. > That will cause the linker to store "liblua52.so" as a DT_NEEDED entry > in the executable. At runtime, the runtime linker ld.so > (/lib64/ld-linux-x86-64.so.2) will read the DT_NEEDED entry, and try to > find the library liblua52.so, which is in /usr/lib64, so it will be > found without any ld.so.conf tricks. This requires no special runtime > support, as the actual name the compiler/linker will end up storing in > the output contains the proper version. This means that if > /usr/bin/binA linked to liblua51.so (via the linker finding > /usr/lib64/lua5.1/liblua.so) and /usr/bin/binB linked to liblua52.so > (likewise), they would both be able to find the correct libraries, as > the DT_SONAMEs are different. > > Jonathan Callen > Yes, this sounds doable and should work! Aisha
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 3:01 PM, Michael Orlitzky wrote: > On 12/23/20 1:14 PM, Aisha Tammy wrote: >> >> I've recently had the same problem for TACC/Lmod which uses >> autotools to get lua versions and lua.cpath and lua.path and did infact >> manage >> to push the horrendously large patch upstream - >> https://github.com/TACC/Lmod/commit/0913bf05dd7e8f478f69d5297e26d744ddb5073a >> >> Maybe something similar can work for your use case? > > Yes, patching the build system works. > Ah, I was unclear. I meant this kind of patch would not be something that upstream will hate, as it is backwards compatible. That's the biggest problem with creating large backwards compatible patches >.< > >> The problem with just using -L/path/to/lua/lib/ -llua would be loading >> library at runtime, unless autotools is smart enough to actually change this >> CFLAGs into a -Wl,-rpath,/path/to/lua/lib -L/path/to/lua/lib -llua. >> I am not sure how intelligent autotools is to be able to do this or not. >> > > We already add entries to /etc/ld.so.conf to make things like slotted LLVM > work. > Hmm, how would that work? I have package A with a binary /usr/bin/binA linked to liblua51.so (which in your suggestion would change to /usr/lib64/lua5.1/liblua.so) and and /usr/bin/binB linked to liblua52.so (or /usr/lib64/lua5.2/liblua.so). I don't see any ordering of /usr/lib64/lua5.1/ and /usr/lib64/lua5.2/ that allows for binA and binB to find their correct lua libraries. (This doesn't need to be for binaries, other shared libraries even) Admittedly, I am not a lua or linker expert... Aisha
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 6:18 PM, Michael Orlitzky wrote: Using pkg-config has a related problem. If lua-5.1 is eselected and if the upstream build system runs $(pkg-config ... lua), it's going to get the information for lua-5.1, even if you're trying to build against lua-5.2. So, first I have to patch upstream to use pkg-config, and then I have to hack it in Gentoo to avoid using pkg-config. Sorry, I was wrong about this. Patching the upstream build system to support pkg-config does work if you use the eclasses.
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 4:51 PM, Mart Raudsepp wrote: Ühel kenal päeval, K, 23.12.2020 kell 07:49, kirjutas Michael Orlitzky: AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"]) How do I pass the name "lua5.2" to that, without hacking configure.ac and running autoreconf? The only option that comes to mind is to build the entire project with LIBS="-llua5.2", but that's not right if only some of the executables are supposed to be linked with lua. You either fix upstream to use pkg-config, or you set up the cache variable for your configure call to tell configure what the answer is, without making it do the work wrong. Using pkg-config has a related problem. If lua-5.1 is eselected and if the upstream build system runs $(pkg-config ... lua), it's going to get the information for lua-5.1, even if you're trying to build against lua-5.2. So, first I have to patch upstream to use pkg-config, and then I have to hack it in Gentoo to avoid using pkg-config. Overriding the cached check variable does work, but you have to override every single call to AC_SEARCH_LIBS(function...) with ac_cv_search_ which depends highly on the implementation details of configure.ac. This is roughly as invasive and hard to maintain as patching, and should just not be the default way to link against a library IMO.
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 6:04 PM, Aisha Tammy wrote: Yes, this sounds doable and should work > Only problem is that if there is an actual liblua.so with the proper SONAME available in /usr/lib64 (I dunno if Gentoo has any provider of liblua.so with that SONAME, IIRC SLOT=0 currently does that) then it will ignore the wrong SONAMEd even with the -L/usr/lib64/lua5.2/ eselect-lua creates a symlink named /usr/lib64/liblua.so, but the -L path should take precedence when looking for liblua.so. When it's looking for e.g. liblua5.2.so at runtime, there is only one copy to find, so no problem there.
[gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 4:34 PM, Aisha Tammy wrote: > On 12/23/20 3:01 PM, Michael Orlitzky wrote: >> On 12/23/20 1:14 PM, Aisha Tammy wrote: >>> >>> I've recently had the same problem for TACC/Lmod which uses >>> autotools to get lua versions and lua.cpath and lua.path and did infact >>> manage >>> to push the horrendously large patch upstream - >>> https://github.com/TACC/Lmod/commit/0913bf05dd7e8f478f69d5297e26d744ddb5073a >>> >>> Maybe something similar can work for your use case? >> >> Yes, patching the build system works. >> > > Ah, I was unclear. > I meant this kind of patch would not be something that > upstream will hate, as it is backwards compatible. > That's the biggest problem with creating large backwards > compatible patches >.< > >> >>> The problem with just using -L/path/to/lua/lib/ -llua would be loading >>> library at runtime, unless autotools is smart enough to actually change this >>> CFLAGs into a -Wl,-rpath,/path/to/lua/lib -L/path/to/lua/lib -llua. >>> I am not sure how intelligent autotools is to be able to do this or not. >>> >> >> We already add entries to /etc/ld.so.conf to make things like slotted LLVM >> work. >> > > Hmm, how would that work? > I have package A with a binary /usr/bin/binA linked to liblua51.so (which in > your suggestion would change to /usr/lib64/lua5.1/liblua.so) and > and /usr/bin/binB linked to liblua52.so (or /usr/lib64/lua5.2/liblua.so). > I don't see any ordering of /usr/lib64/lua5.1/ and /usr/lib64/lua5.2/ > that allows for binA and binB to find their correct lua libraries. > (This doesn't need to be for binaries, other shared libraries even) > > Admittedly, I am not a lua or linker expert... > > Aisha > > My intention with the suggestion was that the actual library be stored in /usr/lib64/liblua52.so (or whatever the appropriate name is), but a symlink used for linking be stored in /usr/lib64/lua5.2/liblua.so. When you pass "-L /usr/lib64/lua5.2 -llua" to the compiler, it will find the file /usr/lib64/lua5.2/liblua.so and read the DT_SONAME entry in it. That will cause the linker to store "liblua52.so" as a DT_NEEDED entry in the executable. At runtime, the runtime linker ld.so (/lib64/ld-linux-x86-64.so.2) will read the DT_NEEDED entry, and try to find the library liblua52.so, which is in /usr/lib64, so it will be found without any ld.so.conf tricks. This requires no special runtime support, as the actual name the compiler/linker will end up storing in the output contains the proper version. This means that if /usr/bin/binA linked to liblua51.so (via the linker finding /usr/lib64/lua5.1/liblua.so) and /usr/bin/binB linked to liblua52.so (likewise), they would both be able to find the correct libraries, as the DT_SONAMEs are different. Jonathan Callen
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 12:13 PM, Jonathan Callen wrote:
> On 12/23/20 9:10 AM, Michael Orlitzky wrote:
>> On 12/23/20 8:35 AM, Jaco Kroon wrote:
>>> Michael,
>>>
>>> I'm busy disecting what Marek has done for asterisk as I need to make
>>> that work for multiple versions of net-misc/asterisk-16.14.0-r100, not
>>> merely the one he did it for - perhaps that would be helpful for you as
>>> well?
>>>
>>> I don't know lua at all.
>>>
>>> Anyway, I'm on IRC as jkroon if you'd like to exchange notes. I don't
>>> particularly like the patch Marek did as I'll NEVER be able to push that
>>> upstream. The patch will work for us, but as a policy I highly prefer
>>> patches which I can get unstreamed such that we don't need to carry them
>>> more than one or two versions.
>>
>> Agreed. This is more of a system library packaging issue than anything
>> to do with Lua specifically.
>>
>>
>>> I'd prefer to patch upstream to keep it's behaviour of detecting a lua
>>> version, but have a --with-lua(version)? type argument that can take an
>>> optional argument which will then be the version, but --with-* generally
>>> takes a prefix where the include and lib folders can be found ... and
>>> I'd prefer to depend on pkg-config as primary and fallback to the
>>> ./configure mess which tries to guess at things ...
>>>
>>
>> This is exactly what I'm trying to get across. Support for multiple
>> versions of libraries in weird locations is not a new thing. Generally,
>> you can put CPPFLAGS="-I/foo/bar" and LIBS="-L/baz/quux" before running
>> ./configure and everything will just work with your header files and
>> libraries in the non-standard paths /foo/bar and /baz/quux,
>> respectively. Those paths take precedence over the defaults (if used
>> correctly...), so you can use them to override the default version of a
>> library and compile/link against a specific copy if you need to do that.
>> Likewise, one could prepend a path to PKG_CONFIG_PATH to override the
>> version that will be detected via pkg-config.
>>
>> The problem that we've gotten ourselves into is that we've copied Debian
>> by not only relocating the library, but renaming it (and its *.pc file)
>> entirely. There is no good way to tell an autotools build system that
>> you've renamed a library completely, and that it should prefer the
>> renamed version over the standard one if the standard one also exists.
>>
>> If you want to build against the eselected version of Lua on Gentoo,
>> then eselect-lua makes everything OK. It creates symlinks from the
>> standard names to the nonstandard ones, and everything just works. But
>> if you want to build against a specific, not-eselected version of Lua?
>> You have to tell the build system what to do. The eselected version
>> lives in the correct locations (via those symlinks), so the build system
>> is going to want to find that first. If the eselected version is not the
>> one you want to build against, oops. Then, since the version you
>> *do* want to link against has the wrong name, you have to tell the build
>> system to link against it somehow. You can't simply override the
>> PKG_CONFIG_PATH, because our *.pc files have the wrong names. You can't
>> just override the library search path, because our libraries have the
>> wrong names.
>>
>> The work I've done so far for OpenDKIM does nothing to address these
>> larger problems. If lua.pc is on the system, it will get used first,
>> regardless of the version you actually want to build against. AFAIK
>> there's no good way around this without patching.
>>
>> If the libraries (and pkg-config files?) were installed to
>> subdirectories instead, then the standard method of overrides could be
>> used to build against a specific version, rather than requiring every
>> upstream build system to support our weird layout.
>>
>>
>
> One way this could be done without breaking things might be to create a
> subdirectory of /usr/$(get_libdir) for each version of Lua and creating
> a liblua.a and/or liblua.so symlink in that directory pointing to the
> liblua${VERSION}.* file in /usr/$(get_libdir). Then you could simply
> point those build systems at that directory and they would still use the
> correct version. As a hack in an ebuild, you probably could just create
> such a setup in (subdirectories of) ${T}, pointing liblua.a, liblua.so,
> and lua.pc at the appropriately-versioned files.
>
> Jonathan Callen
>
I've recently had the same problem for TACC/Lmod which uses
autotools to get lua versions and lua.cpath and lua.path and did infact manage
to push the horrendously large patch upstream -
https://github.com/TACC/Lmod/commit/0913bf05dd7e8f478f69d5297e26d744ddb5073a
Maybe something similar can work for your use case?
The problem with just using -L/path/to/lua/lib/ -llua would be loading
library at runtime, unless autotools is smart enough to actually change this
CFLAGs into a -Wl,-rpath,/path/to/lua/lib -L/path/to/lua/lib -llua.
I am not sure how intelligent autotools is to
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
Ühel kenal päeval, K, 23.12.2020 kell 07:49, kirjutas Michael Orlitzky: > AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"]) > > How do I pass the name "lua5.2" to that, without hacking configure.ac > and running autoreconf? The only option that comes to mind is to > build > the entire project with LIBS="-llua5.2", but that's not right if only > some of the executables are supposed to be linked with lua. You either fix upstream to use pkg-config, or you set up the cache variable for your configure call to tell configure what the answer is, without making it do the work wrong. Should be ac_cv_search_whatever=something in this case. sys-apps/grep ebuild has an example. I'm not sure what package this is to look or test closer. It doesn't appear to be OpenDKIM which is discussed in this thread elsewhere. Mart signature.asc Description: This is a digitally signed message part
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 1:14 PM, Aisha Tammy wrote: I've recently had the same problem for TACC/Lmod which uses autotools to get lua versions and lua.cpath and lua.path and did infact manage to push the horrendously large patch upstream - https://github.com/TACC/Lmod/commit/0913bf05dd7e8f478f69d5297e26d744ddb5073a Maybe something similar can work for your use case? Yes, patching the build system works. The problem with just using -L/path/to/lua/lib/ -llua would be loading library at runtime, unless autotools is smart enough to actually change this CFLAGs into a -Wl,-rpath,/path/to/lua/lib -L/path/to/lua/lib -llua. I am not sure how intelligent autotools is to be able to do this or not. We already add entries to /etc/ld.so.conf to make things like slotted LLVM work.
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On Wed, Dec 23, 2020 at 1:21 PM Michael Orlitzky wrote:
>
> On 12/23/20 12:13 PM, Jonathan Callen wrote:
> >
> > One way this could be done without breaking things might be to create a
> > subdirectory of /usr/$(get_libdir) for each version of Lua and creating
> > a liblua.a and/or liblua.so symlink in that directory pointing to the
> > liblua${VERSION}.* file in /usr/$(get_libdir). Then you could simply
> > point those build systems at that directory and they would still use the
> > correct version. As a hack in an ebuild, you probably could just create
> > such a setup in (subdirectories of) ${T}, pointing liblua.a, liblua.so,
> > and lua.pc at the appropriately-versioned files.
>
> All of these packages are masked, so breakage isn't an issue.
>
> And since these hacks are needed in every package that will link against
> liblua, doesn't it sound like something the eclass should be doing? Even
> the CMake packages would benefit, no longer having to pass in
> -DLUA_INCLUDE_DIR and -DLUA_LIBRARY by hand.
Most ebuilds in the gentoo repo are either using pkg-config, or
already have hacks/patches applied. Perhaps if this idea were offered
earlier in the year, it might have actually helped.
The versioned libdir subdirectory seems like a nice idea for people
building stuff against lua outside of ebuilds. Jonathan's proposal
could be treated as an enhancement at a later date.
Re: [gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 12:13 PM, Jonathan Callen wrote:
One way this could be done without breaking things might be to create a
subdirectory of /usr/$(get_libdir) for each version of Lua and creating
a liblua.a and/or liblua.so symlink in that directory pointing to the
liblua${VERSION}.* file in /usr/$(get_libdir). Then you could simply
point those build systems at that directory and they would still use the
correct version. As a hack in an ebuild, you probably could just create
such a setup in (subdirectories of) ${T}, pointing liblua.a, liblua.so,
and lua.pc at the appropriately-versioned files.
All of these packages are masked, so breakage isn't an issue.
And since these hacks are needed in every package that will link against
liblua, doesn't it sound like something the eclass should be doing? Even
the CMake packages would benefit, no longer having to pass in
-DLUA_INCLUDE_DIR and -DLUA_LIBRARY by hand.
[gentoo-dev] Re: Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 9:10 AM, Michael Orlitzky wrote:
> On 12/23/20 8:35 AM, Jaco Kroon wrote:
>> Michael,
>>
>> I'm busy disecting what Marek has done for asterisk as I need to make
>> that work for multiple versions of net-misc/asterisk-16.14.0-r100, not
>> merely the one he did it for - perhaps that would be helpful for you as
>> well?
>>
>> I don't know lua at all.
>>
>> Anyway, I'm on IRC as jkroon if you'd like to exchange notes. I don't
>> particularly like the patch Marek did as I'll NEVER be able to push that
>> upstream. The patch will work for us, but as a policy I highly prefer
>> patches which I can get unstreamed such that we don't need to carry them
>> more than one or two versions.
>
> Agreed. This is more of a system library packaging issue than anything
> to do with Lua specifically.
>
>
>> I'd prefer to patch upstream to keep it's behaviour of detecting a lua
>> version, but have a --with-lua(version)? type argument that can take an
>> optional argument which will then be the version, but --with-* generally
>> takes a prefix where the include and lib folders can be found ... and
>> I'd prefer to depend on pkg-config as primary and fallback to the
>> ./configure mess which tries to guess at things ...
>>
>
> This is exactly what I'm trying to get across. Support for multiple
> versions of libraries in weird locations is not a new thing. Generally,
> you can put CPPFLAGS="-I/foo/bar" and LIBS="-L/baz/quux" before running
> ./configure and everything will just work with your header files and
> libraries in the non-standard paths /foo/bar and /baz/quux,
> respectively. Those paths take precedence over the defaults (if used
> correctly...), so you can use them to override the default version of a
> library and compile/link against a specific copy if you need to do that.
> Likewise, one could prepend a path to PKG_CONFIG_PATH to override the
> version that will be detected via pkg-config.
>
> The problem that we've gotten ourselves into is that we've copied Debian
> by not only relocating the library, but renaming it (and its *.pc file)
> entirely. There is no good way to tell an autotools build system that
> you've renamed a library completely, and that it should prefer the
> renamed version over the standard one if the standard one also exists.
>
> If you want to build against the eselected version of Lua on Gentoo,
> then eselect-lua makes everything OK. It creates symlinks from the
> standard names to the nonstandard ones, and everything just works. But
> if you want to build against a specific, not-eselected version of Lua?
> You have to tell the build system what to do. The eselected version
> lives in the correct locations (via those symlinks), so the build system
> is going to want to find that first. If the eselected version is not the
> one you want to build against, oops. Then, since the version you
> *do* want to link against has the wrong name, you have to tell the build
> system to link against it somehow. You can't simply override the
> PKG_CONFIG_PATH, because our *.pc files have the wrong names. You can't
> just override the library search path, because our libraries have the
> wrong names.
>
> The work I've done so far for OpenDKIM does nothing to address these
> larger problems. If lua.pc is on the system, it will get used first,
> regardless of the version you actually want to build against. AFAIK
> there's no good way around this without patching.
>
> If the libraries (and pkg-config files?) were installed to
> subdirectories instead, then the standard method of overrides could be
> used to build against a specific version, rather than requiring every
> upstream build system to support our weird layout.
>
>
One way this could be done without breaking things might be to create a
subdirectory of /usr/$(get_libdir) for each version of Lua and creating
a liblua.a and/or liblua.so symlink in that directory pointing to the
liblua${VERSION}.* file in /usr/$(get_libdir). Then you could simply
point those build systems at that directory and they would still use the
correct version. As a hack in an ebuild, you probably could just create
such a setup in (subdirectories of) ${T}, pointing liblua.a, liblua.so,
and lua.pc at the appropriately-versioned files.
Jonathan Callen
Re: [gentoo-dev] possible additional tag for GLEP66: Pending
Hi, Jaco Kroon wrote: Specifically, what I suggest is to flag the PR that fixes the issues (ie, ebuild bump) with the usual Bug: tag, but to then at the same time be able to pre-emptively file a PR removing the vulnerable versions, but only once the security bug has been handled (closed). Towards this end, I'd suggest a tag such as: Pending: https://bugs.gentoo.org/NN — to reference a bug; the bug needs to be closed before this PR will be considered for merging. No, this is not really needed and will make everything more complicated. Keep in mind that you never know what happens: - It's possible that the initial bump wasn't enough. - Maybe during stabilization process we will uncover the need for an additional patch. - It's possible that keywords will change during the process. - It's still possible that the ebuild(s) which will be cleaned up as part of the process changes before cleanup will happen. - It's possible that the process hasn't finished before a new security bug for same package was created (superseded). But if you have created the follow ups in advance, - this will clutter things up - you will have to adjust things all the time - and any additional fix up will create new notifications, comments... someone has to check - proxy-maintainer would have to spend time for review twice because something could have changed between creation of follow up PR and time when the PR will get merged And like you said, current CI would be unable to test these follow up PRs before new ebuild was marked stable or CI would report a lot of NonsolvableDepsInStable problems. Sure, you could delay or re-trigger check once stabilization has happened but I really see no benefits in doing anything of that in advance if the chance is high that you have to spend the same amount of time again before you can finally merge. -- Regards, Thomas Deutschmann / Gentoo Security Team fpr: C4DD 695F A713 8F24 2AA1 5638 5849 7EE5 1D5D 74A5
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 8:35 AM, Jaco Kroon wrote: Michael, I'm busy disecting what Marek has done for asterisk as I need to make that work for multiple versions of net-misc/asterisk-16.14.0-r100, not merely the one he did it for - perhaps that would be helpful for you as well? I don't know lua at all. Anyway, I'm on IRC as jkroon if you'd like to exchange notes. I don't particularly like the patch Marek did as I'll NEVER be able to push that upstream. The patch will work for us, but as a policy I highly prefer patches which I can get unstreamed such that we don't need to carry them more than one or two versions. Agreed. This is more of a system library packaging issue than anything to do with Lua specifically. I'd prefer to patch upstream to keep it's behaviour of detecting a lua version, but have a --with-lua(version)? type argument that can take an optional argument which will then be the version, but --with-* generally takes a prefix where the include and lib folders can be found ... and I'd prefer to depend on pkg-config as primary and fallback to the ./configure mess which tries to guess at things ... This is exactly what I'm trying to get across. Support for multiple versions of libraries in weird locations is not a new thing. Generally, you can put CPPFLAGS="-I/foo/bar" and LIBS="-L/baz/quux" before running ./configure and everything will just work with your header files and libraries in the non-standard paths /foo/bar and /baz/quux, respectively. Those paths take precedence over the defaults (if used correctly...), so you can use them to override the default version of a library and compile/link against a specific copy if you need to do that. Likewise, one could prepend a path to PKG_CONFIG_PATH to override the version that will be detected via pkg-config. The problem that we've gotten ourselves into is that we've copied Debian by not only relocating the library, but renaming it (and its *.pc file) entirely. There is no good way to tell an autotools build system that you've renamed a library completely, and that it should prefer the renamed version over the standard one if the standard one also exists. If you want to build against the eselected version of Lua on Gentoo, then eselect-lua makes everything OK. It creates symlinks from the standard names to the nonstandard ones, and everything just works. But if you want to build against a specific, not-eselected version of Lua? You have to tell the build system what to do. The eselected version lives in the correct locations (via those symlinks), so the build system is going to want to find that first. If the eselected version is not the one you want to build against, oops. Then, since the version you *do* want to link against has the wrong name, you have to tell the build system to link against it somehow. You can't simply override the PKG_CONFIG_PATH, because our *.pc files have the wrong names. You can't just override the library search path, because our libraries have the wrong names. The work I've done so far for OpenDKIM does nothing to address these larger problems. If lua.pc is on the system, it will get used first, regardless of the version you actually want to build against. AFAIK there's no good way around this without patching. If the libraries (and pkg-config files?) were installed to subdirectories instead, then the standard method of overrides could be used to build against a specific version, rather than requiring every upstream build system to support our weird layout.
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
Michael, I'm busy disecting what Marek has done for asterisk as I need to make that work for multiple versions of net-misc/asterisk-16.14.0-r100, not merely the one he did it for - perhaps that would be helpful for you as well? I don't know lua at all. Anyway, I'm on IRC as jkroon if you'd like to exchange notes. I don't particularly like the patch Marek did as I'll NEVER be able to push that upstream. The patch will work for us, but as a policy I highly prefer patches which I can get unstreamed such that we don't need to carry them more than one or two versions. I'd prefer to patch upstream to keep it's behaviour of detecting a lua version, but have a --with-lua(version)? type argument that can take an optional argument which will then be the version, but --with-* generally takes a prefix where the include and lib folders can be found ... and I'd prefer to depend on pkg-config as primary and fallback to the ./configure mess which tries to guess at things ... Kind Regards, Jaco On 2020/12/23 14:49, Michael Orlitzky wrote: > On 12/23/20 4:09 AM, Marek Szuba wrote: >> >> I think what you are looking for is lua_get_shared_lib() from >> lua-utils.eclass. We have already got ebuilds in the tree which use >> it. >> > > Knowing the library name only helps if I patch the build system; > that's what I'm getting at. The few packages where this works use > CMake, and CMake already tries to guess[0] the name of the lua library > (thanks Debian) and so it has a pre-defined variable to store the result. > > Not all build systems are going to work like that. Suppose I know that > the name of the lua library is "lua5.2". An autotools build system is > going to run something like, > > AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"]) > > How do I pass the name "lua5.2" to that, without hacking configure.ac > and running autoreconf? The only option that comes to mind is to build > the entire project with LIBS="-llua5.2", but that's not right if only > some of the executables are supposed to be linked with lua. > > For contrast, if the lua library was stored in /usr/lib64/lua5.2, then > I could just set LIBS="-L/usr/lib64/lua5.2" and let ./configure do its > thing. > > > [0] https://github.com/Kitware/CMake/blob/master/Modules/FindLua.cmake >
[gentoo-dev] possible additional tag for GLEP66: Pending
Hi All, When bumping for security updates, the requirement is that the replacement ebuild be stabilized (the GLSA be issued), and then to clean up the tree of vulnerable versions. As a proxy maintainer, the addition of a tag to queue a PR pending a specific Bug be closed first would in this scenario be potentially beneficial. Specifically, what I suggest is to flag the PR that fixes the issues (ie, ebuild bump) with the usual Bug: tag, but to then at the same time be able to pre-emptively file a PR removing the vulnerable versions, but only once the security bug has been handled (closed). Towards this end, I'd suggest a tag such as: Pending: https://bugs.gentoo.org/NN — to reference a bug; the bug needs to be closed before this PR will be considered for merging. Obviously it's also possible to file a second bug that depends on the security bug, but this doesn't block merging. QA checks doesn't make sense to run (since this remove commit will mostly likely remove all current stable versions). Ideas and thoughts around this? Kind Regards, Jaco
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
On 12/23/20 4:09 AM, Marek Szuba wrote: I think what you are looking for is lua_get_shared_lib() from lua-utils.eclass. We have already got ebuilds in the tree which use it. Knowing the library name only helps if I patch the build system; that's what I'm getting at. The few packages where this works use CMake, and CMake already tries to guess[0] the name of the lua library (thanks Debian) and so it has a pre-defined variable to store the result. Not all build systems are going to work like that. Suppose I know that the name of the lua library is "lua5.2". An autotools build system is going to run something like, AC_SEARCH_LIBS([whatever], [lua], [lua_found="yes"]) How do I pass the name "lua5.2" to that, without hacking configure.ac and running autoreconf? The only option that comes to mind is to build the entire project with LIBS="-llua5.2", but that's not right if only some of the executables are supposed to be linked with lua. For contrast, if the lua library was stored in /usr/lib64/lua5.2, then I could just set LIBS="-L/usr/lib64/lua5.2" and let ./configure do its thing. [0] https://github.com/Kitware/CMake/blob/master/Modules/FindLua.cmake
Re: [gentoo-dev] Slotted Lua + revdeps to be unmasked on Christmas Eve
On December 23, 2020 2:56:05 AM UTC, Michael Orlitzky wrote: >One last design issue that I ran into during the migration. > >The slotted lua ebuilds install the headers into subdirectories like >/usr/include/lua5.2, but otherwise with their upstream names. The >libraries, on the other hand, all get installed directly to $libdir.. >but with NON-standard names like liblua5.2.a (as opposed to simply >liblua.a). > >This makes it impossible to build against a specific version of Lua >without using pkg-config. The usual way would be to add >/usr/include/lua5.2 to the compiler's include path, and $libdir/lua5.2 >to its library path (via -I and -L)... but there's no way to tell the >build system to look for a library of an entirely different name. >Things >like AC_SEARCH_LIBS are going to try -llua, and that's it. I think what you are looking for is lua_get_shared_lib() from lua-utils.eclass. We have already got ebuilds in the tree which use it. -- MS
