Re: [gentoo-dev] Re: Banning modification of pkg-config files
On Sat, May 10, 2014 at 9:36 AM, hasufell wrote: > Longterm, this makes it year after year more difficult to develop > software for "Linux". Instead (like valve), people start to develop for > certain distros only (like Ubuntu), because it's just too much work to > bother with all this hackery-here-hackery-there-incompatible-here > things. Maybe also a reason they start to bundle all libraries for every > single game (among the convenience factor), effectively decreasing > security overall. I'm with you here, but what is the solution? If we say we stick to upstream then we don't provide pkg-config files at all (in these cases). Then when Debian does the other upstreams use them and then those packages break on Gentoo. People are still going to target their favorite distro no matter what we do. The only people with the power to break the distro-targeting behavior are the maintainers of the upstream packages. The linux kernel maintains a few stable branches with well-defined support periods, and as a result you can bet that just about any distro is going to be on one of them. Few other projects take this kind of care. Indeed, some upstreams can't be bothered to change their SONAME when their ABI changes. You could try to get distros to come together, but that tends not to work either. The minor distros all have lots of incentive to do this, but nobody cares about targeting them. The really big distros don't have incentive to play along, because they can just tell everybody that if their software breaks on their distro it is their problem. Then you have companies like RedHat which want to differentiate themselves so the last thing they want is to make other distros as robust, and to be fair they don't want to do the integration work only to have others mooch. So, in your mind what would a sane policy look like? Should packages like lua not provide pkg-config files even though apparently every other distro does? If so, where do we draw the line? Do we follow some particular distro like Debian? Do we list 4 distros and allow the file if 3/4 use it? If we don't allow a pkg-config in general can maintainers still have a "gentoo-foo" file? If we want a firm policy then there needs to be a proposal for one that makes sense. Otherwise the council is 95% likely to just say "we recommend that maintainers use care when creating pkg-config files but we leave it to their discretion," because that is the only thing that makes any sense when you can't come up with a rule that makes sense. Rich
Re: [gentoo-dev] Re: Banning modification of pkg-config files
Rich Freeman: > On Sat, May 10, 2014 at 9:00 AM, hasufell wrote: >> >> Our philosophy states that our tools "should be a joy to use". If we add >> random hackery on stuff that affects portability across distros, then >> this doesn't hold true anymore. >> > > Which one of our tools is at risk of not being a joy to use? Gentoo in general as a _development platform_. I'v also seen people arguing "this library isn't used by any package, let's remove it". Not. A. Reason. > > It sounds like the impact is to upstream developers who use Gentoo not > realizing that a library they depend on doesn't actually provide a > pkg-config file across all distros. How large an issue is this in > practice? It sounds like somebody will build something which works > fine in their testing, and then somebody will get a compiler error on > some other distro and report it, and then they can take 2min to fix > their build system once and for all. > The impact is on users as well if they try to build software with a broken build system. > What solutions do we have? Obviously we should try to get upstream to > change, but when they don't I don't see a universal policy that makes > sense. > We are effectively spreading the philosophy of modifying interfaces, libraries, headers and other things that are not trivial (and with one of the pkgconfig file additions in lua... we also got a library rename that breaks dlopen. So this is just the first step.). This has at first almost no effect on others, but for us, get's a lot of "shit" done. Longterm, this makes it year after year more difficult to develop software for "Linux". Instead (like valve), people start to develop for certain distros only (like Ubuntu), because it's just too much work to bother with all this hackery-here-hackery-there-incompatible-here things. Maybe also a reason they start to bundle all libraries for every single game (among the convenience factor), effectively decreasing security overall. > So, while I agree that the current state isn't ideal, I'm not sure > that it is any worse than the alternatives. > As described above, it is definitely worse. TBH, I don't want to be part of a "get shit done" distro. I could simply switch to debian then and do all the funny "lolpatches". I was hoping that QA is the authority that separates "get bugs fixed" from "get shit done".
Re: [gentoo-dev] Re: Banning modification of pkg-config files
On Sat, May 10, 2014 at 9:00 AM, hasufell wrote: > > Our philosophy states that our tools "should be a joy to use". If we add > random hackery on stuff that affects portability across distros, then > this doesn't hold true anymore. > Which one of our tools is at risk of not being a joy to use? All of the tools we're talking about here have no origin in Gentoo. It sounds like the impact is to upstream developers who use Gentoo not realizing that a library they depend on doesn't actually provide a pkg-config file across all distros. How large an issue is this in practice? It sounds like somebody will build something which works fine in their testing, and then somebody will get a compiler error on some other distro and report it, and then they can take 2min to fix their build system once and for all. What solutions do we have? Obviously we should try to get upstream to change, but when they don't I don't see a universal policy that makes sense. We could ban non-upstream pkg-config files entirely, in which case build systems that work for every other distro that supplies them may fail on Gentoo and we need to patch them (and for users building their own software that hardly sounds like a joy to use). Or we could force them to be renamed to gentoo-foo, in which case again build systems that work fine for every other distro that doesn't do this fail on Gentoo. Or we could leave it up to the maintainer, in which case we basically end up with what we have today. I could see guidelines, but even those are going to be hazy. Maybe recommend using a gentoo prefix on the pkg-config file when we're the only distro doing something. However, then we run into the prefix changing on a later release and then reverse dependencies break. We could have a USE flag which blocks installation of non-upstream pkg-config files. Of course, it might not be practical to use since anything which depends on the library in question might force it to not be set so that its own build system can rely on it. Sure, we could patch the build system to not require it, but most likely the build system does require it is that it is common on other distros so we're the ones standing alone. So, while I agree that the current state isn't ideal, I'm not sure that it is any worse than the alternatives. Rich
Re: [gentoo-dev] Re: Banning modification of pkg-config files
Markos Chandras: >> Gentoo, almost all pkgconfig files come from upstream with minimal >> modification. So a .pc file that is specific to Gentoo is a rare >> exception, and it could cause confusion for users who installed Gentoo >> on their development machine and who wish to develop new portable >> software. > > I don't see how this is a bad thing. This actually makes us look good in > the sense that we stick to upstream code as much as possible. > > In an ideal world, all distros would be compatible :) > Yeah, I was actually under the impression that gentoo is a "meta-distribution" and therefor takes the pain to maintain a complex Package Manager Specification amongst other things. Our philosophy states that our tools "should be a joy to use". If we add random hackery on stuff that affects portability across distros, then this doesn't hold true anymore.
Re: [gentoo-dev] Re: Banning modification of pkg-config files
Markos Chandras: > On 05/09/2014 09:32 PM, Tom Wijsman wrote: >> On Fri, 9 May 2014 16:15:58 -0400 >> Rich Freeman wrote: >> >>> I think fixing upstream is a no-brainer. >> >> It indeed is, this is the goal; you can force them in multiple ways, >> some of which can be found on the Lua bug and previous discussion(s). >> >>> The controversy only exists when upstream refuses to cooperate (which >>> seems to be the case when we're one of six distros patching it). If >>> there are other situations where we supply our own files please speak >>> up. >> >> Not that I know of; the refusal to cooperate is what this is all about, >> see my last response to hwoarang before this mail for a short summary. >> Though, I think that the Lua maintainers can explain all the details... >> >>> When the only issue is maintainer laziness I could see fixing that in >>> a different way... >> >> It has always been an issue; we could always use more manpower, ... >> >> https://wiki.gentoo.org/wiki/Contributing_to_Gentoo >> > > Well to me it feels that gentoo specific .pc files is a similar problem > to any other patch that affects upstream code in order to make the > package compatible with gentoo. Some people may consider downstream pc > files more dangerous because reverse deps are affected. But really, if > there is no other alternative, we shouldn't be treating this as a > special case. We patch upstream packages all the time after all > I don't agree with that. Pkg-config files are like headers and library files. They affect code of developers. If I mess with random configure.ac files in ffmpeg... it doesn't really affect any1 other than gentoo users in 99% of the cases.
Re: [gentoo-dev] Re: Banning modification of pkg-config files
Rich Freeman: > On Fri, May 9, 2014 at 4:08 PM, Tom Wijsman wrote: >> On Fri, 09 May 2014 20:57:29 +0100 >> Markos Chandras wrote: >> >>> I was wondering, is there a good reason we keep our own pkgconfig >>> files instead of communicating that to upstream and resolve that >>> properly? >> >> Yes, when your "instead of ..." is not an option. >> >>> What other distributions do? Or are we a special case and >>> we need our own pc files? >> >> No, see https://bugs.gentoo.org/show_bug.cgi?id=509392#c23 which reads: >> >> "You do realize that out of five distros (Fedora, Debian, >> Slackware, SuSe, Mandriva) I checked five ship a .pc file?" by mabi. > > I think fixing upstream is a no-brainer. The controversy only exists > when upstream refuses to cooperate (which seems to be the case when > we're one of six distros patching it). If there are other situations > where we supply our own files please speak up. > > When the only issue is maintainer laziness I could see fixing that in > a different way... > The fact is... missing pkg-config files are in 99% of the cases all fixable by fixing the build systems of packages that assume those pkg-config files... more specific: provide a fallback (I gave enough links for that in the reponse to the council agenda mail). This improves portability overall, for upstream, for us, for other distros and for random users.
Re: [gentoo-dev] Re: Banning modification of pkg-config files
On 10/05/14 12:39, Markos Chandras wrote: > On 05/10/2014 07:31 AM, Alexandre Rostovtsev wrote: >> On Sat, 2014-05-10 at 13:50 +0800, Ben de Groot wrote: >>> On 10 May 2014 04:34, Markos Chandras wrote: On 05/09/2014 09:32 PM, Tom Wijsman wrote: > On Fri, 9 May 2014 16:15:58 -0400 > Rich Freeman wrote: > >> I think fixing upstream is a no-brainer. > It indeed is, this is the goal; you can force them in multiple ways, > some of which can be found on the Lua bug and previous discussion(s). > >> The controversy only exists when upstream refuses to cooperate (which >> seems to be the case when we're one of six distros patching it). If >> there are other situations where we supply our own files please speak >> up. > Not that I know of; the refusal to cooperate is what this is all about, > see my last response to hwoarang before this mail for a short summary. > Though, I think that the Lua maintainers can explain all the details... > >> When the only issue is maintainer laziness I could see fixing that in >> a different way... > It has always been an issue; we could always use more manpower, ... > > https://wiki.gentoo.org/wiki/Contributing_to_Gentoo > Well to me it feels that gentoo specific .pc files is a similar problem to any other patch that affects upstream code in order to make the package compatible with gentoo. Some people may consider downstream pc files more dangerous because reverse deps are affected. But really, if there is no other alternative, we shouldn't be treating this as a special case. We patch upstream packages all the time after all >>> Exactly. I don't understand why this is an issue at all. Obviously, >>> if upstream does not ship a .pc file or ships a broken one, we try >>> to work with upstream to get it fixed on their end. If they are >>> uncooperative, we fix it on our end. >> Adding a pkgconfig file is a bit of a special case. Some distros have a >> habit of renaming and creating .pc files for various libraries. > Isn't this the same thing? If Debian creates/renames upstream pc files, > and you use Debian as a development box, you have the same problem: > Develop software which is not portable across distros. Say, a package XYZ makes use of xyz.pc and it's distribution specific, then you switch to a distribution that also ships XYZ but without pkg-config file, you can simply... export FOOBAR_LIBS="-lfoo" export FOOBAR_CFLAGS="-I/usr/include/foo" ./configure make make install ...as pkg-config allows using it without the .pc files by design. This is an non-issue. - Samuli
Re: [gentoo-dev] Re: Banning modification of pkg-config files
On 05/10/2014 07:31 AM, Alexandre Rostovtsev wrote: > On Sat, 2014-05-10 at 13:50 +0800, Ben de Groot wrote: >> On 10 May 2014 04:34, Markos Chandras wrote: >>> On 05/09/2014 09:32 PM, Tom Wijsman wrote: On Fri, 9 May 2014 16:15:58 -0400 Rich Freeman wrote: > I think fixing upstream is a no-brainer. It indeed is, this is the goal; you can force them in multiple ways, some of which can be found on the Lua bug and previous discussion(s). > The controversy only exists when upstream refuses to cooperate (which > seems to be the case when we're one of six distros patching it). If > there are other situations where we supply our own files please speak > up. Not that I know of; the refusal to cooperate is what this is all about, see my last response to hwoarang before this mail for a short summary. Though, I think that the Lua maintainers can explain all the details... > When the only issue is maintainer laziness I could see fixing that in > a different way... It has always been an issue; we could always use more manpower, ... https://wiki.gentoo.org/wiki/Contributing_to_Gentoo >>> >>> Well to me it feels that gentoo specific .pc files is a similar problem >>> to any other patch that affects upstream code in order to make the >>> package compatible with gentoo. Some people may consider downstream pc >>> files more dangerous because reverse deps are affected. But really, if >>> there is no other alternative, we shouldn't be treating this as a >>> special case. We patch upstream packages all the time after all >> >> Exactly. I don't understand why this is an issue at all. Obviously, >> if upstream does not ship a .pc file or ships a broken one, we try >> to work with upstream to get it fixed on their end. If they are >> uncooperative, we fix it on our end. > > Adding a pkgconfig file is a bit of a special case. Some distros have a > habit of renaming and creating .pc files for various libraries. Isn't this the same thing? If Debian creates/renames upstream pc files, and you use Debian as a development box, you have the same problem: Develop software which is not portable across distros. I have done very limited upstream development myself, but my opinion has always been that upstream developers who use Debian/Gentoo/Fedora/$FOOlinux as their dev environment shouldn't care about distro peculiarities. That's packagers' job, who are responsible to make the upstream software compatible with each distribution. But in > Gentoo, almost all pkgconfig files come from upstream with minimal > modification. So a .pc file that is specific to Gentoo is a rare > exception, and it could cause confusion for users who installed Gentoo > on their development machine and who wish to develop new portable > software. I don't see how this is a bad thing. This actually makes us look good in the sense that we stick to upstream code as much as possible. In an ideal world, all distros would be compatible :) -- Regards, Markos Chandras
