Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Peter Volkov wrote: | Is there any reason why --as-needed is not enabled by default? There's still about 18 open bugs on the tracker[1] for it. You can see how many problems it had been causing by the huge number of blocking bugs. I've been using it for a pretty long time now (probably a couple weeks after Diego first blogged about it) and don't have many problems at all (now), but every once in a while a version bump or a new package will just fail to compile properly and the problem leads back to as-needed. I'm not sure whether ~arch users would be able to catch all the as-needed bugs before they hit stable, so I couldn't say whether it should be enabled by default or not. I also don't think it would completely eliminate the problems that Diego mentioned with preserve-libs (there could still be instances where bar and foo both NEEDED thing.so, but bar had been compiled more recently and needed thing.so.1 whilst foo needed thing.so.0, and starting bar would break trying to load both)... Mike 5:) [1] http://bugs.gentoo.org/show_bug.cgi?id=129413 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkg/qb4ACgkQu7rWomwgFXpl7wCdFhDuZbQOVy1b12dgXbZbSWtj dIMAn3Z6FDx5HW1KD83JxdboNrQOOap1 =Nca2 -END PGP SIGNATURE- -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Fri, 30 May 2008 10:55:51 +0400 Peter Volkov [EMAIL PROTECTED] wrote: В Чтв, 29/05/2008 в 11:02 +0200, Diego 'Flameeyes' Pettenò пишет: I'm afraid that it will turn, for complex libraries like libexpat and users not using --as-needed, the message telling you the program cannot be started with subtle crashes for symbol collision. Is there any reason why --as-needed is not enabled by default? --as-needed is fundamentally broken by design and causes legitimate code to break. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Fri, May 30, 2008 at 9:16 AM, Mike Auty [EMAIL PROTECTED] wrote: Peter Volkov wrote: | Is there any reason why --as-needed is not enabled by default? There's still about 18 open bugs on the tracker[1] for it. You can see how many problems it had been causing by the huge number of blocking bugs. I've been using it for a pretty long time now (probably a couple weeks after Diego first blogged about it) and don't have many problems at all (now), but every once in a while a version bump or a new package will just fail to compile properly and the problem leads back to as-needed. I'm not sure whether ~arch users would be able to catch all the as-needed bugs before they hit stable, so I couldn't say whether it should be enabled by default or not. That's not a problem at all. If we choose to support --as-needed by default we'd get testing from maintainers when adding new ebuilds, and from arch teams before ebuilds hit stable. --as-needed breaking legitimate code is a problem, though. I wonder if we have that kind of code in any application in the tree and if we have some way to detect it. Regards, -- Santiago M. Mola Jabber ID: [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
Santiago M. Mola [EMAIL PROTECTED] writes: --as-needed breaking legitimate code is a problem, though. I wonder if we have that kind of code in any application in the tree and if we have some way to detect it. You could be looking for code not supposed to work under Solaris or Windows, as the --as-needed behaviour behaves a lot like the standard Sun linker and the forced behaviour of the PE executable format. The only thing that can be broken by using --as-needed is code that assumes the order in calling the .init sections of a set of shared objects. Such an order is not only changed by --as-needed usage but by any other change in the loading mechanism. It is strictly related to glibc at this point as far as I can tell. -- Diego Flameeyes Pettenò http://blog.flameeyes.eu/ pgpNOhQiIWOXd.pgp Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
Santiago M. Mola a écrit : On Fri, May 30, 2008 at 9:16 AM, Mike Auty [EMAIL PROTECTED] wrote: I've been using it for a pretty long time now (probably a couple weeks after Diego first blogged about it) and don't have many problems at all (now), but every once in a while a version bump or a new package will just fail to compile properly and the problem leads back to as-needed. I'm not sure whether ~arch users would be able to catch all the as-needed bugs before they hit stable, so I couldn't say whether it should be enabled by default or not. Our experience in the gnome herd is that most --as-needed issues come up *very* quickly: either the build fails, or application plugins don't get loaded. As such, they're quite easy to catch. Fixing them can be trickier. --as-needed breaking legitimate code is a problem, though. I wonder if we have that kind of code in any application in the tree and if we have some way to detect it. --as-needed breaks legitimate C++ code, I have yet to see it break plain C code (but I could be wrong). Ciaran posted an example code a while ago, either here or on his blog. His example is valid C++ code (basically it was about implicitly loading .so plugins) but IMHO, it's just not a good engineering practice. Plugin loading should be explicit (readdir + dlopen). But that's my opinion. There could be a couple other odd cases but I don't remember them. All I can say is that --as-needed *today* does much more good than harm. Cheers -- Rémi Cardona LRI, INRIA [EMAIL PROTECTED] [EMAIL PROTECTED] -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
Rémi Cardona [EMAIL PROTECTED] writes: --as-needed breaks legitimate C++ code, I have yet to see it break plain C code (but I could be wrong). This because C does not have constructors or static objects. I suppose I can write a couple of C files that can present the problem, but really, they would break as soon as I change the loading rules by moving to a non-ELF system or a different dynamic loader. It's interesting to note that Microsoft _did_ think of this problem when designing the .NET framework, and they answer is ensuring just that the static constructor will be called (right) before any static method or attribute is referenced and (right) before any object of the given class is istantiated. They don't get otherwise an absolute order in which static constructors are called. -- Diego Flameeyes Pettenò http://blog.flameeyes.eu/ pgp6Q7c1XvJEy.pgp Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Friday 30 May 2008 13:22:15 Diego 'Flameeyes' Pettenò wrote: The only thing that can be broken by using --as-needed is code that assumes the order in calling the .init sections of a set of shared objects. Such an order is not only changed by --as-needed usage but by any other change in the loading mechanism. It is strictly related to glibc at this point as far as I can tell. It's not just the order, it also breaks things that rely on the .init section being called at all to register themselves with the core application (with --as-needed, the .so doesn't get loaded in the first place, so it doesn't get a chance to run anything). It's interesting to note that Microsoft _did_ think of this problem when designing the .NET framework, and they answer is ensuring just that the static constructor will be called (right) before any static method or attribute is referenced and (right) before any object of the given class is istantiated. They don't get otherwise an absolute order in which static constructors are called. That's /an/ answer, but it doesn't provide all the functionality that static objects in C++ have. -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
David Leverton [EMAIL PROTECTED] writes: It's not just the order, it also breaks things that rely on the .init section being called at all to register themselves with the core application (with --as-needed, the .so doesn't get loaded in the first place, so it doesn't get a chance to run anything). This really is backward, solution-wise: you expect the core application to know enough of the plugins to link them together, but not enough to call their init functions... And still, it breaks not only with --as-needed but also with the Sun linker, with the PE file format and other non-ELF file formats. So it's really not a problem _limited_ to --as-needed. Beside, you can _force_ a link by using --no-as-needed before a given library, that's what it's supposed to happen if you don't want to be helped by the linker. To a very minimum this can be said to be a _clash_ between two designs. Saying that --as-needed is _broken_ because it breaks this case is _quite_ an exaggeration... -- Diego Flameeyes Pettenò http://blog.flameeyes.eu/ pgpDGGzHF7XVV.pgp Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Fri, 30 May 2008 18:29:49 +0200 [EMAIL PROTECTED] (Diego 'Flameeyes' Pettenò) wrote: David Leverton [EMAIL PROTECTED] writes: It's not just the order, it also breaks things that rely on the .init section being called at all to register themselves with the core application (with --as-needed, the .so doesn't get loaded in the first place, so it doesn't get a chance to run anything). This really is backward, solution-wise: you expect the core application to know enough of the plugins to link them together, but not enough to call their init functions... Actually, no. The core application doesn't have to be doing the linking. The linking can be done by an intermediate library. And still, it breaks not only with --as-needed but also with the Sun linker, with the PE file format and other non-ELF file formats. So it's really not a problem _limited_ to --as-needed. It works with any standard-compliant C++ setup. Beside, you can _force_ a link by using --no-as-needed before a given library, that's what it's supposed to happen if you don't want to be helped by the linker. And next you'll be saying and you can force a compile using --no-broken-behaviour-that-goes-against-the-standard. To a very minimum this can be said to be a _clash_ between two designs. Saying that --as-needed is _broken_ because it breaks this case is _quite_ an exaggeration... Saying it's broken because it goes against an international standard isn't... -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Friday 30 May 2008 17:29:49 Diego 'Flameeyes' Pettenò wrote: This really is backward, solution-wise: you expect the core application to know enough of the plugins to link them together, but not enough to call their init functions... Why should it call their init functions, when a static object with a constructor can do the job just fine? -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
David Leverton wrote: On Friday 30 May 2008 17:29:49 Diego 'Flameeyes' Pettenò wrote: This really is backward, solution-wise: you expect the core application to know enough of the plugins to link them together, but not enough to call their init functions... Why should it call their init functions, when a static object with a constructor can do the job just fine? Talk to the upstream about this, probably getting a satisfying solution isn't that difficult. lu -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Fri, 30 May 2008 21:13:32 +0200 Luca Barbato [EMAIL PROTECTED] wrote: Talk to the upstream about this, probably getting a satisfying solution isn't that difficult. The solution is to use --as-needed in the same way that -ffast-math is used: only with applications specifically designed to support it. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
Ciaran McCreesh wrote: --as-needed is fundamentally broken by design and causes legitimate code to break. Basically C++ quasi-standard corner cases nobody uses, that happen to work on ELF only and that should be removed in the next revision of 0x ? Implicit plugin loading isn't exactly a sane thing. lu - less excuses to laziness please. -- Luca Barbato Gentoo Council Member Gentoo/linux Gentoo/PPC http://dev.gentoo.org/~lu_zero -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Fri, 30 May 2008 21:29:26 +0200 Luca Barbato [EMAIL PROTECTED] wrote: Ciaran McCreesh wrote: --as-needed is fundamentally broken by design and causes legitimate code to break. Basically C++ quasi-standard corner cases nobody uses, that happen to work on ELF only and that should be removed in the next revision of 0x ? It's going to end up being used more, not less, in 0x, thanks to constexpr and the new thread rules. -- Ciaran McCreesh signature.asc Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On R, 2008-05-30 at 20:20 +0100, Ciaran McCreesh wrote: On Fri, 30 May 2008 21:13:32 +0200 Luca Barbato [EMAIL PROTECTED] wrote: Talk to the upstream about this, probably getting a satisfying solution isn't that difficult. The solution is to use --as-needed in the same way that -ffast-math is used: only with applications specifically designed to support it. You mean everything but paludis? Doesn't your grand plan include supporting Prefix and Interix with PE binaries and so on? I know projects that need to work around static initialization not being reliable - they only happen to have done that for other reasons (such as Windows PE format, iirc) years before --as-needed was implemented for binutils. Standards is one thing - reality is something quite different. The reality is that everything designed to work everywhere is just mighty happy with --as-needed and lots of benefits to gain from it. -- Mart Raudsepp Gentoo Developer Mail: [EMAIL PROTECTED] Weblog: http://planet.gentoo.org/developers/leio -- gentoo-dev@lists.gentoo.org mailing list
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Sat, 31 May 2008 00:31:22 +0300 Mart Raudsepp [EMAIL PROTECTED] wrote: On R, 2008-05-30 at 20:20 +0100, Ciaran McCreesh wrote: On Fri, 30 May 2008 21:13:32 +0200 Luca Barbato [EMAIL PROTECTED] wrote: Talk to the upstream about this, probably getting a satisfying solution isn't that difficult. The solution is to use --as-needed in the same way that -ffast-math is used: only with applications specifically designed to support it. You mean everything but paludis? Paludis is fine with as-needed. But hey, don't let reality get in the way of your pathetic attempts at turning everything into Paludis bashing. Doesn't your grand plan include supporting Prefix and Interix with PE binaries and so on? I have no particular interest in supporting any platform that can't ship a Standard-compliant C++ environment. I know projects that need to work around static initialization not being reliable - they only happen to have done that for other reasons (such as Windows PE format, iirc) years before --as-needed was implemented for binutils. Standards is one thing - reality is something quite different. The reality is that everything designed to work everywhere is just mighty happy with --as-needed and lots of benefits to gain from it. And twenty years ago C++ had to work around linkers that only supported eight character symbol names. Reality moves forward, except in situations like these where people try to rice it backwards. -- Ciaran McCreesh signature.asc Description: PGP signature
[gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
Marius Mauch [EMAIL PROTECTED] writes: Also it is not going to be a perfect solution against all runtime link errors, but if enabled it should eliminate the need for revdep-rebuild in most cases. I'm afraid that it will turn, for complex libraries like libexpat and users not using --as-needed, the message telling you the program cannot be started with subtle crashes for symbol collision. preserve-libs would be quite perfect if all libraries out there used versioned symbol, but this is far from true (and some systems Gentoo runs on don't even consider versioned symbol to begin with). Example at hand? When the libexpat transition started, the choice of keeling .so.0 around with .so.1 was discarded right away because: - library libfoo links to libexpat; - program bar links to libfoo; - user is not using --as-needed, so bar has a NEEDED against both libfoo and libexpat; - user rebuilds libfoo, but not bar; or bar and not libfoo, the result is the same; - KABOOM! symbol collision and bar crashes. As much as we want preserve-libs to be an all-curing magic, it's not. When you need to replace a library you need to do so _for all its users at once_, if you allow it to be gradually you're opening the hellgate of symbol collision. My solution would be to disallow _building_ anything that is or depends directly or indirectly on a package on the set until it is removed, or at the request of merging mickeymouse, depending on bar, re-emerging libfoo first, and bar if the user is not using --as-needed (checking the NEEDED lines). With all due respect to everybody, the right course of action here has to be selected by people who knows how the runtime linker works, symbol collision and all the rest, as that's what's at stake here. -- Diego Flameeyes Pettenò http://blog.flameeyes.eu/ pgpR6kEe6MGbq.pgp Description: PGP signature
Re: [gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Thu, 29 May 2008 11:02:55 +0200 [EMAIL PROTECTED] (Diego 'Flameeyes' Pettenò) wrote: Marius Mauch [EMAIL PROTECTED] writes: As much as we want preserve-libs to be an all-curing magic, it's not. When you need to replace a library you need to do so _for all its users at once_, if you allow it to be gradually you're opening the hellgate of symbol collision. That's what `emerge @preserved-rebuild` does, or do you mean something different? My solution would be to disallow _building_ anything that is or depends directly or indirectly on a package on the set until it is removed, or at the request of merging mickeymouse, depending on bar, re-emerging libfoo first, and bar if the user is not using --as-needed (checking the NEEDED lines). Well, with preserve-libs the situation is this (using your example): - user upgrades expat, portage keeps libexpat.so.0 around (some packages might now be linked against both versions if the session included other packages as well) - emerge tells the user to rebuild all affected packages (affected = contains libexpat.so.0 in NEEDED, so includes both libfoo and bar) by using `emerge @preserved-rebuild` (in the future this could also be done automatically, but that won't be before 2.2 final) - when all affected packages have been rebuilt (so their NEEDED entries don't contain libexpat.so.0 anymore) libexpat.so.0 is automatically removed So, if I understand you correctly (probably not), you want portage to prevent the user from building any packages depending on any affected package before the last step is completed? With all due respect to everybody, the right course of action here has to be selected by people who knows how the runtime linker works, symbol collision and all the rest, as that's what's at stake here. Whoever that is is welcome to voice his opinion here, that's the point of this thread after all. Marius -- gentoo-dev@lists.gentoo.org mailing list
[gentoo-dev] Re: RFC: Should preserve-libs be enabled by default?
On Thu, 29 May 2008 01:13:16 +0200 Marius Mauch [EMAIL PROTECTED] wrote: So, do you think it should be enabled by default? Yes please. :) I haven't had any problems in the couple of months i've been using it. -- fonts, gcc-porting, by design, by neglect mips, treecleaner,for a fact or just for effect wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662 signature.asc Description: PGP signature
