Re: A plan to get rid of unnecessary package dependencies
On Mon, Oct 02, 2006, Peter Samuelson wrote:
The problem is that .la files provide a way to pull in all the
dependent libraries for static linking, and unless you also ship .pc
files, there is no other automated way to do this. Some people
apparently care about this capability, which is why we can't just
delete _all_ .la files _now_.
I think we're already aiming at the removal of *.la files, at least
some maintainers are. I'm removing them from leaf packages or as soon
as the rdeps permit it, and I'm also using the dependency_libs
erasing that was introduced in libxml2 when I can't remove the *.la
files -- this renders *.la files useless but at least not harmful.
On the other hand, with your proposal, we end up keeping the *.la
files. I certainly understand that they ease static linking, but this
also means that we need to depend on all libraries down the dependency
tree (${la:Depends}) of each package to ensure that its *.la files are
functional for static linking, even with the modified dependency_libs
line.
Since there are other ways to offer a static linking solution (adding
pkg-config files) and since not all packages use libtool, I think it
might be time to suggest people not to rely on *.la files for static
linking.
--
Loïc Minier [EMAIL PROTECTED]
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Sun, Oct 01, 2006, Peter Samuelson wrote: A first step in that direction would be to fix .la, .pc and -config files so that they only give the needed libraries. binary-arch: build-arch # [something like '$(MAKE) install DESTDIR=$(shell pwd)/debian/tmp'] # sed -i 's:/usr/lib/lib\([^ ]*\).la:-l\1:g' debian/tmp/usr/lib/*.la Can't we just tell people to not use *.la files for static linking? -- Loïc Minier [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
[Loïc Minier] Can't we just tell people to not use *.la files for static linking? The problem is that .la files provide a way to pull in all the dependent libraries for static linking, and unless you also ship .pc files, there is no other automated way to do this. Some people apparently care about this capability, which is why we can't just delete _all_ .la files _now_. signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
On Mon, Oct 02, 2006 at 02:10:01PM -0500, Peter Samuelson [EMAIL PROTECTED] wrote: [Loïc Minier] Can't we just tell people to not use *.la files for static linking? The problem is that .la files provide a way to pull in all the dependent libraries for static linking, and unless you also ship .pc files, there is no other automated way to do this. Some people apparently care about this capability, which is why we can't just delete _all_ .la files _now_. Why not add support for Libs.private in libtool ? Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
[Mike Hommey] Why not add support for Libs.private in libtool ? That's essentially what the .la file already provides, with _debian_ libtool. The main problem we have with libtool these days is packages that use their own shipped libtool rather than debian's. Anyway, it's probably easiest just to get people to stop shipping .la files at all. Easy to lintian for, etc. signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
[Mike Hommey] A first step in that direction would be to fix .la, .pc and -config files so that they only give the needed libraries. The correct fix for .la files for dynamic linking (remove all dependent libraries, relying on the runtime linker to pull them in recursively) does not work for static linking. Some people apparently still care about static linking, don't ask me who or why. .pc files do not have this limitation, as it's possible to specify two separate lists of libraries, one only for static linking. So, the real solution, insofar as there _are_ real solutions, is to phase out .la files entirely. One problem is that if your .la file is referenced within someone else's .la file, you can't remove it without breaking their package. To eliminate _this_ problem, I suggest that everyone add the following (or similar) to debian/rules: binary-arch: build-arch # [something like '$(MAKE) install DESTDIR=$(shell pwd)/debian/tmp'] # sed -i 's:/usr/lib/lib\([^ ]*\).la:-l\1:g' debian/tmp/usr/lib/*.la signature.asc Description: Digital signature
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Martijn van Oosterhout skrev: Which is all crap. Yes, this is the list you need for static, but pkg-config is recursing through modules even for dynamic linking which is wrong. Now either pkg-config of the gtk+2 pc file needs to be fixed, then you can start recompiling all the affected programs... The gtk+2 .pc file needs to be changed to mark a bunch of those Requires as Requires.private, pkg-config provides all the necessary infrastructure now. (If not, please do file bugs.) - tfheen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 01:35:12PM -0700, Kevin B. McCarty wrote: [**] I hope someone out there knows how a library maintainer can reliably determine this! Use the Source, Luke. For C, you should examine all functions marked with __attribute__((__constructor__)) or __attribute__((__destructor__)). For C++ you need to check every static object as well. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences -
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On 9/28/06, Tollef Fog Heen [EMAIL PROTECTED] wrote: Martijn van Oosterhout skrev: Which is all crap. Yes, this is the list you need for static, but pkg-config is recursing through modules even for dynamic linking which is wrong. Now either pkg-config of the gtk+2 pc file needs to be fixed, then you can start recompiling all the affected programs... The gtk+2 .pc file needs to be changed to mark a bunch of those Requires as Requires.private, pkg-config provides all the necessary infrastructure now. (If not, please do file bugs.) Ok, the reduces the libs list, but it also reduces the cflags list. So much so that you can't actually compile anything gtk-related. It'd be nice if there was a way to specify package that should be included for the purposes cflags, but excluded for the libs. Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Thu, Sep 28, 2006 at 04:17:39PM +0200, Martijn van Oosterhout wrote: The gtk+2 .pc file needs to be changed to mark a bunch of those Requires as Requires.private, pkg-config provides all the necessary infrastructure now. (If not, please do file bugs.) Ok, the reduces the libs list, but it also reduces the cflags list. So much so that you can't actually compile anything gtk-related. Note that Requires.private is used for cflags since the last version of pkg-config. Please see http://bugs.debian.org/340904 Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Thu, Sep 28, 2006, Tollef Fog Heen wrote: Which is all crap. Yes, this is the list you need for static, but pkg-config is recursing through modules even for dynamic linking which is wrong. Now either pkg-config of the gtk+2 pc file needs to be fixed, then you can start recompiling all the affected programs... The gtk+2 .pc file needs to be changed to mark a bunch of those Requires as Requires.private, pkg-config provides all the necessary infrastructure now. (If not, please do file bugs.) Patching of gtk+-2.0.pc and friends is on the TODO, but will be implemented in the 2.10.x series, not in 2.8. pango*.pc would have been my guinea pigs, but I'm not sure it's worth the risk so close to release now. Thanks for the reminder! -- Loïc Minier [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On 9/28/06, Kurt Roeckx [EMAIL PROTECTED] wrote: Note that Requires.private is used for cflags since the last version of pkg-config. Please see http://bugs.debian.org/340904 Well, then something wierd is going on. I have 0.21-1 installed and I get this. This first time is with Requires, the second is with Requires.private. My understanding is that the results should be the same? vali:/usr/lib/pkgconfig# dpkg -l pkg-config snip ii pkg-config 0.21-1 manage compile and link flags for libraries vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -DXTHREADS -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/X11R6/include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/libpng12 vali:/usr/lib/pkgconfig# joe gtk+-2.0.pc editted Requires to Requires.private File gtk+-2.0.pc saved. vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Thu, Sep 28, 2006 at 10:20:18PM +0200, Martijn van Oosterhout [EMAIL PROTECTED] wrote: On 9/28/06, Kurt Roeckx [EMAIL PROTECTED] wrote: Note that Requires.private is used for cflags since the last version of pkg-config. Please see http://bugs.debian.org/340904 Well, then something wierd is going on. I have 0.21-1 installed and I get this. This first time is with Requires, the second is with Requires.private. My understanding is that the results should be the same? vali:/usr/lib/pkgconfig# dpkg -l pkg-config snip ii pkg-config 0.21-1 manage compile and link flags for libraries vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -DXTHREADS -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo -I/usr/include/pango-1.0 -I/usr/X11R6/include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/freetype2 -I/usr/include/libpng12 vali:/usr/lib/pkgconfig# joe gtk+-2.0.pc editted Requires to Requires.private File gtk+-2.0.pc saved. vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo I don't know what pkg-config is supposed to do, but surely you don't need the freetype2 nor the pango, nor the X11 includes files to build gtk applications. Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On 9/28/06, Mike Hommey [EMAIL PROTECTED] wrote: vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo I don't know what pkg-config is supposed to do, but surely you don't need the freetype2 nor the pango, nor the X11 includes files to build gtk applications. Have you tried? Try a simple #include gtk/gtk.h and you get errors like this: In file included from /usr/include/gtk-2.0/gdk/gdkcolor.h:31, from /usr/include/gtk-2.0/gdk/gdkcairo.h:23, from /usr/include/gtk-2.0/gdk/gdk.h:30, from /usr/include/gtk-2.0/gtk/gtk.h:31, from gtktest.c:10: /usr/include/gtk-2.0/gdk/gdktypes.h:33:25: error: pango/pango.h: No such file or directory In file included from /usr/include/gtk-2.0/gdk/gdk.h:30, from /usr/include/gtk-2.0/gtk/gtk.h:31, from gtktest.c:10: /usr/include/gtk-2.0/gdk/gdkcairo.h:25:30: error: pango/pangocairo.h: No such file or directory If you add /usr/include/pango-1.0 it works. Perhaps Requires.private is not recursive? It's not mentioned in the manpage, but I would've expected the there to be no difference between Requires and Requires.private when looking at cflags. At least, that's my interpretation of this thread. Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Thu, Sep 28, 2006 at 10:20:18PM +0200, Martijn van Oosterhout wrote: On 9/28/06, Kurt Roeckx [EMAIL PROTECTED] wrote: Note that Requires.private is used for cflags since the last version of pkg-config. Please see http://bugs.debian.org/340904 Well, then something wierd is going on. I have 0.21-1 installed and I get this. This first time is with Requires, the second is with Requires.private. My understanding is that the results should be the same? Afaik, they should, so I suggest you file a bug. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Thu, Sep 28, 2006 at 10:40:43PM +0200, Martijn van Oosterhout [EMAIL PROTECTED] wrote: On 9/28/06, Mike Hommey [EMAIL PROTECTED] wrote: vali:/usr/lib/pkgconfig# pkg-config --cflags gtk+-2.0 -I/usr/include/gtk-2.0 -I/usr/lib/gtk-2.0/include -I/usr/include/atk-1.0 -I/usr/include/cairo I don't know what pkg-config is supposed to do, but surely you don't need the freetype2 nor the pango, nor the X11 includes files to build gtk applications. Have you tried? Try a simple #include gtk/gtk.h and you get errors like this: In file included from /usr/include/gtk-2.0/gdk/gdkcolor.h:31, from /usr/include/gtk-2.0/gdk/gdkcairo.h:23, from /usr/include/gtk-2.0/gdk/gdk.h:30, from /usr/include/gtk-2.0/gtk/gtk.h:31, from gtktest.c:10: /usr/include/gtk-2.0/gdk/gdktypes.h:33:25: error: pango/pango.h: No such file or directory In file included from /usr/include/gtk-2.0/gdk/gdk.h:30, from /usr/include/gtk-2.0/gtk/gtk.h:31, from gtktest.c:10: /usr/include/gtk-2.0/gdk/gdkcairo.h:25:30: error: pango/pangocairo.h: No such file or directory If you add /usr/include/pango-1.0 it works. Oh my... Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Am Donnerstag 28 September 2006 21:14 schrieb Kurt Roeckx: Note that Requires.private is used for cflags since the last version of pkg-config. Interesting but maybe documenting Requires.private would be a good idea? Hint: the manpage only mentions Libs.private, see #341977 #346602. And 9 months is quite some time :-/ How shall I convince upstream to do it right if pkg-config upstream does not document it? Fixing it only in Debian does not quite convince upstream... HS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Tue, Sep 26, 2006 at 06:29:25PM -0700, Russ Allbery wrote: But folding it into shlibdeps at least would remove all those warnings that were created by shlibdeps. What warnings were created by shlibdeps? I'm not sure what you're referring to here. Sorry. I had a short look at some packages and all redundant dependencies were created by shlibdeps. It doesn't seem to make sense to have a list by maintainer when the dependencies weren't added by him/her. Or did I misunderstand what this discussion is about? Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 09:49:57PM +0200, Christian Aichinger wrote: As a start, I've written a script that searches for unnecessary dependencies and reports them. Results are available here: http://rerun.lefant.net/checklib Excellent work :-). I didn't see a link to the checklib script itself. Do you intend to release it some time? Thanks. Ganesan -- Ganesan Rajagopal -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 11:09:37AM +0200, Michael Meskes wrote: On Tue, Sep 26, 2006 at 06:29:25PM -0700, Russ Allbery wrote: But folding it into shlibdeps at least would remove all those warnings that were created by shlibdeps. What warnings were created by shlibdeps? I'm not sure what you're referring to here. Sorry. I had a short look at some packages and all redundant dependencies were created by shlibdeps. It doesn't seem to make sense to have a list by maintainer when the dependencies weren't added by him/her. Or did I misunderstand what this discussion is about? Yes, I'm afraid you did. The behavior of dpkg-shlibdeps is correct: it documents the packages that must be present on the system in order for the binaries to work. The bug we're discussing happens at the ELF linker level -- you can't have dpkg-shlibdeps decide to ignore some of these libs, because the *binary* still embeds references to them and if they're missing, the binary will not work. So the fix is to get our binaries to stop embedding references to libraries they don't need, then dpkg-shlibdeps will do the right thing automatically. Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 02:05:29PM +0530, Ganesan Rajagopal wrote: Excellent work :-). Thanks :) I didn't see a link to the checklib script itself. Do you intend to release it some time? Thanks. It's linked at the bottom of all the pages, the link points to URL:http://greek0.net/div/checklib.tar.gz Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
Sorry. I had a short look at some packages and all redundant dependencies were created by shlibdeps. It doesn't seem to make sense to have a list by maintainer when the dependencies weren't added by him/her. Or did I misunderstand what this discussion is about? Yes, I'm afraid you did. The behavior of dpkg-shlibdeps is correct: it documents the packages that must be present on the system in order for the binaries to work. The bug we're discussing happens at the ELF linker level ... Ah, silly me, I thought the problem was a Depends: entry not a real dependancy. Sorry for the confusion. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Tue, Sep 26, 2006 at 06:28:50PM -0700, Russ Allbery wrote: Christian Aichinger [EMAIL PROTECTED] writes: Maintaining such information somewhere within the library package would be possible, but that sounds like a more complex plan, and I doubt that many library maintainers know if their lib uses such tricks or not. does still apply. As a first approximation perhaps libraries with constructors or destructors could be assumed to always be required? That would be more conservative than required but perhaps so conservative as to be useless. -- You grabbed my hand and we fell into it, like a daydream - or a fever. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 12:50:07PM +0100, Mark Brown wrote: As a first approximation perhaps libraries with constructors or destructors could be assumed to always be required? That would be more conservative than required but perhaps so conservative as to be useless. The problematic sections seem to be .init and .fini. There are sections called .ctors and .dtors, but they aren't marked as executable. However both .init and .fini seem to be always present, and include code too. To see the sections in the ELF file: readelf -S file To see the code in .init and .fini: objdump -D file | sed -n -e '/\.init\|\.fini/,/^Dis/p' What I could imagine was some sort of opt-in system, where library maintainers could put a special marker into their -dev packages indicating yes, please remove me if I'm not needed, it's save. I'm not sure where to put it inside the -dev package though, perhaps in an /usr/share/debhelper/stripdeps/pkgname file? Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 02:57:16AM -0700, Steve Langasek [EMAIL PROTECTED] wrote: On Wed, Sep 27, 2006 at 11:09:37AM +0200, Michael Meskes wrote: On Tue, Sep 26, 2006 at 06:29:25PM -0700, Russ Allbery wrote: But folding it into shlibdeps at least would remove all those warnings that were created by shlibdeps. What warnings were created by shlibdeps? I'm not sure what you're referring to here. Sorry. I had a short look at some packages and all redundant dependencies were created by shlibdeps. It doesn't seem to make sense to have a list by maintainer when the dependencies weren't added by him/her. Or did I misunderstand what this discussion is about? Yes, I'm afraid you did. The behavior of dpkg-shlibdeps is correct: it documents the packages that must be present on the system in order for the binaries to work. The bug we're discussing happens at the ELF linker level -- you can't have dpkg-shlibdeps decide to ignore some of these libs, because the *binary* still embeds references to them and if they're missing, the binary will not work. So the fix is to get our binaries to stop embedding references to libraries they don't need, then dpkg-shlibdeps will do the right thing automatically. A first step in that direction would be to fix .la, .pc and -config files so that they only give the needed libraries. Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 02:44:10PM +0200, Christian Aichinger wrote: The problematic sections seem to be .init and .fini. There are sections called .ctors and .dtors, but they aren't marked as executable. .ctors and .dtors contain function pointers that are called by code in the .init/.fini sections (crtbegin.o, crtend.o). Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences -
Re: A plan to get rid of unnecessary package dependencies
Christian Aichinger [EMAIL PROTECTED] writes: What I could imagine was some sort of opt-in system, where library maintainers could put a special marker into their -dev packages indicating yes, please remove me if I'm not needed, it's save. I'm not sure where to put it inside the -dev package though, perhaps in an /usr/share/debhelper/stripdeps/pkgname file? Well, if we're talking about an external tool to strip unnecessary NEEDED entries from the library, you could call it dh_striplibs, include it in debhelper, and have the opt-in system be whether the maintainer chooses to run it or not. That doesn't help the folks who aren't using debhelper as much, though. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Wed, Sep 27, 2006 at 10:54:34AM -0700, Russ Allbery wrote: I'm not sure where to put it inside the -dev package though, perhaps in an /usr/share/debhelper/stripdeps/pkgname file? Well, if we're talking about an external tool to strip unnecessary NEEDED entries from the library, you could call it dh_striplibs, include it in debhelper, and have the opt-in system be whether the maintainer chooses to run it or not. That's what I was thinking of. That doesn't help the folks who aren't using debhelper as much, though. We could also put the flags in another location then under /usr/share/debhelper (or into the control tarball, though that sounds cumbersome), so other build systems could use it too. Ideas where to put such a thing? Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
to run it or not. That doesn't help the folks who aren't using debhelper = as much, though. But isn't the decision of the maintainer to use dh_striplibs orthogonal to the need for a library not to be stripped? That is, some set of libraries presumably shouldn't be stripped out of NEEDED entries (because they provide constructors/destructors), but that is not likely to correspond to the set of maintainers who decide to strip apparently-superfluous NEEDED entries from their own program or library. I think what's needed is two-fold. First, a library packager should be able to say I'm sure my library doesn't use constructors/destructors or anything else that would make checklib give a false negative for it. Second, any maintainer should be able to take advantage of that fact and make that library's NEEDED entry be stripped from his/her own package if (a) it appears unneeded and (b) the library packager has promised it would be OK. The first item could be implemented with a new debhelper tool, for the sake of argument call it dh_setstrippable. This would just register the fact in some file of the library package, say /var/lib/dpkg/info/libfoo.strippable, similar to the shlibs file. This could just contain a list of sonames that the library maintainer knows actually could be stripped away if checklib thinks they appear unneeded. For instance, the file /var/lib/dpkg/info/libgtk2.0-0.strippable might have these contents: libgdk-x11-2.0.so.0 libgdk_pixbuf-2.0.so.0 libgdk_pixbuf_xlib-2.0.so.0 libgtk-x11-2.0.so.0 caused by the GTK maintainer putting this command in debian/rules: dh_setstrippable -plibgtk2.0-0 libgdk-x11-2.0.so.0 \ libgdk_pixbuf-2.0.so.0 [...] The second item could be a new feature of dh_strip; it would check for apparently unneeded NEEDED entries [*] using the same logic as checklib, and, for each soname that is registered in a strippable file, it would nuke the NEEDED entry. (Whether this is possible, or how this is done, I unfortunately have no idea.) One could of course imagine dh_strip having a new flag that would turn this behavior off if for some reason the maintainer found it unwanted. [*] unneeded NEEDED entries: there's a great oxymoron! Since dh_strip is run in the build of just about every package containing a binary or library, the only effort required for this to work would be the opt-in on the part of library maintainers (who, one hopes, tend to be among the most active and knowledgeable maintainers). As more library maintainers determined it was safe to opt in [**], and new versions of packages were recompiled and uploaded, the problem of unnecessary library dependencies would slowly be reduced. No annoying relibtoolization, no potentially dangerous use of ld's --as-needed flag, and no editing of pkg-config or .la files required! [**] I hope someone out there knows how a library maintainer can reliably determine this! best regards, --=20 Kevin B. McCarty [EMAIL PROTECTED] Physics Department WWW: http://www.princeton.edu/~kmccarty/Princeton University GPG: public key ID 4F83C751 Princeton, NJ 08544 signature.asc Description: OpenPGP digital signature
Re: A plan to get rid of unnecessary package dependencies
ugh, somehow one of Thunderbird, Enigmail or my SMTP server mangled the quoted part of my last email at Message-id: [EMAIL PROTECTED], maybe also the GPG sig. Sorry about that! Everything I wanted to say went through OK though. -- Kevin B. McCarty [EMAIL PROTECTED] Physics Department WWW: http://www.princeton.edu/~kmccarty/Princeton University GPG: public key ID 4F83C751 Princeton, NJ 08544 signature.asc Description: OpenPGP digital signature
Problems with g++ (Was: Re: A plan to get rid of unnecessary package dependencies)
Hello, On Mon, 25 Sep 2006, Christian Aichinger wrote: As Steve Langasek detailed in his d-d-a mail[1], unnecessary dependencies cause lots of problems, as they make transitions bigger then they need to be. Sorry for the noise. I wrote earlier to Christian Aichinger (and submitted a report #389555 against g++) and got no response. On Tue, 26 Sep 2006, I wrote: I have filed an ITA for swish++ so the problems with that package land in my lap! When looking through the makefiles and the libraries that are linked there is no mention of the two offending libraries libm or libgcc. On the other hand I noticed (by giving CC=g++ -v) that it was g++ which was putting these libraries in its call to collect2. If instead of this I link with gcc and add a -lstdc++ in order to get it to link correctly then the extra NEEDED's disappear. I suppose I could fix the problem this way, but is this not a bug with g++? On Tue, 26 Sep 2006, I wrote: Actually, I found the following thread: http://gcc.gnu.org/ml/gcc/2005-01/msg00229.html This seems to indicate that g++ always links in libm and libgcc_*. This will mean that programs that use g++ to link will always create a problem. Perhaps this should be solved by *not* using g++ to link. I have now created a version of the swish++ package that does not use g++ to link and uses gcc instead. My question is whether this is The Way. If so it could be documented as a solution to at least some of the problem cases. Regards, Kapil. P.S. Please CC me I do not subscribe to this list. -- signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
Christian Aichinger [EMAIL PROTECTED] writes: As a start, I've written a script that searches for unnecessary dependencies and reports them. Results are available here: http://rerun.lefant.net/checklib A problem means that the package has useless dependencies on library packages. This causes the kind of trouble outlined above and should be fixed. A HOWTO is here: http://rerun.lefant.net/checklib/howto-fix-problems.html Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? As it happens, your script missed gnucash, which has this problem in spades, and can't just use the Debian libtool, because gnucash's build system actually *depends* on the documented behavior of libtool, and breaks with the Debian one. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Christian Aichinger [EMAIL PROTECTED] writes: A problem means that the package has useless dependencies on library packages. This causes the kind of trouble outlined above and should be fixed. A HOWTO is here: http://rerun.lefant.net/checklib/howto-fix-problems.html Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? How could shlibdeps help? The trouble with these dependencies is that, when the library or binary is loaded, all of those NEEDED libraries will also be loaded, even though they're not actually needed. So the shared library or binary *does* depend on those libraries, in that it won't run unless they're present, despite the fact that it's not actually using any symbols from them. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
Russ Allbery [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: Christian Aichinger [EMAIL PROTECTED] writes: A problem means that the package has useless dependencies on library packages. This causes the kind of trouble outlined above and should be fixed. A HOWTO is here: http://rerun.lefant.net/checklib/howto-fix-problems.html Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? How could shlibdeps help? The trouble with these dependencies is that, when the library or binary is loaded, all of those NEEDED libraries will also be loaded, even though they're not actually needed. So the shared library or binary *does* depend on those libraries, in that it won't run unless they're present, despite the fact that it's not actually using any symbols from them. Right, sorry. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
[Richard Atterer] On Mon, Sep 25, 2006 at 02:40:49PM -0700, Kevin B. McCarty wrote: Thank you for this very cool effort! Might we see checklib packaged for Debian soon? Hmm, maybe the functionality could be included in lintian? See #340934. Henning wrote this lintian check several months ago, but Jeroen explains in the bug log why it was not accepted. I use it locally, and it works fine. (Well, modulo the bug I reported later to that same bug log.) signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
Re: Christian Aichinger 2006-09-25 [EMAIL PROTECTED] As a start, I've written a script that searches for unnecessary dependencies and reports them. Results are available here: http://rerun.lefant.net/checklib DDPO (aka developer.php) has a link to the list of checklib results per maintainer. It's at the top in the 'General information' section. http://qa.debian.org/[EMAIL PROTECTED] http://rerun.lefant.net/checklib/[EMAIL PROTECTED] Christoph -- [EMAIL PROTECTED] | http://www.df7cb.de/ signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 11:12:49PM -0700, Thomas Bushnell BSG wrote: Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? Note that if a shared object has constructor and/or destructor functions then there is NO WAY to automatically decide if it is needed or not, even if it does not even provide exported symbols. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Tue, Sep 26, 2006 at 01:34:50PM +0200, Gabor Gombas wrote: You could invent a mechanism to tag a library as never to remove even if no symbols appear to be referenced. Since this should be quite rare even a single (config?) file listing all problematic libraries would be enough, at least in the beginning. I could maintain such a list for checklib, that wouldn't be problematic. Doing such a thing for debhelper sounds evil though, I surely don't want to be responsible if packages fail to build (or run correctly) because a library was removed that shouldn't have been. Maintaining such information somewhere within the library package would be possible, but that sounds like a more complex plan, and I doubt that many library maintainers know if their lib uses such tricks or not. Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
On Tue, Sep 26, 2006 at 12:35:35PM +0200, Christian Aichinger wrote: On Mon, Sep 25, 2006 at 11:12:49PM -0700, Thomas Bushnell BSG wrote: Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? As pointed out by Russ Allbery, shlibdeps wouldn't do. It would be possible to stuff it into dh_strip or a new debhelper script, which just removes the unnecessary NEEDED entries. But folding it into shlibdeps at least would remove all those warnings that were created by shlibdeps. Michael -- Michael Meskes Email: Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org) ICQ: 179140304, AIM/Yahoo: michaelmeskes, Jabber: [EMAIL PROTECTED] Go SF 49ers! Go Rhein Fire! Use Debian GNU/Linux! Use PostgreSQL! -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 11:12:49PM -0700, Thomas Bushnell BSG wrote: Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? As pointed out by Russ Allbery, shlibdeps wouldn't do. It would be possible to stuff it into dh_strip or a new debhelper script, which just removes the unnecessary NEEDED entries. On Tue, 26 Sep 2006 10:39:29 +0200, Gabor Gombas wrote: Note that if a shared object has constructor and/or destructor functions then there is NO WAY to automatically decide if it is needed or not, even if it does not even provide exported symbols. That's one major problem, which is also the reason why I haven't implemented a dh_ script. AIUI the number of libraries using such tricks is minor, so it is unnerving that we can't use the dh_ method because of that. I don't see any solution though :-/ Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
On Tue, Sep 26, 2006 at 12:35:35PM +0200, Christian Aichinger wrote: AIUI the number of libraries using such tricks is minor, so it is unnerving that we can't use the dh_ method because of that. I don't see any solution though :-/ You could invent a mechanism to tag a library as never to remove even if no symbols appear to be referenced. Since this should be quite rare even a single (config?) file listing all problematic libraries would be enough, at least in the beginning. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences -
Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Christian Aichinger wrote:
As a start, I've written a script that searches for unnecessary
dependencies and reports them. Results are available here:
http://rerun.lefant.net/checklib
More detailed information about the meaning of the results are
available on the web page, the two most important points are
problems and errors.
A problem means that the package has useless dependencies on
library packages. This causes the kind of trouble outlined above and
should be fixed. A HOWTO is here:
http://rerun.lefant.net/checklib/howto-fix-problems.html
In case it's of interest to anyone, I went through the checklib logs
available on the web page for problems and found the libraries that
are most often listed as bogus dependencies. Here are the top twenty
offenders, listed with the number of packages that (according to
checklib) have each as an unnecessary dependency:
1663: libxext6
1275: libxi6
1196: zlib1g
1174: libx11-6
949: libice6
948: libsm6
940: libfontconfig1
923: libxrender1
918: libxinerama1
918: libxcursor1
905: libxrandr2
697: libatk1.0-0
688: libcairo2
636: libgcc1
610: libxfixes3
536: libpango1.0-0
480: libpng12-0
424: libfreetype6
394: libart-2.0-2
386: libxml2
Most of these are X-related, suggesting that quite a lot of .la and .pc
files are pretty indiscriminate about which X libs they link in.
I used the attached (fairly trivial) script to scan through the log
files. It is pretty dependent on the exact file format of the logs, though.
regards,
--
Kevin B. McCarty [EMAIL PROTECTED] Physics Department
WWW: http://www.princeton.edu/~kmccarty/Princeton University
GPG: public key ID 4F83C751 Princeton, NJ 08544
#!/bin/sh
if [ $1 = download ] ; then
wget -r --level=1 'http://rerun.lefant.net/checklib/status.PROBLEM.html'
fi
rm -f reverse-deps.txt
touch reverse-deps.txt
for file in rerun.lefant.net/checklib/log.*.html ; do
for lib in `grep -B 1 '^RESULT: PROBLEMS' $file | head -n 1 | tr -d ,` ; do
line=`grep '[:] '$lib'$' reverse-deps.txt`
if [ -n $line ] ; then
n=`echo $line | awk '{print $1}' | tr -d ':'`
n2=$(( $n + 1 ))
sed -i 's/^'$n'\(: '$lib'\)$/'$n2'\1'/ reverse-deps.txt
else
echo '1: '$lib reverse-deps.txt
fi
done
done
sort -n reverse-deps.txt tmp
mv -f tmp reverse-deps.txt
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
You ([EMAIL PROTECTED]) wrote: KBM Most of these are X-related, suggesting that quite a lot of .la KBM and .pc files are pretty indiscriminate about which X libs they KBM link in. Will this problem disappear if end programs will pass --as-needed flag to the ld command line? -- JID: [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Mikhail Gusarov wrote: You ([EMAIL PROTECTED]) wrote: KBM Most of these are X-related, suggesting that quite a lot of .la KBM and .pc files are pretty indiscriminate about which X libs they KBM link in. Will this problem disappear if end programs will pass --as-needed flag to the ld command line? I asked myself the very same question a few weeks ago, but someone said that too much black magic is involved with this parameter -- whatever this means. The main problem seems to be, that relibtoolizing seems to be too complicated and error prone so many developers just skip this part. So I think we need an easy (automatic) solution in order to get rid of this problem once and for all. Cheers, Bastian -- Bastian Venthur http://venthur.de -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Mikhail Gusarov wrote: You ([EMAIL PROTECTED]) wrote: KBM Most of these are X-related, suggesting that quite a lot of .la KBM and .pc files are pretty indiscriminate about which X libs they KBM link in. Will this problem disappear if end programs will pass --as-needed flag to the ld command line? In many cases, yes, but there may be some breakage. I'm by no means a binutils expert, but following are the caveats I'm aware of. A little before the release of Sarge, there was a discussion about whether to do this by default: http://lists.debian.org/debian-devel/2005/04/msg2.html There is some concern that ld may strip away libraries that provide constructors and destructors using this option. See Gabor's emails in the old thread: http://lists.debian.org/debian-devel/2005/04/msg2.html and in this thread: http://lists.debian.org/debian-devel/2006/09/msg00908.html There is also concern that some libraries do *not* link against other libraries they need, leaving it up to the end program to link them in. Suppose libA.so uses symbols from libB.so, but doesn't link it. (This is arguably broken, but it happens. But IMO a bug should be filed on such libraries.) Someone then writes program C, which uses symbols only from libA.so, not from libB.so. Nevertheless, linking program C requires -lA -lB. I think --as-needed will break in those cases. Further, --as-needed doesn't work when creating shared libraries (at least it didn't at the time I experimented with it; instead it stripped out *all* NEEDED entries from the .so file!) so it won't help with those cases of unnecessary dependencies. Finally, if you use --as-needed in a Debian package, you'll want to have a versioned Build-Depends on binutils (= 2.16.1cvs20050902-1) to ensure you avoid http://bugs.debian.org/320697 . regards, -- Kevin B. McCarty [EMAIL PROTECTED] Physics Department WWW: http://www.princeton.edu/~kmccarty/Princeton University GPG: public key ID 4F83C751 Princeton, NJ 08544 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006 at 11:41:33PM +0700, Mikhail Gusarov wrote: You ([EMAIL PROTECTED]) wrote: KBM Most of these are X-related, suggesting that quite a lot of .la KBM and .pc files are pretty indiscriminate about which X libs they KBM link in. Will this problem disappear if end programs will pass --as-needed flag to the ld command line? --as-needed basicly does the same as checklib. They both might say that some library isn't needed while it actually is. It might work in most cases, but it might break in others. The best solution is to avoid telling the linker it should be linking to that library, but that might be hard in some cases. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 02:40:49PM -0700, Kevin B. McCarty wrote:
One thing I noticed is that there are a lot of problems (in your
terminology) caused by unneeded dependencies on libgcc1
(/lib/libgcc_s.so.1). From my quick investigation, it appears that the
C++ and Fortran compilers (g++, g77, gfortran) introduce this dependency
automatically to programs linked with them. However, if gcc is instead
used in the linking step, no such dependency is created (at least on
amd64 where I'm testing).
That is, programs compiled like this have libgcc_s.so.1 NEEDED:
g++ foo.cc
g77 foo.F
gfortran foo.F
but if they are compiled to object code and then linked with plain
vanilla gcc, like this, they don't:
g++ -c foo.cc -o foo.o gcc foo.o -lstdc++
g77 -c foo.F -o foo.o gcc foo.o -lg2c -lfrtbegin
gfortran -c foo.F -o foo.o gcc foo.o -lgfortran -lgfortranbegin
[CC'ed to debian-gcc to see if someone there can explain why this happens.]
Atleast for g++, it ends up being linked to both -lm and -lgcc_s
Using g++ I get:
[pid 22315] execve(/usr/bin/ld, [/usr/bin/ld, --eh-frame-hdr, -m,
elf_x86_64, -dynamic-linker, /lib64/ld-linux-x86-64.so.2, -o, foo,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crt1.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crti.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtbegin.o,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64, -L/lib/../lib64,
-L/usr/lib/../lib64, foo.o, -lstdc++, -lm, -lgcc_s, -lgcc,
-lc, -lgcc_s, -lgcc, /usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtend.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crtn.o], [/* 18 vars
*/]) = 0
Using gcc I get:
[pid 22347] execve(/usr/bin/ld, [/usr/bin/ld, --eh-frame-hdr, -m,
elf_x86_64, -dynamic-linker, /lib64/ld-linux-x86-64.so.2, -o, foo,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crt1.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crti.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtbegin.o,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2,
-L/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64, -L/lib/../lib64,
-L/usr/lib/../lib64, foo.o, -lstdc++, -lgcc, --as-needed, -lgcc_s,
--no-as-needed, -lc, -lgcc, --as-needed, -lgcc_s, --no-as-needed,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtend.o,
/usr/lib/gcc/x86_64-linux-gnu/4.1.2/../../../../lib64/crtn.o], [/* 18 vars
*/]) = 0
Notice the --as-needed and --no-as-needed around the -lgcc_s's.
From the specs:
*libgcc:
%{static|static-libgcc:-lgcc
-lgcc_eh}%{!static:%{!static-libgcc:%{!shared-libgcc:-lgcc --as-needed -lgcc_s
--no-as-needed}%{shared-libgcc:-lgcc_s%{!shared: -lgcc
g++ seem to be using some other specs, but I have no idea how I
should dump them.
Kurt
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006 at 09:36:08AM -0700, Kevin B. McCarty wrote: In case it's of interest to anyone, I went through the checklib logs available on the web page for problems and found the libraries that are most often listed as bogus dependencies. Here are the top twenty offenders, listed with the number of packages that (according to checklib) have each as an unnecessary dependency: 1663: libxext6 out of 2061 packages that have a Depends on it on unstable/amd64, so that's about 80% 1275: libxi6 out of 1393: 91% 1196: zlib1g out of 1796: 66% 1174: libx11-6 out of 2502: 47% 949: libice6 out of 983: 96% 948: libsm6 out of 976: 97% 940: libfontconfig1 out of 1136: 82% 923: libxrender1 out of 1071: 86% 918: libxinerama1 out of 1093: 84% 918: libxcursor1 out of 1053: 87% 905: libxrandr2 out of 1049: 86% 697: libatk1.0-0 out of 850: 82% 688: libcairo2 out of 865: 79% 636: libgcc1 out of 2277: 28% 610: libxfixes3 out of 653: 93% 536: libpango1.0-0 out of 881: 71% 480: libpng12-0 out of 753: 64% 424: libfreetype6 out of 596: 71% 394: libart-2.0-2 out of 444: 89% 386: libxml2 out of 790: 49% Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006 at 09:36:08AM -0700, Kevin B. McCarty wrote: In case it's of interest to anyone, I went through the checklib logs available on the web page for problems and found the libraries that are most often listed as bogus dependencies. This seriously rocks. Thanks. Actually that should be terribly easy to get this out of the DB checklib uses, but it isn't, mostly due to lazyness on my side. Might I include your script in checklib (under GPLv2+, if possible), and put the results on a seperate page on rerun.lefant.net/checklib? Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006 at 11:41:33PM +0700, Mikhail Gusarov wrote: Will this problem disappear if end programs will pass --as-needed flag to the ld command line? As described in the other mails, --as-needed is a hack, and can cause trouble. I've also thought of a debhelper script stipping out useless NEEDED entries, but that suffers from the same problems as --as-needed (mostly related to init/fini functions). On IRC the idea came up to include such checks in piuparts, however I haven't yet investigated how this would fit with its architecture (conceptionally it should be the right place, though). It's really quite sad that we can't easily automate such checks during package build, as that would be the easiest way to get rid of all the dependencies. Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Christian Aichinger wrote: On Tue, Sep 26, 2006 at 09:36:08AM -0700, Kevin B. McCarty wrote: In case it's of interest to anyone, I went through the checklib logs available on the web page for problems and found the libraries that are most often listed as bogus dependencies. This seriously rocks. Thanks. Actually that should be terribly easy to get this out of the DB checklib uses, but it isn't, mostly due to lazyness on my side. Might I include your script in checklib (under GPLv2+, if possible), and put the results on a seperate page on rerun.lefant.net/checklib? Yes, sure! You'll probably want to make serious improvements to it since it was just something I whipped out very quickly. I hereby license the script under the MIT license, to wit: Copyright (c) 2006 Kevin B. McCarty Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the Software), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. (This is of course GPL-compatible :-) Is there any chance you could also have it output the percentage of dependencies on each library which are unnecessary (as Kurt did in http://lists.debian.org/debian-devel/2006/09/msg00940.html )? best regards, -- Kevin B. McCarty [EMAIL PROTECTED] Physics Department WWW: http://www.princeton.edu/~kmccarty/Princeton University GPG: public key ID 4F83C751 Princeton, NJ 08544 signature.asc Description: OpenPGP digital signature
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On 9/26/06, Mikhail Gusarov [EMAIL PROTECTED] wrote: You ([EMAIL PROTECTED]) wrote: KBM Most of these are X-related, suggesting that quite a lot of .la KBM and .pc files are pretty indiscriminate about which X libs they KBM link in. Will this problem disappear if end programs will pass --as-needed flag to the ld command line? Some of it is also pkg-config's fault. For example, anyone using pkg-config with gtk+2 gets the following: # pkg-config --libs gtk+-2.0 -L/usr/X11R6/lib -lgtk-x11-2.0 -lgdk-x11-2.0 -lXrandr -lXi -lXinerama -lXext -latk-1.0 -lgdk_pixbuf-2.0 -lm -lpangocairo-1.0 -lfontconfig -lXcursor -lpango-1.0 -lcairo -lXrender -lX11 -lgobject-2.0 -lgmodule-2.0 -ldl -lglib-2.0 Which is all crap. Yes, this is the list you need for static, but pkg-config is recursing through modules even for dynamic linking which is wrong. Now either pkg-config of the gtk+2 pc file needs to be fixed, then you can start recompiling all the affected programs... Some of the discussion in bug #340904 is relevent here. Have a nice day, -- Martijn van Oosterhout [EMAIL PROTECTED] http://svana.org/kleptog/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
* Kevin B. McCarty [EMAIL PROTECTED] [060926 18:37]: Most of these are X-related, suggesting that quite a lot of .la and .pc files are pretty indiscriminate about which X libs they link in. There is also AC_PATH_XTRA, which just adds all X stuff when only few may be needed. Also note that Xt libs (Xt, Xaw, Xaw3d) sometimes have funny effects as the order they are linked against can make a difference sometimes. Hochachtungsvoll, Bernhard R. Link -- Sendmail is like emacs: A nice operating system, but missing an editor and a MTA. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
Le mardi 26 septembre 2006 à 09:36 -0700, Kevin B. McCarty a écrit : In case it's of interest to anyone, I went through the checklib logs available on the web page for problems and found the libraries that are most often listed as bogus dependencies. Here are the top twenty offenders, listed with the number of packages that (according to checklib) have each as an unnecessary dependency: [lots of X libraries] For those interested in reducing such dependencies, the GNOME team is using Mike Hommey's hack to strip indirect dependencies from .la files: for file in $(wildcard debian/$(cdbs_curpkg)/usr/lib/*.la); do \ sed -i /dependency_libs/ s/'.*'/''/ $$file ; \ done Currently this is mostly done for experimental GNOME 2.16 packages, but it has dramatically cut off some dependencies. It also helps building a path towards complete removal of .la files, which we hope to achieve for GNOME packages in etch+1. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: Ceci est une partie de message numériquement signée
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006 at 09:23:23PM +0200, Martijn van Oosterhout wrote: Will this problem disappear if end programs will pass --as-needed flag to the ld command line? Some of it is also pkg-config's fault. For example, anyone using pkg-config with gtk+2 gets the following: # pkg-config --libs gtk+-2.0 -L/usr/X11R6/lib -lgtk-x11-2.0 -lgdk-x11-2.0 -lXrandr -lXi -lXinerama -lXext -latk-1.0 -lgdk_pixbuf-2.0 -lm -lpangocairo-1.0 -lfontconfig -lXcursor -lpango-1.0 -lcairo -lXrender -lX11 -lgobject-2.0 -lgmodule-2.0 -ldl -lglib-2.0 Which is all crap. Yes, this is the list you need for static, but gtk1.2 also seems to be noisy; at least two of my packages (libstroke, mtrack) pick up a ton of apparently unneeded deps from gtk-config; [8:48am] [EMAIL PROTECTED]:ham/mtrack/mtrack-0.3 pkg-config gtk --libs -rdynamic -L/usr/lib -L/usr/X11R6/lib -lgtk -lgdk -lgmodule -lglib -ldl -lXi -lXext -lX11 -lm The pkg-config results are the same: [8:48am] [EMAIL PROTECTED]:ham/mtrack/mtrack-0.3 gtk-config --libs -L/usr/lib -L/usr/X11R6/lib -lgtk -lgdk -rdynamic -lgmodule -lglib -ldl -lXi -lXext -lX11 -lm checklib reports that libdl, libXi, libXext and libX11 all become unnecessary dependencies of the mtrack package as a result. Hamish -- Hamish Moffatt VK3SB [EMAIL PROTECTED] [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
Christian Aichinger [EMAIL PROTECTED] writes: I could maintain such a list for checklib, that wouldn't be problematic. Doing such a thing for debhelper sounds evil though, I surely don't want to be responsible if packages fail to build (or run correctly) because a library was removed that shouldn't have been. For a debhelper script, there's a fairly standard way of handling such things, namely the -X option that's supported by various debhelper commands (dh_strip, dh_compress, etc.). However, this problem: Maintaining such information somewhere within the library package would be possible, but that sounds like a more complex plan, and I doubt that many library maintainers know if their lib uses such tricks or not. does still apply. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
Michael Meskes [EMAIL PROTECTED] writes: On Tue, Sep 26, 2006 at 12:35:35PM +0200, Christian Aichinger wrote: On Mon, Sep 25, 2006 at 11:12:49PM -0700, Thomas Bushnell BSG wrote: Hey. If we have a programmatic way of detecting this cases, why not just fold that into the shlibdeps program, and be done with it? As pointed out by Russ Allbery, shlibdeps wouldn't do. It would be possible to stuff it into dh_strip or a new debhelper script, which just removes the unnecessary NEEDED entries. But folding it into shlibdeps at least would remove all those warnings that were created by shlibdeps. What warnings were created by shlibdeps? I'm not sure what you're referring to here. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Top 20 unnecessary dependencies [was: Re: A plan to get rid of unnecessary package dependencies]
On Tue, Sep 26, 2006, Josselin Mouette wrote: for file in $(wildcard debian/$(cdbs_curpkg)/usr/lib/*.la); do \ sed -i /dependency_libs/ s/'.*'/''/ $$file ; \ done To use this feature, if you use CDBS, simply: include /usr/share/gnome-pkg-tools/1/rules/clean-la.mk and build-dep on gnome-pkg-tools = 0.7. I'm using it as follow in my non-CDBS packages: sed -i /dependency_libs/ s/'.*'/''/ debian/$(DEV_PKG)/usr/lib/*.la -- Loïc Minier [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Creo que esto debe ser tomado muy en cuenta para los que tenemos paquetes. Saludos. Christian Aichinger escribió: As Steve Langasek detailed in his d-d-a mail[1], unnecessary dependencies cause lots of problems, as they make transitions bigger then they need to be. The root cause are unnecessary libraries on the linker commandline, which get stored in the resulting binary and which dpkg-shlibdeps happily turns into inter-package dependencies. Thus the goal is to eliminate those unnecessary linker arguments. As a start, I've written a script that searches for unnecessary dependencies and reports them. Results are available here: http://rerun.lefant.net/checklib More detailed information about the meaning of the results are available on the web page, the two most important points are problems and errors. A problem means that the package has useless dependencies on library packages. This causes the kind of trouble outlined above and should be fixed. A HOWTO is here: http://rerun.lefant.net/checklib/howto-fix-problems.html The other, even more important issue, are checklib errors. They indicate that the package check failed. That's usually caused by deps on non-existing packages or missing dependencies. Both are (at least) important bugs in the package! A summary of the possible causes for errors and some hints for fixing them are here: http://rerun.lefant.net/checklib/howto-fix-errors.html Below is the dd-list output of packages which are missing dependencies, and where this can cause real problems (i.e. the package can be installed without pulling in all the necessary library packages). All the packages listed below have (not yet filed) serious bugs. If one of your packages is affected please fix it ASAP. Please take a look at the results for packages you are involved in, and try to fix the issues if you see problems or errors. Cheers, Christian Aichinger [1]: http://lists.debian.org/debian-devel-announce/2005/11/msg00016.html Daniel Baumann [EMAIL PROTECTED] lush Jay Bonci [EMAIL PROTECTED] libxml-libxml-common-perl Marco Cabizza [EMAIL PROTECTED] libgnome-media0 Debian Hebrew Packaging Team [EMAIL PROTECTED] libhdate-perl libhdate-python libhocr-perl libhocr-python Debian Java Maintainers pkg-java-maintainers@lists.alioth.debian.org eclipse-jdt-gcj eclipse-pde-gcj eclipse-platform-gcj eclipse-rcp-gcj Patrick Dreker [EMAIL PROTECTED] python-pyxmpp Dirk Eddelbuettel [EMAIL PROTECTED] r-cran-fmultivar r-cran-fseries Peter Van Eynde [EMAIL PROTECTED] cmucl-clm Gustavo Franco [EMAIL PROTECTED] python-notify David Moreno Garza [EMAIL PROTECTED] rxvt Debian QA Group [EMAIL PROTECTED] rxvt-beta socketapi1 Christoph Haas [EMAIL PROTECTED] fyre Dafydd Harries [EMAIL PROTECTED] libjinglebase0.3-0 libjinglep2p0.3-0 libjinglesession0.3-0 libjinglexmllite0.3-0 libjinglexmpp0.3-0 Nobuhiro Iwamatsu [EMAIL PROTECTED] libflash-mozplugin libflash-swfplayer Matthias Klose [EMAIL PROTECTED] python-pygresql tix Carlos Laviola [EMAIL PROTECTED] fp-ide Chris Lawrence [EMAIL PROTECTED] r-cran-bayesm Ricardo Javier Cardenes Medina [EMAIL PROTECTED] python-kde3 Vittorio Palmisano [EMAIL PROTECTED] python-matplotlib Michael Piefel [EMAIL PROTECTED] python-magic Doug Rintoul [EMAIL PROTECTED] kmflcomp Kevin M. Rosenberg [EMAIL PROTECTED] ctsim ctsim-athlon ctsim-pentium4 Anibal Monsalve Salazar [EMAIL PROTECTED] libgii1-target-x Joop Stakenborg [EMAIL PROTECTED] libhamlib2 William Vera [EMAIL PROTECTED] gsetroot - -- ~ghostbar @ linux/debian 'unstable' on i686 - Linux Counter# 382503 http://ghostbar.ath.cx/ - irc.freenode.net #talug #velug #debian-es http://debianvenezuela.org.ve - irc.debian.org #debian-es #debian-ve San Cristobal - Venezuela. TALUG -- http://linuxtachira.org CHASLUG -- http://chaslug.org.ve - irc.unplug.org.ve #chaslug Fingerprint = 3E7D 4267 AFD5 2407 2A37 20AC 38A0 AD5B CACA B118 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.5 (GNU/Linux) iD8DBQFFGXGWOKCtW8rKsRgRAqO9AJ9ynfTv3Dee6NhwT8pGmH7pcS7ZHgCgzbBQ Pqtxl+61kvT0KsHYGmhhtSU= =y0wp -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 02:40:49PM -0700, Kevin B. McCarty wrote: Thank you for this very cool effort! Might we see checklib packaged for Debian soon? Hmm, maybe the functionality could be included in lintian? Cheers, Richard -- __ _ |_) /| Richard Atterer | GnuPG key: 888354F7 | \/¯| http://atterer.net | 08A9 7B7D 3D13 3EF2 3D25 D157 79E6 F6DC 8883 54F7 ¯ '` ¯ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: A plan to get rid of unnecessary package dependencies
On Mon, Sep 25, 2006 at 02:40:49PM -0700, Kevin B. McCarty wrote: (Sorry if you get this twice, I don't know whether you subscribe to debian-devel.) I am subscribed here, also sorry about the broken MFT in the -announce mail :-/ Thank you for this very cool effort! Might we see checklib packaged for Debian soon? Thanks :) Aurélien GÉRÔME started an alioth project, I'm also member there: https://alioth.debian.org/projects/checklib/ Nothing there yet, as I wanted to get the announcement out before starting to work on the codebase again to clean it up (which is badly needed :-/). The current version of the source is linked on the bottom of the pages (updated 10 minutes ago, so please reload ;) One thing I noticed is that there are a lot of problems (in your terminology) caused by unneeded dependencies on libgcc1 (/lib/libgcc_s.so.1). From my quick investigation, it appears that the C++ and Fortran compilers (g++, g77, gfortran) introduce this dependency automatically to programs linked with them. However, if gcc is instead used in the linking step, no such dependency is created (at least on amd64 where I'm testing). I'm not sure if libgcc_s.so.1 doesn't do some magic that checklib can't detect. Maybe one of the gcc maintainers could clarify if/when it's needed. If it really is needed it could be ignored for the results, that wouldn't be much of a problem. Even though you are technically correct that libgcc1 is an unneeded dependency, it may be hard for anyone (except maybe the gcc maintainers) to do anything to fix this; could you have an option for libcheck to ignore this library? I suspect if this was done, the proportion of problems in your pie chart would shrink a fair bit. Could be done, however if the dependency is really unneeded, it might be interesting to fix it in gcc if possible and desirable (i.e. not unpractical for reasons unknown to me). One general comment: Checklib problems concern binary packages. So if there's a problem on one package, that doesn't necessarily mean that the corresponding source package is the correct place to fix it. As in this case the problem might originate in gcc, another common reason are broken .la/.pc files in -dev packages. Checklib just can't know where the problems come from, it can only see where they manifest themselves. Richard Atterer wrote: Hmm, maybe the functionality could be included in lintian? I'd have to talk with the lintian maintainers, but it's not easy. Checklib needs a local Debian Mirror to work, and that sounds a bit problematic for inclusion in lintian :) There might be ways around this, I'll try to explore that. Cheers, Christian Aichinger signature.asc Description: Digital signature
Re: A plan to get rid of unnecessary package dependencies
Hi Christian, (Sorry if you get this twice, I don't know whether you subscribe to debian-devel.) Christian Aichinger wrote: As Steve Langasek detailed in his d-d-a mail[1], unnecessary dependencies cause lots of problems, as they make transitions bigger then they need to be. The root cause are unnecessary libraries on the linker commandline, which get stored in the resulting binary and which dpkg-shlibdeps happily turns into inter-package dependencies. Thus the goal is to eliminate those unnecessary linker arguments. As a start, I've written a script that searches for unnecessary dependencies and reports them. Results are available here: http://rerun.lefant.net/checklib [snip] Thank you for this very cool effort! Might we see checklib packaged for Debian soon? (I was going to ask about source code, but then I noticed the link to here URL:http://greek0.net/div/checklib.tar.gz at the bottom of the checklib web page. Reposting it for anyone else who might not have noticed it.) One thing I noticed is that there are a lot of problems (in your terminology) caused by unneeded dependencies on libgcc1 (/lib/libgcc_s.so.1). From my quick investigation, it appears that the C++ and Fortran compilers (g++, g77, gfortran) introduce this dependency automatically to programs linked with them. However, if gcc is instead used in the linking step, no such dependency is created (at least on amd64 where I'm testing). That is, programs compiled like this have libgcc_s.so.1 NEEDED: g++ foo.cc g77 foo.F gfortran foo.F but if they are compiled to object code and then linked with plain vanilla gcc, like this, they don't: g++ -c foo.cc -o foo.o gcc foo.o -lstdc++ g77 -c foo.F -o foo.o gcc foo.o -lg2c -lfrtbegin gfortran -c foo.F -o foo.o gcc foo.o -lgfortran -lgfortranbegin [CC'ed to debian-gcc to see if someone there can explain why this happens.] Even though you are technically correct that libgcc1 is an unneeded dependency, it may be hard for anyone (except maybe the gcc maintainers) to do anything to fix this; could you have an option for libcheck to ignore this library? I suspect if this was done, the proportion of problems in your pie chart would shrink a fair bit. best regards, -- Kevin B. McCarty [EMAIL PROTECTED] Physics Department WWW: http://www.princeton.edu/~kmccarty/Princeton University GPG: public key ID 4F83C751 Princeton, NJ 08544 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
