Re: Mandatory -dbg packages for libraries?
On Tue, May 22, 2007 at 11:26:38PM +0200, Wouter Verhelst wrote: On Tue, May 22, 2007 at 04:10:06PM -0500, Steve Greenland wrote: [...] Those are good reasons. Those are different reasons than static libraries are faster, which was the previous argument for keeping them. No, that was one argument for keeping them, and the only one that I could come up with in a short time frame about a subject that I don't care much about. Since you forced me to think more about it, I came up with two more reasons in an equally short timeframe. Of course, that begs the question why I seem to be the only one giving reasons that seem quite obvious to me, but then again... It only needs one person to voice them, but thank you for doing so. Regards, Paddy -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 22-May-07, 13:40 (CDT), Wouter Verhelst [EMAIL PROTECTED] wrote: On Sun, May 20, 2007 at 09:19:52PM -0500, Steve Greenland wrote: Why should we spend time and space to provide something that doesn't do anything useful?[1] I also once heard an argument that static libraries are easier to maintain. While I disagreed at first, it started to make sense when it was explained that in-house software in /usr/local which does not add dependencies to the package management system easily breaks on upgrades. If you really think dropping static libraries is the best way forward, then I think you should make a case for it that also explains why we do give our users all possible and impossible options by linking our software against every library in sight, while at the same time not providing them with the basic option of compiling their software statically. Moreover, remember that most, if not all, other distributions have it, meaning that newbie-oriented documentation assumes it'll work, and that not providing it for no good reason will probably only serve to annoy and confuse the user. Those are good reasons. Those are different reasons than static libraries are faster, which was the previous argument for keeping them. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Tue, May 22, 2007 at 04:10:06PM -0500, Steve Greenland wrote: [...] Those are good reasons. Those are different reasons than static libraries are faster, which was the previous argument for keeping them. No, that was one argument for keeping them, and the only one that I could come up with in a short time frame about a subject that I don't care much about. Since you forced me to think more about it, I came up with two more reasons in an equally short timeframe. Of course, that begs the question why I seem to be the only one giving reasons that seem quite obvious to me, but then again... -- Shaw's Principle: Build a system that even a fool can use, and only a fool will want to use it. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Wed, May 09, 2007 at 11:28:49AM -0500, Steve Greenland wrote: On 09-May-07, 04:02 (CDT), Wouter Verhelst [EMAIL PROTECTED] wrote: I'm not entirely sure about the specifics, and especially not across architectures; but regardless, doing a PLT lookup is more expensive than doing a function call to something that was statically linked in. True. Now, does anyone have measurements to show that this has any actual significance in real world code on modern hardware? I don't see why that would be relevant. We're not providing statically linked binaries; we are providing static libraries so that people who want them can perform static linking for their own in-house software. Anything more than that (including the decision on whether or not it's a good idea to do static linking, considering the fact that the maintenance overhead for statically linked binaries is significantly higher) is up to the user, anyway. -- Shaw's Principle: Build a system that even a fool can use, and only a fool will want to use it. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 20-May-07, 13:41 (CDT), Wouter Verhelst [EMAIL PROTECTED] wrote: On Wed, May 09, 2007 at 11:28:49AM -0500, Steve Greenland wrote: On 09-May-07, 04:02 (CDT), Wouter Verhelst [EMAIL PROTECTED] wrote: I'm not entirely sure about the specifics, and especially not across architectures; but regardless, doing a PLT lookup is more expensive than doing a function call to something that was statically linked in. True. Now, does anyone have measurements to show that this has any actual significance in real world code on modern hardware? I don't see why that would be relevant. We're not providing statically linked binaries; we are providing static libraries so that people who want them can perform static linking for their own in-house software. Why should we spend time and space to provide something that doesn't do anything useful?[1] Nothing prevents the users who want static linking from building their own libraries. My guess is that those who do need these kinds of small gains are compiling the relevant libraries and programs from scratch anyway, using the carefully tuned compiler options that they've measured as performing the best on their particular hardware. Steve [1] Or maybe it does...no one has provided measurements, yet. -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 09-May-07, 04:02 (CDT), Wouter Verhelst [EMAIL PROTECTED] wrote: On Tue, May 08, 2007 at 02:23:37PM +0200, Josselin Mouette wrote: Le lundi 07 mai 2007 ? 13:02 +0200, Wouter Verhelst a ?crit : Dropping most .a libraries is something I agree with. I see no reason why we should have them for most of the libraries. As a courtesy to our users. Statically linked programs are slightly faster (since they don't need to do PLT lookups, so they spare a jump on every function call to a shared object). For people for whom performance is critical, providing .a libraries is a good idea. Hm, I thought the double-jump issue was specific to i386. On other architectures, there is still the need for an extra register, but AIUI the performance impact should be much smaller. I'm not entirely sure about the specifics, and especially not across architectures; but regardless, doing a PLT lookup is more expensive than doing a function call to something that was statically linked in. True. Now, does anyone have measurements to show that this has any actual significance in real world code on modern hardware? In particular, anything *besides* the BLAS routines, which is about the only thing I'd expect to be measurable. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, May 07, 2007 at 08:25:53PM +0200, Kurt Roeckx wrote: On Mon, May 07, 2007 at 01:02:17PM +0200, Wouter Verhelst wrote: On Mon, Apr 23, 2007 at 12:32:37AM +0200, Kurt Roeckx wrote: On Sun, Apr 22, 2007 at 11:15:36PM +0100, Roger Leigh wrote: If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? Dropping most .a libraries is something I agree with. I see no reason why we should have them for most of the libraries. As a courtesy to our users. Statically linked programs are slightly faster (since they don't need to do PLT lookups, so they spare a jump on every function call to a shared object). For people for whom performance is critical, providing .a libraries is a good idea. I agree that for some things it might be useful for have static libraries, but in most cases they're not. It would be reasonable to provide static libraries maybe for some core libs, where justified interest is there and for libraries user request them via wishlist bugs, I guess. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 12:32:37AM +0200, Kurt Roeckx wrote: On Sun, Apr 22, 2007 at 11:15:36PM +0100, Roger Leigh wrote: If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? Dropping most .a libraries is something I agree with. I see no reason why we should have them for most of the libraries. As a courtesy to our users. Statically linked programs are slightly faster (since they don't need to do PLT lookups, so they spare a jump on every function call to a shared object). For people for whom performance is critical, providing .a libraries is a good idea. -- Shaw's Principle: Build a system that even a fool can use, and only a fool will want to use it. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, May 07, 2007 at 01:02:17PM +0200, Wouter Verhelst wrote: On Mon, Apr 23, 2007 at 12:32:37AM +0200, Kurt Roeckx wrote: On Sun, Apr 22, 2007 at 11:15:36PM +0100, Roger Leigh wrote: If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? Dropping most .a libraries is something I agree with. I see no reason why we should have them for most of the libraries. As a courtesy to our users. Statically linked programs are slightly faster (since they don't need to do PLT lookups, so they spare a jump on every function call to a shared object). For people for whom performance is critical, providing .a libraries is a good idea. I agree that for some things it might be useful for have static libraries, but in most cases they're not. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Sat, Apr 28, 2007 at 02:37:46PM -0700, Steve Langasek wrote: write documentation or don't understand the API. Wrong API docs are surely worse than not having no docs, aren't they? snip If I thought putting it in policy would significantly improve the availability of API docs in Debian, I would support doing so. But I don't think this will happen, and anyway if people want to campaign for improving our API documentation they can do that in any number of ways without asking for it to be put in policy first. On Sat, Apr 28, 2007 at 02:48:23PM -0700, Russ Allbery wrote: In my case, because of: binary-without-manpage (1283 packages, 3616 tags) I think we should demonstrate our ability to deliver on tasks we've already promised to do before promising to do even more in the same vein. That's all true, but it fails to convince me that is better not to state this in the policy than to state it (only Steve's point about wrong API docs, but I'm convinced it will be quantitatively small). My approach to this is first to decide whether API docs in the policy is something we want in debian or not. Then, if it is the case, to state it in the policy. Then see how this will be received by DDs and maybe start filing bug with patches for fixing the issues. I don't like thinking nh, it won't work, so please don't try to enforce it. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
[Neil Williams] I chose Debian as a development platform for my own reasons and my decision was not deemed to be wise in the eyes of some of my upstream colleagues. As the newbie to that particular team, I was under significant pressure to upgrade to Fedora or SuSE. Are you saying Fedora and SuSE have API documentation for all the libraries they ship? I must say, that surprises me. My own experience with upstream's opinion of Debian is that they think we should have fixed #291641 a long, long time ago. And I can't disagree. (The problem is that since libtool apparently exposes a shortcoming in binutils, nobody seems willing to add a workaround to libtool, waiting instead for a fix from binutils.) signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Sun, Apr 29, 2007 at 09:23:03AM +0200, Stefano Zacchiroli wrote: docs, but I'm convinced it will be quantitatively small). My approach to this is first to decide whether API docs in the policy is something we want in debian or not. Then, if it is the case, to state it in the policy. Then see how this will be received by DDs and maybe start filing bug with patches for fixing the issues. I don't like thinking nh, it won't work, so please don't try to enforce it. Why do you feel that putting this into policy is a useful first step? To me the approach that you're suggesting seems to be the wrong way round - it really shouldn't require a policy amendment to ask for the inclusion of documentation. It's not something that requires a great deal of coordination between packages. -- You grabbed my hand and we fell into it, like a daydream - or a fever. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On 29-Apr-07, 03:10 (CDT), Peter Samuelson [EMAIL PROTECTED] wrote: [Neil Williams] I chose Debian as a development platform for my own reasons and my decision was not deemed to be wise in the eyes of some of my upstream colleagues. As the newbie to that particular team, I was under significant pressure to upgrade to Fedora or SuSE. Are you saying Fedora and SuSE have API documentation for all the libraries they ship? I must say, that surprises me. And if so, why haven't they submitted them upstream? :-) Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
Stefano Zacchiroli [EMAIL PROTECTED] writes: That's all true, but it fails to convince me that is better not to state this in the policy than to state it (only Steve's point about wrong API docs, but I'm convinced it will be quantitatively small). My approach to this is first to decide whether API docs in the policy is something we want in debian or not. Then, if it is the case, to state it in the policy. Then see how this will be received by DDs and maybe start filing bug with patches for fixing the issues. I don't like thinking nh, it won't work, so please don't try to enforce it. This is generally not how Policy works. Policy modifications that render large numbers of packages buggy are only accepted in very unusual circumstances. Normally, the onus is on the folks wanting to make the change to get widespread adoption first, and then the Policy modification happens afterwards. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Wed, Apr 25, 2007 at 04:23:38PM +0200, Stefano Zacchiroli wrote: On Tue, Apr 24, 2007 at 08:12:46AM -0700, Steve Langasek wrote: If we are talking about hand-written documentation you're of course right. However if you're talking about documentation which can be generated automatically from sources (and not that it was the ideal point of Neil) than you're not. There are ways to generate manpages from --help output of programs. We still have a lot of programs in Debian with no manpages. According to my experience with html2man doesn't work that properly every time, but I see your point. But still I don't get the objection: is the laziness of people a good reason not to state the right path? Is this right path going to describe best practices for providing *good* API documentation? Some maintainers will do anything policy (or lintian) tells them to. If you say libraries must include API documentation in policy, they'll write API documentation -- even if they don't know how to write documentation or don't understand the API. Wrong API docs are surely worse than not having no docs, aren't they? It happened to me many times to find library -dev packages with upstream sources commented with some literate programming stuff but nevertheless missing the corresponding automatically generated API docs in the package. That's a pity. And that's something the policy should address. Hmm, I don't agree that it needs to be addressed in policy, FWIW. Fair enough, but note that having man pages is actually addressed by the policy. Why do you think API doc shouldn't? After all man pages are docs for users and API doc are too, with the only difference that in the latter case the users are programmers. If I thought putting it in policy would significantly improve the availability of API docs in Debian, I would support doing so. But I don't think this will happen, and anyway if people want to campaign for improving our API documentation they can do that in any number of ways without asking for it to be put in policy first. Cheers, -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
Stefano Zacchiroli [EMAIL PROTECTED] writes: Fair enough, but note that having man pages is actually addressed by the policy. Why do you think API doc shouldn't? After all man pages are docs for users and API doc are too, with the only difference that in the latter case the users are programmers. In my case, because of: binary-without-manpage (1283 packages, 3616 tags) I think we should demonstrate our ability to deliver on tasks we've already promised to do before promising to do even more in the same vein. Good library API documentation is in many cases much harder to write than a man page for a binary. Many of those already diagnosed missing man pages are a matter of a few minutes to write a brief man page for a wrapper script or simple utility. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Josselin Mouette [EMAIL PROTECTED] writes: Le mardi 24 avril 2007 à 18:09 +0200, Reinhard Tartler a écrit : Sending core dumps is of course debatable, espec. if you cannot assert that no sensitive information is transmitted. Still, Apport is more than that. It also describes the mechanism of debug packages (*.ddeb), which are generated automatically at build time. This part is definitely something we should grab from Ubuntu, and integrate in our tools. In ubuntu this is done witht he package pkg-create-dbgsym: https://code.beta.launchpad.net/~ubuntu-core-dev/pkg-create-dbgsym/ubuntu You can browse the source here: http://codebrowse.launchpad.net/~ubuntu-core-dev/pkg-create-dbgsym/ubuntu/files (or grab the source package from some ubuntu mirror) The idea is to divert dh_strip and automatically create these *.ddebs at build time. Ubuntu profits here that binaryless uploads are being done. This means it is pretty easy to change the build environment. As in debian source-only uploads are highly disputed, espc. among the ftp-masters, we probably cannot go this way. Therefore we would need to delegate this task to the developers and buildd admins. We need some way to manage these .ddebs. In ubuntu, they aren't integrated into the archive system soyuz, but available on http://people.ubuntu.com/~pitti/ddebs/ We could perhaps do the same: host these ddebs on some people.debian.org space for a start. However, we would need to ask developers and buildd adminst to additionally upload their ddebs to this location (hint: this needs some upload infrastructure). This approach would have the advantage that we don't need to touch dak (yet). As an extension if .ddebs prove to be useful in debian, we can think about having dak enforce the submission of ddebs. But that's future work. Currently we would need some facilities to upload and manage ddebs. Instead of sending core dumps, I'd really like to make gdb able to grab the appropriate data. I'll check whether it is possible to wrap it in some environment that would gather only appropriate parts of the debugging files. Honstly, this sounds exactly to what the command 'apport-retrace' already does. Otherwise, the idea of letting the user choose between centralized operation, downloading debugging symbols or saving the core for future analysis is definitely appropriate. I (obviously) agree. -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 pgpF6rncw8YON.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Tue, Apr 24, 2007 at 08:12:46AM -0700, Steve Langasek wrote: If we are talking about hand-written documentation you're of course right. However if you're talking about documentation which can be generated automatically from sources (and not that it was the ideal point of Neil) than you're not. There are ways to generate manpages from --help output of programs. We still have a lot of programs in Debian with no manpages. According to my experience with html2man doesn't work that properly every time, but I see your point. But still I don't get the objection: is the laziness of people a good reason not to state the right path? It happened to me many times to find library -dev packages with upstream sources commented with some literate programming stuff but nevertheless missing the corresponding automatically generated API docs in the package. That's a pity. And that's something the policy should address. Hmm, I don't agree that it needs to be addressed in policy, FWIW. Fair enough, but note that having man pages is actually addressed by the policy. Why do you think API doc shouldn't? After all man pages are docs for users and API doc are too, with the only difference that in the latter case the users are programmers. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Tue, Apr 24, 2007 at 11:43:02PM +0200, Pierre THIERRY wrote: Scribit Daniel Jacobowitz dies 23/04/2007 hora 16:19: Another possible way to change glibc would be to have libc6-dbg contain full debug symbols, libc6-dev contain -g1 symbols only, and have the -dbg divert the -dev. Why not do that for every library? Just doesn't seem useful enough to go through the hassle. -- Daniel Jacobowitz CodeSourcery -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Le mardi 24 avril 2007 à 23:43 +0200, Pierre THIERRY a écrit : Scribit Daniel Jacobowitz dies 23/04/2007 hora 16:19: Another possible way to change glibc would be to have libc6-dbg contain full debug symbols, libc6-dev contain -g1 symbols only, and have the -dbg divert the -dev. Why not do that for every library? Because diversions are a hack and should not be the rule. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée
Re: Mandatory -dbg packages for libraries?
Le mardi 24 avril 2007 à 18:09 +0200, Reinhard Tartler a écrit : Sending core dumps is of course debatable, espec. if you cannot assert that no sensitive information is transmitted. Still, Apport is more than that. It also describes the mechanism of debug packages (*.ddeb), which are generated automatically at build time. This part is definitely something we should grab from Ubuntu, and integrate in our tools. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. Sure. How about generating ddebs like ubuntu at build time, upload them to ftp-master along the regular binary packages, and install them in a special section on the ftp servers. This way not all mirrors need to serve the ddebs. We can still leave the choice: sending in core dumps to some centralised service which retraces them, or use provided tools so users can download the debug data and retrace their crashed on their own. Instead of sending core dumps, I'd really like to make gdb able to grab the appropriate data. I'll check whether it is possible to wrap it in some environment that would gather only appropriate parts of the debugging files. Otherwise, the idea of letting the user choose between centralized operation, downloading debugging symbols or saving the core for future analysis is definitely appropriate. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile. signature.asc Description: Ceci est une partie de message numériquement signée
Re: Mandatory -dbg packages for libraries?
On Tue, 2007-04-24 at 14:40 +0200, Loïc Minier wrote: On Tue, Apr 24, 2007, Josselin Mouette wrote: Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. But Apport is written already, and it's also the path that Windows crash report and Mozilla's talkback tools have taken; these corporations might not represent our ideals, but they present examples of deployed and working solutions. I don't think it's still 200 MB compressed, but some input from Ubuntu folks could help. apport can send mini core dumps, and will offer to do so if the coredump is very large. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. Yes, it sounds like a good idea; I suppose it might offer less possibilities, but a good stack trace is often good enough. However modifying gdb sounds like a lot of hard work. I don't know how Apport works on the server side, but if this part of Apport could be made to run on the client and to fetch the relevant files, this might have all advantages of not sending the sensitive core dumps, not uploading too much data, and being available without too much development. This would be nice too - I would certainly like to be able to say to apport 'please retrace on my machine now without uploading' for applications that have been exposed to my password. Rob -- GPG key available at: http://www.robertcollins.net/keys.txt. signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Mon, Apr 23, 2007 at 04:27:34PM -0700, Steve Langasek wrote: On 23-Apr-07, 15:51 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. That the existing requirement is already too much for us to keep up with, so adding new requirements, especially ones that require significant attention to detail to get right, dilutes our attention for little benefit? If we are talking about hand-written documentation you're of course right. However if you're talking about documentation which can be generated automatically from sources (and not that it was the ideal point of Neil) than you're not. It happened to me many times to find library -dev packages with upstream sources commented with some literate programming stuff but nevertheless missing the corresponding automatically generated API docs in the package. That's a pity. And that's something the policy should address. Often, even when comments are not properly formatted, doc generation tools can generate useful documents which complements .h files or similar stuff in other languages. Cheers. -- Stefano Zacchiroli -*- PhD in Computer Science ... now what? [EMAIL PROTECTED],debian.org,bononia.it} -%- http://www.bononia.it/zack/ (15:56:48) Zack: e la demo dema ?/\All one has to do is hit the (15:57:15) Bac: no, la demo scema\/right keys at the right time signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Mon, 23 Apr 2007 19:32:46 -0400 Joey Hess [EMAIL PROTECTED] wrote: Neil Williams wrote: I chose Debian as a development platform for my own reasons and my decision was not deemed to be wise in the eyes of some of my upstream colleagues. As the newbie to that particular team, I was under significant pressure to upgrade to Fedora or SuSE. Debian needs to reclaim the respect of upstream development teams and part of that is making it *a lot* easier to do upstream development on Debian without needing to become a DD as well. Debian is respected as a distribution for users because of the multiple architecture support and the patches and bug reports that are forwarded upstream - what is missing (IMHO) is respect for Debian as the distribution of choice for upstream development itself. Are you generalising from your one poor personal experience with a non-Debian-friendly upstream, or do you have a significant body of data that I don't about masses of upsteams who are not Debian friendly? Generalising - which probably isn't giving me the best overview, you're right. My impression has always been that a significant proportion of upstreams use Debian, or are at least familiar with it. I base this on, amoung other things, interacting with hundreds of different upstreams whose packages I have maintained in Debian, as well as working in linux companies and personally knowing a lot of upstream developers. Do you then think that Debian should not require basic API docs if upstream don't provide them - even if the information is available (and properly distributable) from outside the .orig ? The only significant documentation that is missing in Debian that I know of is GFDL licensed docs which have been removed from main. Aside from that, if a library is missing documentation, it's missing it because it's not available upsteam either. The 'big' libraries are very well documented, it's the smaller ones. Individually these may not be significant but collectively, I think there is an appreciable gap in the API docs. Recently, I've been working with libarchive (deb-gview) and libgtkhtml (gtk2 port of quicklist). libarchive has nicely commented headers which could be turned into HTML but aren't, libgtkhtml has only a test program. (I've missed one -doc package myself - just noticed that libgpewidget can build a -doc package so that's just gone on the ToDO list along with a few wishlist bugs for the above packages.) Maybe instead of seeking API docs as a requirement it would be better to seek a section under Best Practise that outlines how man (3) can be used to provide at least an introduction to the API when the .orig does not contain docs itself? -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpNFcyopAoRV.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Mon, 2007-04-23 at 10:37 +0200, Josselin Mouette wrote: Le dimanche 22 avril 2007 à 20:39 +0100, Neil Williams a écrit : I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) I fully concur, and I think we need to go farther, by adding a debugging package to *all* architecture-dependent packages. Of course, this requires changes at the dak level, in the mirror scripts (not all mirrors will want their size to double) and in APT. This was in Sam's platform and I hope to see some efforts happen in this direction. With the introduction of debreaper, I'd like to make automated bug reporting a reality for the majority of unstable users, but it will be useless if there is no solution to install the debugging symbols automatically. snip Installing debugging symbols for all binaries involved in a crash seems... heavyweight. I would expect the user to want to get on with his or her work at this point. Wouldn't it be better - in terms of response rate - to take a minidump (along the lines of Windows error reporting or Mozilla's Quality Feedback Agent) and do symbol lookup on some central server? It might also be worth looking at MS's symbol server. I don't know whether that supports queries by address or whether it just serves up whole debug databases, but it seems to work reasonably quickly. (But unsurprisingly it's providing quite sparse information about Windows binaries.) Ben. -- Ben Hutchings When you say `I wrote a program that crashed Windows', people just stare ... and say `Hey, I got those with the system, *for free*'. - Linus Torvalds signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries?
Ben Hutchings [EMAIL PROTECTED] writes: Installing debugging symbols for all binaries involved in a crash seems... heavyweight. I would expect the user to want to get on with his or her work at this point. Wouldn't it be better - in terms of response rate - to take a minidump (along the lines of Windows error reporting or Mozilla's Quality Feedback Agent) and do symbol lookup on some central server? isn't this quite similar to what ubuntu is doing with Apport? https://wiki.ubuntu.com/Apport -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Le mardi 24 avril 2007 à 12:58 +0200, Reinhard Tartler a écrit : Ben Hutchings [EMAIL PROTECTED] writes: Installing debugging symbols for all binaries involved in a crash seems... heavyweight. I would expect the user to want to get on with his or her work at this point. Wouldn't it be better - in terms of response rate - to take a minidump (along the lines of Windows error reporting or Mozilla's Quality Feedback Agent) and do symbol lookup on some central server? isn't this quite similar to what ubuntu is doing with Apport? https://wiki.ubuntu.com/Apport Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile.
Re: Mandatory -dbg packages for libraries?
On Tue, Apr 24, 2007, Josselin Mouette wrote: Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. But Apport is written already, and it's also the path that Windows crash report and Mozilla's talkback tools have taken; these corporations might not represent our ideals, but they present examples of deployed and working solutions. I don't think it's still 200 MB compressed, but some input from Ubuntu folks could help. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. Yes, it sounds like a good idea; I suppose it might offer less possibilities, but a good stack trace is often good enough. However modifying gdb sounds like a lot of hard work. I don't know how Apport works on the server side, but if this part of Apport could be made to run on the client and to fetch the relevant files, this might have all advantages of not sending the sensitive core dumps, not uploading too much data, and being available without too much development. -- Loïc Minier -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Tue, Apr 24, 2007 at 02:12:39PM +0200, Josselin Mouette wrote: Le mardi 24 avril 2007 à 12:58 +0200, Reinhard Tartler a écrit : Ben Hutchings [EMAIL PROTECTED] writes: Installing debugging symbols for all binaries involved in a crash seems... heavyweight. I would expect the user to want to get on with his or her work at this point. Wouldn't it be better - in terms of response rate - to take a minidump (along the lines of Windows error reporting or Mozilla's Quality Feedback Agent) and do symbol lookup on some central server? isn't this quite similar to what ubuntu is doing with Apport? https://wiki.ubuntu.com/Apport Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. FWIW, Mozilla is going to replace Talkback with Breakpad, which is under development. It would be worth getting involved in that. http://code.google.com/p/google-breakpad/ Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Tue, Apr 24, 2007 at 09:13:18AM +0200, Stefano Zacchiroli wrote: On Mon, Apr 23, 2007 at 04:27:34PM -0700, Steve Langasek wrote: On 23-Apr-07, 15:51 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. That the existing requirement is already too much for us to keep up with, so adding new requirements, especially ones that require significant attention to detail to get right, dilutes our attention for little benefit? If we are talking about hand-written documentation you're of course right. However if you're talking about documentation which can be generated automatically from sources (and not that it was the ideal point of Neil) than you're not. There are ways to generate manpages from --help output of programs. We still have a lot of programs in Debian with no manpages. It happened to me many times to find library -dev packages with upstream sources commented with some literate programming stuff but nevertheless missing the corresponding automatically generated API docs in the package. That's a pity. And that's something the policy should address. Hmm, I don't agree that it needs to be addressed in policy, FWIW. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Tue, Apr 24, 2007 at 02:40:22PM +0200, Loïc Minier wrote: On Tue, Apr 24, 2007, Josselin Mouette wrote: Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. But Apport is written already, and it's also the path that Windows crash report and Mozilla's talkback tools have taken; these corporations might not represent our ideals, but they present examples of deployed and working solutions. I don't think it's still 200 MB compressed, but some input from Ubuntu folks could help. I for one, won't use a solution that sends full core dumps. KDE applications e.g. use many shared memory among the different processes and I will never know if the crash of konqueror I want to report wouldn't include all my accounts passwords from kwallet it just used. Sorry but, well, that's an option I'm not willing to take, and I'm sure others feel the same. There is a major privacy issue here, and well, sorry, but taking Microsoft as an example here is quite err disturbing :) A clean and good backtrace is very often useful enough, and would IMHO be a significant improvement already. One could also imagine a setup where cores remains archived on the user side, so that they can provide more extensive backtraces (values of some variables or so) and see textually if sensitive informations are in there or not. -- ·O· Pierre Habouzit ··O[EMAIL PROTECTED] OOOhttp://www.madism.org pgpNBHNkLffwg.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
Josselin Mouette [EMAIL PROTECTED] writes: Le mardi 24 avril 2007 à 12:58 +0200, Reinhard Tartler a écrit : Ben Hutchings [EMAIL PROTECTED] writes: Installing debugging symbols for all binaries involved in a crash seems... heavyweight. I would expect the user to want to get on with his or her work at this point. Wouldn't it be better - in terms of response rate - to take a minidump (along the lines of Windows error reporting or Mozilla's Quality Feedback Agent) and do symbol lookup on some central server? isn't this quite similar to what ubuntu is doing with Apport? https://wiki.ubuntu.com/Apport Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. Sending core dumps is of course debatable, espec. if you cannot assert that no sensitive information is transmitted. Still, Apport is more than that. It also describes the mechanism of debug packages (*.ddeb), which are generated automatically at build time. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. Sure. How about generating ddebs like ubuntu at build time, upload them to ftp-master along the regular binary packages, and install them in a special section on the ftp servers. This way not all mirrors need to serve the ddebs. We can still leave the choice: sending in core dumps to some centralised service which retraces them, or use provided tools so users can download the debug data and retrace their crashed on their own. -- Gruesse/greetings, Reinhard Tartler, KeyID 945348A4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Am Dienstag 24 April 2007 14:12 schrieb Josselin Mouette: Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. You can't do offline debugging with a central server. Unless you are always-on whereever you are... Additionally, sending in stuff is a non-developer approach. A definition of the Debian user base should be taken into account for the solution (Ubuntu might have another target group). HS pgpXBdtogC6ab.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Tue, 2007-04-24 at 14:40 +0200, Loïc Minier wrote: On Tue, Apr 24, 2007, Josselin Mouette wrote: Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. But Apport is written already, and it's also the path that Windows crash report and Mozilla's talkback tools have taken; these corporations might not represent our ideals, but they present examples of deployed and working solutions. Windows minidumps are typically 100 KB. My understand is that Mozilla is using basically the same format with extensions to cover other platforms and architectures. I don't think it's still 200 MB compressed, but some input from Ubuntu folks could help. Yes, the example shown in the screenshot is much smaller than that. There must be some trimming. Using a central server for symbol lookup like Ben proposed looks like a better idea. It needs gdb to be adapted or wrapped to access them correctly, though. snip In fact, reading through that Wiki page suggests that the various retrace tools do that. So perhaps Apport would be suitable for Debian, though we would need to expand architecture support. Ben. -- Ben Hutchings When you say `I wrote a program that crashed Windows', people just stare ... and say `Hey, I got those with the system, *for free*'. - Linus Torvalds signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries?
Loïc Minier [EMAIL PROTECTED] writes: On Tue, Apr 24, 2007, Josselin Mouette wrote: Apport sends complete core dumps, which is a very bad idea. The dumps can be huge (for desktop applications they often grow beyond 200MB) and they can contain gazillions of sensitive information. But Apport is written already, and it's also the path that Windows crash report and Mozilla's talkback tools have taken; these corporations might not represent our ideals, but they present examples of deployed and working solutions. I don't think it's still 200 MB compressed, but some input from Ubuntu folks could help. I believe sending full core dumps is an acceptable solution if and only if: * The user is notified and voluntarily opts in to sending dumps. * The core dumps are sent encrypted over the network. * The core dumps are stored securely wherever they're sent and only authorized people have access to them. * Every organization that has access to the core dumps is legally bound to not disclose any personal or confidential information in them to anyone. I think the last two are pretty hard for Debian to do. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/
Re: Mandatory -dbg packages for libraries?
Scribit Daniel Jacobowitz dies 23/04/2007 hora 16:19: Another possible way to change glibc would be to have libc6-dbg contain full debug symbols, libc6-dev contain -g1 symbols only, and have the -dbg divert the -dev. Why not do that for every library? Curiously, Pierre -- [EMAIL PROTECTED] OpenPGP 0xD9D50D8A signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Sun, 22 Apr 2007 19:29:55 -0400, Joey Hess [EMAIL PROTECTED] said: So while I'd love to have a way to have -dbg packages available for every binary, I actually am happy with this proposal to do it for only every library (plus whatever other binaries really need it). And it's a direction we're already moving in, with, as I mentioned, 227 lib*-dbg packages already in the archive. That's more than 10% of all our libraries already done[3]. So, making it a should would make 90% of our library packages insta-buggy. So I suggest that we take this as an existing practice, document it as a should in policy for now, document *how* to do separated debugging symbols in the developers reference (which does not currently seem to mention it at all), and go add -dbg versions of our library packages. I would rather add it as a recommended practice in policy, with a note that it will become a should/must as we get better coverage, and _also_ provide examples of what maintainers need to do to create separate debugging symbol packages in an informative footnote. manoj -- Where do forest rangers go to get away from it all? George Carlin Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 2007-04-22, Mark Brown [EMAIL PROTECTED] wrote: I'd be interested to see some numbers on the archive size impact - my experience with C++ suggests that the size inflation caused by debug symbols can be enormous. Openoffice currently ships a -g1 -dbg package. If it wasn't -g1, the deb would be around 400mb. Kdebase: 33mb -dbg package 28mb source 11-12 mb arch all debs. 20 mb arch debs. kdelibs 26 mb -dbg package 18 mb source package 39 (!) mb -doc package 10 mb other arch all debs 10 mb arch debs. For these two deb packages - the debug packages is around 1.5 the size of the arch package. A couple of other kde packages seems to show the same behaviour. /Sune -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Le dimanche 22 avril 2007 à 20:39 +0100, Neil Williams a écrit : I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) I fully concur, and I think we need to go farther, by adding a debugging package to *all* architecture-dependent packages. Of course, this requires changes at the dak level, in the mirror scripts (not all mirrors will want their size to double) and in APT. This was in Sam's platform and I hope to see some efforts happen in this direction. With the introduction of debreaper, I'd like to make automated bug reporting a reality for the majority of unstable users, but it will be useless if there is no solution to install the debugging symbols automatically. Currently I plan to automate installation of the -dbg packages when they are available, but this is grossly suboptimal and there is no real reason to not generate them for more packages. -- .''`. : :' : We are debian.org. Lower your prices, surrender your code. `. `' We will add your hardware and software distinctiveness to `-our own. Resistance is futile.
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 07:38:15AM +0200, Mike Hommey wrote: On Sun, Apr 22, 2007 at 08:26:38PM -0400, Joey Hess [EMAIL PROTECTED] wrote: Mark Brown wrote: I'd be interested to see some numbers on the archive size impact - my experience with C++ suggests that the size inflation caused by debug symbols can be enormous. I don't know about C++, but for C it depends. For example, aalib is a 102 kb library that compresses to 44kb. Its debug symbols are 234 kb and compress to 65 kb. OTOH, people rarely need full debugging information for aalib, it's probably plenty to see its functions in the backtrace, and not have line number info (bear in mind that line number info includes effectively the entire source code of the program). So it I build it with -g1, the debug symbols size drops to 48 kb and compresses to 14 kb. Except that -g1 drops line numbers... Wasn't that Joey's point? cheers, Hamish -- Hamish Moffatt VK3SB [EMAIL PROTECTED] [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 07:29:55PM -0400, Joey Hess wrote: Even with separated debugging symbols, -dbg packages are frequently larger than the package they provide debugging symbols for. See for example xserver-xorg-core-dbg. Looking through the 227 lib*-dbg packages, I found few contain separated debugging symbols, except for packages maintained by the xorg team[1]. I'm not sure if this is due many people still not knowing about separated debugging symbols, or due to technical reasons. For example, is there a tecnical reason why libc6-dbg does not contain separated debugging symbols? Yes, it's deliberate. People rarely need them just because they're debugging something linked to libc.so.6. Having them slows down GDB startup and increases its memory usage, for _every_ debug session. You'll notice if you look closely that libc6-dbg contains two things. One of them is a set of libraries you can use if you want to debug libc6. The other is a set of separate symbol files, but they contain only frame unwind information, no symbolic or line number information. This keeps the size and performance impact of the package down, but makes backtraces out of libc6 hugely more reliable. I've considered before trying to set up a separate, parallel archive that would only hold the -dbg packages, but implementing that without initially using the Debian infrastructure would be tough, and my experiences with setting up[2]/maintaining the separate udeb section of the archive is that it adds a lot of complexity. FWIW, I still think this is the way to go, though it would be hard. They wouldn't need nearly as much mirroring. e.g. they could go into a separate pool directory... -- Daniel Jacobowitz CodeSourcery -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 11:28:06PM +1000, Hamish Moffatt [EMAIL PROTECTED] wrote: On Mon, Apr 23, 2007 at 07:38:15AM +0200, Mike Hommey wrote: On Sun, Apr 22, 2007 at 08:26:38PM -0400, Joey Hess [EMAIL PROTECTED] wrote: Mark Brown wrote: I'd be interested to see some numbers on the archive size impact - my experience with C++ suggests that the size inflation caused by debug symbols can be enormous. I don't know about C++, but for C it depends. For example, aalib is a 102 kb library that compresses to 44kb. Its debug symbols are 234 kb and compress to 65 kb. OTOH, people rarely need full debugging information for aalib, it's probably plenty to see its functions in the backtrace, and not have line number info (bear in mind that line number info includes effectively the entire source code of the program). So it I build it with -g1, the debug symbols size drops to 48 kb and compresses to 14 kb. Except that -g1 drops line numbers... Wasn't that Joey's point? The point is that it's pretty useless to have a backtrace without line numbers. It would be interesting if there was an option similar to -g1, but that would keep the line numbers (without the burden of keeping the whole source code) Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 07:06:55PM +0200, Mike Hommey wrote: The point is that it's pretty useless to have a backtrace without line numbers. It would be interesting if there was an option similar to -g1, but that would keep the line numbers (without the burden of keeping the whole source code) Are you sure the source code is in there with -g? I was pretty sure it wasn't, only line numbers. /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 22-Apr-07, 17:29 (CDT), Kurt Roeckx [EMAIL PROTECTED] wrote: On Sun, Apr 22, 2007 at 04:40:45PM -0500, Steve Greenland wrote: On 22-Apr-07, 16:22 (CDT), Robert Collins [EMAIL PROTECTED] wrote: Because segfaults are often not easily reproduced. Having the ability to analyse a crash that occured when the user did not have the -dbg packages installed is not possible unless you have the original symbols the compiler created. That's an argument in favor of making the base library package built with debug symbols and then stripped[1], not of requiring -dbg packages. Maybe you should take a look at what dh_strip and -dbg packages do. It strips the debug symbols and puts it in a seperate file that's put in /usr/lib/debug. You put those files with debug symbols only in the -dbg package. This means that you can just install the -dbg package and gdb will automaticly pick up the debug symbols, without needing to rebuild anything. Ah, thanks for clarifying. This makes more sense than what I was (mis-)understanding. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 22-Apr-07, 17:01 (CDT), Neil Williams [EMAIL PROTECTED] wrote: 2. Why a seperate -doc? API docs should be part of the -dev package. In practice, such attitudes are commonly expressed as RTSL. (Read The Source, Luke). That does NOT encourage upstream usage of Debian as a distro. Is man (3) really so hard for a DD to provide? I'm going to guess that for *most* libraries, the docs are a trivial part of the size of the -dev package. For those with significant documentation, sure, a seperate -doc makes sense, just as we do now. I think libraries should be encouraged to provide significant documentation - what we have now is simply not enough. You seem to be arguing that the man pages should be in the core library package, yes? My objection is against mandating a *separate* -doc package. Separate doc packages make sense when the documentation is a significant portion of the total binary package size, and thus duplicating them in each binary package (rather than a single arch-all package) causes more hardship on the archive and mirrors than having a new packages causes the Packages file. As for putting the docs in the core library file, I don't actually buy your argument. The *VAST* majority of a libraries users are never going to look at the man pages for that library. People who need the man pages are going to have the -dev installed, or can easily install it. I don't see why upstreams needs this. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 07:10:15PM +0200, Steinar H. Gunderson [EMAIL PROTECTED] wrote: On Mon, Apr 23, 2007 at 07:06:55PM +0200, Mike Hommey wrote: The point is that it's pretty useless to have a backtrace without line numbers. It would be interesting if there was an option similar to -g1, but that would keep the line numbers (without the burden of keeping the whole source code) Are you sure the source code is in there with -g? I was pretty sure it wasn't, only line numbers. You're indeed right. I wrote quicker than my thoughts. Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Tshepang Lekhonkhobe wrote: It would be nice if the standard iso images that Debian makes available could be made to exclude -dbg packages as a trade-off. It actually felt painful someday when doing jigdo on the archive, only to see loads of -dbg packages getting downloaded, and knowing I wasn't ever gonna use 'em. debian-cd orders packages by popcon popularity, so if the -dbg packages are not on the last CDs, they must be used more than the packages that come after them. (The first -dbg package put on a CD is libc6-dbg, which comes just after exim in popularity. That's exim 3, not 4.) -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Manoj Srivastava wrote: On Sun, 22 Apr 2007 19:29:55 -0400, Joey Hess [EMAIL PROTECTED] said: So while I'd love to have a way to have -dbg packages available for every binary, I actually am happy with this proposal to do it for only every library (plus whatever other binaries really need it). And it's a direction we're already moving in, with, as I mentioned, 227 lib*-dbg packages already in the archive. That's more than 10% of all our libraries already done[3]. So, making it a should would make 90% of our library packages insta-buggy. So I suggest that we take this as an existing practice, document it as a should in policy for now, document *how* to do separated debugging symbols in the developers reference (which does not currently seem to mention it at all), and go add -dbg versions of our library packages. I would rather add it as a recommended practice in policy, with a note that it will become a should/must as we get better coverage, and _also_ provide examples of what maintainers need to do to create separate debugging symbol packages in an informative footnote. Well, we've made more than ~300 packages insta-buggy with policy changes before. It's not insta-rc-buggy. OTOH, I don't really care; 300 bug reports could be mass-filed w/o it being a should in policy. Note that I've already written some documentation for developers-reference in #420540. The policy-relevant bits are that we use /usr/lib/debug/path-to-object, that the files should not be executable (possibly a common mistake since objcopy preserves executable bits IIRC), and that the package names end in -dbg and the debug packages depend on an equal version of the package they provide debugging symbols for. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Daniel Jacobowitz wrote: Yes, it's deliberate. People rarely need them just because they're debugging something linked to libc.so.6. Having them slows down GDB startup and increases its memory usage, for _every_ debug session. Ok. Of course, this is also generally an argument against having -dbg packages for libraries with separated symbols.. You'll notice if you look closely that libc6-dbg contains two things. One of them is a set of libraries you can use if you want to debug libc6. The other is a set of separate symbol files, but they contain only frame unwind information, no symbolic or line number information. This keeps the size and performance impact of the package down, but makes backtraces out of libc6 hugely more reliable. What are your feelings on only including the -g1 information in library -dbg packages in general? It does save a lot of space, but the potential utility also goes way down. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Steinar H. Gunderson wrote: Are you sure the source code is in there with -g? I was pretty sure it wasn't, only line numbers. You're right, it doesn't include the actual code, but the size is still roughly porportional to the size of the code in the library. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Mike Hommey wrote: The point is that it's pretty useless to have a backtrace without line numbers. That depends on whether the problem you're debugging is a bug in the library itself, or only a bug triggered by code that calls the library, or in code called by the library. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Steinar H. Gunderson wrote: Are you sure the source code is in there with -g? I was pretty sure it wasn't, only line numbers. I was fooled by the fact that the debug packages I was using have: [EMAIL PROTECTED]:/usr/lib/debug/usr/libstrings libaa.so.1.0.4|grep joey /home/joey/src/packages/aalib/src [EMAIL PROTECTED]:/usr/lib/debug/usr/libstrings xorg/modules/*.so |grep home|head /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/hw/xfree86/dixmods /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/afb /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/hw/xfree86/dixmods /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/cfb /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/mfb /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/hw/xfree86/dixmods /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/cfb32 /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/hw/xfree86/exa /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/exa /home/julien/src/xsf/git/xserver/xorg-server/obj-i486-linux-gnu/hw/xfree86/dixmods Hmm, I hope this isn't a potential security hole.. I'd be happy if there were a way to remove that info from the package, actually. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 03:08:41PM -0400, Joey Hess wrote: Daniel Jacobowitz wrote: Yes, it's deliberate. People rarely need them just because they're debugging something linked to libc.so.6. Having them slows down GDB startup and increases its memory usage, for _every_ debug session. Ok. Of course, this is also generally an argument against having -dbg packages for libraries with separated symbols.. Yes. It's a tradeoff question. Pretty much any other library will affect a smaller overall percentage of users :-) You'll notice if you look closely that libc6-dbg contains two things. One of them is a set of libraries you can use if you want to debug libc6. The other is a set of separate symbol files, but they contain only frame unwind information, no symbolic or line number information. This keeps the size and performance impact of the package down, but makes backtraces out of libc6 hugely more reliable. What are your feelings on only including the -g1 information in library -dbg packages in general? It does save a lot of space, but the potential utility also goes way down. I don't remember exactly what -g1 produces, but I think libc6-dbg is even less - it's only .debug_frame and .symtab, nothing else at all. I think libc6-dbg is a special case here, and we should use -g (-g2) in general. Another possible way to change glibc would be to have libc6-dbg contain full debug symbols, libc6-dev contain -g1 symbols only, and have the -dbg divert the -dev. -- Daniel Jacobowitz CodeSourcery -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 03:19:23PM -0400, Joey Hess wrote: Hmm, I hope this isn't a potential security hole.. I'd be happy if there were a way to remove that info from the package, actually. I have done some work with debugedit, which is shipped with RPM - it's supposed to be able to do this, but it needs a little love and to be integrated into the post-install process. -- Daniel Jacobowitz CodeSourcery -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Mon, Apr 23, 2007 at 12:22:40PM -0500, Steve Greenland wrote: On 22-Apr-07, 17:01 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think libraries should be encouraged to provide significant documentation - what we have now is simply not enough. You seem to be arguing that the man pages should be in the core library package, yes? My objection is against mandating a *separate* -doc package. Separate doc packages make sense when the documentation is a significant portion of the total binary package size, and thus Reading between the lines I think Neil is assuming that good documentation has to be large enough to be worth splitting out. I don't think that's a good assumption - for example, one reason people make libraries is to hide some complex algorithm or nasty portability problem. This often results in a library with a very thin API that doesn't require much documentation. -- You grabbed my hand and we fell into it, like a daydream - or a fever. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Mon, 23 Apr 2007 12:22:40 -0500 Steve Greenland [EMAIL PROTECTED] wrote: You seem to be arguing that the man pages should be in the core library package, yes? I would prefer a -doc package that covers the entire API in a comprehensive and detailed manner, registered with helper programs like dwww and/or devhelp. In limited cases (like perl) where the library itself is very small, a man (3) in the -dev would be sufficient IF it covers the complete API. As for putting the docs in the core library file, I don't actually buy your argument. The *VAST* majority of a libraries users are never going to look at the man pages for that library. People who need the man pages are going to have the -dev installed, or can easily install it. I don't see why upstreams needs this. Sorry, I didn't intend to give the impression that the man (3) would go into the libfooSONAME package, instead I expect it to be present in the -dev package. HOWEVER, with libraries like libfoo-perl, the documentation IS part of the core library file already - most perl libraries (/modules) do not have a -dev package, nor would they need one. Most would also not need a -doc package but some might. Perl documentation is trivial to generate when using POD. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgp2tEz4LwxUV.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Mon, 23 Apr 2007 21:19:39 +0100 Mark Brown [EMAIL PROTECTED] wrote: Reading between the lines I think Neil is assuming that good documentation has to be large enough to be worth splitting out. In a lot of cases, yes. I do accept that some libraries - and particularly perl modules - do not need separate -doc packages. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgppwRToTm8D3.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Sun, 22 Apr 2007 20:39:26 +0100 Neil Williams [EMAIL PROTECTED] wrote: After reading the responses so far, the -doc element of my original idea needs modification. I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) The -doc is overkill for all packages but I still feel it is important for upstream development that all libraries have at least basic API documentation within Debian, not just somewhere in the Google cache. I think a -dbg package should be mandatory for all libraries that can support it, albeit with an introductory period where it is advisable, recommended then required. I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. Where such documentation makes a -doc package worthwhile, that should be done but a man (3) page in the -dev would be sufficient in some cases. I would like to see -doc packages - or at least substantive and reliable API documentation - being the norm for all libraries in Debian. As with the -dbg, the 'must' could be introduced as a recommendation prior to being mandatory. The tools exist for all these changes - only the incentive to use the tools is needed. Things like that make Debian a nicer environment to be upstream, not just a nice environment to be a DD or user. I'm upstream for many of my Debian packages and I'd like to think that Debian unstable would be the distribution of choice for upstream development. I was using Debian for upstream development long before I even considered being a DD. I chose Debian as a development platform for my own reasons and my decision was not deemed to be wise in the eyes of some of my upstream colleagues. As the newbie to that particular team, I was under significant pressure to upgrade to Fedora or SuSE. Debian needs to reclaim the respect of upstream development teams and part of that is making it *a lot* easier to do upstream development on Debian without needing to become a DD as well. Debian is respected as a distribution for users because of the multiple architecture support and the patches and bug reports that are forwarded upstream - what is missing (IMHO) is respect for Debian as the distribution of choice for upstream development itself. Possible policy amendment: Any library source package capable of building with debug information (i.e. with -g) must do so. Any such library source package must strip the debug symbols into separate objects, provide a binary package librarynamesoversion-dbg containing these separate objects as /usr/lib/debug/path/to/ELF/object for each /path/to/ELF/object in the main library package, and reference these separate objects in a .gnu_debuglink section in the corresponding unstripped object. (Thanks to Josh Triplett) I'd like to add something on -doc to that proposition but haven't decided how just yet. All library packages must include at least basic API documentation either in the -dev package or in a dedicated -doc package where sufficient documentation exists. Wherever possible, documentation should cover the entire library API, be generated from the source code of the library and be registered with helper programs like dwww and/or devhelp etc. (subject to being introduced as a recommendation prior to being made mandatory.) What happens now? Would these changes need a GR? Would it be sufficient to generate a bug report against Policy? Or submit these ideas to -policy and take from there? -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpRNdsh1sZHy.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On 23-Apr-07, 15:51 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. That's not a Debian issue. All we can do is include the documentation provided by upstream. Sure, a DD *can* write docs when they are missing, but we don't (and shouldn't) require it. Is there any case where existing valid distributable documentation is *not* in the appropriate Debian package? (Not including issues like the GDL). Debian needs to reclaim the respect of upstream development teams and part of that is making it *a lot* easier to do upstream development on Debian without needing to become a DD as well. Huh? Why do upstreams think that they need to be DDs to use Debian? Because we discourage non-DD upstreams from distributing crappy non-conforming .debs alongside their crappy non-conforming .rpms? (Not that I blame upstreams for having crappy .debs; there's a lot of policy and a lot of technology to understand - better to let a specialist take care of it.) All library packages must include at least basic API documentation either in the -dev package or in a dedicated -doc package where sufficient documentation exists. Wherever possible, documentation should cover the entire library API, be generated from the source code of the library and be registered with helper programs like dwww and/or devhelp etc. I'd remove the generated from the source code clause. Yes, many projects choose to do their docs that way. Some don't. Would these changes need a GR? No. Or submit these ideas to -policy and take from there? Yes. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Mon, 23 Apr 2007 16:15:02 -0500 Steve Greenland [EMAIL PROTECTED] wrote: On 23-Apr-07, 15:51 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. That's not a Debian issue. All we can do is include the documentation provided by upstream. Sure, a DD *can* write docs when they are missing, but we don't (and shouldn't) require it. Why not? What is wrong with writing a basic man (3) for a library when we already have a requirement for a man (1) for the application? A lack of application documentation makes a system hard for users - we know that so Policy mandates a man (1) for these people whether one exists in the .orig or not, even if it is almost empty. A lack of library documentation makes a system hard for upstream developers - these people are also Debian users and deserve support too. True, they may be more 'advanced' than the average user in terms of their knowledge of C or the autotools - doesn't mean that DD's can assume that they know how a DD would find out the information they need. Upstream generally tries to be as distro-neutral as possible - this can be very difficult with inadequate documentation. It is a Debian issue - it is precisely because the impression has got about that Debian is unfriendly to upstream development that this kind of change is absolutely necessary. The information isn't being made accessible and that results in people thinking you have to know what a DD knows in order to work out what is wrong with the upstream build on your system. These are development builds, not releases - not even snapshots. Things often break and if we want upstream releases that build nicely on Debian, it is good to encourage more upstream developers to use Debian. DD's win in the end - cleaner upstream development builds make for cleaner released .orig builds, meaning less patches! That has to be AGoodThing (tm)! Conversely, a lack of information can result in those upstream developers who choose to use Debian being flamed as a result of that choice because their commits break the build for other distros (another snippet from my pre-DD days!) I can assure you, claiming that your changes haven't broken the build on your Debian system cuts no ice when the change has broken everyone else's build!!! Without the docs, the Debian user is left without the information required to either defend their change or fix the build. Not good. If there is no documentation, file a bug upstream and ask. If the response is RTSL, this should at least be documented in Debian so that users of the -dev know who to blame for the lack of docs. Having a Debian man (3) also provides a focus for contribution of suitable content for the man (3) which in turn could be forwarded upstream. It's easier to add content to an existing man (3) than to create a patch to create a whole one from scratch - especially when the contributors are not necessarily familiar with Debian packaging beyond apt-get. Is there any case where existing valid distributable documentation is *not* in the appropriate Debian package? (Not including issues like the GDL). There is a distinct lack of man (3) and coordinated documentation for libraries in Debian. True, some poorly documented packages include test programs or examples somewhere under /usr/share/doc/ but it isn't simple to track these down. At the very least, the Debian maintainer should make it clear where these files are located in a man (3) for the library. Where possible though, a full -doc package is a far, far better option if Debian actually does want to support upstream development on Debian. That's why I stressed the registration of docs with dww and/or devhelp. Some libraries have very, very good docs - libglib2, libgtk2 - the problem comes with the smaller libraries, specialised tasks that are only used by a few applications. By definition, few people will know these API's yet these are often the very libraries that have the least documentation - hence mandatory -doc or man (3). Debian needs to reclaim the respect of upstream development teams and part of that is making it *a lot* easier to do upstream development on Debian without needing to become a DD as well. Huh? Why do upstreams think that they need to be DDs to use Debian? I can only speak from my own experience upstream on this - I wasn't new to Debian at the time but most questions that I asked (or problems that I experienced) came back with a response of must be something wrong with Debian because something everyone else did wasn't working for me. I didn't have the documentation available to find out what was wrong and it was very frustrating. In the end, I resorted to a laborious method involving rsync and simultaneous builds on Debian and Fedora prior to commits. That kind of hack can seriously drain
Re: Mandatory -dbg packages for libraries? (and API docs too)
Neil Williams wrote: Would these changes need a GR? Why would a policy change need a GR? How could a GR possibly be the best way to choose a sound technical policy? -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
On Tue, Apr 24, 2007 at 12:00:59AM +0100, Neil Williams wrote: On Mon, 23 Apr 2007 16:15:02 -0500 Steve Greenland [EMAIL PROTECTED] wrote: On 23-Apr-07, 15:51 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I think that all libraries - without exception - must come with some API documentation and the docs should be as complete and as accurate as possible - ideally generated from the source itself. That's not a Debian issue. All we can do is include the documentation provided by upstream. Sure, a DD *can* write docs when they are missing, but we don't (and shouldn't) require it. Why not? What is wrong with writing a basic man (3) for a library when we already have a requirement for a man (1) for the application? That the existing requirement is already too much for us to keep up with, so adding new requirements, especially ones that require significant attention to detail to get right, dilutes our attention for little benefit? A lack of library documentation makes a system hard for upstream developers - these people are also Debian users and deserve support too. Feel free to support them by writing any manpages you think are missing. It is a Debian issue - it is precisely because the impression has got about that Debian is unfriendly to upstream development that this kind of change is absolutely necessary. Huh? There is a distinct lack of man (3) and coordinated documentation for libraries in Debian. True, some poorly documented packages include test programs or examples somewhere under /usr/share/doc/ but it isn't simple to track these down. At the very least, the Debian maintainer should make it clear where these files are located in a man (3) for the library. That sounds to me like an abuse of section 3 of the man hierarchy. Where possible though, a full -doc package is a far, far better option if Debian actually does want to support upstream development on Debian. I think it's preposterous to assert that it's Debian's responsibility to provide upstream documentation for libraries in order to make Debian appealing as a platform to other upstreams. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. [EMAIL PROTECTED] http://www.debian.org/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries? (and API docs too)
Neil Williams wrote: I chose Debian as a development platform for my own reasons and my decision was not deemed to be wise in the eyes of some of my upstream colleagues. As the newbie to that particular team, I was under significant pressure to upgrade to Fedora or SuSE. Debian needs to reclaim the respect of upstream development teams and part of that is making it *a lot* easier to do upstream development on Debian without needing to become a DD as well. Debian is respected as a distribution for users because of the multiple architecture support and the patches and bug reports that are forwarded upstream - what is missing (IMHO) is respect for Debian as the distribution of choice for upstream development itself. Are you generalising from your one poor personal experience with a non-Debian-friendly upstream, or do you have a significant body of data that I don't about masses of upsteams who are not Debian friendly? My impression has always been that a significant proportion of upstreams use Debian, or are at least familiar with it. I base this on, amoung other things, interacting with hundreds of different upstreams whose packages I have maintained in Debian, as well as working in linux companies and personally knowing a lot of upstream developers. The only significant documentation that is missing in Debian that I know of is GFDL licensed docs which have been removed from main. Aside from that, if a library is missing documentation, it's missing it because it's not available upsteam either. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Sun, 2007-04-22 at 22:31 +0200, Hendrik Sattler wrote: Am Sonntag 22 April 2007 22:12 schrieb Russ Allbery: Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. Did you ever try to debug an application compiled with optimizations? snip Of course. There are many undefined behaviour bugs that will only show up in optimised builds. You have to stop relying on source lines and look at the disassembly, of course. Also, oprofile wants debugging information, and there's no sense in turning optimisations off for that! Ben. -- Ben Hutchings When you say `I wrote a program that crashed Windows', people just stare ... and say `Hey, I got those with the system, *for free*'. - Linus Torvalds signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries?
On Sun, 2007-04-22 at 22:14 +0200, Florian Weimer wrote: * Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? GCC's debugging information at -O2 will continue to worsen (in part as a result of -O2 getting better). It's now quite capable of telling the debugger that a source line has ended up in several different chunks of object code, and that a variable moves around between memory and registers. Ben. -- Ben Hutchings When you say `I wrote a program that crashed Windows', people just stare ... and say `Hey, I got those with the system, *for free*'. - Linus Torvalds signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries?
On Mon, 23 Apr 2007 15:05:22 -0400, Joey Hess [EMAIL PROTECTED] said: I would rather add it as a recommended practice in policy, with a note that it will become a should/must as we get better coverage, and _also_ provide examples of what maintainers need to do to create separate debugging symbol packages in an informative footnote. Well, we've made more than ~300 packages insta-buggy with policy changes before. It's not insta-rc-buggy. OTOH, I don't really care; 300 bug reports could be mass-filed w/o it being a should in policy. If I have inadvertently done so in the past, I feel the need to apologize, but my past mistakes do not condone me making the same errors again. Note that I've already written some documentation for developers-reference in #420540. Thanks. Now that we have released Etch, I need to be getting back at updating policy again, there are a number of issues sitting on my TODO list. The policy-relevant bits are that we use /usr/lib/debug/path-to-object, that the files should not be executable (possibly a common mistake since objcopy preserves executable bits IIRC), and that the package names end in -dbg and the debug packages depend on an equal version of the package they provide debugging symbols for. Actually, the whole writeup seems good, and some of it can be kept informative rather than normative. I'll queue this up for things to do when I do get some round tuits to spend on policy, and see about adding -dbg packages for the libraries I have. manoj -- My Boss needs a surge protector. That way her mouth would be buffered from surprise spikes in her brain. Manoj Srivastava [EMAIL PROTECTED] http://www.debian.org/~srivasta/ 1024D/BF24424C print 4966 F272 D093 B493 410B 924B 21BA DABB BF24 424C -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 07:29:55PM -0400, Joey Hess wrote: Neil Williams wrote: Certain packages have already had bug reports requesting a -dbg package. I'd rather see some offline debug-symbol infrastructure for all packages implemented, so that you can download the debug symbols when you need them. But the -dbg package only depends on the same version of the library - the library won't depend on the -dbg so those who need the -dbg are the only ones who would download and install them. Each time this has come up before, the concern has been that adding -dbg versions of every binary package would greatly inflate the size of the archive, and nearly double the total number of packages, with associated scalability problems. Even with separated debugging symbols, -dbg packages are frequently larger than the package they provide debugging symbols for. See for example xserver-xorg-core-dbg. Looking through the 227 lib*-dbg packages, I found few contain separated debugging symbols, except for packages maintained by the xorg team[1]. I'm not sure if this is due many people still not knowing about separated debugging symbols, or due to technical reasons. For example, is there a tecnical reason why libc6-dbg does not contain separated debugging symbols? Anyway, doubling the size of the archive is less of an issue than it might have been in the past, since we've done the archive split, and since ftp-master has 1.4 Terabytes of disk with half that unused, but it is still a concern, for mirrors, number of DVDs, etc. What about some special parts on the archive for this, somethings like what is actually used for source packages, but of course arch dependant: deb-dbg http://http.us.debian.org/debian sid main could translate to http://http.us.debian.org/debian/dist/sid/main/dbg-$arch where all the -dbg packages could live. That would minimize the impact making -dbg packages easier not to mirror and avoid cluttering packages lists for non-developers users while only adding a 'add a deb-dbg mirror' instruction for requesting a backtrace on a bug report. Scalability issues with the number of packages have also been reduced in some areas. apt no longer has to download the while Packages files on each update, so it wouldn't take 2x the bandwidth to add -dbg packages for every package to the Packages files. There would still be significant impact in apt's memory usage, in the disk space used to store the Packages files, in the UIs that have to somehow present or hide all these -dbg packages, etc. With the above approach this impact is minimized quite nicely. I've considered before trying to set up a separate, parallel archive that would only hold the -dbg packages, but implementing that without initially using the Debian infrastructure would be tough, and my experiences with setting up[2]/maintaining the separate udeb section of the archive is that it adds a lot of complexity. Someone made a very good point that it's often and increasingly painful to rebuild debugging versions for the whole library chain of a binary. OTOH, rebuilding a debug version of the binary itself is not especially hard. So while I'd love to have a way to have -dbg packages available for every binary, I actually am happy with this proposal to do it for only every library (plus whatever other binaries really need it). And it's a direction we're already moving in, with, as I mentioned, 227 lib*-dbg packages already in the archive. That's more than 10% of all our libraries already done[3]. So I suggest that we take this as an existing practice, document it as a should in policy for now, document *how* to do separated debugging symbols in the developers reference (which does not currently seem to mention it at all), and go add -dbg versions of our library packages. -- see shy jo, doing so for aalib now [1] Who are doing a really nice job on their -dbg packages. [2] Actually, the ftp-masters did all the real setup work. [3] Conversely, there are only 62 -dbg packages for non-libraries.. -- Damián Viano(Des) ¯ ¯ - _ _ - ¯ ¯ GPG: 0x6EB95A6F Debian ¯-_GNU_-¯ Linux Web: http://damianv.com.ar/ ¯-¯ signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries? (and API docs too)
Neil Williams wrote: snip There is a distinct lack of man (3) and coordinated documentation for libraries in Debian. True, some poorly documented packages include test programs or examples somewhere under /usr/share/doc/ but it isn't simple to track these down. Is it unreasonable to expect libfoo's docs (independent of format or quality) to be located under /usr/share/doc/libfoo, and not somewhere else? snip Noting if the Debian library package differs significantly to the library upstream and why - not somewhere in NEWS.gz or README.gz, but in the one place application developers will look, the API docs. AFAIK, README.Debian is _the_ place where divergences from upstream should be noted. I can't find (yes, I only glanced through the documents, please correct me if I'm wrong) any reference to that neither in the Debian FAQ nor the Debian Reference, which I assume are the 2 most important user documents. Maybe a note should be added? -- Felipe Sateler -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Mandatory -dbg packages for libraries?
After getting some positive feedback on this from a blog entry and personal mail, I wondered if the idea deserves a wider airing. http://www.linux.codehelp.co.uk/serendipity/index.php?/archives/18-Mandatory-dbg-packages-for-libraries.html Others have expressed the same idea: http://ze-dinosaur.livejournal.com/7036.html Certain packages have already had bug reports requesting a -dbg package. I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) Alternatively, all lib packages should at least provide a -dev and -dbg package provide and either man (3) content or preferably a full -doc package. Currently, only a -dev is mandated. I'm currently writing a GtkHTML extension for the new Gtk2 version of quicklist (in experimental) and writing code to use an undocumented library is a PITA. Libraries without a -dbg are worse because Debian doesn't need any upstream involvement to create -dbg packages. Things like that make Debian a nicer environment to be upstream, not just a nice environment to be a DD or user. I'm upstream for many of my Debian packages and I'd like to think that Debian unstable would be the distribution of choice for upstream development. Of course, the problem is that -dbg packages are necessarily larger than the ordinary lib and putting all the affected libraries into the NEW queue to get the -dbg package added to the override file is going to make the NEW queue truly enormous. (Then again if our new DPL is able to put some of his platform ideas into practise, maybe new binaries from an existing source package can be fast-tracked through NEW.) Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? Possible policy amendment: Any library source package capable of building with debug information (i.e. with -g) must do so. Any such library source package must strip the debug symbols into separate objects, provide a binary package librarynamesoversion-dbg containing these separate objects as /usr/lib/debug/path/to/ELF/object for each /path/to/ELF/object in the main library package, and reference these separate objects in a .gnu_debuglink section in the corresponding unstripped object. (Thanks to Josh Triplett) I'd like to add something on -doc to that proposition but haven't decided how just yet. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpHWFVDMN4uP.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 08:39:26PM +0100, Neil Williams wrote: After getting some positive feedback on this from a blog entry and personal mail, I wondered if the idea deserves a wider airing. http://www.linux.codehelp.co.uk/serendipity/index.php?/archives/18-Mandatory-dbg-packages-for-libraries.html Others have expressed the same idea: http://ze-dinosaur.livejournal.com/7036.html Certain packages have already had bug reports requesting a -dbg package. I'd rather see some offline debug-symbol infrastructure for all packages implemented, so that you can download the debug symbols when you need them. Michael -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 09:44:28PM +0200, Michael Banck [EMAIL PROTECTED] wrote: On Sun, Apr 22, 2007 at 08:39:26PM +0100, Neil Williams wrote: After getting some positive feedback on this from a blog entry and personal mail, I wondered if the idea deserves a wider airing. http://www.linux.codehelp.co.uk/serendipity/index.php?/archives/18-Mandatory-dbg-packages-for-libraries.html Others have expressed the same idea: http://ze-dinosaur.livejournal.com/7036.html Certain packages have already had bug reports requesting a -dbg package. I'd rather see some offline debug-symbol infrastructure for all packages implemented, so that you can download the debug symbols when you need them. Well, to download debug symbols, they have to be kept somehow at build time, and put somewhere... Mike -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Any examples for debian/rules file? But if you mandate -dbg packages for libs, why not also for apps? However, most users will never use them. HS pgpI3LwPuIDCE.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 10:02:37PM +0200, Hendrik Sattler wrote: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Any examples for debian/rules file? But if you mandate -dbg packages for libs, why not also for apps? However, most users will never use them. Also, is there any figure on mirror size impact? -- Damián Viano(Des) ¯ ¯ - _ _ - ¯ ¯ GPG: 0x6EB95A6F Debian ¯-_GNU_-¯ Linux Web: http://damianv.com.ar/ ¯-¯ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 22:02:37 +0200, Hendrik Sattler wrote: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Why would you do that? (btw, you already build them twice if you provide a shared and a static library, but adding debugging symbols doesn't change that) Cheers, Julien -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
* Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? GCC's debugging information at -O2 will continue to worsen (in part as a result of -O2 getting better). Hence, -dbg libraries would need to be compiled with different optimization settings in some cases. It might not a good idea to feature them prominently because they might behave differently in corner cases. And compiling everything with something close to -O1 isn't acceptable, either. For our internal libraries at work, we simply compile with -g and do not strip the binaries, but program-related disk space is not an issue we face. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, 22 Apr 2007 22:02:37 +0200 Hendrik Sattler [EMAIL PROTECTED] wrote: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Any examples for debian/rules file? dh_strip --dbg-package=libfooSONAME-dbg But if you mandate -dbg packages for libs, why not also for apps? Because, as an upstream developer, you are generally working with the upstream package, you have the debug code for your own package available. Some applications could benefit from a -dbg package, probably large programs like rhythmbox, gnucash. However, most users will never use them. That is precisely what I want to change - I want Debian to be attractive for upstream development so that more Debian users are also free software developers. Why not make life easier for developing software ON Debian? It's only going to make it easier for Debian to package upstream code if it was developed on Debian in the first place. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpgfTRmv4FA3.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, 22 Apr 2007 21:44:28 +0200 Michael Banck [EMAIL PROTECTED] wrote: Certain packages have already had bug reports requesting a -dbg package. I'd rather see some offline debug-symbol infrastructure for all packages implemented, so that you can download the debug symbols when you need them. But the -dbg package only depends on the same version of the library - the library won't depend on the -dbg so those who need the -dbg are the only ones who would download and install them. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpZ8OILkHyV0.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
Am Sonntag 22 April 2007 22:12 schrieb Russ Allbery: Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 21:39 schrieb Neil Williams: Apart from those limitations, is there a *technical* reason why -dbg packages should not be available? Is it worth taking to -policy? You essentially need to build all library packages 2 times, then. Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. Did you ever try to debug an application compiled with optimizations? No, either you build an optimized version (-O2) or you build a debug version (-g). If you want to debug what was coded, you better compile without optimizations. HS -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 22:12 schrieb Russ Allbery: Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. Did you ever try to debug an application compiled with optimizations? Yes, I do it all the time. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On 22-Apr-07, 14:39 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) 1. Rather than cluttering the archive and Packages file with -dbg packages that will (mostly) never be used, how about mandating a debug target in library debian/rules files, so that when someone does need the debug package, it's trivial to build. Since the person most likely to need the target is the package maintainer, there would be some incentive to make sure it works. 2. Why a seperate -doc? API docs should be part of the -dev package. I'm going to guess that for *most* libraries, the docs are a trivial part of the size of the -dev package. For those with significant documentation, sure, a seperate -doc makes sense, just as we do now. Steve -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Am Sonntag 22 April 2007 22:56 schrieb Russ Allbery: Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 22:12 schrieb Russ Allbery: Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. Did you ever try to debug an application compiled with optimizations? Yes, I do it all the time. Hmm, must be personal preference, then. It makes debugging loops and such things rather a strange thing. CMake obviously agrees as its default setting between release build and debug build act exactly like that. HS pgpVnKdBOvBPs.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, 2007-04-22 at 16:14 -0500, Steve Greenland wrote: On 22-Apr-07, 14:39 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) 1. Rather than cluttering the archive and Packages file with -dbg packages that will (mostly) never be used, how about mandating a debug target in library debian/rules files, so that when someone does need the debug package, it's trivial to build. Since the person most likely to need the target is the package maintainer, there would be some incentive to make sure it works. Because segfaults are often not easily reproduced. Having the ability to analyse a crash that occured when the user did not have the -dbg packages installed is not possible unless you have the original symbols the compiler created. -Rob -- GPG key available at: http://www.robertcollins.net/keys.txt. signature.asc Description: This is a digitally signed message part
Re: Mandatory -dbg packages for libraries?
On 22-Apr-07, 16:22 (CDT), Robert Collins [EMAIL PROTECTED] wrote: On Sun, 2007-04-22 at 16:14 -0500, Steve Greenland wrote: On 22-Apr-07, 14:39 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) 1. Rather than cluttering the archive and Packages file with -dbg packages that will (mostly) never be used, how about mandating a debug target in library debian/rules files, so that when someone does need the debug package, it's trivial to build. Since the person most likely to need the target is the package maintainer, there would be some incentive to make sure it works. Because segfaults are often not easily reproduced. Having the ability to analyse a crash that occured when the user did not have the -dbg packages installed is not possible unless you have the original symbols the compiler created. That's an argument in favor of making the base library package built with debug symbols and then stripped[1], not of requiring -dbg packages. Steve [1] This all seems vaguely familiar. Didn't we go through this argument back in the late 90s? IIRC, we originally did ship libs with (stripped) debug symbols, for just the reason that Robert is promoting. -- Steve Greenland The irony is that Bill Gates claims to be making a stable operating system and Linus Torvalds claims to be trying to take over the world. -- seen on the net -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 10:31:04PM +0200, Hendrik Sattler wrote: Am Sonntag 22 April 2007 22:12 schrieb Russ Allbery: Actually, you don't. See the features of dh_strip introduced at debhelper level V5. And of course you can do the same thing by hand. gdb will read the resulting detached debugging symbols automatically. Did you ever try to debug an application compiled with optimizations? Irrelevant. We're talking about *libraries* here, not applications. The main reason why you want debugging symbols in a library is so that you can get a reasonable backtrace in a bug report. If you need to step into the library to figure out how a few function calls work, then often you're debugging an application that uses the library rather than the library itself. Stepping in and through the library is only useful insofar that you can figure out whether you're calling the library's API calls correctly; that doesn't need a whole lot of going inside the library, at least not so much that having an optimized library isn't annoying yet. If you *are* debugging the library itself and not an application that uses the library, then you probably need to recompile the library anyway; at that point, doing it before throwing stuff through the debugger (so that make after changing one or two lines in a file doesn't take an hour) is a good idea; and while you're at it, compiling with -O0 isn't very hard to do. YMMV, of course. -- Fun will now commence -- Seven Of Nine, Ashes to Ashes, stardate 53679.4 -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, 22 Apr 2007 16:14:04 -0500 Steve Greenland [EMAIL PROTECTED] wrote: On 22-Apr-07, 14:39 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) 1. Rather than cluttering the archive and Packages file with -dbg packages that will (mostly) never be used, Don't underestimate how many people either want to help fix bugs or want to develop their own software. Why push them to Fedora? It's bad enough that many upstream teams think that they have to test on Fedora, Debian and SuSE. how about mandating a debug target in library debian/rules files, so that when someone does need the debug package, it's trivial to build. Since the person most likely to need the target is the package maintainer, there would be some incentive to make sure it works. It complicates bug testing enormously if the bug tester has to rebuild A DEPENDENCY of the package just to get the debug symbols - it's a DIFFERENT build, it may hide or even replace certain bugs. The person needing the lib -dbg is NOT the package maintainer, that's my point - it's needed by upstream of the applications that use the library. Separate DD's from upstream - the needs of DD's do not always match the needs of an upstream developer. In this case, the needs of the DD are largely irrelevant - the needs of upstream are important. Upstream are using SourceForge or Berlios, not Alioth. Upstream don't use dh_strip or debhelper - they need us to provide what dh_strip can give them. When I'm fixing bugs in the Gtk2 port of quicklist, why should I have to rebuild gtk +extra2? It's a MUCH bigger package. Plus, rebuilding it on my system may introduce bugs that would cause build problems on other distributions that used the original gtk+extra2. 2. Why a seperate -doc? API docs should be part of the -dev package. In practice, such attitudes are commonly expressed as RTSL. (Read The Source, Luke). That does NOT encourage upstream usage of Debian as a distro. Is man (3) really so hard for a DD to provide? I'm going to guess that for *most* libraries, the docs are a trivial part of the size of the -dev package. For those with significant documentation, sure, a seperate -doc makes sense, just as we do now. I think libraries should be encouraged to provide significant documentation - what we have now is simply not enough. -- Neil Williams = http://www.data-freedom.org/ http://www.nosoftwarepatents.com/ http://www.linux.codehelp.co.uk/ pgpdTuUFSpODP.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, 22 Apr 2007 20:39:26 +0100, Neil Williams wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) For most lib*-perl packages a separate -doc package seems like an overkill to me; the package's overhead would probably be larger than the actual contents for most of them. gregor -- .''`. http://info.comodo.priv.at/ | gpg key ID: 0x00F3CFE4 : :' : debian: the universal operating system - http://www.debian.org/ `. `' member of https://www.vibe.at/ | how to reply: http://got.to/quote/ `-NP: Bob Dylan: Subterranean Homesick Blues signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 04:40:45PM -0500, Steve Greenland wrote: That's an argument in favor of making the base library package built with debug symbols and then stripped[1], not of requiring -dbg packages. Depends what you put in the -dbg package; it could be the symbols stripped out of the main binary. -- You grabbed my hand and we fell into it, like a daydream - or a fever. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007, Neil Williams wrote: 2. Why a seperate -doc? API docs should be part of the -dev package. In practice, such attitudes are commonly expressed as RTSL. (Read The Source, Luke). That does NOT encourage upstream usage of Debian as a distro. Is man (3) really so hard for a DD to provide? The idea is to provide man(3) *in* the -dev package if it's not big enough to justify a package split. -- Sam. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Steve Greenland [EMAIL PROTECTED] writes: On 22-Apr-07, 14:39 (CDT), Neil Williams [EMAIL PROTECTED] wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) 1. Rather than cluttering the archive and Packages file with -dbg packages that will (mostly) never be used, how about mandating a debug target in library debian/rules files, so that when someone does need the debug package, it's trivial to build. Since the person most likely to need the target is the package maintainer, there would be some incentive to make sure it works. When I have needed to debug problems in the past, this has sometimes meant over a day of rebuilding large libraries when the problem covers several shared libraries. I would have saved hours if I could have just installed some prebuilt -dbg packages and run gdb. Even more importantly, I want to use the debug package matching the version *and build* the user reporting the problem has. If it's due to e.g. a buggy header in a build dependency causing a misbuild, I would miss this when I rebuild it. I have had this bite me in the past. I would personally support the requirement for every binary library package to have a separate -dbg package containing the debug symbols. Even if they don't get regularly used, they are worth it for when things go wrong. We can then just ask the user to install libfoo-dbg and send us a backtrace, whatever the package or library. For big and/or popular applications, it would also be nice to have -dbg symbols for them as well. If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? 2. Why a seperate -doc? API docs should be part of the -dev package. I'm going to guess that for *most* libraries, the docs are a trivial part of the size of the -dev package. For libraries using doxygen, it can generate megabytes of documentation, even for small libraries if you enable all the call graphs and things. Unless I'm actually developing using a particular library directly, I don't want disk space wasted by its docs. It also means less to download when autobuilding stuff. For those with significant documentation, sure, a seperate -doc makes sense, just as we do now. Agreed. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. pgpHH33cqVk9H.pgp Description: PGP signature
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 04:40:45PM -0500, Steve Greenland wrote: On 22-Apr-07, 16:22 (CDT), Robert Collins [EMAIL PROTECTED] wrote: Because segfaults are often not easily reproduced. Having the ability to analyse a crash that occured when the user did not have the -dbg packages installed is not possible unless you have the original symbols the compiler created. That's an argument in favor of making the base library package built with debug symbols and then stripped[1], not of requiring -dbg packages. Maybe you should take a look at what dh_strip and -dbg packages do. It strips the debug symbols and puts it in a seperate file that's put in /usr/lib/debug. You put those files with debug symbols only in the -dbg package. This means that you can just install the -dbg package and gdb will automaticly pick up the debug symbols, without needing to rebuild anything. See objcopy's --only-keep-debug and --add-gnu-debuglink options. (It's also in strip's manpage) Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 11:15:36PM +0100, Roger Leigh wrote: If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? Dropping most .a libraries is something I agree with. I see no reason why we should have them for most of the libraries. Kurt -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
Hendrik Sattler [EMAIL PROTECTED] writes: Am Sonntag 22 April 2007 22:56 schrieb Russ Allbery: Hendrik Sattler [EMAIL PROTECTED] writes: Did you ever try to debug an application compiled with optimizations? Yes, I do it all the time. Hmm, must be personal preference, then. It makes debugging loops and such things rather a strange thing. CMake obviously agrees as its default setting between release build and debug build act exactly like that. It's certainly not ideal, and the control flow is often obscure. But it's better than nothing, and it's way easier to just use dh_strip than to do a second compile of all the libraries, stash them somewhere, and then play LD_LIBRARY_PATH games. I've tracked down a lot of bugs with nothing more than the current -dbg packages. Indeed, the reason why there's a libkrb5-dbg is precisely because I needed it to debug other Kerberos packages I maintain. -- Russ Allbery ([EMAIL PROTECTED]) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 10:31:04PM +0200, Hendrik Sattler wrote: Did you ever try to debug an application compiled with optimizations? No, either you build an optimized version (-O2) or you build a debug version (-g). If you want to debug what was coded, you better compile without optimizations. Often, single-stepping isn't required -- given a proper backtrace, you can solve a _lot_ of problems. Heck, even just knowing what line things crashed on will help for almost all segfaults that aren't related to heap corruption. /* Steinar */ -- Homepage: http://www.sesse.net/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 11:15:36PM +0100, Roger Leigh wrote: If there are concerns over archive size, why don't we drop all static .a libraries at the same time. Given that in Debian we typically always link dynamically, is there a need for .a libraries in all but a handful of cases? I'd be interested to see some numbers on the archive size impact - my experience with C++ suggests that the size inflation caused by debug symbols can be enormous. -- You grabbed my hand and we fell into it, like a daydream - or a fever. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Neil Williams wrote: Certain packages have already had bug reports requesting a -dbg package. I'd rather see some offline debug-symbol infrastructure for all packages implemented, so that you can download the debug symbols when you need them. But the -dbg package only depends on the same version of the library - the library won't depend on the -dbg so those who need the -dbg are the only ones who would download and install them. Each time this has come up before, the concern has been that adding -dbg versions of every binary package would greatly inflate the size of the archive, and nearly double the total number of packages, with associated scalability problems. Even with separated debugging symbols, -dbg packages are frequently larger than the package they provide debugging symbols for. See for example xserver-xorg-core-dbg. Looking through the 227 lib*-dbg packages, I found few contain separated debugging symbols, except for packages maintained by the xorg team[1]. I'm not sure if this is due many people still not knowing about separated debugging symbols, or due to technical reasons. For example, is there a tecnical reason why libc6-dbg does not contain separated debugging symbols? Anyway, doubling the size of the archive is less of an issue than it might have been in the past, since we've done the archive split, and since ftp-master has 1.4 Terabytes of disk with half that unused, but it is still a concern, for mirrors, number of DVDs, etc. Scalability issues with the number of packages have also been reduced in some areas. apt no longer has to download the while Packages files on each update, so it wouldn't take 2x the bandwidth to add -dbg packages for every package to the Packages files. There would still be significant impact in apt's memory usage, in the disk space used to store the Packages files, in the UIs that have to somehow present or hide all these -dbg packages, etc. I've considered before trying to set up a separate, parallel archive that would only hold the -dbg packages, but implementing that without initially using the Debian infrastructure would be tough, and my experiences with setting up[2]/maintaining the separate udeb section of the archive is that it adds a lot of complexity. Someone made a very good point that it's often and increasingly painful to rebuild debugging versions for the whole library chain of a binary. OTOH, rebuilding a debug version of the binary itself is not especially hard. So while I'd love to have a way to have -dbg packages available for every binary, I actually am happy with this proposal to do it for only every library (plus whatever other binaries really need it). And it's a direction we're already moving in, with, as I mentioned, 227 lib*-dbg packages already in the archive. That's more than 10% of all our libraries already done[3]. So I suggest that we take this as an existing practice, document it as a should in policy for now, document *how* to do separated debugging symbols in the developers reference (which does not currently seem to mention it at all), and go add -dbg versions of our library packages. -- see shy jo, doing so for aalib now [1] Who are doing a really nice job on their -dbg packages. [2] Actually, the ftp-masters did all the real setup work. [3] Conversely, there are only 62 -dbg packages for non-libraries.. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
I demand that Neil Williams may or may not have written... [snip] Upstream are using SourceForge or Berlios, not Alioth. Upstream don't use dh_strip or debhelper And, of course, upstream is not a Debian package maintainer. [snip] -- | Darren Salt| linux or ds at | nr. Ashington, | Toon | RISC OS, Linux | youmustbejoking,demon,co,uk | Northumberland | Army | URL:http://www.youmustbejoking.demon.co.uk/ (PGP 2.6, GPG keys) I'm writing a book... I've got the page numbers done. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: Mandatory -dbg packages for libraries?
On Sun, Apr 22, 2007 at 08:39:26PM +0100, Neil Williams wrote: I'd like to see all library source packages having a minimum of 4 binary packages required by Policy: the SONAME, the -dev, the -dbg and a -doc package. (Libraries for perl or other non-compiled languages would be exempt from -dbg packages but not -doc.) Mandating a -doc package seems excessive unless it can just be a Provides. For the two libraries I maintain (both of which are pretty straightforward) the provided documentation consists of a single manual page and in one case some example code. Putting them in a separate package wouldn't feel terribly constructive to me. What benefit do you see from forcing such a split? -- You grabbed my hand and we fell into it, like a daydream - or a fever. signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
Mark Brown wrote: I'd be interested to see some numbers on the archive size impact - my experience with C++ suggests that the size inflation caused by debug symbols can be enormous. I don't know about C++, but for C it depends. For example, aalib is a 102 kb library that compresses to 44kb. Its debug symbols are 234 kb and compress to 65 kb. OTOH, people rarely need full debugging information for aalib, it's probably plenty to see its functions in the backtrace, and not have line number info (bear in mind that line number info includes effectively the entire source code of the program). So it I build it with -g1, the debug symbols size drops to 48 kb and compresses to 14 kb. -- see shy jo signature.asc Description: Digital signature
Re: Mandatory -dbg packages for libraries?
On 4/23/07, Joey Hess [EMAIL PROTECTED] wrote: Anyway, doubling the size of the archive is less of an issue than it might have been in the past, since we've done the archive split, and since ftp-master has 1.4 Terabytes of disk with half that unused, but it is still a concern, for mirrors, number of DVDs, etc. It would be nice if the standard iso images that Debian makes available could be made to exclude -dbg packages as a trade-off. It actually felt painful someday when doing jigdo on the archive, only to see loads of -dbg packages getting downloaded, and knowing I wasn't ever gonna use 'em. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
