Re: How to proceed with MiniDebugInfo
On Fri, 2012-05-25 at 10:10 +0300, Panu Matilainen wrote:
On 05/25/2012 09:26 AM, Alexander Larsson wrote:
On Thu, 2012-05-24 at 13:20 -0400, Casey Dahlin wrote:
On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote:
I'm at a loss to how to proceed with the MiniDebugInfo work. I have
patches to rpmbuild that creates the compressed minidebuginfo putting
them in the main binaries, and I have patches to gdb that reads the
compressed debuginfo on demand.
However, the whole thing is useless unless we agree that we want to
enable this by default. It seems some people like the idea, whereas
others disagree that its worth the increased binary size. It doesn't
look like either side is gonna be able to convince the other side, so
how do we get to a decision here?
Just go do it. See who actually shows up to stop you.
To actually get the debuginfo in the builds all I need is a minor patch
to the find-debuginfo.sh script in rpm-build. However, since the effect
of it is global it seems to me that its a wider decision than just the
maintainer of rpm-build.
As long as the behavior is optional and configurable, just send the
patch to rpm-maint and we'll see. Fedora isn't the only consumer of rpm,
and somebody else might find it useful regardless of whether Fedora
wants to enable the thing or not.
Ok, here goes:
* rpm-minidebuginfo.patch
Patches find-debuginfo.sh and adds new _include_minidebuginfo option
defaulting to off
* redhat-rpm-config-minidebug.patch
Example patch that enables building minidebuginfo by default on Fedora
* 0001-configure.ac-Look-for-and-use-libzma-on-ELF-systems.patch
* 0002-Support-lzma-compressed-.gnu_debugdata-sections.patch
Gdb patches enabling on-demand lzma decompression of
the .gnu_debugdata section looking for separate debuginfo there.
With these patches all applied any rpm you build (that doesn't
explicitly disable debuginfo generation) will get xz compresses minimal
debug info that gdb will find automatically (unless the real debuginfo
is installed, then that will be used).
diff -up rpm-4.9.1.3/macros.in.minidebug rpm-4.9.1.3/macros.in
--- rpm-4.9.1.3/macros.in.minidebug 2012-06-11 11:16:21.216952339 +0200
+++ rpm-4.9.1.3/macros.in 2012-06-11 11:16:23.686912455 +0200
@@ -175,7 +175,7 @@
# the script. See the script for details.
#
%__debug_install_post \
- %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_find_debuginfo_opts} %{_builddir}/%{?buildsubdir}\
+ %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} %{_builddir}/%{?buildsubdir}\
%{nil}
# Template for debug information sub-package.
@@ -418,6 +418,12 @@ package or when debugging this package.\
#%_missing_build_ids_terminate_build 1
#
+# Include minimal debug information in build binaries.
+# Requires _enable_debug_packages.
+#
+#%_include_minidebuginfo 1
+
+#
# Use internal dependency generator rather than external helpers?
%_use_internal_dependency_generator 1
diff -up rpm-4.9.1.3/scripts/find-debuginfo.sh.minidebug rpm-4.9.1.3/scripts/find-debuginfo.sh
--- rpm-4.9.1.3/scripts/find-debuginfo.sh.minidebug 2012-06-11 11:16:09.698138273 +0200
+++ rpm-4.9.1.3/scripts/find-debuginfo.sh 2012-06-11 11:16:13.399078526 +0200
@@ -2,7 +2,7 @@
#find-debuginfo.sh - automagically generate debug info and file list
#for inclusion in an rpm spec file.
#
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
# [-o debugfiles.list]
# [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
# [builddir]
@@ -29,6 +29,9 @@ strip_g=false
# with -r arg, pass --reloc-debug-sections to eu-strip.
strip_r=false
+# with -m arg, add minimal debuginfo to binary.
+include_minidebug=false
+
# Barf on missing build IDs.
strict=false
@@ -43,6 +46,9 @@ while [ $# -gt 0 ]; do
-g)
strip_g=true
;;
+ -m)
+include_minidebug=true
+;;
-o)
if [ -z ${lists[$nout]} -a -z ${ptns[$nout]} ]; then
out=$2
@@ -105,6 +111,32 @@ strip_to_debug()
chmod 444 $1 || exit
}
+add_minidebug()
+{
+ local debuginfo=$1
+ local binary=$2
+
+ local dynsyms=`mktemp`
+ local funcsyms=`mktemp`
+ local keep_symbols=`mktemp`
+ local mini_debuginfo=`mktemp`
+
+ # Extract the dynamic symbols from the main binary, there is no need to also have these
+ # in the normal symbol table
+ nm -D $binary --format=posix --defined-only | awk '{ print $1 }' | sort $dynsyms
+ # Extract all the text (i.e. function) symbols from the debuginfo
+ nm $debuginfo --format=posix --defined-only | awk '{ if ($2 == T || $2 == t) print $1 }' | sort $funcsyms
+ # Keep all the function symbols not already in the dynamic symbol table
+ comm -13 $dynsyms $funcsyms $keep_symbols
+ # Copy the full debuginfo, keeping only a minumal set of symbols and
Re: How to proceed with MiniDebugInfo
On Tue, 2012-05-29 at 21:49 +0200, Lennart Poettering wrote: On Thu, 24.05.12 09:28, Alexander Larsson (al...@redhat.com) wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? The right way is probably to write a feature page for F18 for it, and then get it through Fedora 18 feature process. With FESCO accepting the feature you should have all you need to get your work accepted by the various stakeholders. I did write a feature page for it. Thats how these threads started. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Tue, Jun 5, 2012 at 11:08 AM, Alexander Larsson al...@redhat.com wrote: On Tue, 2012-05-29 at 21:49 +0200, Lennart Poettering wrote: The right way is probably to write a feature page for F18 for it, and then get it through Fedora 18 feature process. With FESCO accepting the feature you should have all you need to get your work accepted by the various stakeholders. I did write a feature page for it. Thats how these threads started. It is in Category:FeaturePageIncomplete , which means FESCo will not see it on its agenda. Please see http://fedoraproject.org/wiki/Features/Policy/Process . Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Sat, 2012-05-26 at 16:12 +0200, Jan Kratochvil wrote: On Sat, 26 May 2012 12:50:31 +0200, Richard W.M. Jones wrote: - You're in IRC or email, and all the bug reporter has given you is a random copy and paste from their terminal. They don't care to open a bug; they don't much care about anything except getting a fix. Minidebuginfo would help here. This is a bug of ABRT again. The crash should have been already reported without asking anything. (Not going to discuss some security claims.) You could just look up BZ(*) with the person's bugreport at that moment. (*) Maybe such mass bugreports should not end up in BZ as the first place. And how are you going to stop the user _then_ creating a manual report with the random copy/pasted message _after_ the automatic report has reported it without them noticing? How is the maintainer to know it's a dupe without minidebuginfo? Your answer appears to be insufficient. :) -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24.05.12 09:28, Alexander Larsson (al...@redhat.com) wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? The right way is probably to write a feature page for F18 for it, and then get it through Fedora 18 feature process. With FESCO accepting the feature you should have all you need to get your work accepted by the various stakeholders. Lennart -- Lennart Poettering - Red Hat, Inc. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Sat, 2012-05-26 at 11:50 +0100, Richard W.M. Jones wrote: On Thu, May 24, 2012 at 04:43:12PM +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 16:35:57 +0200, Frank Ch. Eigler wrote: jan.kratochvil wrote: If your feature does not solve any problem it is just a bloat. This overstates the case. Alex's proposal clearly solves some problems. This is just about wording. My reaction was to: I don't think there has to be a specific problem. ... but then he goes on to list 4 or 5 different features, which are all nice to haves at a very small cost. I'll add one more case, which seems to happen to me all the time: - You're in IRC or email, and all the bug reporter has given you is a random copy and paste from their terminal. They don't care to open a bug; they don't much care about anything except getting a fix. Minidebuginfo would help here. Yeah, in general the goal of MiniDebugInfo is to raise the *minimum* quality of all backtraces. If all the stars align we should be able to have a system that gives perfect backtraces, using server magic in ABRT and other things, but in all the cases where this for some reason doesn't happen we should be able to *always* at least get full function names in the backtrace instead of ??? (unless the stack is totally corrupted of course). -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, May 24, 2012 at 04:43:12PM +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 16:35:57 +0200, Frank Ch. Eigler wrote: jan.kratochvil wrote: If your feature does not solve any problem it is just a bloat. This overstates the case. Alex's proposal clearly solves some problems. This is just about wording. My reaction was to: I don't think there has to be a specific problem. ... but then he goes on to list 4 or 5 different features, which are all nice to haves at a very small cost. I'll add one more case, which seems to happen to me all the time: - You're in IRC or email, and all the bug reporter has given you is a random copy and paste from their terminal. They don't care to open a bug; they don't much care about anything except getting a fix. Minidebuginfo would help here. Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://et.redhat.com/~rjones/virt-top -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Sat, 26 May 2012 12:50:31 +0200, Richard W.M. Jones wrote: - You're in IRC or email, and all the bug reporter has given you is a random copy and paste from their terminal. They don't care to open a bug; they don't much care about anything except getting a fix. Minidebuginfo would help here. This is a bug of ABRT again. The crash should have been already reported without asking anything. (Not going to discuss some security claims.) You could just look up BZ(*) with the person's bugreport at that moment. (*) Maybe such mass bugreports should not end up in BZ as the first place. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Sat, May 26, 2012 at 12:50 PM, Richard W.M. Jones rjo...@redhat.com wrote: I'll add one more case, which seems to happen to me all the time: - You're in IRC or email, and all the bug reporter has given you is a random copy and paste from their terminal. They don't care to open a bug; they don't much care about anything except getting a fix. Minidebuginfo would help here. It seems you (and others in the thread) are hoping for an automatic backtrace on any crash - I don't think that's in the currently discussed scope. Even with this feature you'll have to walk them through 1) enabling core file creation, or finding the abrt-maintained core, and 2) running gdb or abrt on the core. I'm not sure whether minidebuginfo would make the situation actually simpler... Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 22:24 +0200, Jan Kratochvil wrote: Wrt upstreaming the patch to FSF GDB first it can be posted but I would keep it for a release or two only downstream, it is simple enough patch, there may be found some issues with its practical use (if any) etc. I agree with Jan. The patch is specifically meant to target a new form of debuginfo that would (initially) only exist in Fedora. I don't think it make sense to upstream that before its had some time to bake. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 13:20 -0400, Casey Dahlin wrote: On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? Just go do it. See who actually shows up to stop you. To actually get the debuginfo in the builds all I need is a minor patch to the find-debuginfo.sh script in rpm-build. However, since the effect of it is global it seems to me that its a wider decision than just the maintainer of rpm-build. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On 05/25/2012 09:26 AM, Alexander Larsson wrote: On Thu, 2012-05-24 at 13:20 -0400, Casey Dahlin wrote: On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? Just go do it. See who actually shows up to stop you. To actually get the debuginfo in the builds all I need is a minor patch to the find-debuginfo.sh script in rpm-build. However, since the effect of it is global it seems to me that its a wider decision than just the maintainer of rpm-build. As long as the behavior is optional and configurable, just send the patch to rpm-maint and we'll see. Fedora isn't the only consumer of rpm, and somebody else might find it useful regardless of whether Fedora wants to enable the thing or not. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
How to proceed with MiniDebugInfo
I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 09:28:16 +0200, Alexander Larsson wrote: However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? It is difficult to agree on something when you still have not accepted why some people disagree with it. I do not mind the size, as for example we lose already 5-10% by not using gold (unused + duplicate template methods). I mind that in all aspects better solution is ABRT and we should put more effort to it and not to some temporary poor solutions. (This is very generalized to avoid the discussion again.) Thanks, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 09:35 +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 09:28:16 +0200, Alexander Larsson wrote: However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? It is difficult to agree on something when you still have not accepted why some people disagree with it. I do not mind the size, as for example we lose already 5-10% by not using gold (unused + duplicate template methods). I mind that in all aspects better solution is ABRT and we should put more effort to it and not to some temporary poor solutions. (This is very generalized to avoid the discussion again.) And its difficult for me to understand how do you continue to claim in all aspects better when comparing the two, An offline solution that always produces at least something usable to a online one that requires all-star alignment of circumstances to produce the perfect backtrace result. There is no basis for one-or-the-other comparison. IMHO its is a good thing for lightweight, kernel-like userspace backtraces to become widely desseminated across the webs. Regards Yanko -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On 05/24/2012 10:28 AM, Alexander Larsson wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? Pass the feature to FESCo? For anything remotely controversial (and judging by past discussion on the subject, this counts as one), you're unlikely to get what amounts to an actual decision on fedora-devel... - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote: On Thu, 2012-05-24 at 09:35 +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 09:28:16 +0200, Alexander Larsson wrote: However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? It is difficult to agree on something when you still have not accepted why some people disagree with it. I do not mind the size, as for example we lose already 5-10% by not using gold (unused + duplicate template methods). I mind that in all aspects better solution is ABRT and we should put more effort to it and not to some temporary poor solutions. (This is very generalized to avoid the discussion again.) And its difficult for me to understand how do you continue to claim in all aspects better when comparing the two, An offline solution that always produces at least something usable to a online one that requires all-star alignment of circumstances to produce the perfect backtrace result. There is no basis for one-or-the-other comparison. IMHO its is a good thing for lightweight, kernel-like userspace backtraces to become widely desseminated across the webs. I obviously agree with this, and disagree with Jan, but I'd like to avoid just repeating the previous discussion. The disagreement seems to be about two things: 1) Any binary size increase is bad (as it affects cd sizes, etc) 2) The results of the MiniDebugInfo is not perfect, and there is a theoretically perfect approach. So we should not spend time/energy/space/bits/whatever on the non-perfect appraoch. However, the perfect approach has other disadvantages due to being online/centralized, so I and others think its worth having both. The increased space is clearly a project global wide question that probably has to be decided by Fesco. The duplication of effort less so IMHO, as different people are doing the work. If we don't do minidebug I will not be spending any resources on the ABRT server anyway. So, not doing minidebug will not affect ABRT positively, and doing it will not affect it negatively (in fact, it might have a slight positive effect as it can use the low quality info when offline). But still, as this is mainly a resource/project management disagreement it might make sense to have Fesco look at it too. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On 05/24/2012 11:07 AM, Alexander Larsson wrote: On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote: On Thu, 2012-05-24 at 09:35 +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 09:28:16 +0200, Alexander Larsson wrote: However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? It is difficult to agree on something when you still have not accepted why some people disagree with it. I do not mind the size, as for example we lose already 5-10% by not using gold (unused + duplicate template methods). I mind that in all aspects better solution is ABRT and we should put more effort to it and not to some temporary poor solutions. (This is very generalized to avoid the discussion again.) And its difficult for me to understand how do you continue to claim in all aspects better when comparing the two, An offline solution that always produces at least something usable to a online one that requires all-star alignment of circumstances to produce the perfect backtrace result. There is no basis for one-or-the-other comparison. IMHO its is a good thing for lightweight, kernel-like userspace backtraces to become widely desseminated across the webs. I obviously agree with this, and disagree with Jan, but I'd like to avoid just repeating the previous discussion. The disagreement seems to be about two things: 1) Any binary size increase is bad (as it affects cd sizes, etc) 2) The results of the MiniDebugInfo is not perfect, and there is a theoretically perfect approach. So we should not spend time/energy/space/bits/whatever on the non-perfect appraoch. However, the perfect approach has other disadvantages due to being online/centralized, so I and others think its worth having both. The increased space is clearly a project global wide question that probably has to be decided by Fesco. The duplication of effort less so IMHO, as different people are doing the work. If we don't do minidebug I will not be spending any resources on the ABRT server anyway. So, not doing minidebug will not affect ABRT positively, and doing it will not affect it negatively (in fact, it might have a slight positive effect as it can use the low quality info when offline). But still, as this is mainly a resource/project management disagreement it might make sense to have Fesco look at it too. In fact it will affect ABRT positively - the calltrace with function names is a pretty good for duplicate checking, so ABRT will be able to find the dupes in already filled bugzilla tickets without requiring the full debuginfo. Jirka -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 11:17 +0200, Jiri Moskovcak wrote: On 05/24/2012 11:07 AM, Alexander Larsson wrote: On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote: The duplication of effort less so IMHO, as different people are doing the work. If we don't do minidebug I will not be spending any resources on the ABRT server anyway. So, not doing minidebug will not affect ABRT positively, and doing it will not affect it negatively (in fact, it might have a slight positive effect as it can use the low quality info when offline). But still, as this is mainly a resource/project management disagreement it might make sense to have Fesco look at it too. In fact it will affect ABRT positively - the calltrace with function names is a pretty good for duplicate checking, so ABRT will be able to find the dupes in already filled bugzilla tickets without requiring the full debuginfo. Well, theoretically it could do that by retracing the backtrace on the server. It seems much simpler and more efficient to just do that locally though, but this is partly where the disagreement is. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On 05/24/2012 11:24 AM, Alexander Larsson wrote: On Thu, 2012-05-24 at 11:17 +0200, Jiri Moskovcak wrote: On 05/24/2012 11:07 AM, Alexander Larsson wrote: On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote: The duplication of effort less so IMHO, as different people are doing the work. If we don't do minidebug I will not be spending any resources on the ABRT server anyway. So, not doing minidebug will not affect ABRT positively, and doing it will not affect it negatively (in fact, it might have a slight positive effect as it can use the low quality info when offline). But still, as this is mainly a resource/project management disagreement it might make sense to have Fesco look at it too. In fact it will affect ABRT positively - the calltrace with function names is a pretty good for duplicate checking, so ABRT will be able to find the dupes in already filled bugzilla tickets without requiring the full debuginfo. Well, theoretically it could do that by retracing the backtrace on the server. It seems much simpler and more efficient to just do that locally though, but this is partly where the disagreement is. I know we could retrace the calltrace on a server to get the function names, but for the use case we want to use it it's critical to have it fast. And having it stored locally and available at the moment of crash is imho faster than retracing it on a server. I actually don't see a reason (except the space, but the numbers doesn't look so horrible..) why not to have it. No one says it's going to be used in tickets created by ABRT in bugzilla and no one says we're going to force maintainers to work only with these low quality backtraces. ABRT can still require the full backtrace to create a bz ticket and I can easily imagine a situation where knowing just the function name helps me to find a problem. So what's the worry about? Jirka -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 11:07:06 +0200, Alexander Larsson wrote: 2) The results of the MiniDebugInfo is not perfect, and there is a theoretically perfect approach. So we should not spend time/energy/space/bits/whatever on the non-perfect appraoch. However, the perfect approach has other disadvantages due to being online/centralized, so I and others think ^^^ / is not + here, other solutions require online connectivity but they do not have to be centralized. its worth having both. There are many ways how to solve this problem, unfortunately nobody knows what is your problem, there are too many close but still different problems in this basket. You have delivered solution without stating the problem first. (1) There are two kinds of developers: (a) PC (instruction) of the crash only (b) full context with parameters, variables and wanting even more. (2) There are also two different kinds of developers: (a) dependency on network services (like ABRT Retrace Server) is OK (b) everything must work with full feature set even offline. (3) And yet more two different kinds of developers: (a) we must not upload anything for security reasons (b) we can freely upload anything because Fedora already has no security. - we already freely download+execute Mozilla binary plugins not reviewed + signed by Fedora Project, 99%(?) people install Flash anyway etc. (4) And yet more two different kinds of developers: (a) desktop ones who have thousands of BZs and ignore any ABRT BZ anyway They are more interested in stats in which parts it crashes the most. (b) at least Tools ones who check each BZ and have more problem that some crashes repeat but they are not reproducible and even rich backtraces in many cases do not contain enough info and it would be best to extend the backtraces/bugreports even more. There are various other solutions still keeping high quality backtraces such as: (x) Already stated fast core files upload to ABRT Retrace Server via gdbserver. (y) With F-18 shorted .debug files http://fedoraproject.org/wiki/Features/DwarfCompressor downloading specific .debug.xz files can be much faster than whole *-debuginfo.rpm. (z) Symbol server for GDB - no need to download full debuginfos, GDB fetches only parts on demand. Maybe other solutions but it depends which of the 16 combinations of the (1), (2), (3) and (4) use cases above you choose. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 14:46 +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 11:07:06 +0200, Alexander Larsson wrote: There are many ways how to solve this problem, unfortunately nobody knows what is your problem, there are too many close but still different problems in this basket. You have delivered solution without stating the problem first. I don't think there has to be a specific problem. In fact, I think Fedora shouldn't really care what *my* problem is. What is interesting is: I have this feature; It has a certain cost (increase in size) and it gives certain features. Is the price worth the features it gives? Now, I don't want to repeat everything said before about what minidebuginfo can do, but I'll give some short examples of things that are nice to have and hard to do well without local debuginfo. * Write backtraces to syslog on coredumps * Allow ABRT to do better duplication matching (the ABRT developers even want minidebuginfo!) * Always get some minimal level of backtrace quality, even for rpms built locally or from other repositories which are not availible on the retrace servers. (Assuming they are built on a F18 or later which has this feature.) * Do system wide profiling and tracing without having to install a lot of debuginfo. * Help developers by always having at least some level of debuginfo, even for e.g. uncommon dependencies that you don't typically have debuginfo for, or when you don't have a network connection to get debuginfo packages. So, does these advantages outweigh the cost? -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 15:34:28 +0200, Alexander Larsson wrote: I don't think there has to be a specific problem. In fact, I think Fedora shouldn't really care what *my* problem is. What is interesting is: I have this feature; It has a certain cost (increase in size) and it gives certain features. Is the price worth the features it gives? If your feature does not solve any problem it is just a bloat. * Write backtraces to syslog on coredumps backtrace is overloaded here. Minidebuginfo provides only bare unwinds. * Allow ABRT to do better duplication matching (the ABRT developers even want minidebuginfo!) do better is too ambiguous and probably not right. Duplication matching can be always done server-side. Minidebuginfo may give less load for ABRT servers for example, this does not match the do better phrase. * Always get some minimal level of backtrace quality, even for rpms built locally or from other repositories which are not availible on the retrace servers. (Assuming they are built on a F18 or later which has this feature.) I do not limit possible solutions only to retrace servers. Cores can be backtraced even locally with full quality by (y) or (z) from my last mail. * Do system wide profiling and tracing without having to install a lot of debuginfo. But a poor quality again, there won't be line-specific data for example. * Help developers by always having at least some level of debuginfo, even for e.g. uncommon dependencies that you don't typically have debuginfo for, debuginfo-install does everything on its own, user does not have to care. or when you don't have a network connection to get debuginfo packages. This is the only valid point and pre-requisite of all your claims. But I do not find a machine without network connectivity to be useful for anything. So, does these advantages outweigh the cost? Sure in no way. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, May 24, 2012 at 04:19:15PM +0200, Jan Kratochvil wrote: do better is too ambiguous and probably not right. Duplication matching can be always done server-side. Minidebuginfo may give less load for ABRT servers for example, this does not match the do better phrase. And the symbols for the duplication matching aren't as important, it is enough if the reporting client transforms addresses in the unwind address list into build-id + offset form, then it can be matched on the server side too. Only if you want to fuzzy match bugreports from different NVRs, then offsets are not useful and symbol names only sometimes useful. But the build-id + offset form can be transformed into more detailed forms on the server easily. Jakub -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
jan.kratochvil wrote: If your feature does not solve any problem it is just a bloat. This overstates the case. Alex's proposal clearly solves some problems. - FChE -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 16:35:57 +0200, Frank Ch. Eigler wrote: jan.kratochvil wrote: If your feature does not solve any problem it is just a bloat. This overstates the case. Alex's proposal clearly solves some problems. This is just about wording. My reaction was to: I don't think there has to be a specific problem. Alexander talks about Minidebuginfo as about a feature but I find it to be a bugfix category. Specifically to fix what ABRT should have delivered. The primary goal are crash backtraces, tracing is not the Minidebuginfo goal. Tracing is only misused as a way how to push Minidebuginfo through. And bugfix without a problem is just that bloat. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
IMHO administrators would benefit much more from the minidebuginfo feature than developers. The advantage for admins is that for every crash the computer would also give a name of the crash. So it's no longer just httpd: Core dumped., but you get a unique sequence of functions (a name) and you can talk about this particular crash with other admins (it's no longer just our webserver is randomly crashing), and search the Internet for other victims. It would be very cool if the bottom of the stack (last few functions) is printed to the terminal together with the usual Core dumped. message. For developers, it is unappealing to attempt fixing a bug just from an ordered list of function names. Karel Alexander Larsson wrote: Now, I don't want to repeat everything said before about what minidebuginfo can do, but I'll give some short examples of things that are nice to have and hard to do well without local debuginfo. * Write backtraces to syslog on coredumps * Allow ABRT to do better duplication matching (the ABRT developers even want minidebuginfo!) * Always get some minimal level of backtrace quality, even for rpms built locally or from other repositories which are not availible on the retrace servers. (Assuming they are built on a F18 or later which has this feature.) * Do system wide profiling and tracing without having to install a lot of debuginfo. * Help developers by always having at least some level of debuginfo, even for e.g. uncommon dependencies that you don't typically have debuginfo for, or when you don't have a network connection to get debuginfo packages. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 2012-05-24 at 11:06 -0400, Karel Klic wrote: For developers, it is unappealing to attempt fixing a bug just from an ordered list of function names. Sometimes. Other times, it's all I need if I have other data at hand (for example, the git log for the code that's crashing). -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On 05/24/2012 07:09 PM, Colin Walters wrote: On Thu, 2012-05-24 at 11:06 -0400, Karel Klic wrote: For developers, it is unappealing to attempt fixing a bug just from an ordered list of function names. Sometimes. Other times, it's all I need if I have other data at hand (for example, the git log for the code that's crashing). Nod. There are always going to be cases where you wont be able to get the perfect backtrace, and a backtrace with nothing but function names is helluva lot more useful than it segfaulted. - Panu - -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote: I'm at a loss to how to proceed with the MiniDebugInfo work. I have patches to rpmbuild that creates the compressed minidebuginfo putting them in the main binaries, and I have patches to gdb that reads the compressed debuginfo on demand. However, the whole thing is useless unless we agree that we want to enable this by default. It seems some people like the idea, whereas others disagree that its worth the increased binary size. It doesn't look like either side is gonna be able to convince the other side, so how do we get to a decision here? Just go do it. See who actually shows up to stop you. --CJD -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 19:20:15 +0200, Casey Dahlin wrote: Just go do it. See who actually shows up to stop you. I am sure this is significant enough distro change to require FESCo decision. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, May 24, 2012 at 07:27:03PM +0200, Jan Kratochvil wrote: On Thu, 24 May 2012 19:20:15 +0200, Casey Dahlin wrote: Just go do it. See who actually shows up to stop you. I am sure this is significant enough distro change to require FESCo decision. Still cuts his workload down from convincing an open mailing list to convincing 7 people. --CJD -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, May 24, 2012 at 3:34 PM, Alexander Larsson al...@redhat.com wrote: I don't think there has to be a specific problem. In fact, I think Fedora shouldn't really care what *my* problem is. What is interesting is: I have this feature; It has a certain cost (increase in size) and it gives certain features. Is the price worth the features it gives? It's only this simple if you are adding a new package. Otherwise, there are two more questions: - Is Fedora the right place for the feature? - You contribute the initial code; do the people who would end up maintaining it (if it is someone else than you) accept that code and agree to maintain it? For minidebuginfo, it is a Fedora decision whether to include minidebuginfo in built RPMs - but the inclusion of gdb support is, ideally, up to gdb upstream, or alternatively, up to gdb's package maintainers (in this case, Jan and Sergio). It's not infrequent that a feature happens in Fedora first and is integrated upstream later - but it's not quite the preferred path. Fairly frequently FESCo or FPC agrees to a feature and makes it effectively required for all packages, even if some package maintainers disagree (ideally after gathering input from interested package maintainers first) - but that should IMHO be the case primarily for system-wide features where individual agreement with all affected parties is infeasible. So, where to go from here? For the gdb change, I think the ideal case would be to push the gdb support upstream (I have no idea what upstream thinks, though), second best is to convince Jan and Sergio. Only a third best is asking FESCo to overrule the gdb package maintainers. OTOH, FESCo will probably need to vote on the RPM packaging change in any case, so it would be possible to start with this as well - with the caveat that opposition from gdb package maintainers is an additional risk for the feature and its acceptance. My personal opinion is that this feature is net positive, but not compelling enough to overrule the gdb maintainers and ask them to maintain a Fedora-specific patch they don't like; of course, others on FESCo may, and some probably do, disagree. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: How to proceed with MiniDebugInfo
On Thu, 24 May 2012 21:53:48 +0200, Miloslav Trmač wrote: So, where to go from here? For the gdb change, I think the ideal case would be to push the gdb support upstream (I have no idea what upstream thinks, though), second best is to convince Jan and Sergio. I have no problems accepting the patch to Fedora GDB (after some adjustments). I just expect FESCo decision considering its pros and cons distro-wide. I am trying to provide FESCo enough information for the feature as Alexander's presentation of it seems biased to me. Wrt upstreaming the patch to FSF GDB first it can be posted but I would keep it for a release or two only downstream, it is simple enough patch, there may be found some issues with its practical use (if any) etc. Regards, Jan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
