from:"Alexander Larsson"

On Wed, 2016-06-15 at 21:38 +0200, Mark Wielaard wrote:

> O! I see in builder/builder-utils.c "This code is based on
> debugedit.c
> from rpm". And I am just hacking on that for rpm (see some patches on
> rpm-ma...@rpm.org). 

Its just part of the debugedit stuff though, because we don't have to
rewrite the paths.

> Maybe it is an idea to extract that code and provide
> a "standalone" debugedit program that different packaging programs
> (rpmbuild, flatpak-builder, ...) could use to collect build-ids and
> debuginfo source path cleanups? 

Seems like a great idea.

> What would be a good list to discuss
> that? rpm-ecosystem, flatpak-devel?

I dunno. The flatpak list is at:
 https://lists.freedesktop.org/mailman/listinfo/xdg-app
and I'm on that, but it doesn't strike me as an ideal place.
I'm not on rpm-ecosystem though.
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Fedora development of Snap packages

On Wed, 2016-06-15 at 16:32 +0100, James Hogarth wrote:
> 
> 
> > Snappy fundamentally relies on apparmour to do confinement (i.e. it
> > doesn't use filesystem namespaces like flatpak), how does this work
> on
> > fedora? You can't use selinux and apparmour at the same time, so
> this
> > shouldn't be able to work, unless they disable the containment
> feature.
> >
> That's precisely what they are doing on non-Ubuntu distributions,
> disabling confinement.

Thats is pretty crappy. That means things will keep accidentally being
packaged that depends on things not in the ubuntu core. It also means
that there is zero sandboxing.
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Fedora development of Snap packages

On Wed, 2016-06-15 at 08:24 +0200, Florian Weimer wrote:
> On 06/15/2016 06:27 AM, Andrew Lutomirski wrote:
> > On Tue, Jun 14, 2016 at 9:07 PM, Florian Weimer  > > wrote:
> > > On 06/15/2016 04:11 AM, Andrew Lutomirski wrote:
> > > 
> > > > I *strongly* disagree here.  The xdg-app folks seem to be doing
> > > > a
> > > > pretty good job with their sandbox.  The kernel attack surface
> > > > is
> > > > reduced considerably, as is the attack surface against the user
> > > > via
> > > > ptrace and filesystem access.  If Wayland is available (which
> > > > is
> > > > should be!) then so is the attack surface against X.
> > > 
> > > 
> > > What about the direct access to DRI device nodes?  Why isn't this
> > > a problem
> > > that reduces the effectiveness of the sandbox considerably?
> > 
> > It's certainly a meaningful attack surface.  That being said, if it
> > works well, it should be about as dangerous as Chromium's render
> > sandbox, and the latter seems to work fairly well in practice.  I'm
> > assuming that xdg-app grants access to render nodes, not to the
> > legacy
> > node.
> 
> I'm not sure what kind of sandboxing there is.  I was just able to
> open 
> ~/.ssh/id_rsa from a Flatpak application, and change ~/.bash_profile 
> (both outside the sandbox, obviously).

You can opt-out of parts of the sandbox for flatpak apps, and right now
most apps allow access to the home-directory, because we've not yet
finished the work on portals which will allow mediated access to user
files. That said, simple apps like a game will work fine without
granting them access to the users file.

This means current applications packaged with flatpak is mostly about
distribution, and not so much sandboxing. However, we're working on
fixing this.
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Fedora development of Snap packages

On Tue, 2016-06-14 at 19:26 +0100, James Hogarth wrote:
> So I was rather surprised by this Softpaedia article today:
> http://news.softpedia.com/news/snap-packages-become-the-universal-bin
> ary-format-for-all-gnu-linux-distributions-505241.shtml
> It claims that Canonical state that they have been working with
> Fedora developers to make this the universal packaging format.
> The snapcraft.io site instructions say to use a COPR by a Canonical
> employee who is not a Fedora packager.
> Does anyone in marketing or development now what the article is
> referring to and what's going on?

Snappy fundamentally relies on apparmour to do confinement (i.e. it
doesn't use filesystem namespaces like flatpak), how does this work on
fedora? You can't use selinux and apparmour at the same time, so this
shouldn't be able to work, unless they disable the containment feature.
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: Source, symbols and debuginfo for flatpaks/snaps

On Wed, 2016-06-15 at 12:19 +0200, Mark Wielaard wrote:
> Hi,
> 
> On Tue, 2016-06-14 at 22:02 +0200, Igor Gnatenko wrote:
> > When DNF will be able to install flatpack pkgs then we can stop
> > supporting
> > distro packages for that.
> 
> One of the things I am working on is making access to sources,
> symbols
> and debuginfo easier for rpm packages as distributed by Fedora. That
> helps users profiling, debugging and tracing the things they run on
> their systems. For some background see:
> https://gnu.wildebeest.org/blog/mjw/2016/02/02/where-are-your-symbols
> -debuginfo-and-sources/
> https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo

Flatpak supports something called "extensions", where an app (or a
runtime) can specify extension points which are then optionally there
when running the app. One use of this in flatpak is debuginfo
extensions for the case above (another is locale data). Also, if you
use flatpak-builder to build your app then these are automatically
built for you, similar to how rpm does it (which support for was also
written by me).
--
devel mailing list
devel@lists.fedoraproject.org
https://lists.fedoraproject.org/admin/lists/devel@lists.fedoraproject.org

Re: F23 System Wide Change: Glibc locale subpackaging

2015-08-25 Thread Alexander Larsson

On fre, 2015-08-21 at 15:14 -0400, Carlos O'Donell wrote:
 On 06/22/2015 10:59 AM, Alexander Larsson wrote:

  Do they really have to modify locale-archive? Can't each package
  install separate archive files (say, based on the locale name).
  Packaging optional extra files is a lot easier for me in my work
  with
  an xdg-app runtime based on fedora.
  
 For now we are modifying locale-archive, but we are aware the this
 causes
 problems with container overlays, and are looking for a solution
 where
 this works on a per-file basis with each new subpackage langauge
 contributing
 a new loadable optimized binary locale file.
 
 The stepping stone will likely be:
 
 _install_langs (to limit languages)
   - subpackages that modify locale-archive 
   - subpackages that install their own files
 
 Please remember the core runtimes are key part of the OS, and we make
 these transitions slowly and gather metrics about how effective each
 step was and how many bugs we got, and if the intermediate changes
 broke anything.

I understand that, and thanks for doing this work, its pretty
important imho. I just wanted to make sure that you were aware of what
the best end goal was.
Btw, do you have any measures of the performance improvements of using
the locale archive is in general? It seems a like many other distros
chose not to use it.
-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander LarssonRed Hat, Inc 
   al...@redhat.comalexander.lars...@gmail.com 
He's a lounge-singing skateboarding dwarf on his last day in the job. 
She's a green-fingered African-American magician's assistant who can talk 
to animals. They fight crime! 


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: F23 System Wide Change: Glibc locale subpackaging

2015-06-22 Thread Alexander Larsson

On mån, 2015-06-22 at 06:16 -0400, Jan Kurik wrote:
 = Proposed System Wide Change: Glibc locale subpackaging  =
 https://fedoraproject.org/wiki/Changes/Glibc_locale_subpackaging
 
 Change owner(s):
 * Mike Fabian mfabian At redhat DOT com
 * Siddhesh Poyarekar spoyarek AT redhat DOT com
 * Carlos O’Donell codonell AT redhat DOT com
 
 This change should make it possible to install or uninstall locales 
 individually. 
 
 == Detailed Description ==
 Currently the file /usr/lib/locale/locale-archive contains all 
 locales and is thus huge (103MB).
 For small systems (and containers) it would be useful to be able to 
 install only a small number of locales.
 Recently we made it possible to install a small number of locales by 
 supplying the rpm-macro “_install_langs”, for example
 
rpm -i -D _install_langs=en:de_DE glibc-common.rpm
 
 will install all English locales and all German locales which start 
 with “de_DE”,
 
rpm -i -D _install_langs=en_US.utf8 glibc-common.rpm
 
 will install only the en_US.utf8 locale,
 
rpm -i -D _install_langs=POSIX glibc-common.rpm
 
 will install nothing (but the POSIX/C is still available because it 
 is builtin into glibc).
 
 But this approach works only during an Anaconda based install when 
 Anaconda supplies the _install_langs rpm-macro.
 When glibc is updated later, the _install_langs macro will not be 
 supplied on the command line during the update and the default value 
 “all” of “_install_langs” from /usr/lib/rpm/macros will be used and 
 all locales come back during an update.
 Therefore, this solution is far from perfect.
 It should be made possible to install and uninstall locales 
 individually, for example by having a separate package for the 
 locales for each language. Installing such a package would add these 
 locales to locale-archive, uninstalling it would remove them.

Do they really have to modify locale-archive? Can't each package
install separate archive files (say, based on the locale name).
Packaging optional extra files is a lot easier for me in my work with
an xdg-app runtime based on fedora.


-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander LarssonRed Hat, Inc 
   al...@redhat.comalexander.lars...@gmail.com 
He's an immortal small-town gentleman spy on his last day in the job. 
She's a high-kicking belly-dancing journalist from a different time and 
place. They fight crime! 

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: F21 System Wide Change: Default Local DNS Resolver

2014-04-30 Thread Alexander Larsson

On tis, 2014-04-29 at 11:24 -0400, Simo Sorce wrote:
 On Tue, 2014-04-29 at 17:15 +0200, Alexander Larsson wrote:
  On tis, 2014-04-29 at 14:15 +0200, Jaroslav Reznik wrote:
   = Proposed System Wide Change:  Default Local DNS Resolver = 
   https://fedoraproject.org/wiki/Changes/Default_Local_DNS_Resolver
   
   Change owner(s): P J P p...@fedoraproject.org, Pavel Šimerda 
   pav...@pavlix.net,   Tomas Hozza tho...@redhat.com
   
   To install a local DNS resolver trusted for the DNSSEC validation running 
   on 
   127.0.0.1:53. This must be the only name server entry in /etc/resolv.conf.
  
  This is gonna conflict a bit with docker, and other  users of network
  namespaces, like systemd-nspawn. When docker runs, it picks up the
  current /etc/resolv.conf and puts it in the container, but the container
  itself runs in a network namespace, so it gets its own loopback device.
  This will mean 127.0.0.1:53 points to the container itself, not the
  host, so dns resolving in the container will not work.
  
  Not sure how to fix something like that though...
 
 Any way we can redirect the connection to the host ?
 
 On the host we cannot listen on 0.0.0.0 so we cannot make unbound
 available through normal routing on a different interface.
 
 However we can perhaps make it listen on a special virtual interface
 dedicated to let containers talk to other processes on the host maybe ?
 (could even be other privileged containers). There is a question of what
 addresses to use though ...

I don't see any nice way to make this just work in docker (i.e.
without changes to docker). Docker as well as the host sets up
127.0.0.1/8 for the loopback device, so any 127.0.0.* will get routed to
the local loopback. 

The only ways to have a ip available to both the host and the container
are to either have a ip not in the 127.0.0.x range (thus this will be
forwarded to the default gw, i.e. the host) or to set up some kind of
forwarding of a port in 127.0.0.x (i.e. use iptables). The later needs
to be done by docker, as its what sets up the network interfaces for the
container.

So, without changes to docker the option seems to be to set up another
local interface with address range different than 127.0.0.1 and have the
dns server listen to that.



-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: F21 System Wide Change: Default Local DNS Resolver

2014-04-29 Thread Alexander Larsson

On tis, 2014-04-29 at 14:15 +0200, Jaroslav Reznik wrote:
 = Proposed System Wide Change:  Default Local DNS Resolver = 
 https://fedoraproject.org/wiki/Changes/Default_Local_DNS_Resolver
 
 Change owner(s): P J P p...@fedoraproject.org, Pavel Šimerda 
 pav...@pavlix.net,   Tomas Hozza tho...@redhat.com
 
 To install a local DNS resolver trusted for the DNSSEC validation running on 
 127.0.0.1:53. This must be the only name server entry in /etc/resolv.conf.

This is gonna conflict a bit with docker, and other  users of network
namespaces, like systemd-nspawn. When docker runs, it picks up the
current /etc/resolv.conf and puts it in the container, but the container
itself runs in a network namespace, so it gets its own loopback device.
This will mean 127.0.0.1:53 points to the container itself, not the
host, so dns resolving in the container will not work.

Not sure how to fix something like that though...




-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: We want to stop systemd from being added to docker images, because of rpm requiring systemctl.

2014-04-29 Thread Alexander Larsson

On tis, 2014-04-29 at 11:21 -0400, Josh Boyer wrote:
 On Tue, Apr 29, 2014 at 10:58 AM, Alexander Larsson al...@redhat.com wrote:
  On tis, 2014-04-29 at 12:33 +0200, Lennart Poettering wrote:
  On Mon, 28.04.14 17:01, Daniel J Walsh (dwa...@redhat.com) wrote:
 
   The problem  is lots of services require systemd because they ship a
   unit file and want systemctl reload to happen.  Systemd then triggers a
   require for udev and kmod, which docker containers do not need.
 
  If you discount the docs/man pages of the RPMs, how much does kmod,
  udev, systemd actually contribtue in bytes to your docker images?
 
  Its around 15 megs or so, although on rhel7 its 20 megs larger because
  of a dependency that kmod has on /usr/bin/nm (binutils) that doesn't
  seem to be there on fedora kmod. This seems like a bug in fedora though,
  as kmod ships /usr/sbin/weak-modules which calls nm, so once fixed
  fedora would be at 35 meg too.
 
 It's a bug one way or another.  Lacking the dep on nm would break
 weak-modules as you suggest, but afaik Fedora doesn't ship anything
 that actually uses or leverages weak-modules anyway.  So either the
 dep could be added or the script could be dropped.

Another alternative is to use eu-nm, which is part of elfutils, which is
not 20 meg.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: We want to stop systemd from being added to docker images, because of rpm requiring systemctl.

2014-04-29 Thread Alexander Larsson

On tis, 2014-04-29 at 17:40 +0200, Lennart Poettering wrote:
 On Tue, 29.04.14 16:58, Alexander Larsson (al...@redhat.com) wrote:
 
  On tis, 2014-04-29 at 12:33 +0200, Lennart Poettering wrote:
   On Mon, 28.04.14 17:01, Daniel J Walsh (dwa...@redhat.com) wrote:
   
The problem  is lots of services require systemd because they ship a
unit file and want systemctl reload to happen.  Systemd then triggers a
require for udev and kmod, which docker containers do not need.
   
   If you discount the docs/man pages of the RPMs, how much does kmod,
   udev, systemd actually contribtue in bytes to your docker images?
  
  Its around 15 megs or so, although on rhel7 its 20 megs larger because
  of a dependency that kmod has on /usr/bin/nm (binutils) that doesn't
  seem to be there on fedora kmod. This seems like a bug in fedora though,
  as kmod ships /usr/sbin/weak-modules which calls nm, so once fixed
  fedora would be at 35 meg too.
 
 I am pretty sure that the weak-modules thing should just go. It's
 outdated cruft, for some enterprise thing, and inused in Fedora. I'd
 really recommend to just drop it from the Fedora package...
 
  But, even if the size is small that is not the full picture. There are a
  bunch of dependencies like dbus (the daemon), device-mapper, kmod, and
  iptables that are recursively pulled in by systemd that don't really
 
 device-mapper? iptables? That sounds wrong... Any idea how that gets
 pulled in? the dm libs might get pulled in indirectly via libcryptsetup,
 but the other dm tools really shouldn't be. And iptables i really don't
 see how that's pulled in?

systemd = cryptsetup-libs = device-mapper-libs = device-mapper

Don't have time to look up the details atm, but iptable was reached via
initscripts somehow.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: Packages requiring Xorg BackingStore true

2013-09-11 Thread Alexander Larsson

On ons, 2013-09-11 at 12:16 +0200, Sandro Mani wrote:
 Hello,
 
 Some time ago I've packaged Xfoil, Avl and Xrotor (which are popular 
 codes for foil/fluid-dynamics related computations), but I never ended 
 up posting a package review for one reason: the plot window of those 
 programs needs BackingStore true set in xorg.conf, or otherwise the 
 contents of the plot windows will disappear as soon as something else 
 covers the window. Now, this is not a terribly huge blocker, in the 
 sense that the package basically also works without the setting, but 
 still, it is not pretty. Hence I wonder on possible approaches to pursue:

It probably works without backingstore on most modern desktops anyway
since they use compositing managers which doesn't throw away non-visible
window content.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct

Re: minimal install set tuning [was Re: systemd requires HTTP server and serves QR codes]

2012-10-10 Thread Alexander Larsson

On tis, 2012-10-09 at 12:34 -0400, Adam Jackson wrote:
 On 10/9/12 9:18 AM, Lennart Poettering wrote:
 
  From the list of packages this minimal set still installs, that I'd
  really like to see gone:
 
  chkconfig
  gamin
  info
  systemd-sysv
 
 chkconfig seems like it could have the 'alternatives' bit split off. 
 I've not investigated this in detail.
 
 gamin is glib2's fault.  Strictly it's an implementation detail, it's 
 not like the glib headers expose gamin types.  It's a little irritating 
 that you end up with both libfam.so and libgamin-1.so installed, with 
 literally identical APIs.

Honestly, we should be building glib2 with --disable-fam, since glib
will prefer the inotify notification module anyway (it has prio 20 and
fam prio 10).


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

2012-06-11 Thread Alexander Larsson

On Fri, 2012-05-25 at 10:10 +0300, Panu Matilainen wrote:
 On 05/25/2012 09:26 AM, Alexander Larsson wrote:
  On Thu, 2012-05-24 at 13:20 -0400, Casey Dahlin wrote:
  On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote:
  I'm at a loss to how to proceed with the MiniDebugInfo work. I have
  patches to rpmbuild that creates the compressed minidebuginfo putting
  them in the main binaries, and I have patches to gdb that reads the
  compressed debuginfo on demand.
 
  However, the whole thing is useless unless we agree that we want to
  enable this by default. It seems some people like the idea, whereas
  others disagree that its worth the increased binary size. It doesn't
  look like either side is gonna be able to convince the other side, so
  how do we get to a decision here?
 
 
  Just go do it. See who actually shows up to stop you.
 
  To actually get the debuginfo in the builds all I need is a minor patch
  to the find-debuginfo.sh script in rpm-build. However, since the effect
  of it is global it seems to me that its a wider decision than just the
  maintainer of rpm-build.
 
 As long as the behavior is optional and configurable, just send the 
 patch to rpm-maint and we'll see. Fedora isn't the only consumer of rpm, 
 and somebody else might find it useful regardless of whether Fedora 
 wants to enable the thing or not.

Ok, here goes:

* rpm-minidebuginfo.patch 
  Patches find-debuginfo.sh and adds new _include_minidebuginfo option
  defaulting to off
* redhat-rpm-config-minidebug.patch
  Example patch that enables building minidebuginfo by default on Fedora

* 0001-configure.ac-Look-for-and-use-libzma-on-ELF-systems.patch
* 0002-Support-lzma-compressed-.gnu_debugdata-sections.patch
  Gdb patches enabling on-demand lzma decompression of
  the .gnu_debugdata section looking for separate debuginfo there.

With these patches all applied any rpm you build (that doesn't
explicitly disable debuginfo generation) will get xz compresses minimal
debug info that gdb will find automatically (unless the real debuginfo
is installed, then that will be used).

diff -up rpm-4.9.1.3/macros.in.minidebug rpm-4.9.1.3/macros.in
--- rpm-4.9.1.3/macros.in.minidebug	2012-06-11 11:16:21.216952339 +0200
+++ rpm-4.9.1.3/macros.in	2012-06-11 11:16:23.686912455 +0200
@@ -175,7 +175,7 @@
 #	the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_find_debuginfo_opts} %{_builddir}/%{?buildsubdir}\
+   %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} %{_builddir}/%{?buildsubdir}\
 %{nil}
 
 #	Template for debug information sub-package.
@@ -418,6 +418,12 @@ package or when debugging this package.\
 #%_missing_build_ids_terminate_build	1
 
 #
+# Include minimal debug information in build binaries.
+# Requires _enable_debug_packages.
+#
+#%_include_minidebuginfo	1
+
+#
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator	1
 
diff -up rpm-4.9.1.3/scripts/find-debuginfo.sh.minidebug rpm-4.9.1.3/scripts/find-debuginfo.sh
--- rpm-4.9.1.3/scripts/find-debuginfo.sh.minidebug	2012-06-11 11:16:09.698138273 +0200
+++ rpm-4.9.1.3/scripts/find-debuginfo.sh	2012-06-11 11:16:13.399078526 +0200
@@ -2,7 +2,7 @@
 #find-debuginfo.sh - automagically generate debug info and file list
 #for inclusion in an rpm spec file.
 #
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 #	 		   [-o debugfiles.list]
 #			   [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 #			   [builddir]
@@ -29,6 +29,9 @@ strip_g=false
 # with -r arg, pass --reloc-debug-sections to eu-strip.
 strip_r=false
 
+# with -m arg, add minimal debuginfo to binary.
+include_minidebug=false
+
 # Barf on missing build IDs.
 strict=false
 
@@ -43,6 +46,9 @@ while [ $# -gt 0 ]; do
   -g)
 strip_g=true
 ;;
+  -m)
+include_minidebug=true
+;;
   -o)
 if [ -z ${lists[$nout]} -a -z ${ptns[$nout]} ]; then
   out=$2
@@ -105,6 +111,32 @@ strip_to_debug()
   chmod 444 $1 || exit
 }
 
+add_minidebug()
+{
+  local debuginfo=$1
+  local binary=$2
+
+  local dynsyms=`mktemp`
+  local funcsyms=`mktemp`
+  local keep_symbols=`mktemp`
+  local mini_debuginfo=`mktemp`
+
+  # Extract the dynamic symbols from the main binary, there is no need to also have these
+  # in the normal symbol table
+  nm -D $binary --format=posix --defined-only | awk '{ print $1 }' | sort  $dynsyms
+  # Extract all the text (i.e. function) symbols from the debuginfo 
+  nm $debuginfo --format=posix --defined-only | awk '{ if ($2 == T || $2 == t) print $1 }' | sort  $funcsyms
+  # Keep all the function symbols not already in the dynamic symbol table
+  comm -13 $dynsyms $funcsyms  $keep_symbols
+  # Copy the full debuginfo, keeping only a minumal set of symbols

Re: How to proceed with MiniDebugInfo

2012-06-05 Thread Alexander Larsson

On Tue, 2012-05-29 at 21:49 +0200, Lennart Poettering wrote:
 On Thu, 24.05.12 09:28, Alexander Larsson (al...@redhat.com) wrote:
 
  I'm at a loss to how to proceed with the MiniDebugInfo work. I have
  patches to rpmbuild that creates the compressed minidebuginfo putting
  them in the main binaries, and I have patches to gdb that reads the
  compressed debuginfo on demand. 
  
  However, the whole thing is useless unless we agree that we want to
  enable this by default. It seems some people like the idea, whereas
  others disagree that its worth the increased binary size. It doesn't
  look like either side is gonna be able to convince the other side, so
  how do we get to a decision here?
 
 The right way is probably to write a feature page for F18 for it, and
 then get it through Fedora 18 feature process. With FESCO accepting the
 feature you should have all you need to get your work accepted by the
 various stakeholders.

I did write a feature page for it. Thats how these threads started.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

2012-05-28 Thread Alexander Larsson

On Sat, 2012-05-26 at 11:50 +0100, Richard W.M. Jones wrote:
 On Thu, May 24, 2012 at 04:43:12PM +0200, Jan Kratochvil wrote:
  On Thu, 24 May 2012 16:35:57 +0200, Frank Ch. Eigler wrote:
   jan.kratochvil wrote:
If your feature does not solve any problem it is just a bloat.
   
   This overstates the case.  Alex's proposal clearly solves some problems.
  
  This is just about wording.
  
  My reaction was to:
  I don't think there has to be a specific problem.
 
 ... but then he goes on to list 4 or 5 different features,
 which are all nice to haves at a very small cost.
 
 I'll add one more case, which seems to happen to me all the time:
 
 - You're in IRC or email, and all the bug reporter has given you is a
   random copy and paste from their terminal.  They don't care to open
   a bug; they don't much care about anything except getting a fix.
 
 Minidebuginfo would help here.

Yeah, in general the goal of MiniDebugInfo is to raise the *minimum*
quality of all backtraces. If all the stars align we should be able to
have a system that gives perfect backtraces, using server magic in ABRT
and other things, but in all the cases where this for some reason
doesn't happen we should be able to *always* at least get full function
names in the backtrace instead of ??? (unless the stack is totally
corrupted of course).


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

2012-05-25 Thread Alexander Larsson

On Thu, 2012-05-24 at 22:24 +0200, Jan Kratochvil wrote:

 Wrt upstreaming the patch to FSF GDB first it can be posted but I would
 keep it for a release or two only downstream, it is simple enough patch, there
 may be found some issues with its practical use (if any) etc.

I agree with Jan. The patch is specifically meant to target a new form
of debuginfo that would (initially) only exist in Fedora. I don't think
it make sense to upstream that before its had some time to bake.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

2012-05-25 Thread Alexander Larsson

On Thu, 2012-05-24 at 13:20 -0400, Casey Dahlin wrote:
 On Thu, May 24, 2012 at 09:28:16AM +0200, Alexander Larsson wrote:
  I'm at a loss to how to proceed with the MiniDebugInfo work. I have
  patches to rpmbuild that creates the compressed minidebuginfo putting
  them in the main binaries, and I have patches to gdb that reads the
  compressed debuginfo on demand. 
  
  However, the whole thing is useless unless we agree that we want to
  enable this by default. It seems some people like the idea, whereas
  others disagree that its worth the increased binary size. It doesn't
  look like either side is gonna be able to convince the other side, so
  how do we get to a decision here?
  
 
 Just go do it. See who actually shows up to stop you.

To actually get the debuginfo in the builds all I need is a minor patch
to the find-debuginfo.sh script in rpm-build. However, since the effect
of it is global it seems to me that its a wider decision than just the
maintainer of rpm-build.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

How to proceed with MiniDebugInfo

I'm at a loss to how to proceed with the MiniDebugInfo work. I have
patches to rpmbuild that creates the compressed minidebuginfo putting
them in the main binaries, and I have patches to gdb that reads the
compressed debuginfo on demand. 

However, the whole thing is useless unless we agree that we want to
enable this by default. It seems some people like the idea, whereas
others disagree that its worth the increased binary size. It doesn't
look like either side is gonna be able to convince the other side, so
how do we get to a decision here?


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote:
 On Thu, 2012-05-24 at 09:35 +0200, Jan Kratochvil wrote:
  On Thu, 24 May 2012 09:28:16 +0200, Alexander Larsson wrote:
   However, the whole thing is useless unless we agree that we want to
   enable this by default. It seems some people like the idea, whereas
   others disagree that its worth the increased binary size. It doesn't
   look like either side is gonna be able to convince the other side, so
   how do we get to a decision here?
  
  It is difficult to agree on something when you still have not accepted why
  some people disagree with it.
  
  I do not mind the size, as for example we lose already 5-10% by not using 
  gold
  (unused + duplicate template methods).  I mind that in all aspects better
  solution is ABRT and we should put more effort to it and not to some 
  temporary
  poor solutions.  (This is very generalized to avoid the discussion again.)
 
 And its difficult for me to understand how do you continue to claim in
 all aspects better when comparing the two, An offline solution that
 always produces at least something usable to a online one that requires
 all-star alignment of circumstances to produce the perfect backtrace
 result.  There is no basis for one-or-the-other comparison. 
 
 IMHO its is a good thing for lightweight, kernel-like userspace
 backtraces to become widely desseminated across the webs.

I obviously agree with this, and disagree with Jan, but I'd like to
avoid just repeating the previous discussion. The disagreement seems to
be about two things:

1) Any binary size increase is bad (as it affects cd sizes, etc)
2) The results of the MiniDebugInfo is not perfect, and
   there is a theoretically perfect approach. So we should not
   spend time/energy/space/bits/whatever on the non-perfect
   appraoch. 
   However, the perfect approach has other disadvantages
   due to being online/centralized, so I and others think
   its worth having both.

The increased space is clearly a project global wide question that
probably has to be decided by Fesco. 

The duplication of effort less so IMHO, as different people are doing
the work. If we don't do minidebug I will not be spending any resources
on the ABRT server anyway. So, not doing minidebug will not affect ABRT
positively, and doing it will not affect it negatively (in fact, it
might have a slight positive effect as it can use the low quality info
when offline). But still, as this is mainly a resource/project
management disagreement it might make sense to have Fesco look at it
too.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

On Thu, 2012-05-24 at 11:17 +0200, Jiri Moskovcak wrote:
 On 05/24/2012 11:07 AM, Alexander Larsson wrote:
  On Thu, 2012-05-24 at 11:22 +0300, Yanko Kaneti wrote:
 
  The duplication of effort less so IMHO, as different people are doing
  the work. If we don't do minidebug I will not be spending any resources
  on the ABRT server anyway. So, not doing minidebug will not affect ABRT
  positively, and doing it will not affect it negatively (in fact, it
  might have a slight positive effect as it can use the low quality info
  when offline). But still, as this is mainly a resource/project
  management disagreement it might make sense to have Fesco look at it
  too.
 
 In fact it will affect ABRT positively - the calltrace with function 
 names is a pretty good for duplicate checking, so ABRT will be able to 
 find the dupes in already filled bugzilla tickets without requiring the 
 full debuginfo.

Well, theoretically it could do that by retracing the backtrace on the
server. It seems much simpler and more efficient to just do that locally
though, but this is partly where the disagreement is.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: How to proceed with MiniDebugInfo

On Thu, 2012-05-24 at 14:46 +0200, Jan Kratochvil wrote:
 On Thu, 24 May 2012 11:07:06 +0200, Alexander Larsson wrote:

 There are many ways how to solve this problem, unfortunately nobody knows what
 is your problem, there are too many close but still different problems in this
 basket.  You have delivered solution without stating the problem first.

I don't think there has to be a specific problem. In fact, I think
Fedora shouldn't really care what *my* problem is. What is interesting
is: I have this feature; It has a certain cost (increase in size) and it
gives certain features. Is the price worth the features it gives?

Now, I don't want to repeat everything said before about what
minidebuginfo can do, but I'll give some short examples of things that
are nice to have and hard to do well without local debuginfo.

* Write backtraces to syslog on coredumps
* Allow ABRT to do better duplication matching (the ABRT developers even
  want minidebuginfo!)
* Always get some minimal level of backtrace quality, even for rpms
  built locally or from other repositories which are not availible
  on the retrace servers. (Assuming they are built on a F18 or later
  which has this feature.)
* Do system wide profiling and tracing without having to install a lot
  of debuginfo.
* Help developers by always having at least some level of debuginfo,
  even for e.g. uncommon dependencies that you don't typically have
  debuginfo for, or when you don't have a network connection to get
  debuginfo packages.

So, does these advantages outweigh the cost?

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: x32 abi support?

2012-05-17 Thread Alexander Larsson

On Thu, 2012-05-17 at 02:28 +0200, Kevin Kofler wrote:
 Nicolas Chauvet wrote:
  So that make me wonder if we really need to built the whole collection
  as x32 ? Or if we only wants a selection of components to be optimized
  by x32. For example does it matter to move the whole Xorg server
  infrastructure as x32 or is it possible to leave it as x86_64 ? or is
  there any benifit for the move to x32 in this area ?
 
 As already explained in this thread by others, if you have a mix of x32 and 
 normal x86_64, you end up with both multilibs in memory, negating the memory 
 savings from the bastard x32 ABI.

I don't think this is always the case. Obviously if you run a lot of
both kinds of apps it matters, but if there are a few, small x86-64
processes (say e.g. the xserver, or a database instance) the doubled mem
use from some shared libraries should not affect things overly much. In
most practical uses of computers (i.e. those that actually *do*
something) most memory is used for the users data, and for that, x32
uses less ram, as all pointers in the ram structures are half the size.

Obviously, it all depends. But I think x32 makes a lot of sense as an
option, especially in the virtual machine space.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: x32 abi support?

2012-05-16 Thread Alexander Larsson

On Wed, 2012-05-16 at 15:30 +0100, Matthew Garrett wrote:
 On Wed, May 16, 2012 at 04:28:31PM +0200, Lennart Poettering wrote:
 
  Mhmm, so I was under the impression that x32 was mostly about increasing
  the scalability of virtualized systems. i.e. run a higher number of
  x32 containers/VM on an x86_64 host. Most server software that is run in
  containers/VMs does not require 64bit address space, and hence using x32
  for them should be quite benificial so that you can run more
  containers/VMs per host. After all this would reduce memory and CPU
  consumption of each, and due to smaller memory usage also result in less
  IO?
 
 I was under the impression that it was to make Android work better on 
 Intel. Scalable VMs are an interesting idea, but for a typical session 
 how much RAM are we talking about?

If you look at some contemporary VM hosting providers, like e.g.:
http://bloggerkhan.com/vps-hosting-vps-servers/185

You'll see that most of them are in the 384-1024 meg of ram range, and
almost none are larger than 4 gigs. I'm sure most of these run 32bit
images, but the hosts are likely 64bit, so it seems to make quite some
sense to use an x32 ABI here.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: x32 abi support?

2012-05-16 Thread Alexander Larsson

On Wed, 2012-05-16 at 21:37 +0200, drago01 wrote:
 On Wed, May 16, 2012 at 9:33 PM, Alexander Larsson al...@redhat.com wrote:
  On Wed, 2012-05-16 at 15:30 +0100, Matthew Garrett wrote:
  On Wed, May 16, 2012 at 04:28:31PM +0200, Lennart Poettering wrote:
 
   Mhmm, so I was under the impression that x32 was mostly about increasing
   the scalability of virtualized systems. i.e. run a higher number of
   x32 containers/VM on an x86_64 host. Most server software that is run in
   containers/VMs does not require 64bit address space, and hence using x32
   for them should be quite benificial so that you can run more
   containers/VMs per host. After all this would reduce memory and CPU
   consumption of each, and due to smaller memory usage also result in less
   IO?
 
  I was under the impression that it was to make Android work better on
  Intel. Scalable VMs are an interesting idea, but for a typical session
  how much RAM are we talking about?
 
  If you look at some contemporary VM hosting providers, like e.g.:
  http://bloggerkhan.com/vps-hosting-vps-servers/185
 
  You'll see that most of them are in the 384-1024 meg of ram range, and
  almost none are larger than 4 gigs. I'm sure most of these run 32bit
  images, but the hosts are likely 64bit, so it seems to make quite some
  sense to use an x32 ABI here.
 
 Can x32 run i686 software (multilib) ?
 Because not being able to run existing software might be a reason for
 many to want such a host.

x32 is not a kernel, so it doesn't run software. The kernel you run is
a standard x86-64 one which can access all your physical memory, then
you have a 32bit userspace mode similar to i386-on-x86-64 but with a
different ABI that lets it use more x86-64 features while only using
32bit pointers (so each process can only use 4 gig of virtual memory).


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Size of official media handouts (was Re: Proposed F18 feature: MiniDebugInfo)

2012-05-14 Thread Alexander Larsson

On Fri, 2012-05-11 at 16:42 +0200, Christoph Wickert wrote:
 Am Mittwoch, den 09.05.2012, 11:57 -0400 schrieb Adam Jackson:
  On Wed, 2012-05-09 at 11:45 -0400, Matthias Clasen wrote:
   On Wed, 2012-05-09 at 13:35 +0200, Kevin Kofler wrote:
Alexander Larsson wrote:
 The feature page lists some of the background and statistics. It also
 lists some options in how to implement this, which all have various
 different pros and cons. I'd like to hear what peoples opinions on 
 these
 are.

There is no room left on the KDE live image for installing any sort of 
debugging information by default.
   
   We could easily drop some of less-than-half-complete translations to
   make room for a bit of minidebuginfo. Last time I looked, translations,
   fonts, etc made up upwards of 25% of the livecd. Or we could just drop
   the obsolescent cdrom size limitation...
  
  I know I've said this before, but: we should break the CD size barrier
  precisely so people can't burn things to CDs.  If you must burn to
  optical media, do yourself a favor and burn a DVD, the reduced seek time
  is entirely worth it.
  
  1G fits on both the smallest MiniDVD format and most extant USB sticks.
  Let's do it already.
 
 As an ambassador and former EMEA media wrangler I tend to agree.
 
 Currently both EMEA and NA only do dual layer DVDs, both for live and
 the installer. EMEA did separate installer vor i386 and x86_64, but
 after NA had no problems with exclusively providing dual layer, we
 decided to do the same.
 
 This being said I don't care how big we grow as long as we can still fit
 all 4 desktops (GNOME, KDE, Xfce, LXDE) in 2 arches each on one multi
 desktop live image. A dual layer DVD has a maximum capacity of 8.5 GB,
 so fitting 8 x 1 GB is not a problem.
 
 We might have to drop Sugar, but if only GNOME and KDE go for 1 GB and
 Xfce and LXDE still target 700 MB or less, we should even be able to
 keep it.

I understand that you want to ship isolated images for each of the
desktops in this combination image, as that is what is tested, etc.
However, there is gonna be a lot of duplicated bits in those images.
Can't we use some form of image where the duplicated blocks can be
shared. Seems like an obvious win to me.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

2012-05-10 Thread Alexander Larsson

On Thu, 2012-05-10 at 10:02 +0200, drago01 wrote:
 On Thu, May 10, 2012 at 9:20 AM, Kevin Kofler kevin.kof...@chello.at wrote:

  I'd rather we just don't add yet another size overhead to every package. Our
  packages keep growing and growing even without that. A few KiB here, a few
  KiB there, in many packages, adding up to a few MiB, and we keep running
  into size issues with our live image at every single release. Size matters!
 
 Not really, you are restricting yourself by the artificial CD size limit.
 You don't have to use the full size of whatever bigger medium you
 choose (DVD, 1 or 2GB stick) but you are currently providing a poorer
 user experience because you insist on a medium from the last century.

I agree, I think bumping the image size to 1GB and use
DVD/mini-dvd/usb-stick is the sane way forward, since we consistently
run into the cd limit and are forced to make changes that negatively
affect the user experience in various ways.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

2012-05-09 Thread Alexander Larsson

On Wed, 2012-05-09 at 13:32 +0200, Jiri Moskovcak wrote:

 Appart from that I see two questions here:
 1. Whether to add the minidebuginfo in Fedora
 2. Whether to use this stripped backtrace when reporting a bug.
 
 
 For 1: The decision to use it or not should be based on some real-life 
 tests like how it impacts the current gnome/kde live cd or other 
 spins. If the additional payload is really small then I don't see a 
 problem here (but I'm glad the decision is not mine ;))

That requires us to rebuild the entire distro to get the minidebuginfo
rpms. Its certainly doable, but some work. I can produce a patch to
rpm-build that does this, but I can't really do the rebuild stuff, that
would need help from someone on the build team.

 For 2: At this point (F18 timeframe) probably not. From ABRT point of 
 view the minidebug is not a problem at all if we can use gdb to generate 
 some backtrace using the mindebuginfo. But what matters are developers 
 who will need to deal with this stripped backtrace and I can guarantee 
 that there will be many unhappy devels. And also the ABRT server 
 projects rely on the coredumps: 
 http://git.fedorahosted.org/git?p=abrt.git;a=blob_plain;f=doc/project/abrt.pdf;hb=HEAD
  
 And once put into life these server side projects will be a great help 
 in bugfixing.

I'm not proposing that we drop the existing backtraces with full debug
info, but (appart from the other places where backtraces are also
useful) I'd like it if ABRT could somehow catch all the cases where
people abort a bugreport because things are scary/slow/bad
network/whatever and at least report the low quality backtrace, which
should be very quick and require little work from the user.

I don't have a full design in mind, but I'm thinking that as soon as the
user acks that he wants to report the bug we would start by just
uploading the low quality backtrace, and *then* start retracing the bug
and show the user the backtrace with full data etc, asking them if its
ok to submit the data. That way we get at least *something* in all
crashes, and perfect reports for users that goes all the way.




-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

2012-05-09 Thread Alexander Larsson

On Wed, 2012-05-09 at 13:35 +0200, Kevin Kofler wrote:
 Alexander Larsson wrote:
  The feature page lists some of the background and statistics. It also
  lists some options in how to implement this, which all have various
  different pros and cons. I'd like to hear what peoples opinions on these
  are.
 
 There is no room left on the KDE live image for installing any sort of 
 debugging information by default.

Its not particularly hard to strip the debuginfo when constructing the
live image, although then installation from it will not really work as
the rpms checksums will be wrong.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Mon, 2012-05-07 at 23:54 +0200, Jan Kratochvil wrote:
On Mon, 07 May 2012 23:36:04 +0200, Lennart Poettering wrote:

But anyway, I don't think it's worth continuing this discussion, this is
a bit like a dialogue between two wet towels...

I also do not think we can ever find an agreement. I only wanted to post here
the opposite side of oppinions on this formal feature request.

I think this is the one thing we can agree on.

With Alex' work you need very very little working, just a small unwinder.

Yes, just an unwinder. Not backtrace for debugging the problem.

This is your opinion. I rarely need the full backtrace in a bug report,
because it you can get one its generally something thats easily
reproduced and I can just run it in gdb myself. When you need it is when
something weird is happening and you have to rely on the bugreport only.
This is sometimes doable even without debug info, I even wrote a blog
post about this:

http://blogs.gnome.org/alexl/2005/08/26/the-art-of-decoding-backtraces-without-debug-info/

But, having the full symbol names for all libraries and apps in all
backtraces I'll ever see in the future would help me immensely. Even if
its just an unwinder.

I am pretty sure I don't want my local developer machine always talk to
the fedora server

Again, as a developer you can affort several GBs of debuginfo.

Not only developers are interested in backtraces, and not only on their
development machine. Administrators are too, and developers are
interested in backtraces from live systems in deployment etc. It just
makes more sense to have solid reliable client side backtraces.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Mon, 2012-05-07 at 18:55 +0100, Peter Robinson wrote:
 On Mon, May 7, 2012 at 2:07 PM, Alexander Larsson al...@redhat.com wrote:
  I just wrote a new Feature proposal for shipping minimal debug info by
  default:
  https://fedoraproject.org/wiki/Features/MiniDebugInfo
 
  The feature page lists some of the background and statistics. It also
  lists some options in how to implement this, which all have various
  different pros and cons. I'd like to hear what peoples opinions on these
  are.
 
  My personal opinion is that we should go with compressed data, in the
  original files without the line number information. This means we use
  minimal space (i.e. an installation increase by only 0.5%) while being
  completely transparent to users. It does however make the normal
  packages larger in a non-optional way which some people disagree with.
 
 What sort of size impact are we talking about here, there's a lot of
 devices that people are starting to use Fedora on such as ARM devices
 that don't have a lot of storage space. One of the most widely
 deployed devices running Fedora for example is the OLPC XO-1 which
 only has 1gb of space so every size increase is a hit and Fedora is
 already starting to have quite a large muffin top to deal with.

See the feature page for detail on the space use. On my F17 desktop
install with and 8 gigabytes /usr it would add 43 megabytes of data.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Mon, 2012-05-07 at 16:24 -0400, Bill Nottingham wrote:
 Alexander Larsson (al...@redhat.com) said: 
  I just wrote a new Feature proposal for shipping minimal debug info by
  default:
  https://fedoraproject.org/wiki/Features/MiniDebugInfo
  
  The feature page lists some of the background and statistics. It also
  lists some options in how to implement this, which all have various
  different pros and cons. I'd like to hear what peoples opinions on these
  are.
  
  My personal opinion is that we should go with compressed data, in the
  original files without the line number information. This means we use
  minimal space (i.e. an installation increase by only 0.5%) while being
  completely transparent to users. It does however make the normal
  packages larger in a non-optional way which some people disagree with.
 
 1) minidebuginfo.rpm is silly. Either it's small enough (and 0.5% is
 certainly that, IMO) that it goes in the main package, or it's too big and
 we should just do regular debuginfo packages.

I completely agree.

 2) It will also make it easier to do things like system wide profiling,
 userspace dynamic probes and causual debugging.
 
 However, the Scope: is only gdb and rpm. Wouldn't said tools also need
 changes? Would this be done in libdwarf, or similar?

I'm not sure what these tools use to unwind, I expect that we'd have to
implement it in libunwind too (added it to the deps) at the very least.
However, anything that already supports separate debug info should be
able to also load this with very little work as it is very similar.

 3) You mention this being done in find-debuginfo.sh, via injection(?). Is
 this possible to be done automatically even for non-rpm-packaged code?

It surely is, the actual change is just a few lines of added shell code.

Basically, when you've separated out the normal separate debug info
you make a copy of it, then run some strip operations on the copy to
remove all but the minimal debug info, then you do:
 xz $debuginfofile
 objcopy --add-section .gnu_debugdata=$debuginfofile.xz $executable

 4) I disagree with the contention that this should all be done via the
 retrace server.

 For this to provide a reasonable amount of information, all you need is:
 - an unwinder
 
 Simpler is usually better.

Agree.


-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Mon, 2012-05-07 at 22:44 +0200, Jan Kratochvil wrote:
 On Mon, 07 May 2012 22:24:15 +0200, Bill Nottingham wrote:
  4) I disagree with the contention that this should all be done via the
  retrace server. For the retrace server to work, you have to have
  all of the following:
  
  - all relevant binaries and DSOs built in Fedora
 
 When we are considering Fedora Bugzilla bugreports then it is valid.
 Custom downloaded binaries will not have this compressed-.symtab anyway.

Any rpm built by anyone with this feature will have this information in
it. Be it a locally rebuild fedora rpm such as a scratch build or a
totally custom rpm. Just like we build debuginfo rpms for such rpms.

  For this to provide a reasonable amount of information, all you need is:
  - an unwinder
 
 The problem is .symtab is not sufficient information for a backtrace.

You keep saying this, but I and several others think that having it is a
sufficient for a great many things.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Tue, 2012-05-08 at 08:30 +0200, Jakub Jelinek wrote:
On Tue, May 08, 2012 at 08:09:04AM +0200, Alexander Larsson wrote:
This is your opinion. I rarely need the full backtrace in a bug report,
because it you can get one its generally something thats easily
reproduced and I can just run it in gdb myself. When you need it is when
something weird is happening and you have to rely on the bugreport only.
This is sometimes doable even without debug info, I even wrote a blog
post about this:

http://blogs.gnome.org/alexl/2005/08/26/the-art-of-decoding-backtraces-without-debug-info/

But, having the full symbol names for all libraries and apps in all
backtraces I'll ever see in the future would help me immensely. Even if
its just an unwinder.

But for that you really don't need the symtabs stored in the binaries/shared
libraries, you can just have the backtrace without symbols printed + print
relevant build-ids at the beginning, a script at any time can reconstruct
that into not just the symbol names, but also lineinfo. And the build-ids
will help even if you want to look at further details (.debug_info, source
files).

Its true that that is all the information you need from the
process/core. But you need to have the rest of the information availible
*somewhere*, such as on a global retrace server or just having it
locally in the minidebuginfo. The later is far more robust and simple.
It lets you directly get a reasonable backtrace given *only* the actual
binaries in the running process.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo

On Tue, 2012-05-08 at 13:08 +0200, Miloslav Trmač wrote:
 My take:
 
 1) Developers of the software in question: Bluntly, the ~1-100 users
 in the whole world shouldn't matter in our discussion - if they are
 even running the RPM, they can and probably will install complete
 debuginfo, enable logging and do other non-default things to make
 their job easier; The Fedora defaults don't matter that much for them,
 and the mini debuginfo is not that useful either.

I generally agree with this. When i'm working on an app I generally have
custom builds of it and its dependencies. However, at some point down
the dependency chain i start relying on distro packages, and it would be
kind of nice to have some info for that when e.g. profiling or
something.

 2) Non-programming end-users.  _This_ is the case that we need to get
 right by default.In many cases, a developer is lucky if the end
 user ever sends any crash report, they often don't respond to
 follow-up questions, and the problem does not have to be reproducible
 at all.  From such users we definitely want as full crash information
 as possible (IOW, including the variable contents information) because
 there won't be a second change to get it.  The mini debuginfo is
 therefore irrelevant, we need to steer users to the retrace server (or
 to attaching full core files to reports, which has much worse privacy
 impact).

I agree that we need to get this right, and that its the most important
problem. However, I don't agree with your reasoning. Its true that it
would be nice to have as much information as possible, and having the
retraced data availible when it works is nice. However, the details with
retracing, having to show the full backtrace letting you ack the
backtrace for privacy issue, the waiting for the retracing to happen,
etc, risks scaring the user away resulting in nothing being reported at
all. 

Take this post for instance:

https://plus.google.com/110933625728671692704/posts/iFXggK7Q8KJ

It show exactly why this is a problem. We try to get more information,
but the result is less information. 

A report based on the minidebuginfo already existing on the system will
give you a basic backtrace that is quite useful, and this should be
reportable with a single, fast operation just sending the data to the
server (as well as logging it to the system logs). The developer can
then do the retrace from that on the server side to get line numbers if
they are needed. We can also have an optional method of reporting that
gives the full stacktrace information, does the retracing over the
network and whatnot, but I don't think its a good idea to do by default.

 BTW, the feature suggests mini debuginfo would be useful for userspace
 tracing - AFAIK such uses, e.g. systemtap, use the variable location
 information very extensively, and would thus not benefit from mini
 debuginfo.

True.

-- 
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Re: Proposed F18 feature: MiniDebugInfo