Re: [Rpm-maint] [PATCH] Support eu-strip --reloc-debug-sections with find-debuginfo -r

[Mark Wielaard]

On Thu, 2011-05-26 at 16:39 +0200, devzero2000 wrote:
 On Wed, May 25, 2011 at 11:36 PM, Mark Wielaard m...@redhat.com wrote:
  The attached patch adds an -r argument flag to find-debuginfo. When that
  flag is given then find-debuginfo will invoke eu-strip with
  --reloc-debug-sections. This is a new option to eu-strip that strips
  out/resolves relocations between .debug_* elf sections in ET_REL files.
  This can save a lot of space for kernel module. For example rebuilding
  kernel-debuginfo-2.6.35.13-91.fc14.x86_64 showed a nice reduction in
  size when using find-debuginfo -r with a new eu-strip from elfutils was
  installed.
  Before: Compressed: 238 M Expanded: 1381 M
  After : Compressed: 207 M Expanded: 908 M
 
 I was noticed, at least on Fedora, that this option leads to a reduction of
 the size of many shared lib or binary of 40% . There would be
 drawbacks to enable it in every package if it is supported by elfutils ?

The option does nothing for shared library (ET_DYN) or executable
(ET_EXEC) files. It is specific to ET_REL files. Which normally are (.o)
objects files created by the compiler that can be relinked by the linker
with other object files (to create shared libraries or executables). It
just happens that kernel modules (.ko files) are also ET_REL files, and
those .ko files also still contain all the relocations for the .debug_*
elf sections. Since they will never be relinked with other object files
(especially after stripping the debug sections away), it just makes
sense to just apply all relocations to the debug sections to saves
space.

So it only makes sense to enable this option in the SPEC file for the
kernel modules. It should be harmless when enabled for other packages.

Cheers,

Mark

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] plugins/systemd_inhibit.c: Define DBUS_TYPE_UNIX_FD if undefined.

[Mark Wielaard]

Older dbus versions (at least 1.2.24) don't define it by default.
---
 plugins/systemd_inhibit.c |4 
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/plugins/systemd_inhibit.c b/plugins/systemd_inhibit.c
index badcc9e..1dd66e6 100644
--- a/plugins/systemd_inhibit.c
+++ b/plugins/systemd_inhibit.c
@@ -3,6 +3,10 @@
 #include rpm/rpmts.h
 #include plugin.h
 
+#ifndef DBUS_TYPE_UNIX_FD
+#define DBUS_TYPE_UNIX_FD ((int) 'h')
+#endif
+
 rpmPluginHook PLUGIN_HOOKS = (
 PLUGINHOOK_INIT |
 PLUGINHOOK_CLEANUP |
-- 
1.7.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] plugins/systemd_inhibit.c: Define DBUS_TYPE_UNIX_FD if undefined.

[Mark Wielaard]

On Tue, Mar 05, 2013 at 07:15:30PM +0200, Panu Matilainen wrote:
 Hum... dbus 1.2.24 would be RHEL-6'ish, right? In which case the
 whole plugin makes no sense at all because it actually needs to have
 a fairly recent systemd running on the system to do anything at all.
 
 Dunno, I can certainly apply that but then again it might make more
 sense to not even try compiling it with dbus that old, as anything
 with that old dbus version is highly unlikely to be running systemd
 in the first place (which however is something that cannot be tested
 for during build)

Yes, RHEL 6.4 to be precise. An alternative would be checking for
dbus 1.3+ which has DBUS_TYPE_UNIX_FD. See attached.

Cheers,

Mark
From 5220c09a6a41e1b7cd9ec721409a7a069f9cc2b6 Mon Sep 17 00:00:00 2001
From: Mark Wielaard m...@redhat.com
Date: Tue, 5 Mar 2013 20:35:19 +0100
Subject: [PATCH] Require dbus = 1.3 for DBUS_TYPE_UNIX_FD usage.

---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 7fd7c5e..ba68d84 100644
--- a/configure.ac
+++ b/configure.ac
@@ -747,7 +747,7 @@ AM_CONDITIONAL(ENABLE_PLUGINS,[test $enable_plugins = 
yes])
 with_dbus=no
 AS_IF([test $enable_plugins != no],[
   PKG_CHECK_MODULES([DBUS],
-[dbus-1 = 1.0],
+[dbus-1 = 1.3],
 [AC_DEFINE(DBUS, 1, [Build with dbus support?]) with_dbus=yes],
 [with_dbus=no])
   AC_SUBST(DBUS_CFLAGS)
-- 
1.8.1.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] Getting make check to work

[Mark Wielaard]

Hi,

I am probably doing something wrong with configuring, but I cannot get
make check to work with the attached patch. And even with that half of
the tests fail because for some reason there is a literal '${prefix}'
string used somewhat that cause the creation of an extra directory
tests/testing\$\{prefix\}/ next to the tests/testing one which contains
just var/lib/rpm/

I also tried with srcdir == builddir and with the latest release
4.10.3.1 instead of a git checkout and on fedora 18. But the issue
is always the same with lots of failures because something seems to
use '${prefix}' literally.

I am configuring with: --with-external-db \
  CPPFLAGS=-I/usr/include/nspr4 -I/usr/include/nss3 -I/usr/include/db4

Does this ring any bell?

Thanks,

Mark
From 8ee4a07b970a834547dff1d95343a9ff86cb4597 Mon Sep 17 00:00:00 2001
From: Mark Wielaard m...@redhat.com
Date: Tue, 5 Mar 2013 21:52:36 +0100
Subject: [PATCH] tests: Make sure testing/usr/bin exists and to copy data from
 srcdir.

When srcdir != builddir the data has to be copied over from the srcdir.
When any of the required progs comes from /usr/bin then there also needs
to be a usr/bin under testing. Just use the same symlink as for testing/bin.
---
 tests/Makefile.am | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 61c18af..6a9f0e1 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -101,10 +101,11 @@ testing$(bindir)/rpmbuild: ../rpmbuild
ln -s ./$(bindir) testing/bin
(cd ${top_builddir}  \
  $(MAKE) DESTDIR=`pwd`/${subdir}/testing install)
-   cp -r data/ testing/
+   cp -r ${srcdir}/data/ testing/
mkdir testing/{dev,etc,magic,tmp}
for node in stdin stderr stdout null; do ln -s /dev/$${node} 
testing/dev/$${node}; done
for cf in hosts resolv.conf passwd shadow group gshadow mtab fstab; do 
[ -f /etc/$${cf} ]  ln -s /etc/$${cf} testing/etc/$${cf}; done
+   ln -s ../$(bindir) testing/usr/bin
for prog in gzip cat patch tar sh ln chmod rm mkdir uname grep sed find 
file mktemp cut sort diff; do p=`which $${prog}`; ln -s $${p} testing/$${p}; 
done
for d in /proc /sys /selinux /etc/selinux; do if [ -d $${d} ]; then ln 
-s $${d} testing/$${d}; fi; done
(cd testing/magic  file -C)
-- 
1.8.1.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] Getting make check to work

[Mark Wielaard]

On Wed, 2013-03-06 at 15:28 +0200, Panu Matilainen wrote:
 ./configure CPPFLAGS=`pkg-config --cflags nss` --with-external-db
 make
 make check
 [...]
 And finally, thanks for complaining about this, really. One can only 
 explain known bugs so many times before it gets so irritating as to 
 mandate a fix, which was the case here :)

Thank you! All 266 tests were successful.

Awesome,

Mark

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Don't use hardcoded paths to tools/scripts in find-debuginfo.sh.

[Mark Wielaard]

This prevents installation and testing in any other location than
/usr/lib/rpm.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 scripts/find-debuginfo.sh | 9 ++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 0fbeb9b..c93b246 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -29,6 +29,9 @@
 # All file names in switches are relative to builddir (. if not given).
 #
 
+# Figure out where we are installed so we can call other helper scripts.
+lib_rpm_dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
+
 # With -g arg, pass it to strip on libraries or executables.
 strip_g=false
 
@@ -235,7 +238,7 @@ while read nlinks inum f; do
   fi
 
   echo "extracting debug info from $f"
-  id=$(/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
+  id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
  -i -l "$SOURCEFILE" "$f") || exit
   if [ $nlinks -gt 1 ]; then
 eval linkedid_$inum=\$id
@@ -245,7 +248,7 @@ while read nlinks inum f; do
 $strict && exit 2
   fi
 
-  [ -x /usr/bin/gdb-add-index ] && /usr/bin/gdb-add-index "$f" > /dev/null 2>&1
+  [ type gdb-add-index >/dev/null 2>&1 && gdb-add-index "$f" > /dev/null 2>&1
 
   # A binary already copied into /usr/lib/debug doesn't get stripped,
   # just has its file names collected and adjusted.
@@ -300,7 +303,7 @@ if $run_dwz && type dwz >/dev/null 2>&1 \
 # dwz invalidates .gnu_debuglink CRC32 in the main files.
 cat "$ELFBINSFILE" |
 (cd "$RPM_BUILD_ROOT"; \
- xargs -d '\n' /usr/lib/rpm/sepdebugcrcfix usr/lib/debug)
+ xargs -d '\n' ${lib_rpm_dir}/sepdebugcrcfix usr/lib/debug)
   fi
 fi
 
-- 
2.5.5

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH 3/4] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC checksum.

[Mark Wielaard]

On Tue, 2016-06-07 at 10:33 +0200, Thierry Vignaud wrote:
> On 6 June 2016 at 23:00, Mark Wielaard <m...@redhat.com> wrote:
> > Some old tools might still use the .gnu_debuglink section to find
> > separate debuginfo files instead of build-id style lookups. When
> > dwz has compresses the .debug files the original CRC in the main
> > ELF file will no longer match. Make sure to run sepdebugcrcfix
> > after dwz to recalculate the CRC.
> >
> > The original fix was created by Jan Kratochvil based on code
> > from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
> > I added a testcase to make sure the CRCs were all correctly
> > updated after dwz has run to compress a debuginfo package.
> 
> WARNING: Note that the original Fedora patch has issues!
> Panu (the then upstream rpm.org maintainer) said it should _not_ be
> upstreamed when we bring the issue up:
> 
> Once we applied this patch to rpm-4.12 in Mageia, rpm silently dropped
> the suid/sgid bits from files if they were re not explicitely listed with 
> %attr
> This was breaking packages...
> See https://bugs.mageia.org/show_bug.cgi?id=14691

Could you provide a small example? From the bug report it isn't clear if
it really is a bug with that particular fix or that it is caused by bad
usage of attributes. If we have a small example we can create a test
case for it.

> Panu answered:
> 
> == BEGIN
> As for the "are you sure we (still) need this patch" question, the
> answer has always been "almost certainly not".
> 
> The short summary is that the issue it fixes only is present if you
> use dwz to compress the debuginfo (like Fedora does), and even then
> case it only affects a handful of toolchain developers in the entire
> world.
> 
> See https://bugzilla.redhat.com/show_bug.cgi?id=971119 for further
> explanation of the issue, comment #9 answers the who is affected part.
> 
> Personally my opinion is "just kick out the stupid patch already" :)
> but judge for yourselves.
> == END

I am missing some context here. But it reads like if you don't use dwz
support then it doesn't fix any real issue. So, then you can drop that
patch also. Which is true.

But you do seem to use compressed debuginfo packages. In which case it
does fix a real issue. The issue is that if you create .debug files
with .gnu_debuglink support then you need to make sure that the CRC is
calculated correctly.  I included a testcase to show what that issue is
(the testcase will fail without the patch applied).

Also note that one of the goals of improving and cleaning up debuginfo
packages is so more people can easily use them, in which case it will
impact more than just a few hackers that manage to get them properly
installed.

You can certainly argue that all tools should use build-id based
identification and lookup and so we should just completely drop .debug
files and .gnu_debuglink entirely. But that seems a rather big change. I
don't know of any distro who did this or anybody who audited all the
tools and processes to make sure .debug files and .gnu_debuglink
(including correct CRCs) are unnecessary in all situations.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 2/4] Add dwz debuginfo compression support.

[Mark Wielaard]

Support for dwz compression has been in Fedora since a couple of years.
https://fedoraproject.org/wiki/Features/DwarfCompressor

The original find-debuginfo.sh patch was written by Jakub Jelinek.
https://bugzilla.redhat.com/show_bug.cgi?id=833311
The new testcase using the macros.debug was added by me.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 macros.debug | 13 
 macros.in|  2 +-
 scripts/find-debuginfo.sh| 53 ++
 tests/data/SPECS/hello2.spec | 62 +++
 tests/rpmbuild.at| 78 
 5 files changed, 207 insertions(+), 1 deletion(-)
 create mode 100644 tests/data/SPECS/hello2.spec

diff --git a/macros.debug b/macros.debug
index bb2c02f..6a8432e 100644
--- a/macros.debug
+++ b/macros.debug
@@ -28,3 +28,16 @@
 
 # Should missing buildids terminate a build?
 %_missing_build_ids_terminate_build1
+
+# Number of debugging information entries (DIEs) above which
+# dwz will stop considering file for multifile optimizations
+# and enter a low memory mode, in which it will optimize
+# in about half the memory needed otherwise.
+%_dwz_low_mem_die_limit 1000
+# Number of DIEs above which dwz will stop processing
+# a file altogether.
+%_dwz_max_die_limit 5000
+
+%_find_debuginfo_dwz_opts --run-dwz\\\
+   --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\\
+   --dwz-max-die-limit %{_dwz_max_die_limit}
diff --git a/macros.in b/macros.in
index 58b122f..8e831c5 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_dwz_opts} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 5c2c381..8de7bad 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -4,6 +4,8 @@
 #
 # Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 # [-o debugfiles.list]
+# [--run-dwz] [--dwz-low-mem-die-limit N]
+# [--dwz-max-die-limit N]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
 #
@@ -20,6 +22,10 @@
 # The -p argument is an grep -E -style regexp matching the a file name,
 # and must not use anchors (^ or $).
 #
+# The --run-dwz flag instructs find-debuginfo.sh to run the dwz utility
+# if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit
+# provide detailed limits.  See dwz(1) -l and -L option for details.
+#
 # All file names in switches are relative to builddir (. if not given).
 #
 
@@ -35,6 +41,11 @@ include_minidebug=false
 # Barf on missing build IDs.
 strict=false
 
+# DWZ parameters.
+run_dwz=false
+dwz_low_mem_die_limit=
+dwz_max_die_limit=
+
 BUILDDIR=.
 out=debugfiles.list
 nout=0
@@ -43,6 +54,17 @@ while [ $# -gt 0 ]; do
   --strict-build-id)
 strict=true
 ;;
+  --run-dwz)
+run_dwz=true
+;;
+  --dwz-low-mem-die-limit)
+dwz_low_mem_die_limit=$2
+shift
+;;
+  --dwz-max-die-limit)
+dwz_max_die_limit=$2
+shift
+;;
   -g)
 strip_g=true
 ;;
@@ -302,6 +324,37 @@ while read nlinks inum f; do
   fi
 done || exit
 
+# Invoke the DWARF Compressor utility.
+if $run_dwz && type dwz >/dev/null 2>&1 \
+   && [ -d "${RPM_BUILD_ROOT}/usr/lib/debug" ]; then
+  dwz_files="`cd "${RPM_BUILD_ROOT}/usr/lib/debug"; find -type f -name 
\*.debug`"
+  if [ -n "${dwz_files}" ]; then
+
dwz_multifile_name="${RPM_PACKAGE_NAME}-${RPM_PACKAGE_VERSION}-${RPM_PACKAGE_RELEASE}.${RPM_ARCH}"
+dwz_multifile_suffix=
+dwz_multifile_idx=0
+while [ -f 
"${RPM_BUILD_ROOT}/usr/lib/debug/.dwz/${dwz_multifile_name}${dwz_multifile_suffix}"
 ]; do
+  let ++dwz_multifile_idx
+  dwz_multifile_suffix=".${dwz_multifile_idx}"
+done
+dwz_multfile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
+dwz_opts="-h -q -r -m .dwz/${dwz_multifile_name}"
+mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
+[ -n "${dwz_low_mem_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -l ${dwz_low_mem_die_limit}"
+[ -n "${dwz_max_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -L ${dwz_max_die_limit}"
+( cd "${RPM_BUILD_ROOT}/usr/lib/debug" && dwz $dw

[Rpm-maint] Some rpm debuginfo improvements

[Mark Wielaard]

Hi,

As discussed on rpm-ecosystem I am working on some debuginfo improvements.
Some improvements have been accepted as new feature for Fedora 25:
https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo
I am keeping track of all ideas here:
http://taiga.fedorainfracloud.org/project/mjw-better-rpm-debuginfo-package-creation/kanban

Here are the first 4 patches that have been sent to this list previously:

 [PATCH 1/4] Add find-debuginfo.sh -m minisymtab support.
 [PATCH 2/4] Add dwz debuginfo compression support.
 [PATCH 3/4] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC
 [PATCH 4/4] Add build-id links to rpm for all ELF files.

All fixes come with new testcases.
Let me know if there is anything missing to get this integrated.

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] RFC: Add build-id links to rpm for all ELF.

[Mark Wielaard]

On Thu, Jun 02, 2016 at 04:35:55PM +0200, Mark Wielaard wrote:
> Yeah, I had to move the symlinking code from find-debuginfo.sh (which
> uses debugedit to get the build-ids) to the main rpm code since there is
> no way for find-debuginfo.sh to add anything to an existing package file
> list. And the whole goal of the patch is to not have all the build-id
> symlinks in the same place in the debuginfo package since they will go
> stale there when the main subpackage and the debuginfo package don't
> match up (which we do want to allow so you can create parallel
> installable debuginfo packages).
> 
> Let me know how you like me to improve the code/logic.
> I just posted a new version of the patch which still needs some extra
> tests and handle duplicate build-ids like the find-debuginfo.sh script
> did.

Here is the final version. It uses rasprintf, hopefully fixes all the
indentation issues, adds duplicate/linked files support and adds lots
of testcases (which found some bugs in the original code).

Since this depends on the various backports from fedora for other
debuginfo improvements for the testcases and find-debuginfo.sh fixups
I'll also post it in context as a full patch series.

Please let me know if there are any outstanding issues with the patch.

Thanks,

Mark
>From f3fa91926924f569424a8c3ad00639a700b663ec Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@redhat.com>
Date: Mon, 6 Jun 2016 22:23:04 +0200
Subject: [PATCH] Add build-id links to rpm for all ELF files.

This patch moves the main ELF file build-id symlinks from the
debuginfo package into the main package. And uses different
base directories for the main ELF file build-id symlink.
For the main build-id use /usr/lib/.build-id and for the debug
build-id use /usr/lib/debug/.build-id.

There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.

To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.

This also includes support for a new config %_build_id_links that
defaults to compat. The other settings are none, alldebug (the old
style) and separate. compat is like separate, but adds a compatibility
link under /usr/lib/debug/.build-id for the main build-id symlink.

There are several new testcases added to test the various settings
using the new keyword "buildid".

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 build/Makefile.am  |   4 +
 build/files.c  | 375 
 configure.ac   |  15 +
 macros.in  |  28 ++
 scripts/find-debuginfo.sh  |  60 
 tests/Makefile.am  |   1 +
 tests/data/SPECS/hello2cp.spec |  64 
 tests/data/SPECS/hello2ln.spec |  63 
 tests/rpmbuildid.at| 761 +
 tests/rpmtests.at  |   1 +
 10 files changed, 1312 insertions(+), 60 deletions(-)
 create mode 100644 tests/data/SPECS/hello2cp.spec
 create mode 100644 tests/data/SPECS/hello2ln.spec
 create mode 100644 tests/rpmbuildid.at

diff --git a/build/Makefile.am b/build/Makefile.am
index 8318cc8..f0f55a9 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -27,3 +27,7 @@ librpmbuild_la_LIBADD = \
@LTLIBICONV@ \
@WITH_POPT_LIB@ \
@WITH_MAGIC_LIB@
+
+if LIBDW
+librpmbuild_la_LIBADD += @WITH_LIBELF_LIB@ @WITH_LIBDW_LIB@
+endif
diff --git a/build/files.c b/build/files.c
index 07bc94e..b3d821f 100644
--- a/build/files.c
+++ b/build/files.c
@@ -14,6 +14,11 @@
 #include 
 #endif
 
+#if HAVE_LIBDW
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -1544,6 +1549,368 @@ exit:
 return rc;
 }
 
+#if HAVE_LIBDW
+/* How build id links are generated.  See macros.in for description.  */
+#define BUILD_IDS_NONE 0
+#define BUILD_IDS_ALLDEBUG 1
+#define BUILD_IDS_SEPARATE 2
+#define BUILD_IDS_COMPAT   3
+
+static int addNewIDSymlink(FileList fl,
+  char *targetpath, char *idlinkpath,
+  int isDbg, int isCompat)
+{
+const char *linkerr = _("failed symlink");
+int rc = 0;
+int nr = 0;
+char *origpath, *linkpath;
+
+if (isDbg)
+   rasprintf(, "%s.debug", idlinkpath);
+else
+   linkpath = idlinkpath;
+origpath = linkpath;
+
+while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 0) {
+   if (nr > 0)
+   free(linkpath);
+   nr++;
+   rasprintf(, "%s.%d%s", idlinkpath, nr,
+   

[Rpm-maint] [PATCH 4/4] Add build-id links to rpm for all ELF files.

[Mark Wielaard]

This patch moves the main ELF file build-id symlinks from the
debuginfo package into the main package. And uses different
base directories for the main ELF file build-id symlink.
For the main build-id use /usr/lib/.build-id and for the debug
build-id use /usr/lib/debug/.build-id.

There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.

To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.

This also includes support for a new config %_build_id_links that
defaults to compat. The other settings are none, alldebug (the old
style) and separate. compat is like separate, but adds a compatibility
link under /usr/lib/debug/.build-id for the main build-id symlink.

There are several new testcases added to test the various settings
using the new keyword "buildid".

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 build/Makefile.am  |   4 +
 build/files.c  | 375 
 configure.ac   |  15 +
 macros.in  |  28 ++
 scripts/find-debuginfo.sh  |  60 
 tests/Makefile.am  |   1 +
 tests/data/SPECS/hello2cp.spec |  64 
 tests/data/SPECS/hello2ln.spec |  63 
 tests/rpmbuildid.at| 761 +
 tests/rpmtests.at  |   1 +
 10 files changed, 1312 insertions(+), 60 deletions(-)
 create mode 100644 tests/data/SPECS/hello2cp.spec
 create mode 100644 tests/data/SPECS/hello2ln.spec
 create mode 100644 tests/rpmbuildid.at

diff --git a/build/Makefile.am b/build/Makefile.am
index 8318cc8..f0f55a9 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -27,3 +27,7 @@ librpmbuild_la_LIBADD = \
@LTLIBICONV@ \
@WITH_POPT_LIB@ \
@WITH_MAGIC_LIB@
+
+if LIBDW
+librpmbuild_la_LIBADD += @WITH_LIBELF_LIB@ @WITH_LIBDW_LIB@
+endif
diff --git a/build/files.c b/build/files.c
index 07bc94e..b3d821f 100644
--- a/build/files.c
+++ b/build/files.c
@@ -14,6 +14,11 @@
 #include 
 #endif
 
+#if HAVE_LIBDW
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -1544,6 +1549,368 @@ exit:
 return rc;
 }
 
+#if HAVE_LIBDW
+/* How build id links are generated.  See macros.in for description.  */
+#define BUILD_IDS_NONE 0
+#define BUILD_IDS_ALLDEBUG 1
+#define BUILD_IDS_SEPARATE 2
+#define BUILD_IDS_COMPAT   3
+
+static int addNewIDSymlink(FileList fl,
+  char *targetpath, char *idlinkpath,
+  int isDbg, int isCompat)
+{
+const char *linkerr = _("failed symlink");
+int rc = 0;
+int nr = 0;
+char *origpath, *linkpath;
+
+if (isDbg)
+   rasprintf(, "%s.debug", idlinkpath);
+else
+   linkpath = idlinkpath;
+origpath = linkpath;
+
+while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 0) {
+   if (nr > 0)
+   free(linkpath);
+   nr++;
+   rasprintf(, "%s.%d%s", idlinkpath, nr,
+ isDbg ? ".debug" : "");
+}
+
+char *symtarget = targetpath;
+if (nr > 0 && isCompat)
+   rasprintf (, "%s.%d", targetpath, nr);
+
+if (symlink(symtarget, linkpath) < 0) {
+   rc = 1;
+   rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
+  linkerr, linkpath, symtarget);
+} else {
+   fl->cur.isDir = 0;
+   rc = addFile(fl, linkpath, NULL);
+}
+
+/* Don't warn (again) if this is a compat id-link, we retarget it. */
+if (nr > 0 && !isCompat) {
+   /* Lets see why there are multiple build-ids. If the original
+  targets are hard linked, then it is OK, otherwise warn
+  something fishy is going on. Would be nice to call
+  something like eu-elfcmp to see if they are really the same
+  ELF file or not. */
+   struct stat st1, st2;
+   if (stat (origpath, ) != 0) {
+   rpmlog(RPMLOG_WARNING, _("Duplicate build-id, stat %s: %m\n"),
+  origpath);
+   } else if (stat (linkpath, ) != 0) {
+   rpmlog(RPMLOG_WARNING, _("Duplicate build-id, stat %s: %m\n"),
+  linkpath);
+   } else if (!(S_ISREG(st1.st_mode) && S_ISREG(st2.st_mode)
+ && st1.st_nlink > 1 && st2.st_nlink == st1.st_nlink
+ && st1.st_ino == st2.st_ino && st1.st_dev == st2.st_dev)) {
+   char *rpath1 = realpath(origpath, NULL);
+   char *rpa

[Rpm-maint] [PATCH 3/4] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC checksum.

[Mark Wielaard]

Some old tools might still use the .gnu_debuglink section to find
separate debuginfo files instead of build-id style lookups. When
dwz has compresses the .debug files the original CRC in the main
ELF file will no longer match. Make sure to run sepdebugcrcfix
after dwz to recalculate the CRC.

The original fix was created by Jan Kratochvil based on code
from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
I added a testcase to make sure the CRCs were all correctly
updated after dwz has run to compress a debuginfo package.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   |   4 +
 scripts/find-debuginfo.sh |   9 ++
 tests/rpmbuild.at |  36 +
 tools/sepdebugcrcfix.c| 344 ++
 4 files changed, 393 insertions(+)
 create mode 100644 tools/sepdebugcrcfix.c

diff --git a/Makefile.am b/Makefile.am
index 157e79d..08d8c8e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -170,6 +170,10 @@ rpmlibexec_PROGRAMS += elfdeps
 elfdeps_SOURCES =  tools/elfdeps.c
 elfdeps_LDADD =rpmio/librpmio.la
 elfdeps_LDADD +=   @WITH_LIBELF_LIB@ @WITH_POPT_LIB@
+
+rpmlibexec_PROGRAMS += sepdebugcrcfix
+sepdebugcrcfix_SOURCES = tools/sepdebugcrcfix.c
+sepdebugcrcfix_LDADD = @WITH_LIBELF_LIB@
 endif
 endif
 
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 8de7bad..c5d3ce6 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -114,10 +114,12 @@ done
 LISTFILE="$BUILDDIR/$out"
 SOURCEFILE="$BUILDDIR/debugsources.list"
 LINKSFILE="$BUILDDIR/debuglinks.list"
+ELFBINSFILE="$BUILDDIR/elfbins.list"
 
 > "$SOURCEFILE"
 > "$LISTFILE"
 > "$LINKSFILE"
+> "$ELFBINSFILE"
 
 debugdir="${RPM_BUILD_ROOT}/usr/lib/debug"
 
@@ -318,6 +320,8 @@ while read nlinks inum f; do
   # strip -g implies we have full symtab, don't add mini symtab in that case.
   $strip_g || ($include_minidebug && add_minidebug "${debugfn}" "$f")
 
+  echo "./${f#$RPM_BUILD_ROOT}" >> "$ELFBINSFILE"
+
   if [ -n "$id" ]; then
 make_id_link "$id" "$dn/$(basename $f)"
 make_id_link "$id" "/usr/lib/debug$dn/$bn" .debug
@@ -352,6 +356,11 @@ if $run_dwz && type dwz >/dev/null 2>&1 \
   [ -n "$id" ] \
&& make_id_link "$id" "/usr/lib/debug/.dwz/${dwz_multifile_name}" .debug
 fi
+
+# dwz invalidates .gnu_debuglink CRC32 in the main files.
+cat "$ELFBINSFILE" |
+(cd "$RPM_BUILD_ROOT"; \
+ xargs -d '\n' /usr/lib/rpm/sepdebugcrcfix usr/lib/debug)
   fi
 fi
 
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 2fea1b6..5672279 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -459,3 +459,39 @@ test "$canonmultiref" = "$canonmultifile" || exit 1
 [],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that old style gnu_debuglink CRC checksums are correct even after
+# using dwz to compress the debuginfo files.
+AT_SETUP([rpmbuild debuginfo dwz gnu_debuglink crc])
+AT_KEYWORDS([build] [debuginfo])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Build a package that
+cp "${abs_srcdir}"/data/SOURCES/hello-1.0.tar.gz 
"${abs_srcdir}"/data/SOURCES/hello-1.0-modernize.patch ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  -ba "${abs_srcdir}"/data/SPECS/hello2.spec
+
+# Unpack the main and debuginfo rpms so we can check binaries and .debug files.
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello2-debuginfo-1.0-1.*.rpm \
+  | cpio -diu
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello2-1.0-1.*.rpm \
+  | cpio -diu
+
+# Check that dwz has ran and a multi file has been produced
+test -f ./usr/lib/debug/.dwz/hello2-1.0-1.* || exit 1
+
+# Run sepdbugcrcfix on the binaries, both should have correct CRC already.
+${abs_top_builddir}/sepdebugcrcfix ./usr/lib/debug \
+  ./usr/local/bin/hello ./usr/local/bin/hello2 | grep CRC32 | cut -f2 -d:
+],
+[0],
+[ Updated 0 CRC32s, 2 CRC32s did match.
+],
+[ignore])
+AT_CLEANUP
diff --git a/tools/sepdebugcrcfix.c b/tools/sepdebugcrcfix.c
new file mode 100644
index 000..8e45abf
--- /dev/null
+++ b/tools/sepdebugcrcfix.c
@@ -0,0 +1,344 @@
+/* Copyright (C) 2013 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distr

[Rpm-maint] [PATCH 1/4] Add find-debuginfo.sh -m minisymtab support.

[Mark Wielaard]

Support for minisymtab (a minimal function symbol table in a compressed
section in the main binary) has been in gdb and elfutils based tools
since some years. Fedora has had this as rpm-4.10.0-minidebuginfo.patch
since 2012.

The patch adjusts macros to pass -m to find-debuginfo.sh when
_include_minidebuginfo has been set. find-debuginfo.sh now takes -m
as argument to generate the .gnu_debugdata ELF section to be added
to the main executable.

To support the testcases a new macros.debug is added that is used to
generate debuginfo packages in the rpmbuild.at testsuite.

The original support was added to Fedora rpm by Alexander Larsson.
Lubos Kardos fixed a bug in it when strip -g was used. I added some
configuration macros and two testcases to check the basic support works
and for the strip -g bug.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   |  3 ++
 macros.debug  | 30 ++
 macros.in |  8 -
 scripts/find-debuginfo.sh | 37 ++-
 tests/rpmbuild.at | 77 +++
 5 files changed, 153 insertions(+), 2 deletions(-)
 create mode 100644 macros.debug

diff --git a/Makefile.am b/Makefile.am
index 939a6a0..157e79d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -198,6 +198,9 @@ macros: $(top_srcdir)/macros.in
 CLEANFILES += macros
 EXTRA_DIST += macros.in
 
+# Used for the testsuite to test creating debuginfo packages
+EXTRA_DIST += macros.debug
+
 noinst_DATA += platform
 platform: $(top_srcdir)/platform.in
@$(SED) \
diff --git a/macros.debug b/macros.debug
new file mode 100644
index 000..bb2c02f
--- /dev/null
+++ b/macros.debug
@@ -0,0 +1,30 @@
+# macros to include to generate debuginfo
+
+%_enable_debug_packages 1
+%_include_minidebuginfo 1
+
+# Expanded at end of %install scriptlet
+
+%__arch_install_post %{nil}
+
+%__os_install_post \
+%{_rpmconfigdir}/brp-compress \
+%{!?__debug_package:\
+%{_rpmconfigdir}/brp-strip %{__strip} \
+%{_rpmconfigdir}/brp-strip-comment-note %{__strip} %{__objdump} \
+} \
+%{_rpmconfigdir}/brp-strip-static-archive %{__strip} \
+%{nil}
+
+%__spec_install_post\
+%{?__debug_package:%{__debug_install_post}}\
+%{__arch_install_post}\
+%{__os_install_post}\
+%{nil}
+
+%install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
+%%install\
+%{nil}
+
+# Should missing buildids terminate a build?
+%_missing_build_ids_terminate_build1
diff --git a/macros.in b/macros.in
index f9172d0..58b122f 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -441,6 +441,12 @@ package or when debugging this package.\
 #%_missing_build_ids_terminate_build   1
 
 #
+# Include minimal debug information in build binaries.
+# Requires _enable_debug_packages.
+#
+#%_include_minidebuginfo   1
+
+#
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator1
 
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 17522e0..5c2c381 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -2,7 +2,7 @@
 #find-debuginfo.sh - automagically generate debug info and file list
 #for inclusion in an rpm spec file.
 #
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 # [-o debugfiles.list]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
@@ -29,6 +29,9 @@ strip_g=false
 # with -r arg, pass --reloc-debug-sections to eu-strip.
 strip_r=false
 
+# with -m arg, add minimal debuginfo to binary.
+include_minidebug=false
+
 # Barf on missing build IDs.
 strict=false
 
@@ -43,6 +46,9 @@ while [ $# -gt 0 ]; do
   -g)
 strip_g=true
 ;;
+  -m)
+include_minidebug=true
+;;
   -o)
 if [ -z "${lists[$nout]}" -a -z "${ptns[$nout]}" ]; then
   out=$2
@@ -106,6 +112,32 @@ strip_to_debug()
   chmod 444 "$1" || exit
 }
 
+add_minidebug()
+{
+  local debuginfo="$1"
+  local binary="$2"
+
+  local dynsyms=`mktemp`
+  local funcsyms=`mktemp`
+  local keep_symbols=`mktemp`
+  local mini_debuginfo=`mktemp`
+
+  # Extract the dynamic symbols from the main binary, there is no need to also 
have these
+  # in the normal symbol table
+  nm -D "$binary" --format=posix --defined-only | awk '{ print $1 }' | sort &g

Re: [Rpm-maint] [PATCH] RFC: Add build-id links to rpm for all ELF.

[Mark Wielaard]

Hi all,

On Wed, 2016-04-13 at 20:59 +0200, Mark Wielaard wrote:
> This is an RFC patch to move the main ELF file build-id symlinks
> from the debuginfo package into the main package and to use a
> different base directory for the main ELF file build-id symlink.
> Use /usr/lib/.build-id instead of /usr/lib/debug/.build-id
> 
> There are two reasons for doing this. The main package and the
> debuginfo package might get out of sync, or the debuginfo package
> might not be installed at all. In which case finding the main ELF
> file through the build-id symlink becomes impossible. Secondly by
> moving the main ELF build-id symlink in its own directory the
> /usr/lib/debug directory gets populated with only debuginfo files
> which is convenient if the user might want to have that directory
> populated through a network mountpoint.
> 
> This patch is still a work in progress. It seems to work, but might
> not be the best way to implement the above idea. The idea itself
> might not be perfect (comments more than welcome). I tried to
> follow the rpm coding style as much as I understood it. Please let
> me know if I use some wrong constructs, or if there are alternate
> ways to do what I am doing. It contains one testcase, that is fairly
> minimal. I'll add more if we agree on the semantics of the idea
> (help writing better testcases also very welcome). It obviously will
> need some config setting (pointers to examples that use configs
> for this kind of rpmbuild settings would be appreciated). And it
> needs corresponding adjustments to scripts/find-debuginfo.sh,
> including, when enabled, a backwards compatible link from the old
> /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy
> 
> Comments, ideas and suggestions how to move a change like this
> forward more than welcome.

I am integrating this patch with the patches I posted that integrate the
debuginfo package handling from fedora and the new testcases and wanted
to make the behavior configurable so a distro can setup a macro to
define what they want. Does the following make sense?

# Defines how and if build_id links are generated for ELF files.
# The following settings are supported:
#
# - none
#   No build_id links are generated.
#
# - debug
#   build_id links are generated only when the __debug_package global is
#   defined. This will generate build_id links in the -debuginfo package
#   for both the main file as /usr/lib/debug/.build-id/xx/yyy and for
#   the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug.
#   This is the old style build_id links as generated by the original
#   find-debuginfo.sh script.
#
# - separate
#   build_id links are generate for all binary packages. If this is a
#   main package (the __debug_package global isn't set) then the
#   build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is
#   a -debuginfo package (the __debug_package global is set) then the
#   build_id link is generated as /usr/lib/debug/.build-id/xx/yyy.
#
# - compat
#   Same as for "separate" but if the __debug_package global is set then
#   the -debuginfo package will have a compatibility link for the main
#   ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy
%_build_id_links compat

Let me know if this makes sense and I will implement it and write some
testcases for the different settings.

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 6/7] Make it possible to have unique build-ids across build versions/releases.

[Mark Wielaard]

Introduce a new macro _unique_build_ids that when set will pass the
version and release to find-debuginfo.sh and debugedit to recalculate
the build-id of ELF files.

Includes two new testcases to make sure the new setting works as expected
both when set and unset.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 macros.in  |  8 +++-
 scripts/find-debuginfo.sh  | 20 -
 tests/data/SPECS/hello-r2.spec | 58 +
 tests/rpmbuildid.at| 96 +-
 tools/debugedit.c  | 24 ++-
 5 files changed, 201 insertions(+), 5 deletions(-)
 create mode 100644 tests/data/SPECS/hello-r2.spec

diff --git a/macros.in b/macros.in
index 508c50f..c79cfcc 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_dwz_opts} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_unique_build_ids:--ver-rel 
"%{version}-%{release}"} %{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -481,6 +481,12 @@ package or when debugging this package.\
 #   ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy
 %_build_id_links compat
 
+# Whether build-ids should be made unique between package version/releases
+# when generating debuginfo packages. If set to 1 this will pass
+# --ver-rel "%{version}-%{release}" to find-debuginfo.sh which will pass it
+# onto debugedit --build-id-seed to be used to prime the build-id note hash.
+%_unique_build_ids 1
+
 #
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator1
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index c93b246..9783715 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -6,6 +6,7 @@
 # [-o debugfiles.list]
 # [--run-dwz] [--dwz-low-mem-die-limit N]
 # [--dwz-max-die-limit N]
+# [--ver-rel VERSION-RELEASE]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
 #
@@ -26,6 +27,12 @@
 # if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit
 # provide detailed limits.  See dwz(1) -l and -L option for details.
 #
+# If --ver-rel VERSION-RELEASE is given then debugedit is called to
+# update the build-ids it finds adding the VERSION-RELEASE string as
+# seed to recalculate the build-id hash.  This makes sure the
+# build-ids in the ELF files are unique between versions and releases
+# of the same package.
+#
 # All file names in switches are relative to builddir (. if not given).
 #
 
@@ -49,6 +56,9 @@ run_dwz=false
 dwz_low_mem_die_limit=
 dwz_max_die_limit=
 
+# Version and release of the spec. Given by --ver-rel
+ver_rel=
+
 BUILDDIR=.
 out=debugfiles.list
 nout=0
@@ -68,6 +78,10 @@ while [ $# -gt 0 ]; do
 dwz_max_die_limit=$2
 shift
 ;;
+  --ver-rel)
+ver_rel=$2
+shift
+;;
   -g)
 strip_g=true
 ;;
@@ -238,8 +252,12 @@ while read nlinks inum f; do
   fi
 
   echo "extracting debug info from $f"
+  build_id_seed=
+  if [ ! -z "$ver_rel" ]; then
+build_id_seed="--build-id-seed=$ver_rel"
+  fi
   id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
- -i -l "$SOURCEFILE" "$f") || exit
+ -i $build_id_seed -l "$SOURCEFILE" "$f") || exit
   if [ $nlinks -gt 1 ]; then
 eval linkedid_$inum=\$id
   fi
diff --git a/tests/data/SPECS/hello-r2.spec b/tests/data/SPECS/hello-r2.spec
new file mode 100644
index 000..ca5091d
--- /dev/null
+++ b/tests/data/SPECS/hello-r2.spec
@@ -0,0 +1,58 @@
+Summary: hello -- hello, world rpm
+Name: hello
+Version: 1.0
+Release: 2
+Group: Utilities
+License: GPL
+Distribution: RPM test suite.
+Vendor: Red Hat Software
+Packager: Red Hat Software <b...@redhat.com>
+URL: http://www.redhat.com
+Source0: hello-1.0.tar.gz
+Patch0: hello-1.0-modernize.patch
+Excludearch: lsi
+Excludeos: cpm
+Provides: hi
+Conflicts: goodbye
+Obsoletes: howdy
+Prefix: /usr
+
+%description
+Simple rpm demonstration.
+
+%prep
+%setup -q
+%patch0 -p1 -b .modernize
+
+%build
+make
+
+%install
+rm -rf $RPM_BUILD_ROOT
+mkdir -p $RPM_BUILD_ROOT/usr/local/bin
+make DESTDIR=$RPM_BUILD_ROOT install
+
+%clean
+rm -rf $RPM_BUILD_ROOT
+
+%pre
+
+%post
+
+%preun
+
+%postun
+
+%files
+%defattr(-,root,root)
+%doc  

[Rpm-maint] [PATCH 5/7] Add build-id links to rpm for all ELF files.

[Mark Wielaard]

This patch moves the main ELF file build-id symlinks from the
debuginfo package into the main package. And uses different
base directories for the main ELF file build-id symlink.
For the main build-id use /usr/lib/.build-id and for the debug
build-id use /usr/lib/debug/.build-id.

There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.

To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.

This also includes support for a new config %_build_id_links that
defaults to compat. The other settings are none, alldebug (the old
style) and separate. compat is like separate, but adds a compatibility
link under /usr/lib/debug/.build-id for the main build-id symlink.

There are several new testcases added to test the various settings
using the new keyword "buildid".

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 build/Makefile.am  |   4 +
 build/files.c  | 375 
 configure.ac   |  15 +
 macros.in  |  28 ++
 scripts/find-debuginfo.sh  |  60 
 tests/Makefile.am  |   1 +
 tests/data/SPECS/hello2cp.spec |  64 
 tests/data/SPECS/hello2ln.spec |  63 
 tests/rpmbuildid.at| 761 +
 tests/rpmtests.at  |   1 +
 10 files changed, 1312 insertions(+), 60 deletions(-)
 create mode 100644 tests/data/SPECS/hello2cp.spec
 create mode 100644 tests/data/SPECS/hello2ln.spec
 create mode 100644 tests/rpmbuildid.at

diff --git a/build/Makefile.am b/build/Makefile.am
index 8318cc8..f0f55a9 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -27,3 +27,7 @@ librpmbuild_la_LIBADD = \
@LTLIBICONV@ \
@WITH_POPT_LIB@ \
@WITH_MAGIC_LIB@
+
+if LIBDW
+librpmbuild_la_LIBADD += @WITH_LIBELF_LIB@ @WITH_LIBDW_LIB@
+endif
diff --git a/build/files.c b/build/files.c
index 07bc94e..b3d821f 100644
--- a/build/files.c
+++ b/build/files.c
@@ -14,6 +14,11 @@
 #include 
 #endif
 
+#if HAVE_LIBDW
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -1544,6 +1549,368 @@ exit:
 return rc;
 }
 
+#if HAVE_LIBDW
+/* How build id links are generated.  See macros.in for description.  */
+#define BUILD_IDS_NONE 0
+#define BUILD_IDS_ALLDEBUG 1
+#define BUILD_IDS_SEPARATE 2
+#define BUILD_IDS_COMPAT   3
+
+static int addNewIDSymlink(FileList fl,
+  char *targetpath, char *idlinkpath,
+  int isDbg, int isCompat)
+{
+const char *linkerr = _("failed symlink");
+int rc = 0;
+int nr = 0;
+char *origpath, *linkpath;
+
+if (isDbg)
+   rasprintf(, "%s.debug", idlinkpath);
+else
+   linkpath = idlinkpath;
+origpath = linkpath;
+
+while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 0) {
+   if (nr > 0)
+   free(linkpath);
+   nr++;
+   rasprintf(, "%s.%d%s", idlinkpath, nr,
+ isDbg ? ".debug" : "");
+}
+
+char *symtarget = targetpath;
+if (nr > 0 && isCompat)
+   rasprintf (, "%s.%d", targetpath, nr);
+
+if (symlink(symtarget, linkpath) < 0) {
+   rc = 1;
+   rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
+  linkerr, linkpath, symtarget);
+} else {
+   fl->cur.isDir = 0;
+   rc = addFile(fl, linkpath, NULL);
+}
+
+/* Don't warn (again) if this is a compat id-link, we retarget it. */
+if (nr > 0 && !isCompat) {
+   /* Lets see why there are multiple build-ids. If the original
+  targets are hard linked, then it is OK, otherwise warn
+  something fishy is going on. Would be nice to call
+  something like eu-elfcmp to see if they are really the same
+  ELF file or not. */
+   struct stat st1, st2;
+   if (stat (origpath, ) != 0) {
+   rpmlog(RPMLOG_WARNING, _("Duplicate build-id, stat %s: %m\n"),
+  origpath);
+   } else if (stat (linkpath, ) != 0) {
+   rpmlog(RPMLOG_WARNING, _("Duplicate build-id, stat %s: %m\n"),
+  linkpath);
+   } else if (!(S_ISREG(st1.st_mode) && S_ISREG(st2.st_mode)
+ && st1.st_nlink > 1 && st2.st_nlink == st1.st_nlink
+ && st1.st_ino == st2.st_ino && st1.st_dev == st2.st_dev)) {
+   char *rpath1 = realpath(origpath, NULL);
+   char *rpa

Re: [Rpm-maint] [PATCH 3/4] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC checksum.

[Mark Wielaard]

Hi Thierry,

On Mon, 2016-06-13 at 14:33 +0200, Thierry Vignaud wrote:
> Could you provide a small example? From the bug report it isn't clear if
> > it really is a bug with that particular fix or that it is caused by bad
> > usage of attributes. If we have a small example we can create a test
> > case for it.
> 
> Here's one example attached.
> With current FC patch applied to rpm (aka uncommenting the patch at
> http://svnweb.mageia.org/packages/cauldron/rpm/current/SPECS/rpm.spec?revision=1018103=markup#l116)
> 
> With "%define debug_package %{nil}", "ll /bin/test2" shows:
> -rwsr-xr-x 1 root root 6128 Eve  13 13:56 /bin/test2*
> With "#define debug_package %{nil}", "ll /bin/test2" shows:
> -rwxr-xr-x 1 root root 7184 Eve  13 13:55 /bin/test2*
> 
> When the rpm patch is not applied, both cases works OK (aka ls -[ol]
> reports the suid bit)
 
Aha, ok. I see what is happening. Since you don't set the attributes
explicitly you rely on the suid bit being picked up. But since we change
the file the suid flag gets reset. The fix is similar to how other helpers
get and reset all mods. I added a testcase for this issue making sure that
it triggers all helpers and still has the suid binaries coming out as
expected. I also rearranged the patchset a little, to make sure the test
actually tests the right binary by bringing forward the "Don't use
hardcoded paths to tools/scripts in find-debuginfo.sh." fix.

 [PATCH 1/7] Add find-debuginfo.sh -m minisymtab support.
 [PATCH 2/7] Add dwz debuginfo compression support.
 [PATCH 3/7] Don't use hardcoded paths to tools/scripts in
 [PATCH 4/7] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC
 [PATCH 5/7] Add build-id links to rpm for all ELF files.
 [PATCH 6/7] Make it possible to have unique build-ids across build
 [PATCH 7/7] Make adding GDB index sections configurable.

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 2/7] Add dwz debuginfo compression support.

[Mark Wielaard]

Support for dwz compression has been in Fedora since a couple of years.
https://fedoraproject.org/wiki/Features/DwarfCompressor

The original find-debuginfo.sh patch was written by Jakub Jelinek.
https://bugzilla.redhat.com/show_bug.cgi?id=833311
The new testcase using the macros.debug was added by me.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 macros.debug | 13 
 macros.in|  2 +-
 scripts/find-debuginfo.sh| 53 ++
 tests/data/SPECS/hello2.spec | 62 +++
 tests/rpmbuild.at| 78 
 5 files changed, 207 insertions(+), 1 deletion(-)
 create mode 100644 tests/data/SPECS/hello2.spec

diff --git a/macros.debug b/macros.debug
index bb2c02f..6a8432e 100644
--- a/macros.debug
+++ b/macros.debug
@@ -28,3 +28,16 @@
 
 # Should missing buildids terminate a build?
 %_missing_build_ids_terminate_build1
+
+# Number of debugging information entries (DIEs) above which
+# dwz will stop considering file for multifile optimizations
+# and enter a low memory mode, in which it will optimize
+# in about half the memory needed otherwise.
+%_dwz_low_mem_die_limit 1000
+# Number of DIEs above which dwz will stop processing
+# a file altogether.
+%_dwz_max_die_limit 5000
+
+%_find_debuginfo_dwz_opts --run-dwz\\\
+   --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\\
+   --dwz-max-die-limit %{_dwz_max_die_limit}
diff --git a/macros.in b/macros.in
index 94d0aa8..e94f476 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_dwz_opts} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 5c2c381..8de7bad 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -4,6 +4,8 @@
 #
 # Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 # [-o debugfiles.list]
+# [--run-dwz] [--dwz-low-mem-die-limit N]
+# [--dwz-max-die-limit N]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
 #
@@ -20,6 +22,10 @@
 # The -p argument is an grep -E -style regexp matching the a file name,
 # and must not use anchors (^ or $).
 #
+# The --run-dwz flag instructs find-debuginfo.sh to run the dwz utility
+# if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit
+# provide detailed limits.  See dwz(1) -l and -L option for details.
+#
 # All file names in switches are relative to builddir (. if not given).
 #
 
@@ -35,6 +41,11 @@ include_minidebug=false
 # Barf on missing build IDs.
 strict=false
 
+# DWZ parameters.
+run_dwz=false
+dwz_low_mem_die_limit=
+dwz_max_die_limit=
+
 BUILDDIR=.
 out=debugfiles.list
 nout=0
@@ -43,6 +54,17 @@ while [ $# -gt 0 ]; do
   --strict-build-id)
 strict=true
 ;;
+  --run-dwz)
+run_dwz=true
+;;
+  --dwz-low-mem-die-limit)
+dwz_low_mem_die_limit=$2
+shift
+;;
+  --dwz-max-die-limit)
+dwz_max_die_limit=$2
+shift
+;;
   -g)
 strip_g=true
 ;;
@@ -302,6 +324,37 @@ while read nlinks inum f; do
   fi
 done || exit
 
+# Invoke the DWARF Compressor utility.
+if $run_dwz && type dwz >/dev/null 2>&1 \
+   && [ -d "${RPM_BUILD_ROOT}/usr/lib/debug" ]; then
+  dwz_files="`cd "${RPM_BUILD_ROOT}/usr/lib/debug"; find -type f -name 
\*.debug`"
+  if [ -n "${dwz_files}" ]; then
+
dwz_multifile_name="${RPM_PACKAGE_NAME}-${RPM_PACKAGE_VERSION}-${RPM_PACKAGE_RELEASE}.${RPM_ARCH}"
+dwz_multifile_suffix=
+dwz_multifile_idx=0
+while [ -f 
"${RPM_BUILD_ROOT}/usr/lib/debug/.dwz/${dwz_multifile_name}${dwz_multifile_suffix}"
 ]; do
+  let ++dwz_multifile_idx
+  dwz_multifile_suffix=".${dwz_multifile_idx}"
+done
+dwz_multfile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
+dwz_opts="-h -q -r -m .dwz/${dwz_multifile_name}"
+mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
+[ -n "${dwz_low_mem_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -l ${dwz_low_mem_die_limit}"
+[ -n "${dwz_max_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -L ${dwz_max_die_limit}"
+( cd "${RPM_BUILD_ROOT}/usr/lib/debug" && dwz $dw

Re: [Rpm-maint] [PATCH] RFC: Add build-id links to rpm for all ELF.

[Mark Wielaard]

On Thu, 2016-05-26 at 17:00 +0200, Mark Wielaard wrote:
> I am integrating this patch with the patches I posted that integrate the
> debuginfo package handling from fedora and the new testcases and wanted
> to make the behavior configurable so a distro can setup a macro to
> define what they want. Does the following make sense?
> 
> # Defines how and if build_id links are generated for ELF files.
> # The following settings are supported:
> #
> # - none
> #   No build_id links are generated.
> #
> # - debug
> #   build_id links are generated only when the __debug_package global is
> #   defined. This will generate build_id links in the -debuginfo package
> #   for both the main file as /usr/lib/debug/.build-id/xx/yyy and for
> #   the .debug file as /usr/lib/debug/.build-id/xx/yyy.debug.
> #   This is the old style build_id links as generated by the original
> #   find-debuginfo.sh script.
> #
> # - separate
> #   build_id links are generate for all binary packages. If this is a
> #   main package (the __debug_package global isn't set) then the
> #   build_id link is generated as /usr/lib/.build-id/xx/yyy. If this is
> #   a -debuginfo package (the __debug_package global is set) then the
> #   build_id link is generated as /usr/lib/debug/.build-id/xx/yyy.
> #
> # - compat
> #   Same as for "separate" but if the __debug_package global is set then
> #   the -debuginfo package will have a compatibility link for the main
> #   ELF /usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy
> %_build_id_links compat

I rebased my patch on top of the other debuginfo improvement patches to
implemented this logic. I renamed debug to alldebug to make it more
clear what that setting does. The default ("compat") is now tested
because all tests from the fedora debuginfo patches now PASS. I still
need to write tests for the other settings. It also doesn't handle
duplicate build-ids, which the old find-debuginfo.sh code did handle. So
that has to be added too.

Feedback welcome.

Cheers,

Mark
From 3df87d2908a8ae795ba28332ce587f56f4ef5612 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@redhat.com>
Date: Wed, 13 Apr 2016 20:37:44 +0200
Subject: [PATCH] Add build-id links to rpm for all ELF.

This is an RFC patch to move the main ELF file build-id symlinks
from the debuginfo package into the main package and to use a
different base directory for the main ELF file build-id symlink.
Use /usr/lib/.build-id instead of /usr/lib/debug/.build-id

There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.

To support the new logic the symlink code has been moved from
find-debuginfo.sh to build/files.c.

This also includes support for a new config %_build_id_links that
defaults to compat.
---
 build/Makefile.am |   4 +
 build/files.c | 331 ++
 configure.ac  |  15 +++
 macros.in |  28 
 scripts/find-debuginfo.sh |   6 -
 tests/Makefile.am |   1 +
 tests/rpmbuildid.at   |  47 +++
 tests/rpmtests.at |   1 +
 8 files changed, 427 insertions(+), 6 deletions(-)
 create mode 100644 tests/rpmbuildid.at

diff --git a/build/Makefile.am b/build/Makefile.am
index 8318cc8..f0f55a9 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -27,3 +27,7 @@ librpmbuild_la_LIBADD = \
 	@LTLIBICONV@ \
 	@WITH_POPT_LIB@ \
 	@WITH_MAGIC_LIB@
+
+if LIBDW
+librpmbuild_la_LIBADD += @WITH_LIBELF_LIB@ @WITH_LIBDW_LIB@
+endif
diff --git a/build/files.c b/build/files.c
index 07bc94e..3d40d09 100644
--- a/build/files.c
+++ b/build/files.c
@@ -14,6 +14,11 @@
 #include 
 #endif
 
+#if HAVE_LIBDW
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -1544,6 +1549,324 @@ exit:
 return rc;
 }
 
+#if HAVE_LIBDW
+/* How build id links are generated.  See macros.in for description.  */
+#define BUILD_IDS_NONE 0
+#define BUILD_IDS_ALLDEBUG 1
+#define BUILD_IDS_SEPARATE 2
+#define BUILD_IDS_COMPAT   3
+
+static int generateBuildIDs(FileList fl)
+{
+int rc = 0;
+int i;
+FileListRec flp;
+char **ids = NULL;
+char **paths = NULL;
+size_t nr_ids, allocated;
+nr_ids = allocated = 0;
+
+/* How are we supposed to create the build-id links?  */
+char *build_id_links_macro = rpmExpand("%{?_build_id_links}", NULL);
+int build_id_links;
+if (build_id_links_macro == NULL) {
+	rpmlog(RPMLOG_WARNING,
+	   _("_build_id_li

[Rpm-maint] [PATCH] Add option to have unique debug file names across version/release/arch.

[Mark Wielaard]

Introduce a new macro _unique_debug_names that when set will pass
--unique-debug-arch "%{_arch}" to find-debuginfo.sh to create debuginfo
files which end in "--..debug" instead of simply ".debug".

Adds testcases for dwz and buildid with and without unique debug file names.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 build/files.c |  19 ++-
 macros.debug  |   3 +
 macros.in |   9 +-
 scripts/find-debuginfo.sh |  18 ++-
 tests/rpmbuild.at | 168 +++-
 tests/rpmbuildid.at   | 316 +-
 6 files changed, 494 insertions(+), 39 deletions(-)

diff --git a/build/files.c b/build/files.c
index b3d821f..2a65ed3 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1783,6 +1783,16 @@ static int generateBuildIDs(FileList fl)
}
}
 
+   /* In case we need ALLDEBUG links we might need the vra as
+  tagged onto the .debug file name. */
+   char *vra = NULL;
+   if (rc == 0 && needDbg && build_id_links == BUILD_IDS_ALLDEBUG) {
+   int unique_debug_names =
+   rpmExpandNumeric("%{?_unique_debug_names}");
+   if (unique_debug_names == 1)
+   vra = rpmExpand("-%{version}-%{release}.%{_arch}", NULL);
+   }
+
/* Now add a subdir and symlink for each buildid found.  */
for (i = 0; i < nr_ids; i++) {
/* Don't add anything more when an error occured. But do
@@ -1877,15 +1887,17 @@ static int generateBuildIDs(FileList fl)
int pathlen = strlen(paths[i]);
int debuglen = strlen(".debug");
int prefixlen = strlen("/usr/lib/debug");
-   if (pathlen > prefixlen
-   && strcmp (paths[i] + pathlen - debuglen,
+   int vralen = vra == NULL ? 0 : strlen(vra);
+   if (pathlen > prefixlen + debuglen + vralen
+   && strcmp ((paths[i] + pathlen - debuglen),
   ".debug") == 0) {
free(linkpath);
free(targetpath);
char *targetstr = xstrdup (paths[i]
   + prefixlen);
int targetlen = pathlen - prefixlen;
-   targetstr[targetlen - debuglen] = '\0';
+   int targetend = targetlen - debuglen - vralen;
+   targetstr[targetend] = '\0';
rasprintf(, "%s/%s",
  buildidsubdir, [i][2]);
rasprintf(, "../../../../..%s",
@@ -1904,6 +1916,7 @@ static int generateBuildIDs(FileList fl)
free(paths[i]);
free(ids[i]);
}
+   free(vra);
free(paths);
free(ids);
 }
diff --git a/macros.debug b/macros.debug
index d273c08..ee0cc9e 100644
--- a/macros.debug
+++ b/macros.debug
@@ -1,4 +1,7 @@
 # macros to include to generate debuginfo
+# Note don't define/enable a feature here if it is already the default in
+# macros.in. Otherwise it cannot simply be --undefined on the command line
+# in the tests (it needs to be undefined multiple times then).
 
 %_enable_debug_packages 1
 %_include_minidebuginfo 1
diff --git a/macros.in b/macros.in
index 205a1f3..edc02b6 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i} 
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i} 
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"} 
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -493,6 +493,13 @@ package or when debugging this package.\
 # onto debugedit --build-id-seed to be used to prime the build-id note hash.
 %_unique_build_ids 1
 
+# Whether .debug files should be made unique between package version,
+# release and architecture. If set to 1 this will pass
+# --unique-debug-arch "%{_arch}" to find-debuginfo.sh to create
+# debuginfo files which end in

[Rpm-maint] [PATCH] configure --with-external-db should fall back to internal if not specified.

[Mark Wielaard]

configure.ac implies that there is a fall back to the internal db if
no external one is specified or found. But that doesn't work since
with_external_db defaults to no when not --with[out]-external-db isn't
given. Fix that by defaulting to "maybe" and then first check for an
external db before falling back to the internal db if available.
---
 configure.ac | 19 ---
 1 file changed, 16 insertions(+), 3 deletions(-)

diff --git a/configure.ac b/configure.ac
index 3befb15..3f693c8 100644
--- a/configure.ac
+++ b/configure.ac
@@ -374,10 +374,10 @@ AC_ARG_WITH(external_db, 
[AS_HELP_STRING([--with-external-db],[build against an
 yes|no) ;;
 *) AC_MSG_ERROR([invalid argument to --with-external-db]) ;;
 esac],
-[with_external_db=no])
+[with_external_db=maybe])
 
 case "$with_external_db" in
-yes )
+yes|maybe )
   AC_CHECK_HEADERS([db.h],[
 AC_PREPROC_IFELSE([
   AC_LANG_SOURCE([
@@ -390,10 +390,23 @@ yes )
   [ AC_MSG_ERROR([Berkeley DB version >= 4.5 required])
 ])
   ],[
+case "$with_external_db" in
+maybe)
+# Fall back to internal db if available
+if [ test -x db/dist/configure ]; then
+  AC_DEFINE(HAVE_DB_H, 1, [Define if you have the  header file])
+  with_external_db=no
+else
+  AC_MSG_ERROR([internal Berkeley DB directory not present, see INSTALL])
+fi
+;;
+yes)
 AC_MSG_ERROR([missing required header db.h])
+;;
+esac
   ])
   ;;
-* ) # Fall back to internal db if available
+no )
   if [ test -x db/dist/configure ]; then
 AC_DEFINE(HAVE_DB_H, 1, [Define if you have the  header file])
   else
-- 
2.5.0

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Make sure CPPFLAGS are setup correctly for finding nspr.h

[Mark Wielaard]

If we have pkgconfig make sure CPPFLAGS are setup correctly for the nss
-I include path. Otherwise the checks to find nspr.h will fail.
---
 configure.ac | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/configure.ac b/configure.ac
index fd73b33..3befb15 100644
--- a/configure.ac
+++ b/configure.ac
@@ -281,6 +281,13 @@ AC_SUBST(WITH_BEECRYPT_INCLUDE)
 WITH_NSS_INCLUDE=
 WITH_NSS_LIB=
 if test "$with_beecrypt" != yes ; then
+# If we have pkgconfig make sure CPPFLAGS are setup correctly for the nss
+# -I include path. Otherwise the below checks will fail because nspr.h
+# cannot be found.
+AC_PATH_TOOL([PKGCONFIG], [pkg-config], [no], [$PATH:/usr/bin:/usr/local/bin])
+if test "x$PKGCONFIG" != "xno"; then
+  CPPFLAGS="$CPPFLAGS $($PKGCONFIG --cflags nss)"
+fi
 AC_CHECK_HEADERS([nspr.h nss.h sechash.h], [], [
   AC_MSG_ERROR([missing required NSPR / NSS header])
 ])
-- 
2.5.0

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] configure --with-external-db should fall back to internal if not specified.

[Mark Wielaard]

On Fri, 2016-03-18 at 17:17 +0100, Mark Wielaard wrote:
> configure.ac implies that there is a fall back to the internal db if
> no external one is specified or found. But that doesn't work since
> with_external_db defaults to no when not --with[out]-external-db isn't
> given. Fix that by defaulting to "maybe" and then first check for an
> external db before falling back to the internal db if available.

On irc (#rpm.org) ffesti said he liked to keep the current behavior of
first checking for an internal db if neither --with-external-db nor
--without-external-db was given. So the attached patch does that (move
the fallback from yes|maybe to no|maybe).

Cheers,

Mark
From 2a6d1be3796f6b14fa24a23ba09976f561f50f54 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@redhat.com>
Date: Fri, 18 Mar 2016 17:14:43 +0100
Subject: [PATCH] configure --with-external-db should fall back to external if
 unspecified.

configure.ac implies that there is a fall back to the internal db if
no external one is specified or found. But that doesn't work since
with_external_db defaults to no when not --with[out]-external-db isn't
given. Fix that by defaulting to "maybe" and then after the check for an
internal db fails fall back to the external db.h if available.

This keeps the current behavior of defaulting to --without-external-db (no)
if nothing is specified, but falls back to trying with the external one if
there is no in tree internal db. Giving an explicit --with-external-db or
--without-external-db doesn't change and produces an error if no external
or no internal db is found.
---
 configure.ac | 28 +---
 1 file changed, 25 insertions(+), 3 deletions(-)

diff --git a/configure.ac b/configure.ac
index 3befb15..b77160b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -374,7 +374,7 @@ AC_ARG_WITH(external_db, [AS_HELP_STRING([--with-external-db],[build against an
 yes|no) ;;
 *) AC_MSG_ERROR([invalid argument to --with-external-db]) ;;
 esac],
-[with_external_db=no])
+[with_external_db=maybe])
 
 case "$with_external_db" in
 yes )
@@ -393,11 +393,33 @@ yes )
 AC_MSG_ERROR([missing required header db.h])
   ])
   ;;
-* ) # Fall back to internal db if available
+no|maybe )
+  # Try internal database first, then fall back to external
+  # unless --without-external-db (no) was explicitly given.
   if [ test -x db/dist/configure ]; then
 AC_DEFINE(HAVE_DB_H, 1, [Define if you have the  header file])
   else
-AC_MSG_ERROR([internal Berkeley DB directory not present, see INSTALL])
+case "$with_external_db" in
+maybe)
+  AC_CHECK_HEADERS([db.h],[
+AC_PREPROC_IFELSE([
+  AC_LANG_SOURCE([
+	#include 
+	#if ((DB_VERSION_MAJOR < 4) || (DB_VERSION_MAJOR == 4 && DB_VERSION_MINOR < 5))
+	#error Berkeley DB too old
+	#endif
+  ])
+],[ WITH_DB_LIB=-ldb ],
+  [ AC_MSG_ERROR([Berkeley DB version >= 4.5 required])
+])
+  ],[
+AC_MSG_ERROR([missing required header db.h])
+  ])
+;;
+no)
+  AC_MSG_ERROR([internal Berkeley DB directory not present, see INSTALL])
+;;
+esac
   fi
   ;;
 esac
-- 
2.5.0

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] RFC: Add build-id links to rpm for all ELF.

[Mark Wielaard]

This is an RFC patch to move the main ELF file build-id symlinks
from the debuginfo package into the main package and to use a
different base directory for the main ELF file build-id symlink.
Use /usr/lib/.build-id instead of /usr/lib/debug/.build-id

There are two reasons for doing this. The main package and the
debuginfo package might get out of sync, or the debuginfo package
might not be installed at all. In which case finding the main ELF
file through the build-id symlink becomes impossible. Secondly by
moving the main ELF build-id symlink in its own directory the
/usr/lib/debug directory gets populated with only debuginfo files
which is convenient if the user might want to have that directory
populated through a network mountpoint.

This patch is still a work in progress. It seems to work, but might
not be the best way to implement the above idea. The idea itself
might not be perfect (comments more than welcome). I tried to
follow the rpm coding style as much as I understood it. Please let
me know if I use some wrong constructs, or if there are alternate
ways to do what I am doing. It contains one testcase, that is fairly
minimal. I'll add more if we agree on the semantics of the idea
(help writing better testcases also very welcome). It obviously will
need some config setting (pointers to examples that use configs
for this kind of rpmbuild settings would be appreciated). And it
needs corresponding adjustments to scripts/find-debuginfo.sh,
including, when enabled, a backwards compatible link from the old
/usr/lib/debug/.build-id/xx/yyy -> /usr/lib/.build-id/xx/yyy

Comments, ideas and suggestions how to move a change like this
forward more than welcome.
---
 build/Makefile.am   |   4 ++
 build/files.c   | 129 
 configure.ac|  15 ++
 tests/Makefile.am   |   1 +
 tests/rpmbuildid.at |  47 +++
 tests/rpmtests.at   |   1 +
 6 files changed, 197 insertions(+)
 create mode 100644 tests/rpmbuildid.at

diff --git a/build/Makefile.am b/build/Makefile.am
index 8318cc8..f0f55a9 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -27,3 +27,7 @@ librpmbuild_la_LIBADD = \
@LTLIBICONV@ \
@WITH_POPT_LIB@ \
@WITH_MAGIC_LIB@
+
+if LIBDW
+librpmbuild_la_LIBADD += @WITH_LIBELF_LIB@ @WITH_LIBDW_LIB@
+endif
diff --git a/build/files.c b/build/files.c
index 07bc94e..0586503 100644
--- a/build/files.c
+++ b/build/files.c
@@ -14,6 +14,11 @@
 #include 
 #endif
 
+#if HAVE_LIBDW
+#include 
+#include 
+#endif
+
 #include 
 #include 
 #include 
@@ -1544,6 +1549,122 @@ exit:
 return rc;
 }
 
+#if HAVE_LIBDW
+static int generateBuildIDs(FileList fl)
+{
+int rc = 0;
+int i;
+FileListRec flp;
+char **ids = NULL;
+char **paths = NULL;
+size_t nr_ids, allocated;
+nr_ids = allocated = 0;
+
+for (i = 0, flp = fl->files.recs; i < fl->files.used; i++, flp++) {
+   int fd;
+   fd = open (flp->diskPath, O_RDONLY);
+   if (fd >= 0) {
+   struct stat sbuf;
+   if (fstat (fd, ) == 0 && S_ISREG (sbuf.st_mode)) {
+   Elf *elf = elf_begin (fd, ELF_C_READ, NULL);
+   if (elf != NULL) {
+   const void *build_id;
+   ssize_t len = dwelf_elf_gnu_build_id (elf, _id);
+   /* len == -1 means error (but could also mean, not
+  really an ELF file), Zero means no build-id. We
+  want at least a length of 2 so we have at least
+  a xx/yy (hex) dir/file.  */
+   if (len >= 2) {
+   const unsigned char *p = build_id;
+   const unsigned char *end = p + len;
+   char *id_str;
+   if (allocated <= nr_ids) {
+   allocated += 16;
+   paths = xrealloc (paths,
+ allocated * sizeof(char *));
+   ids = xrealloc (ids, allocated * sizeof(char *));
+   }
+   paths[nr_ids] = xstrdup(flp->cpioPath);
+   id_str = ids[nr_ids] = xmalloc(2 * len + 1);
+   while (p < end)
+   id_str += sprintf(id_str, "%02x", (unsigned)*p++);
+   *id_str = '\0';
+   nr_ids++;
+   } else if (len > 0) {
+   rpmlog(RPMLOG_WARNING,
+  _("build-id found in %s too small\n"),
+  flp->diskPath);
+   }
+   elf_end (elf);
+   }
+   }
+   close (fd);
+   }
+}
+
+if (nr_ids > 0) {
+   /* Add /usr/lib/.build-id to hold the subdirs/symlinks.  */
+   #define BUILD_ID_DIR "/usr/lib/.build-id"
+const char *errmsg = _("failed to create directory");
+char *buildiddir = 

[Rpm-maint] [PATCH 2/2] Add sepdebugcrcfix to fixup old style gnu_debuglink CRC checksum.

[Mark Wielaard]

Some old tools might still use the .gnu_debuglink section to find
separate debuginfo files instead of build-id style lookups. When
dwz has compresses the .debug files the original CRC in the main
ELF file will no longer match. Make sure to run sepdebugcrcfix
after dwz to recalculate the CRC.

The original fix was created by Jan Kratochvil based on code
from GNU binutils BFD. https://bugzilla.redhat.com/show_bug.cgi?id=971119
I added a testcase to make sure the CRCs were all correctly
updated after dwz has run to compress a debuginfo package.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   |   4 +
 scripts/find-debuginfo.sh |   9 ++
 tests/rpmbuild.at |  36 +
 tools/sepdebugcrcfix.c| 344 ++
 4 files changed, 393 insertions(+)
 create mode 100644 tools/sepdebugcrcfix.c

diff --git a/Makefile.am b/Makefile.am
index 157e79d..08d8c8e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -170,6 +170,10 @@ rpmlibexec_PROGRAMS += elfdeps
 elfdeps_SOURCES =  tools/elfdeps.c
 elfdeps_LDADD =rpmio/librpmio.la
 elfdeps_LDADD +=   @WITH_LIBELF_LIB@ @WITH_POPT_LIB@
+
+rpmlibexec_PROGRAMS += sepdebugcrcfix
+sepdebugcrcfix_SOURCES = tools/sepdebugcrcfix.c
+sepdebugcrcfix_LDADD = @WITH_LIBELF_LIB@
 endif
 endif
 
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 8de7bad..c5d3ce6 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -114,10 +114,12 @@ done
 LISTFILE="$BUILDDIR/$out"
 SOURCEFILE="$BUILDDIR/debugsources.list"
 LINKSFILE="$BUILDDIR/debuglinks.list"
+ELFBINSFILE="$BUILDDIR/elfbins.list"
 
 > "$SOURCEFILE"
 > "$LISTFILE"
 > "$LINKSFILE"
+> "$ELFBINSFILE"
 
 debugdir="${RPM_BUILD_ROOT}/usr/lib/debug"
 
@@ -318,6 +320,8 @@ while read nlinks inum f; do
   # strip -g implies we have full symtab, don't add mini symtab in that case.
   $strip_g || ($include_minidebug && add_minidebug "${debugfn}" "$f")
 
+  echo "./${f#$RPM_BUILD_ROOT}" >> "$ELFBINSFILE"
+
   if [ -n "$id" ]; then
 make_id_link "$id" "$dn/$(basename $f)"
 make_id_link "$id" "/usr/lib/debug$dn/$bn" .debug
@@ -352,6 +356,11 @@ if $run_dwz && type dwz >/dev/null 2>&1 \
   [ -n "$id" ] \
&& make_id_link "$id" "/usr/lib/debug/.dwz/${dwz_multifile_name}" .debug
 fi
+
+# dwz invalidates .gnu_debuglink CRC32 in the main files.
+cat "$ELFBINSFILE" |
+(cd "$RPM_BUILD_ROOT"; \
+ xargs -d '\n' /usr/lib/rpm/sepdebugcrcfix usr/lib/debug)
   fi
 fi
 
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 2fea1b6..5672279 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -459,3 +459,39 @@ test "$canonmultiref" = "$canonmultifile" || exit 1
 [],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that old style gnu_debuglink CRC checksums are correct even after
+# using dwz to compress the debuginfo files.
+AT_SETUP([rpmbuild debuginfo dwz gnu_debuglink crc])
+AT_KEYWORDS([build] [debuginfo])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Build a package that
+cp "${abs_srcdir}"/data/SOURCES/hello-1.0.tar.gz 
"${abs_srcdir}"/data/SOURCES/hello-1.0-modernize.patch ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  -ba "${abs_srcdir}"/data/SPECS/hello2.spec
+
+# Unpack the main and debuginfo rpms so we can check binaries and .debug files.
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello2-debuginfo-1.0-1.*.rpm \
+  | cpio -diu
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello2-1.0-1.*.rpm \
+  | cpio -diu
+
+# Check that dwz has ran and a multi file has been produced
+test -f ./usr/lib/debug/.dwz/hello2-1.0-1.* || exit 1
+
+# Run sepdbugcrcfix on the binaries, both should have correct CRC already.
+${abs_top_builddir}/sepdebugcrcfix ./usr/lib/debug \
+  ./usr/local/bin/hello ./usr/local/bin/hello2 | grep CRC32 | cut -f2 -d:
+],
+[0],
+[ Updated 0 CRC32s, 2 CRC32s did match.
+],
+[ignore])
+AT_CLEANUP
diff --git a/tools/sepdebugcrcfix.c b/tools/sepdebugcrcfix.c
new file mode 100644
index 000..8e45abf
--- /dev/null
+++ b/tools/sepdebugcrcfix.c
@@ -0,0 +1,344 @@
+/* Copyright (C) 2013 Free Software Foundation, Inc.
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distr

[Rpm-maint] [PATCH 1/2] Add dwz debuginfo compression support.

[Mark Wielaard]

Support for dwz compression has been in Fedora since a couple of years.
https://fedoraproject.org/wiki/Features/DwarfCompressor

The original find-debuginfo.sh patch was written by Jakub Jelinek.
https://bugzilla.redhat.com/show_bug.cgi?id=833311
The new testcase using the macros.debug was added by me.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 macros.debug | 13 
 macros.in|  2 +-
 scripts/find-debuginfo.sh| 53 ++
 tests/data/SPECS/hello2.spec | 62 +++
 tests/rpmbuild.at| 78 
 5 files changed, 207 insertions(+), 1 deletion(-)
 create mode 100644 tests/data/SPECS/hello2.spec

diff --git a/macros.debug b/macros.debug
index bb2c02f..6a8432e 100644
--- a/macros.debug
+++ b/macros.debug
@@ -28,3 +28,16 @@
 
 # Should missing buildids terminate a build?
 %_missing_build_ids_terminate_build1
+
+# Number of debugging information entries (DIEs) above which
+# dwz will stop considering file for multifile optimizations
+# and enter a low memory mode, in which it will optimize
+# in about half the memory needed otherwise.
+%_dwz_low_mem_die_limit 1000
+# Number of DIEs above which dwz will stop processing
+# a file altogether.
+%_dwz_max_die_limit 5000
+
+%_find_debuginfo_dwz_opts --run-dwz\\\
+   --dwz-low-mem-die-limit %{_dwz_low_mem_die_limit}\\\
+   --dwz-max-die-limit %{_dwz_max_die_limit}
diff --git a/macros.in b/macros.in
index 58b122f..8e831c5 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_dwz_opts} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 5c2c381..8de7bad 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -4,6 +4,8 @@
 #
 # Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 # [-o debugfiles.list]
+# [--run-dwz] [--dwz-low-mem-die-limit N]
+# [--dwz-max-die-limit N]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
 #
@@ -20,6 +22,10 @@
 # The -p argument is an grep -E -style regexp matching the a file name,
 # and must not use anchors (^ or $).
 #
+# The --run-dwz flag instructs find-debuginfo.sh to run the dwz utility
+# if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit
+# provide detailed limits.  See dwz(1) -l and -L option for details.
+#
 # All file names in switches are relative to builddir (. if not given).
 #
 
@@ -35,6 +41,11 @@ include_minidebug=false
 # Barf on missing build IDs.
 strict=false
 
+# DWZ parameters.
+run_dwz=false
+dwz_low_mem_die_limit=
+dwz_max_die_limit=
+
 BUILDDIR=.
 out=debugfiles.list
 nout=0
@@ -43,6 +54,17 @@ while [ $# -gt 0 ]; do
   --strict-build-id)
 strict=true
 ;;
+  --run-dwz)
+run_dwz=true
+;;
+  --dwz-low-mem-die-limit)
+dwz_low_mem_die_limit=$2
+shift
+;;
+  --dwz-max-die-limit)
+dwz_max_die_limit=$2
+shift
+;;
   -g)
 strip_g=true
 ;;
@@ -302,6 +324,37 @@ while read nlinks inum f; do
   fi
 done || exit
 
+# Invoke the DWARF Compressor utility.
+if $run_dwz && type dwz >/dev/null 2>&1 \
+   && [ -d "${RPM_BUILD_ROOT}/usr/lib/debug" ]; then
+  dwz_files="`cd "${RPM_BUILD_ROOT}/usr/lib/debug"; find -type f -name 
\*.debug`"
+  if [ -n "${dwz_files}" ]; then
+
dwz_multifile_name="${RPM_PACKAGE_NAME}-${RPM_PACKAGE_VERSION}-${RPM_PACKAGE_RELEASE}.${RPM_ARCH}"
+dwz_multifile_suffix=
+dwz_multifile_idx=0
+while [ -f 
"${RPM_BUILD_ROOT}/usr/lib/debug/.dwz/${dwz_multifile_name}${dwz_multifile_suffix}"
 ]; do
+  let ++dwz_multifile_idx
+  dwz_multifile_suffix=".${dwz_multifile_idx}"
+done
+dwz_multfile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
+dwz_opts="-h -q -r -m .dwz/${dwz_multifile_name}"
+mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
+[ -n "${dwz_low_mem_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -l ${dwz_low_mem_die_limit}"
+[ -n "${dwz_max_die_limit}" ] \
+  && dwz_opts="${dwz_opts} -L ${dwz_max_die_limit}"
+( cd "${RPM_BUILD_ROOT}/usr/lib/debug" && dwz $dw

[Rpm-maint] [PATCH] Add find-debuginfo.sh -m minisymtab support.

[Mark Wielaard]

Support for minisymtab (a minimal function symbol table in a compressed
section in the main binary) has been in gdb and elfutils based tools
since some years. Fedora has had this as rpm-4.10.0-minidebuginfo.patch
since 2012. I added two testcases to check the basic support works and
for a bug found in an earlier version when strip -g was used.

macros has been adjusted to pass -m to find-debuginfo.sh when
_include_minidebuginfo has been set. find-debuginfo.sh now takes -m
as argument to generate the .gnu_debugdata ELF section to be added
to the main executable.

To support the testcase a new macros.debug is added that is used to
generate debuginfo packages in the rpmbuild.at testsuite.

The original rpm-4.10.0-minidebuginfo.patch in Fedora was created by
Panu Matilainen and Lubos Kardos.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   |  3 ++
 macros.debug  | 30 ++
 macros.in |  8 -
 scripts/find-debuginfo.sh | 37 ++-
 tests/rpmbuild.at | 77 +++
 5 files changed, 153 insertions(+), 2 deletions(-)
 create mode 100644 macros.debug

diff --git a/Makefile.am b/Makefile.am
index 939a6a0..157e79d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -198,6 +198,9 @@ macros: $(top_srcdir)/macros.in
 CLEANFILES += macros
 EXTRA_DIST += macros.in
 
+# Used for the testsuite to test creating debuginfo packages
+EXTRA_DIST += macros.debug
+
 noinst_DATA += platform
 platform: $(top_srcdir)/platform.in
@$(SED) \
diff --git a/macros.debug b/macros.debug
new file mode 100644
index 000..bb2c02f
--- /dev/null
+++ b/macros.debug
@@ -0,0 +1,30 @@
+# macros to include to generate debuginfo
+
+%_enable_debug_packages 1
+%_include_minidebuginfo 1
+
+# Expanded at end of %install scriptlet
+
+%__arch_install_post %{nil}
+
+%__os_install_post \
+%{_rpmconfigdir}/brp-compress \
+%{!?__debug_package:\
+%{_rpmconfigdir}/brp-strip %{__strip} \
+%{_rpmconfigdir}/brp-strip-comment-note %{__strip} %{__objdump} \
+} \
+%{_rpmconfigdir}/brp-strip-static-archive %{__strip} \
+%{nil}
+
+%__spec_install_post\
+%{?__debug_package:%{__debug_install_post}}\
+%{__arch_install_post}\
+%{__os_install_post}\
+%{nil}
+
+%install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
+%%install\
+%{nil}
+
+# Should missing buildids terminate a build?
+%_missing_build_ids_terminate_build1
diff --git a/macros.in b/macros.in
index f9172d0..58b122f 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -441,6 +441,12 @@ package or when debugging this package.\
 #%_missing_build_ids_terminate_build   1
 
 #
+# Include minimal debug information in build binaries.
+# Requires _enable_debug_packages.
+#
+#%_include_minidebuginfo   1
+
+#
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator1
 
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 17522e0..5c2c381 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -2,7 +2,7 @@
 #find-debuginfo.sh - automagically generate debug info and file list
 #for inclusion in an rpm spec file.
 #
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m]
 # [-o debugfiles.list]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
@@ -29,6 +29,9 @@ strip_g=false
 # with -r arg, pass --reloc-debug-sections to eu-strip.
 strip_r=false
 
+# with -m arg, add minimal debuginfo to binary.
+include_minidebug=false
+
 # Barf on missing build IDs.
 strict=false
 
@@ -43,6 +46,9 @@ while [ $# -gt 0 ]; do
   -g)
 strip_g=true
 ;;
+  -m)
+include_minidebug=true
+;;
   -o)
 if [ -z "${lists[$nout]}" -a -z "${ptns[$nout]}" ]; then
   out=$2
@@ -106,6 +112,32 @@ strip_to_debug()
   chmod 444 "$1" || exit
 }
 
+add_minidebug()
+{
+  local debuginfo="$1"
+  local binary="$2"
+
+  local dynsyms=`mktemp`
+  local funcsyms=`mktemp`
+  local keep_symbols=`mktemp`
+  local mini_debuginfo=`mktemp`
+
+  # Extract the dynamic symbols from the main binary, there is no need to also 
have these
+  # in the normal symbol table
+  nm -D "$binary" --format=posix --defined-only | awk '{ print $1 }' | sort > 
"$d

Re: [Rpm-maint] [PATCH] Add find-debuginfo.sh -m minisymtab support.

[Mark Wielaard]

On Wed, 2016-05-11 at 22:29 +0300, Panu Matilainen wrote:
> On 05/11/2016 05:41 PM, Mark Wielaard wrote:
> > The original rpm-4.10.0-minidebuginfo.patch in Fedora was created by
> > Panu Matilainen and Lubos Kardos.
> 
> Credit where credit is due: I merely added the patch into Fedora, 
> minidebuginfo was actually implemented by Alexander Larsson, the 
> original submission being here I think: 
> http://lists.rpm.org/pipermail/rpm-maint/2012-June/003214.html

Sorry, I should have remembered. I discussed things with Alex when I
implemented the elfutils support. My apologies.

I have updated the commit message to more accurately describe the
origin. Do you need a signed-off-by from Alex too? 

Alex, the patch below is what you submitted originally in the thread
above, plus a bug fix when using strip -g by Lubos and some macro tweaks
and two testcases for upstream rpm written by me.

Thanks,

Mark
From ed8f32d8cfe4eec4558c17ca66df8a642aa6e690 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@redhat.com>
Date: Wed, 11 May 2016 16:24:10 +0200
Subject: [PATCH] Add find-debuginfo.sh -m minisymtab support.

Support for minisymtab (a minimal function symbol table in a compressed
section in the main binary) has been in gdb and elfutils based tools
since some years. Fedora has had this as rpm-4.10.0-minidebuginfo.patch
since 2012.

The patch adjusts macros to pass -m to find-debuginfo.sh when
_include_minidebuginfo has been set. find-debuginfo.sh now takes -m
as argument to generate the .gnu_debugdata ELF section to be added
to the main executable.

To support the testcases a new macros.debug is added that is used to
generate debuginfo packages in the rpmbuild.at testsuite.

The original support was added to Fedora rpm by Alexander Larsson.
Lubos Kardos fixed a bug in it when strip -g was used. I added some
configuration macros and two testcases to check the basic support works
and for the strip -g bug.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   |  3 ++
 macros.debug  | 30 ++
 macros.in |  8 -
 scripts/find-debuginfo.sh | 37 ++-
 tests/rpmbuild.at | 77 +++
 5 files changed, 153 insertions(+), 2 deletions(-)
 create mode 100644 macros.debug

diff --git a/Makefile.am b/Makefile.am
index 939a6a0..157e79d 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -198,6 +198,9 @@ macros: $(top_srcdir)/macros.in
 CLEANFILES += macros
 EXTRA_DIST += macros.in
 
+# Used for the testsuite to test creating debuginfo packages
+EXTRA_DIST += macros.debug
+
 noinst_DATA += platform
 platform: $(top_srcdir)/platform.in
 	@$(SED) \
diff --git a/macros.debug b/macros.debug
new file mode 100644
index 000..bb2c02f
--- /dev/null
+++ b/macros.debug
@@ -0,0 +1,30 @@
+# macros to include to generate debuginfo
+
+%_enable_debug_packages 1
+%_include_minidebuginfo 1
+
+# Expanded at end of %install scriptlet
+
+%__arch_install_post %{nil}
+
+%__os_install_post \
+%{_rpmconfigdir}/brp-compress \
+%{!?__debug_package:\
+%{_rpmconfigdir}/brp-strip %{__strip} \
+%{_rpmconfigdir}/brp-strip-comment-note %{__strip} %{__objdump} \
+} \
+%{_rpmconfigdir}/brp-strip-static-archive %{__strip} \
+%{nil}
+
+%__spec_install_post\
+%{?__debug_package:%{__debug_install_post}}\
+%{__arch_install_post}\
+%{__os_install_post}\
+%{nil}
+
+%install %{?_enable_debug_packages:%{?buildsubdir:%{debug_package}}}\
+%%install\
+%{nil}
+
+# Should missing buildids terminate a build?
+%_missing_build_ids_terminate_build1
diff --git a/macros.in b/macros.in
index f9172d0..58b122f 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
 #	the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh %{?_missing_build_ids_terminate_build:--strict-build-id} %{?_include_minidebuginfo:-m} %{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #	Template for debug information sub-package.
@@ -441,6 +441,12 @@ package or when debugging this package.\
 #%_missing_build_ids_terminate_build	1
 
 #
+# Include minimal debug information in build binaries.
+# Requires _enable_debug_packages.
+#
+#%_include_minidebuginfo	1
+
+#
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator	1
 
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 17522e0..5c2c381 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -2,7 +2,7 @@
 #find-debuginfo.sh - automagically generate debug info and file list
 #for inclusion in an rpm spec file.
 #
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g]

Re: [Rpm-maint] [PATCH 0/3] find-debuginfo.sh speedup

[Mark Wielaard]

Hi,

On Fri, 2016-07-01 at 22:43 +0200, Michal Marek wrote:
> this series allows find-debuginfo.sh to run in parallel. It also makes the
> duplicate build-id links predictable.

The build-id link code was completely rewritten in one of the patches I
wrote some weeks ago:
http://lists.rpm.org/pipermail/rpm-maint/2016-June/004365.html

Could you look at how your changes interact with that? I think your "3/3
find-debuginfo.sh: Process files in parallel" patch should work as is
with that cleanup.

And do you have any testcases for this or could you check with the
testcases in the above patch set to see if it has the right behavior?
It adds a rpmbuildid.at that explicitly checks generating build-id
symlinks for binaries that are hard linked and/or have duplicate
build-ids.

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Add option to have unique debug source dirs across version/release/arch.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

Introduce a new macro _unique_debug_srcs that when set will pass
--unique-debug-src-base "%{name}" to find-debuginfo.sh which will
move sources into a unique "--." directory
under /usr/src/debug/ and makes debugedit rewrite the source paths
in the debuginfo to use that unique directory name.

Traditionally the debug src dir was named after the builddir which
was defined through the %setup macro which used the -n name argument
to define the builddir name and source archive to use. The builddir
might not be unique though between package versions.

Now that debugedit doesn't have strict base and dest dir length
restrictions for rewriting the source dir paths this can now be made
more flexible.

The added testcases show the difference between the old and new way.
The hello2.spec file defines the name of the package as hello2, but
uses the %setup marcro with -n hello-1.0 to use the hello-1.0.tar.gz
archive. This would traditionally result in a hello-1.0 builddir
which would be moved under /usr/src/debug. Possibly conflicting
with any other package (version) that used the same builddir name.
When defining _unique_debug_srcs to 1 that builddir will be moved
to --. instead (hello2-1.0-1.).

The testcases check that both the actual package source filess under
/usr/debug/src/ and the source paths as found in the .debug files are
under the traditional or new unique directory names depending on whether
the new _unique_debug_srcs macro is defined.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 macros.in |  8 -
 scripts/find-debuginfo.sh | 39 ---
 tests/rpmbuild.at | 80 +++
 tests/rpmbuildid.at   |  5 +++
 4 files changed, 127 insertions(+), 5 deletions(-)

diff --git a/macros.in b/macros.in
index a838875..72d4a51 100644
--- a/macros.in
+++ b/macros.in
@@ -172,7 +172,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh %{?_smp_mflags} 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i} 
%{?_unique_build_ids:--ver-rel "%{VERSION}-%{RELEASE}"} 
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh %{?_smp_mflags} 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i} 
%{?_unique_build_ids:--ver-rel "%{VERSION}-%{RELEASE}"} 
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"} 
%{?_unique_debug_srcs:--unique-debug-src-base "%{name}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -505,6 +505,12 @@ package or when debugging this package.\
 # Requires _unique_build_ids.
 %_unique_debug_names   1
 
+# Whether the /usr/debug/src/ directories should be unique between
+# package version, release and architecture. If set to 1 this will pass
+# --unique-debug-src-base "%{name}" to find-debuginfo.sh to name the
+# directory under /usr/debug/src as --.
+%_unique_debug_srcs1
+
 #
 # Use internal dependency generator rather than external helpers?
 %_use_internal_dependency_generator1
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 1524ac2..fd63779 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -67,6 +67,9 @@ ver_rel=
 # Arch given by --unique-debug-arch
 unique_debug_arch=
 
+# Base given by --unique-debug-src-base
+unique_debug_src_base=
+
 # Number of parallel jobs to spawn
 n_jobs=1
 
@@ -97,6 +100,10 @@ while [ $# -gt 0 ]; do
 unique_debug_arch=$2
 shift
 ;;
+  --unique-debug-src-base)
+unique_debug_src_base=$2
+shift
+;;
   -g)
 strip_g=true
 ;;
@@ -147,6 +154,11 @@ if test -z "$ver_rel" -a -n "$unique_debug_arch"; then
   exit 2
 fi
 
+if test -z "$unique_debug_arch" -a -n "$unique_debug_src_base"; then
+  echo >&2 "*** ERROR: --unique-debug-src-base (${unique_debug_src_base}) 
needs --unique-debug-arch (${unique_debug_arch})"
+  exit 2
+fi
+
 i=0
 while ((i < nout)); do
   outs[$i]="$BUILDDIR/${outs[$i]}"
@@ -305,7 +317,18 @@ do_file()
   if [ ! -z "$ver_rel" ]; then
 build_id_seed="--build-id-seed=$ver_rel"
   fi
-  id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
+  # See also cpio SOURCEFILE copy. Directories must match up.
+  debug_base_name="$RPM_BUILD_DIR"
+  debug_dest_name="/usr/src/debug"
+  if [ ! -z "$unique_debug_src_base" ]; then
+debug_base_name="$BUILDDIR"
+
debug_dest_name=&

Re: [Rpm-maint] [PATCH] Add option to have unique debug source dirs across version/release/arch.

[Mark Wielaard]

On Tue, 2017-02-28 at 21:34 +0100, Mark Wielaard wrote:
> @@ -305,7 +317,18 @@ do_file()
>if [ ! -z "$ver_rel" ]; then
>  build_id_seed="--build-id-seed=$ver_rel"
>fi
> -  id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
> +  # See also cpio SOURCEFILE copy. Directories must match up.
> +  debug_base_name="$RPM_BUILD_DIR"
> +  debug_dest_name="/usr/src/debug"
> +  if [ ! -z "$unique_debug_src_base" ]; then
> +debug_base_name="$BUILDDIR"
> +
> debug_dest_name="/usr/src/debug/${unique_debug_src_base}-${ver_rel}.${unique_debug_arch}"
> +  fi
> +echo "RPM_BUILD_DIR: $RPM_BUILD_DIR"
> +echo "BUILDDIR: $BUILDDIR"
> +  echo ${lib_rpm_dir}/debugedit -b $debug_base_name -d $debug_dest_name \
> +   -i $build_id_seed -l "$SOURCEFILE" "$f"
> +  id=$(${lib_rpm_dir}/debugedit -b $debug_base_name -d $debug_dest_name \
> -i $build_id_seed -l "$SOURCEFILE" "$f") || exit
>if [ -z "$id" ]; then
>  echo >&2 "*** ${strict_error}: No build ID note found in $f"

Sorry, those three extra echo lines before the actual debugedit
invocation obviously shouldn't have been there. They were just for
debugging and I forgot to remove them in the final patch.


___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Include new test data spec files in EXTRA_DIST.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

Commit bbfe1f8 (Add build-id links to rpm for all ELF files) and
Commit bbfe1f8 (Make it possible to have unique build-ids across build
versions/releases)
Introduced new test spec files (hello-r2.spec, hello2cp.spec and
hello2ln.spec). Make sure they are added to EXTRA_DIST so the testcases
pass again with make distcheck.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 8c036d6..815a390 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -35,8 +35,11 @@ EXTRA_DIST += $(TESTSUITE_AT)
 ## testsuite data
 EXTRA_DIST += data/SPECS/attrtest.spec
 EXTRA_DIST += data/SPECS/hello.spec
+EXTRA_DIST += data/SPECS/hello-r2.spec
 EXTRA_DIST += data/SPECS/hello-script.spec
 EXTRA_DIST += data/SPECS/hello2.spec
+EXTRA_DIST += data/SPECS/hello2cp.spec
+EXTRA_DIST += data/SPECS/hello2ln.spec
 EXTRA_DIST += data/SPECS/hello2-suid.spec
 EXTRA_DIST += data/SPECS/foo.spec
 EXTRA_DIST += data/SPECS/globtest.spec
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Include new test data spec files in EXTRA_DIST.

[Mark Wielaard]

On Wed, 2017-03-01 at 15:28 +0100, Mark Wielaard wrote:
> From: Mark Wielaard <m...@klomp.org>
> 
> Commit bbfe1f8 (Add build-id links to rpm for all ELF files) and
> Commit bbfe1f8 (Make it possible to have unique build-ids across build
> versions/releases)

Sorry, copy/paste error in commit IDs. Correct IDs in the fixed up
commit.

From 509059d3c2ecfb969ec020dae04e7e335dd5bafc Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@klomp.org>
Date: Wed, 1 Mar 2017 15:24:41 +0100
Subject: [PATCH] Include new test data spec files in EXTRA_DIST.

Commit bbfe1f8 (Add build-id links to rpm for all ELF files) and
Commit 5ef1166 (Make it possible to have unique build-ids across build
versions/releases)
Introduced new test spec files (hello-r2.spec, hello2cp.spec and
hello2ln.spec). Make sure they are added to EXTRA_DIST so the testcases
pass again with make distcheck.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 8c036d6..815a390 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -35,8 +35,11 @@ EXTRA_DIST += $(TESTSUITE_AT)
 ## testsuite data
 EXTRA_DIST += data/SPECS/attrtest.spec
 EXTRA_DIST += data/SPECS/hello.spec
+EXTRA_DIST += data/SPECS/hello-r2.spec
 EXTRA_DIST += data/SPECS/hello-script.spec
 EXTRA_DIST += data/SPECS/hello2.spec
+EXTRA_DIST += data/SPECS/hello2cp.spec
+EXTRA_DIST += data/SPECS/hello2ln.spec
 EXTRA_DIST += data/SPECS/hello2-suid.spec
 EXTRA_DIST += data/SPECS/foo.spec
 EXTRA_DIST += data/SPECS/globtest.spec
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] debugedit: Support String/Line table rewriting for larger/smaller paths.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

debugedit --base to --dest rewriting of debug source file paths only
supported dest paths that were smaller or equal than the base path
(and the size should differ more than 1 character for correct debug lines).
All paths were changed "in place". Which could in theory mess up debug str
sharing.

This rewrite supports base and dest strings of any size (some limitations,
see below). This is done by reconstructing the debug_str and debug_line
tables and updating the references in the debug_info attributes pointing
to these tables. Plus, if necessary (only for ET_REL kernel modules),
updating any relocations for the debug_info and debug_line sections.

This has the nice benefit of merging any duplicate strings in the
debug_str table which might resulting on slightly smaller files.
kernel modules are ET_REL files that often contain a lot of duplicate
strings.

The rewrite uses elfutils (either libebl or libdw) to reconstruct the
debug_str table. Since we are changing some section sizes now we cannot
just use mmap and rawdata to poke the values, but need to read in and
write out the changed sections. This does take a bit more memory because
we now also need to keep track of all string/line references.

There are still some limitations (already in the original debugedit)
not fixed by this rewrite:
- DW_AT_comp_dir in .debug_info using DW_FORM_string can not be made
  larger. We only warn about that now instead of failing. The only
  producer of DW_FORM_string comp_dirs is binutils gas. It seems simpler
  to fix gas than to try to support resizing the debug_info section.
- A DW_AT_name on a DW_TAG_compile_unit is only rewritten for DW_FORM_strp
  not for DW_FORM_string. Probably no problem in practice since this
  wasn't supported originally either.
- The debug_line program isn't scanned for DW_LNE_define_file which
  could in theory define an absolute path that might need rewriting.
  Again probably not a problem because this wasn't supported before
  and there are no know producers for this construct.

To support the upcoming DWARFv5 in gcc 7 (not on by default), we will
need to add support for the new debug_line format and scan the new
debug_macro section that can have references to the debug_str table.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 Makefile.am   |8 +-
 configure.ac  |6 +
 tools/debugedit.c | 1569 -
 3 files changed, 1330 insertions(+), 253 deletions(-)

diff --git a/Makefile.am b/Makefile.am
index b99da12..aeb2c33 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -154,13 +154,18 @@ rpm2archive_LDADD +=  @WITH_BEECRYPT_LIB@ 
@WITH_NSS_LIB@ @WITH_POPT_LIB@ @WITH_ZL
 
 if LIBELF
 if LIBDWARF
+if LIBDW
 rpmconfig_SCRIPTS += scripts/find-debuginfo.sh
 
 rpmlibexec_PROGRAMS += debugedit
 debugedit_SOURCES =tools/debugedit.c tools/hashtab.c tools/hashtab.h
 debugedit_LDADD =  rpmio/librpmio.la
 debugedit_LDADD += @WITH_LIBELF_LIB@ @WITH_POPT_LIB@
-
+if HAVE_LIBDW_STRTAB
+debugedit_LDADD += @WITH_LIBDW_LIB@
+else
+debugedit_LDADD += @WITH_LIBDW_LIB@ -lebl
+endif
 rpmlibexec_PROGRAMS += elfdeps
 elfdeps_SOURCES =  tools/elfdeps.c
 elfdeps_LDADD =rpmio/librpmio.la
@@ -171,6 +176,7 @@ sepdebugcrcfix_SOURCES = tools/sepdebugcrcfix.c
 sepdebugcrcfix_LDADD = @WITH_LIBELF_LIB@
 endif
 endif
+endif
 
 rpmlibexec_PROGRAMS += rpmdeps
 rpmdeps_SOURCES =  tools/rpmdeps.c
diff --git a/configure.ac b/configure.ac
index 4f3be87..5c24ac6 100644
--- a/configure.ac
+++ b/configure.ac
@@ -371,18 +371,24 @@ AM_CONDITIONAL(WITH_ARCHIVE,[test "$with_archive" = yes])
 #=
 # Check for elfutils libdw library with dwelf_elf_gnu_build_id.
 WITH_LIBDW_LIB=
+HAVE_LIBDW_STRTAB=
 AS_IF([test "$WITH_LIBELF" = yes],[
   AC_CHECK_HEADERS([elfutils/libdwelf.h],[
+# dwelf_elf_gnu_build_id was introduced in elfutils 0.159
 AC_CHECK_LIB(dw, dwelf_elf_gnu_build_id, [
   AC_DEFINE(HAVE_LIBDW, 1,
 [Define to 1 if you have elfutils libdw library])
   WITH_LIBDW_LIB="-ldw"
   WITH_LIBDW=yes
+  # If possible we also want the strtab functions from elfutils 0.167.
+  # But we can fall back on the (unsupported) ebl alternatives if not.
+  AC_CHECK_LIB(dw, dwelf_strtab_init, [HAVE_LIBDW_STRTAB=yes])
 ])
   ])
 ])
 AC_SUBST(WITH_LIBDW_LIB)
 AM_CONDITIONAL(LIBDW,[test "$WITH_LIBDW" = yes])
+AM_CONDITIONAL(HAVE_LIBDW_STRTAB,[test "$HAVE_LIBDW_STRTAB" = yes])
 
 #=
 # Process --with/without-external-db
diff --git a/tools/debugedit.c b/tools/debugedit.c
index c0147f0..4798c63 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -1,6 +1,7 @@
-/* Copyright (C) 2001-2003, 2005, 2007, 2009-2011, 2016 Red Hat, Inc.
+/* Copyright (C) 2001-2003, 2005, 2007, 2009-2011, 2016, 2017 Red Hat, Inc.
Written by Alexander Larsson <al...@redhat.com>

[Rpm-maint] [PATCH] Fix misleading-indentation in rpmplugins.c

[Mark Wielaard]

GCC6 will warn about:

lib/rpmplugins.c: In function ‘rpmpluginsCallInit’:
lib/rpmplugins.c:217:5: warning: this ‘if’ clause does not guard...
 if (hookFunc)
 ^~
lib/rpmplugins.c:219:9: note: ...this statement, but the latter is misleadingly 
indented as if it is guarded by the ‘if’
 if (rc != RPMRC_OK && rc != RPMRC_NOTFOUND)
 ^~

GCC is right, this is misleading and causes the if statement to be executed
even when it isn't necessary. It doesn't actually cause wrong results
because rc is initialized to RPMRC_OK earlier and can only change when
hookFunc != NULL.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 lib/rpmplugins.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lib/rpmplugins.c b/lib/rpmplugins.c
index 1022024..97e5d30 100644
--- a/lib/rpmplugins.c
+++ b/lib/rpmplugins.c
@@ -214,10 +214,11 @@ static rpmRC rpmpluginsCallInit(rpmPlugin plugin, rpmts 
ts)
 rpmRC rc = RPMRC_OK;
 plugin_init_func hookFunc;
 RPMPLUGINS_SET_HOOK_FUNC(init);
-if (hookFunc)
+if (hookFunc) {
 rc = hookFunc(plugin, ts);
 if (rc != RPMRC_OK && rc != RPMRC_NOTFOUND)
 rpmlog(RPMLOG_ERR, "Plugin %s: hook init failed\n", plugin->name);
+}
 return rc;
 }
 
-- 
2.7.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Fix mini-symtab in find-debuginfo.sh for arches with function descriptors.

[Mark Wielaard]

add_minidebug uses nm to select the function symbols to include in the
mini-symtab table. But on arches that use function descriptors (like ppc64)
nm --format-posix doesn't make it clear which symbols are real functions
The symbols point to the (stripped away) function descriptor table).

Use --format=sysv style to match the ELF symbol type directly instead of
using the somewhat ambiguous symbol type char used in --format=posix style
in binutils nm.

https://bugzilla.redhat.com/show_bug.cgi?id=1052415

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 scripts/find-debuginfo.sh | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 2016222..5f6e0bb 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -197,7 +197,10 @@ add_minidebug()
   # in the normal symbol table
   nm -D "$binary" --format=posix --defined-only | awk '{ print $1 }' | sort > 
"$dynsyms"
   # Extract all the text (i.e. function) symbols from the debuginfo
-  nm "$debuginfo" --format=posix --defined-only | awk '{ if ($2 == "T" || $2 
== "t") print $1 }' | sort > "$funcsyms"
+  # Use format sysv to make sure we can match against the actual ELF FUNC
+  # symbol type. The binutils nm posix format symbol type chars are
+  # ambigous for architectures that might use function descriptors.
+  nm "$debuginfo" --format=sysv --defined-only | awk -F \| '{ if ($4 ~ "FUNC") 
print $1 }' | sort > "$funcsyms"
   # Keep all the function symbols not already in the dynamic symbol table
   comm -13 "$dynsyms" "$funcsyms" > "$keep_symbols"
   # Copy the full debuginfo, keeping only a minumal set of symbols and 
removing some unnecessary sections
-- 
2.7.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Don't copy extra sections into .gnu_debugdata.

[Mark Wielaard]

When creating the compressed mini-symtab section in find-debuginfo
add_minidebug we explicitly remove .gdb_index and .comment. But there
can be other non-empty sections in the debuginfo that shouldn't be
copied. For example rust binaries might have a .rustc section.

Explicitly remove any non-allocated PROGBITS or NOTE sections.

https://bugzilla.redhat.com/show_bug.cgi?id=1382394

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 scripts/find-debuginfo.sh | 10 +-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 5f6e0bb..1524ac2 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -193,6 +193,14 @@ add_minidebug()
   local keep_symbols=`mktemp`
   local mini_debuginfo=`mktemp`
 
+  # In the minisymtab we don't need the .debug_ sections (already removed
+  # by -S) but also not any other non-allocated PROGBITS or NOTE sections.
+  # List and remove them explicitly. We do want to keep the allocated,
+  # symbol and NOBITS sections so cannot use --keep-only because that is
+  # too agressive. Field $2 is the section name, $3 is the section type
+  # and $8 are the section flags.
+  local remove_sections=`readelf -W -S "$debuginfo" | awk '{ if 
(index($2,".debug_") != 1 && ($3 == "PROGBITS" || $3 == "NOTE") && 
index($8,"A") == 0) printf "--remove-section "$2" " }'`
+
   # Extract the dynamic symbols from the main binary, there is no need to also 
have these
   # in the normal symbol table
   nm -D "$binary" --format=posix --defined-only | awk '{ print $1 }' | sort > 
"$dynsyms"
@@ -204,7 +212,7 @@ add_minidebug()
   # Keep all the function symbols not already in the dynamic symbol table
   comm -13 "$dynsyms" "$funcsyms" > "$keep_symbols"
   # Copy the full debuginfo, keeping only a minumal set of symbols and 
removing some unnecessary sections
-  objcopy -S --remove-section .gdb_index --remove-section .comment 
--keep-symbols="$keep_symbols" "$debuginfo" "$mini_debuginfo" &> /dev/null
+  objcopy -S $remove_sections --keep-symbols="$keep_symbols" "$debuginfo" 
"$mini_debuginfo" &> /dev/null
   #Inject the compressed data into the .gnu_debugdata section of the original 
binary
   xz "$mini_debuginfo"
   mini_debuginfo="${mini_debuginfo}.xz"
-- 
2.7.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Fix libdw configure check.

[Mark Wielaard]

commit a82119 "configure.ac: use LIBDW always conditionally" contained
a typo that caused WITH_LIBDW_LIB never to be set when you were using
libelf. Fixed by reverting the "!=" to "=" again.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 configure.ac | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/configure.ac b/configure.ac
index 6935c6e..6ece8c9 100644
--- a/configure.ac
+++ b/configure.ac
@@ -370,7 +370,7 @@ AM_CONDITIONAL(WITH_ARCHIVE,[test "$with_archive" = yes])
 #=
 # Check for elfutils libdw library with dwelf_elf_gnu_build_id.
 WITH_LIBDW_LIB=
-AS_IF([test "$WITH_LIBELF" != yes],[
+AS_IF([test "$WITH_LIBELF" = yes],[
   AC_CHECK_HEADERS([elfutils/libdwelf.h],[
 AC_CHECK_LIB(dw, dwelf_elf_gnu_build_id, [
   AC_DEFINE(HAVE_LIBDW, 1,
-- 
2.7.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] RPM 4.13.0 rc2 released

[Mark Wielaard]

On Fri, 2016-10-21 at 12:33 +0300, Panu Matilainen wrote:
> On 10/21/2016 12:09 PM, Thierry Vignaud wrote:
> > On 21 October 2016 at 08:51, Panu Matilainen  wrote:
> >
>  Please test and report any anomalies. From rc2 to final only regression 
>  fixes will be allowed. Unless of course something really strange comes 
>  up, you never know. Or I forgot something absolutely vital. THAT would 
>  not count as a strange incident though.
> >>>
> >>>
> >>> Humm Testsuite won't pass
> >>> 2 tests are failing because of missing files...
> >>> Please apply the attached patch
> >>> It fixes:
> >>> 84: rpmbuild debuginfo dwz  FAILED 
> >>> (rpmbuild.at:396)
> >>> 86: rpmbuild implicit suid binary   FAILED 
> >>> (rpmbuild.at:509)
> >>
> >>
> >> Ah, I did run the test-suite but not from the created tarball. One more 
> >> thing to remember when cutting releases. Or rather *cough* to document 
> >> *cough*.

In theory make distcheck should take care of that (I haven't actually
tried if rpm is make distcheck clean though).

> >> Applied (with a slightly expanded comments).
> >
> > If you could cherry-pick in rpm-4.13.x branch too, that would be nice :-)
> 
> I will, eventually. Lets see what other things turn up first.

Sorry for missing adding the new files to Makefile.am when adding those
new tests.

> >>> After that, 2 unexpected failures remain:
> >>>
> >>>  85: rpmbuild debuginfo dwz gnu_debuglink crcFAILED 
> >>> (rpmbuild.at:468)
> >>
> >>
> >> ...but this I had just missed. It appears to be cured by commit
> >> cherry-picking commit 3929ce4d58ab18c810b9a776f5c49d56cb436026.
> >
> > This commit doen't exist in git
> > The attached patch fixes it anyway
> 
> Sorry, should've been 41c4dcf507e2208585d6dc0952f59686a3a69d69, which 
> commit 4ec7c396fb9464a3fcff006408871f8175ab169b apparently depends on.

Yes, that commit is necessary to make sure that tests don't need to run
under fakechroot to pass. I had much trouble with fakechroot and so
wanted to avoid adding more tests that depended on it. It also helps if
you want to install rpm under some other path than /usr/lib/rpm.

Cheers,

mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] Build fixes related to (#108)

[Mark Wielaard]

On Thu, Dec 08, 2016 at 11:57:50PM -0800, DNF Bot wrote:
> Can one of the admins verify this patch?

Could you please include the patch you want to see verified in
your message?

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Link testing/usr/bin to testing/bin.

[Mark Wielaard]

Some tests running under fakechroot started failing because scripts tried
to execute /usr/bin/rm but inside the testing chroot /usr/bin and /bin
aren't linked. Just symlink testing/usr/bin to testing/bin to simulate
a unified /usr setup.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 tests/Makefile.am | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7091220..8c036d6 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -104,6 +104,8 @@ testing$(bindir)/rpmbuild: ../rpmbuild
rm -rf testing
mkdir -p testing/$(bindir)
ln -s ./$(bindir) testing/bin
+   mkdir -p testing/usr
+   ln -s ../bin testing/usr/bin
(cd ${top_builddir} && \
  $(MAKE) DESTDIR=`pwd`/${subdir}/testing install)
cp -r ${srcdir}/data/ testing/
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] Unable to build after Stop messing with CFLAGS from configure patch

[Mark Wielaard]

Hi,

commit bd3eb213 "Stop messing with CFLAGS from configure" renamed
rpm.am to rpm.am.in. rpm.am is included am into all Makefile.am files.
But rpm.am is now generated by configure. Which means it is generated
in the builddir so the includes should reference top_builddir instead of
top_srcdir now. But when running automake (or autoreconf) configure
doesn't even exist yet, so there is no rpm.am file yet and automake
will fail to generate proper Makefile.in files.

Could we instead do it like the attached by just defining the
AM_CFLAGS substitution inside the Makefile.am files themselves?

Thanks,

Mark
>From 7a17a1eeb058cbafe59863ea40e8b0f697ca6bc8 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@redhat.com>
Date: Fri, 9 Dec 2016 09:31:58 +0100
Subject: [PATCH] Define AM_CFLAGS inside the Makefile.am files themselves.

Trying to include AM_CFLAGS through a configure generated rpm.am file
doesn't really work because at the time automake runs configure doesn't
exist yet to process rpm.am.in. Just define the AM_CFLAGS substitution
inside the Makefile.am files themselves.

Rename rpm.am.in back to rpm.am.

Signed-off-by: Mark Wielaard <m...@redhat.com>
---
 Makefile.am   | 1 +
 build/Makefile.am | 1 +
 configure.ac  | 2 +-
 fileattrs/Makefile.am | 1 +
 lib/Makefile.am   | 1 +
 plugins/Makefile.am   | 1 +
 python/Makefile.am| 1 +
 rpm.am.in => rpm.am   | 2 --
 rpmio/Makefile.am | 1 +
 scripts/Makefile.am   | 1 +
 sign/Makefile.am  | 1 +
 tests/Makefile.am | 1 +
 12 files changed, 11 insertions(+), 3 deletions(-)
 rename rpm.am.in => rpm.am (92%)

diff --git a/Makefile.am b/Makefile.am
index 47bb4bd..eef2376 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -5,6 +5,7 @@ ACLOCAL_AMFLAGS = -I m4
 DISTCHECK_CONFIGURE_FLAGS = --with-external-db
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 CLEANFILES =
 EXTRA_DIST = ChangeLog CREDITS INSTALL \
diff --git a/build/Makefile.am b/build/Makefile.am
index f0f55a9..2fa3c3d 100644
--- a/build/Makefile.am
+++ b/build/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpmbuild library.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 AM_CPPFLAGS =  -I$(top_builddir) -I$(top_srcdir) -I$(top_builddir)/include/
 AM_CPPFLAGS += @WITH_BEECRYPT_INCLUDE@
diff --git a/configure.ac b/configure.ac
index e08096d..4f3be87 100644
--- a/configure.ac
+++ b/configure.ac
@@ -892,7 +892,7 @@ AC_PATH_PROG(AUTOM4TE,autom4te,:)
 
 AC_SUBST([dirstamp],[\${am__leading_dot}dirstamp])
 
-AC_CONFIG_FILES([Makefile rpm.am
+AC_CONFIG_FILES([Makefile
rpmio/Makefile lib/Makefile build/Makefile sign/Makefile
po/Makefile.in scripts/Makefile fileattrs/Makefile
misc/Makefile 
diff --git a/fileattrs/Makefile.am b/fileattrs/Makefile.am
index 9c55aa1..1895f44 100644
--- a/fileattrs/Makefile.am
+++ b/fileattrs/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm file attributes
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 fattrsdir = $(rpmconfigdir)/fileattrs
 
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 7fd6f94..c95d674 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm library.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir) -I$(top_builddir)/include/
 AM_CPPFLAGS += @WITH_BEECRYPT_INCLUDE@
diff --git a/plugins/Makefile.am b/plugins/Makefile.am
index 5ddc174..98eb154 100644
--- a/plugins/Makefile.am
+++ b/plugins/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm library.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir) -I$(top_builddir)/include/
 AM_CPPFLAGS += -I$(top_srcdir)/misc
diff --git a/python/Makefile.am b/python/Makefile.am
index cae66f0..af8b089 100644
--- a/python/Makefile.am
+++ b/python/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm library.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 EXTRA_DIST = rpm/__init__.py rpm/transaction.py
 
diff --git a/rpm.am.in b/rpm.am
similarity index 92%
rename from rpm.am.in
rename to rpm.am
index c91bebf..1f43ad8 100644
--- a/rpm.am.in
+++ b/rpm.am
@@ -8,5 +8,3 @@ rpmconfigdir = $(prefix)/lib/rpm
 
 # Libtool version (current-revision-age) for all our libraries
 rpm_version_info = 7:0:0
-
-AM_CFLAGS = @RPMCFLAGS@
diff --git a/rpmio/Makefile.am b/rpmio/Makefile.am
index 68a8219..5ad4d68 100644
--- a/rpmio/Makefile.am
+++ b/rpmio/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm library.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 AM_CPPFLAGS = -I$(top_builddir) -I$(top_srcdir) -I$(top_builddir)/include/
 AM_CPPFLAGS += @WITH_NSS_INCLUDE@
diff --git a/scripts/Makefile.am b/scripts/Makefile.am
index f204c51..b1360e1 100644
--- a/scripts/Makefile.am
+++ b/scripts/Makefile.am
@@ -1,6 +1,7 @@
 # Makefile for rpm scripts.
 
 include $(top_srcdir)/rpm.am
+AM_CFLAGS = @RPMCFLAGS@
 
 CLEANFILES =
 
diff --git a/sign/Makefile.am b/sign/Makefile.am
index 863

Re: [Rpm-maint] [rpm-software-management/rpm] Build fixes related to (#108)

[Mark Wielaard]

Hi rpm-maint,

On Thu, 2016-12-08 at 15:08 -0800, Thomas Petazzoni wrote:
> You can view, comment on, or merge this pull request online at:
> 
>   https://github.com/rpm-software-management/rpm/pull/108

I am not sure what to make of this mail message, or how to properly
reply. It is somewhat inconvenient to not have the patch to review in
the message or attached but having to click-through and download it
separately. Assuming these emails are generated automatically could they
be changed to just include the patch.

>   * Detect bfd.h to enable/disable sepdebugcrcfix building
>   * tools/sepdebugcrcfix.c: fix build with recent binutils

I think the reliance on bfd.h was a mistake. The code was lifted from
bfd, but should be totally independent (it just calculates a CRC). The
attached fixes the type to be a normal size_t and include sys/stat.h
(which was included through bfd.h) to get the definitions of stat and
chmod.

Cheers,

Mark

diff --git a/tools/sepdebugcrcfix.c b/tools/sepdebugcrcfix.c
index cd7fa02..1ce4f61 100644
--- a/tools/sepdebugcrcfix.c
+++ b/tools/sepdebugcrcfix.c
@@ -20,6 +20,7 @@
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -28,7 +29,6 @@
 #include 
 #include 
 #include 
-#include 
 
 #define _(x) x
 #define static_assert(expr) \
@@ -46,7 +46,7 @@ static const bool false = 0, true = 1;
 static unsigned long
 calc_gnu_debuglink_crc32 (unsigned long crc,
 			  const unsigned char *buf,
-			  bfd_size_type len)
+			  size_t len)
 {
   static const unsigned long crc32_table[256] =
 {
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Unbreak short-circuited binary builds

[Mark Wielaard]

On Thu, Jan 05, 2017 at 03:40:42PM +0200, Panu Matilainen wrote:
> > > When checking for pre-existing links see if they already point to
> > > the right file and in that case just reuse it instead of creating new 
> > > ones.
> > 
> > > @@ -1572,6 +1573,16 @@ static int addNewIDSymlink(FileList fl,
> > >  origpath = linkpath;
> > > 
> > >  while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 
> > > 0) {
> > > + char ltarget[PATH_MAX];
> > > + ssize_t llen;
> > > + /* In short-circuited builds the link might already exist  */
> > > + if ((llen = readlink(linkpath, ltarget, sizeof(ltarget)-1)) != -1) {
> > > + ltarget[llen] = '\0';
> > > + if (rstreq(ltarget, targetpath)) {
> > > + exists = 1;
> > > + break;
> > > + }
> > > + }
> > 
> > Allocating PATH_MAX size on the stack might be a bit much
> > (and on some systems PATH_MAX might not exist). Might be
> > good to use lstat first to see if it is a symbolic link
> > and use the st_size i+1 as buffer size. I would malloc/free
> > it to not accidentially blow up the stack.
> 
> Yeah, using PATH_MAX is broken but then rpm is getting away with it (on
> stack and all) in many other places so one more here and there... but good
> point using lstat() there, will change it to use that, thanks.

Ah, indeed. rpm does allocated PATH_MAX bufs on the stack already.
O well. One day we will clean this all up.

> Other than that - does it look like doing the right thing to you? I'm really
> not familiar with the debuginfo business at all :)

You made me look at the whole thing in context!
(Spoiler. Yes, I think the patch is correct. But first I am going
to bore you by explaining the whole function and why it is called
in the first place.)

OK. addNewIDSymlink () is called from generateBuildIDs () at the
end of processPackageFiles () for the FileList. It is called for
each (regular) loadable ELF image file that contains a build-id.

The symlinks are named after the (hex) build-id so they actual
ELF image file associated with the build-id can be easily found.
But they are not all put in the same directory, the first two
hex-digits are used as sub-directory say they are spread out
over 255 different directories. There are two kinds of build-id
symlinks. Those that point to the actual ELF image and those
(ending in .debug) pointing at the associated separate debuginfo
file.

Both kind of build-id files used to be put together under
/usr/lib/debug/.buildid/... directories in the debuginfo
package. But this causes strange issues when the debuginfo
package isn't installed, if the main package and debuginfo
package are out of sync or if the debuginfo package is
installed, but the main package isn't.

So with the latest rpm the main build-id files are put in
their own separate directories in the main (sub)-package
under /usr/lib/.build-id/ with the .debug build-ids still in
the -debuginfo package under /usr/lib/debug/.build-id/. There
is also a compatibility setting %_build_id_links compat that
puts an extra symlink under /usr/lib/debug/.build-id that
just points to the /usr/lib/.build-id symlink. See the comments
in macros.in for the different settings for %_build_id_links.

So addNewIDSymlink () is called with the FileList, the target
(which is the actual ELF file), the idlinkpath (which includes
the whole prefix and 2-digit hex subdir) and whether a debug
or compat link is requested. If it is a debug build-id link
then .debug is added to the idlinkpath.

The build-ids should be globally unique. Some of my recent
patches makes sure that is always the case between packages
even if they differ only very slightly by hashing in the nvr
into the build-id put in the ELF images (through debugedit).
But a package could still generate two identical ELF images
by having the the same ELF file in the package (either hard
linked or copied). So then addNewIDSymlink checks (in a loop)
if the idlinkpath already exists. If so it adds a counter
(.) to the linkpath.  If it is a compat link then both
the target and link get the same counter.

Then, since duplicate build-ids really shouldn't exist, if
this wasn't a compat link, a warning is generated about the
files that contain the same build-id unless the files were
hard links.

So summing up the above I think your patch is correct.
The loop that checks if the idlinkpath already exists should
indeed check that if it exists it points to a different target
than expected. Normally this would indeed never happen.
Or can a FileList contain the same file more than once?
But in a short-circuited build it will already have been
created.

I would just change the comment to explicitly say
"Check if the symlink already points to the expected target."

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Unbreak short-circuited binary builds

[Mark Wielaard]

On Thu, Jan 05, 2017 at 01:47:28PM +0200, Panu Matilainen wrote:
> Commit bbfe1f86b2e4b5c0bd499d9f3dd9de9c9c20fff2 broke short-circuited
> binary builds (which can be handy for testing when working on large
> packages), eg:
>  rpmbuild -bi foo.spec; rpmbuild -bb --short-circuit foo.spec
> 
> The problem is that in a short-circuited build all the links already
> exist and point to the right place, but the code doesn't realize this
> and creates new links instead, which leaves the old links unowned
> in the buildroot which ultimately causes the build to fail with
> "Installed (but unpackaged) file(s) found" for the previously created
> build-id links.

Sorry about that. I didn't even know rpmbuild --short-circuit existed.
Looks like there are lots of subtle ways this can get broken. Might be
good to add a testcase for it.

> When checking for pre-existing links see if they already point to
> the right file and in that case just reuse it instead of creating new ones.

> @@ -1572,6 +1573,16 @@ static int addNewIDSymlink(FileList fl,
>  origpath = linkpath;
>  
>  while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 0) {
> + char ltarget[PATH_MAX];
> + ssize_t llen;
> + /* In short-circuited builds the link might already exist  */
> + if ((llen = readlink(linkpath, ltarget, sizeof(ltarget)-1)) != -1) {
> + ltarget[llen] = '\0';
> + if (rstreq(ltarget, targetpath)) {
> + exists = 1;
> + break;
> + }
> + }

Allocating PATH_MAX size on the stack might be a bit much
(and on some systems PATH_MAX might not exist). Might be
good to use lstat first to see if it is a symbolic link
and use the st_size i+1 as buffer size. I would malloc/free
it to not accidentially blow up the stack.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Only process regular files when generating build-ids

[Mark Wielaard]

On Thu, Jan 05, 2017 at 12:27:52PM +0200, Panu Matilainen wrote:
> It looks commit bbfe1f86b2e4b5c0bd499d9f3dd9de9c9c20fff2 intends to skip
> symlinks since it filters on S_ISREG(), but since uses fstat()
> on already open()'ed file it ends up stat()'ing the symlink target.
> Flip stat() + open() around and use lstat() instead to fix it.

Oops. You are right.
I think I was trying to guard for TOCTOU (time of check time of use)
issues (where the file might change between the stat check and the open
call.

I think just changing fstat into an lstat without flipping the
open/stat around would also do the right thing (given that the lstat
will be on the diskPath not the fd). But that would be the TOCTOU in
reverse of course. And I assume that if the build dir is still
changing at this point we have other issues creating the file list.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Link testing/usr/bin to testing/bin.

[Mark Wielaard]

On Fri, Dec 09, 2016 at 11:36:19AM +0100, Mark Wielaard wrote:
> Some tests running under fakechroot started failing because scripts tried
> to execute /usr/bin/rm but inside the testing chroot /usr/bin and /bin
> aren't linked. Just symlink testing/usr/bin to testing/bin to simulate
> a unified /usr setup.

Ping.

> Signed-off-by: Mark Wielaard <m...@redhat.com>
> ---
>  tests/Makefile.am | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/tests/Makefile.am b/tests/Makefile.am
> index 7091220..8c036d6 100644
> --- a/tests/Makefile.am
> +++ b/tests/Makefile.am
> @@ -104,6 +104,8 @@ testing$(bindir)/rpmbuild: ../rpmbuild
>   rm -rf testing
>   mkdir -p testing/$(bindir)
>   ln -s ./$(bindir) testing/bin
> + mkdir -p testing/usr
> + ln -s ../bin testing/usr/bin
>   (cd ${top_builddir} && \
> $(MAKE) DESTDIR=`pwd`/${subdir}/testing install)
>   cp -r ${srcdir}/data/ testing/
> -- 
> 1.8.3.1
> 
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH 2/5] Only build bundled fts if system has a bad version that doesn't handle LFS

[Mark Wielaard]

On Fri, 2017-03-24 at 11:15 +0200, Panu Matilainen wrote:
> On 03/23/2017 08:21 PM, Gleb Fotengauer-Malinovskiy wrote:
> > diff --git a/configure.ac b/configure.ac
> > index bdcb741..687d58c 100644
> > --- a/configure.ac
> > +++ b/configure.ac
> > @@ -717,6 +717,9 @@ AC_CHECK_FUNCS(
> > [mkstemp getcwd basename dirname realpath setenv unsetenv regcomp 
> > lchown utimes],
> > [], [AC_MSG_ERROR([function required by rpm])])
> >
> > +AC_CHECK_HEADERS([fts.h])
> > +AM_CONDITIONAL([USE_BUNDLED_FTS_KLUDGE], [test "$ac_cv_header_fts_h" = no])
> > +
> >  AC_LIBOBJ(fnmatch)
> >
> 
> How exactly does this ensure the system fts.h is LFS compatible?

The configure.ac has an AC_SYS_LARGEFILE fairly early. Which will define
_FILE_OFFSET_BITS 64 on 32-bit systems that need it. So on such systems
when AC_CHECK_HEADERS([fts.h]) is ran it will have that define in the
conftest.c and the test compile will fail because old fts.h have an
explicit check (error out on) -D_FILE_OFFSET_BITS==64.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH v2] Unbreak short-circuited binary builds (RhBug:1434235)

[Mark Wielaard]

On Wed, 2017-03-22 at 14:37 +0200, Panu Matilainen wrote:
> Guess there's some unhandled corner left still :)

Yeah, that was because my fix was bogus :{

It didn't construct the linkpath correctly (because it was relative). It
checked with readlink (), but that doesn't fully expand the links, it
should have used canonicalize_file_name () on both the link and the
target (because the target is itself a symlink). And even then it
wouldn't have worked because we don't know which duplicate the compat
link is targeting, so the first existing target would always win.

So back to the drawing board. The problem is the compat links in case
there are duplicate build-ids. We don't want the compat links to
"duplicate" the duplication search already done for the buildids already
done. So in this new patch instead of saying we want to create a compat
link when calling addNewBuildIDSymlink, we ask to search (and return)
the number of duplicates found. We can then use that to create the
compat links directly (with the correct duplication counter).

Attached is your original patch plus my don't do duplication search for
compat links change. That makes the testsuite pass and the popt
short-circuit example work for me.

Cheers,

Mark
From e6c0dbf9c6e56bfae6a596929b69d4474c3d69f1 Mon Sep 17 00:00:00 2001
From: Panu Matilainen <pmati...@redhat.com>
Date: Thu, 5 Jan 2017 13:47:28 +0200
Subject: [PATCH] Unbreak short-circuited binary builds

Commit bbfe1f86b2e4b5c0bd499d9f3dd9de9c9c20fff2 broke short-circuited
binary builds (which can be handy for testing when working on large
packages), eg:
 rpmbuild -bi foo.spec; rpmbuild -bb --short-circuit foo.spec

The problem is that in a short-circuited build all the links already
exist and point to the right place, but the code doesn't realize this
and creates new links instead, which leaves the old links unowned
in the buildroot which ultimately causes the build to fail with
"Installed (but unpackaged) file(s) found" for the previously created
build-id links.

When checking for pre-existing links see if they already point to
the right file and in that case just reuse it instead of creating new ones.
Keep track of duplicate build-ids found by noticing existing links that
point to different targets. But don't do this for compat links, they should
just point to the last (duplicate) main build-id symlink found.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 71 ++-
 1 file changed, 51 insertions(+), 20 deletions(-)

diff --git a/build/files.c b/build/files.c
index cca14b9..93021d1 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1592,11 +1592,12 @@ exit:
 
 static int addNewIDSymlink(FileList fl,
 			   char *targetpath, char *idlinkpath,
-			   int isDbg, int isCompat)
+			   int isDbg, int *dups)
 {
 const char *linkerr = _("failed symlink");
 int rc = 0;
 int nr = 0;
+int exists = 0;
 char *origpath, *linkpath;
 
 if (isDbg)
@@ -1606,6 +1607,26 @@ static int addNewIDSymlink(FileList fl,
 origpath = linkpath;
 
 while (faccessat(AT_FDCWD, linkpath, F_OK, AT_SYMLINK_NOFOLLOW) == 0) {
+/* We don't care about finding dups for compat links, they are
+	   OK as is.  Otherwise we will need to double check if
+	   existing link points to the correct target. */
+	if (dups == NULL)
+	  {
+	exists = 1;
+	break;
+	  }
+
+	char ltarget[PATH_MAX];
+	ssize_t llen;
+	/* In short-circuited builds the link might already exist  */
+	if ((llen = readlink(linkpath, ltarget, sizeof(ltarget)-1)) != -1) {
+	ltarget[llen] = '\0';
+	if (rstreq(ltarget, targetpath)) {
+		exists = 1;
+		break;
+	}
+	}
+
 	if (nr > 0)
 	free(linkpath);
 	nr++;
@@ -1613,21 +1634,16 @@ static int addNewIDSymlink(FileList fl,
 		  isDbg ? ".debug" : "");
 }
 
-char *symtarget = targetpath;
-if (nr > 0 && isCompat)
-	rasprintf (, "%s.%d", targetpath, nr);
-
-if (symlink(symtarget, linkpath) < 0) {
+if (!exists && symlink(targetpath, linkpath) < 0) {
 	rc = 1;
 	rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
-	   linkerr, linkpath, symtarget);
+	   linkerr, linkpath, targetpath);
 } else {
 	fl->cur.isDir = 0;
 	rc = addFile(fl, linkpath, NULL);
 }
 
-/* Don't warn (again) if this is a compat id-link, we retarget it. */
-if (nr > 0 && !isCompat) {
+if (nr > 0) {
 	/* Lets see why there are multiple build-ids. If the original
 	   targets are hard linked, then it is OK, otherwise warn
 	   something fishy is going on. Would be nice to call
@@ -1656,8 +1672,8 @@ static int addNewIDSymlink(FileList fl,
 	free(origpath);
 if (nr > 0)
 	free(linkpath);
-if (nr > 0 && isCompat)
-	free(symtarget);
+if (dups != NULL)
+  *dups = nr;
 
 return rc;
 }
@@ -1897,6 +1913,7 @@ static int generate

[Rpm-maint] [PATCH] build/files.c (generateBuildIDs): Fix small memory leak.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

mainiddir and debugiddir are allocated through rpmGetPath ()
and should be released when done.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/build/files.c b/build/files.c
index 93021d1..d7b140e 100644
--- a/build/files.c
+++ b/build/files.c
@@ -2022,6 +2022,8 @@ static int generateBuildIDs(FileList fl)
free(paths[i]);
free(ids[i]);
}
+   free(mainiddir);
+   free(debugiddir);
free(vra);
free(paths);
free(ids);
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] debugedit: Fix cross-endian build-id reading and updating section data.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

debugedit doesn't read raw mmap data any longer. Which made the complex
way to read the build-id unnecessary (and it was broken for cross-endian).
Just use gelf_getnote to read the notes.

Also in some special cases when only the debug_info or build_id data
was updated, but no section changed size and we had to preserve the
allocated section headers we could hit a bug in elfutils that could
trash some section data in case there were gaps between non-dirty and
dirty sections. See https://sourceware.org/bugzilla/show_bug.cgi?id=21199
Add a workaround for that issue.

This fixes the kompose package build on fedora ppc64.
And makes it possible to replicate that issue on x86_64.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tools/debugedit.c | 63 +++
 1 file changed, 31 insertions(+), 32 deletions(-)

diff --git a/tools/debugedit.c b/tools/debugedit.c
index 4798c63..47e5bbf 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -2581,40 +2581,25 @@ main (int argc, char *argv[])
  break;
case SHT_NOTE:
  if (do_build_id
- && build_id == NULL && (dso->shdr[i].sh_flags & SHF_ALLOC))
+ && build_id == 0 && (dso->shdr[i].sh_flags & SHF_ALLOC))
{
  /* Look for a build-ID note here.  */
+ size_t off = 0;
+ GElf_Nhdr nhdr;
+ size_t name_off;
+ size_t desc_off;
  Elf_Data *data = elf_getdata (elf_getscn (dso->elf, i), NULL);
- Elf32_Nhdr nh;
- Elf_Data dst =
-   {
- .d_version = EV_CURRENT, .d_type = ELF_T_NHDR,
- .d_buf = , .d_size = sizeof nh
-   };
- Elf_Data src = dst;
- src.d_buf = data->d_buf;
- assert (sizeof (Elf32_Nhdr) == sizeof (Elf64_Nhdr));
- while ((char *) data->d_buf + data->d_size -
-(char *) src.d_buf > (int) sizeof nh
-&& elf32_xlatetom (, , dso->ehdr.e_ident[EI_DATA]))
-   {
- Elf32_Word len = sizeof nh + nh.n_namesz;
- len = (len + 3) & ~3;
-
- if (nh.n_namesz == sizeof "GNU" && nh.n_type == 3
- && !memcmp ((char *) src.d_buf + sizeof nh, "GNU", sizeof 
"GNU"))
-   {
- build_id = data;
- build_id_offset = (char *) src.d_buf + len -
-   (char *) data->d_buf;
- build_id_size = nh.n_descsz;
- break;
-   }
-
- len += nh.n_descsz;
- len = (len + 3) & ~3;
- src.d_buf = (char *) src.d_buf + len;
-   }
+ while ((off = gelf_getnote (data, off,
+ , _off, _off)) > 0)
+   if (nhdr.n_type == NT_GNU_BUILD_ID
+   && nhdr.n_namesz == sizeof "GNU"
+   && (memcmp ((char *)data->d_buf + name_off, "GNU",
+   sizeof "GNU") == 0))
+ {
+   build_id = data;
+   build_id_offset = desc_off;
+   build_id_size = nhdr.n_descsz;
+ }
}
  break;
default:
@@ -2622,6 +2607,20 @@ main (int argc, char *argv[])
}
 }
 
+  /* Normally we only need to explicitly update the section headers
+ and data when any section data changed size. But because of a bug
+ in elfutils before 0.169 we will have to update and write out all
+ section data if any data has changed (when ELF_F_LAYOUT was
+ set). https://sourceware.org/bugzilla/show_bug.cgi?id=21199 */
+  bool need_update = need_strp_update || need_stmt_update;
+
+#if !_ELFUTILS_PREREQ (0, 169)
+  /* string replacements or build_id updates don't change section size. */
+  need_update = (need_update
+|| need_string_replacement
+|| (do_build_id && build_id != NULL));
+#endif
+
   /* We might have changed the size of some debug sections. If so make
  sure the section headers are updated and the data offsets are
  correct. We set ELF_F_LAYOUT above because we don't want libelf
@@ -2631,7 +2630,7 @@ main (int argc, char *argv[])
  anything for the phdrs allocated sections. Keep the offset of
  allocated sections so they are at the same place in the file. Add
  unallocated ones after the allocated ones. */
-  if (dso->phnum != 0 && (need_strp_update || need_stmt_update))
+  if (dso->phnum != 0 && need_update)
 {
   Elf *elf = dso->elf;
   GElf_Off last_offset;
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] error: Missing build-id in /builddir/build/BUILDROOT/... (#177)

[Mark Wielaard]

On Wed, 2017-03-15 at 20:50 +0100, Mark Wielaard wrote:
> I couldn't replicate this with the current fedora guile-2.0.14-1
> package. There the .go files are not actual ELF files. So rpm will not
> complain about them not having a build-id. If that has changed in with
> guile 2.1.8 and the .go files are ELF binaries now and it is intended
> that they don't have a build-id then you could work around it by
> undefining %_missing_build_ids_terminate_build.
> 
> I'll try with the src.rpm you provided.

And indeed it seems guile switched to producing ELF files.
But note the following from
https://wingolog.org/archives/2014/01/19/elf-in-guile

Note that although Guile uses ELF on all platforms, we do not
use platform support for ELF. Guile implements its own linker
and loader. The advantage of using ELF is not sharing code, but
sharing ideas. ELF is simply a well-designed object file format.

So rpm is correct to warn that the ELF images don't contain build-ids.
As long as guile doesn't produce them you will need to add something
like the following to your spec file:

# Guile produces ELF images that are just containers for guile and don't
# include build-ids. 
https://wingolog.org/archives/2014/01/19/elf-in-guile
%undefine _missing_build_ids_terminate_build

That lets me rebuild your new guile srpm.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] build/files.c: Only check build-ids for executable files in the main package.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

generateBuildIDs should mimic what find-debuginfo.sh does. Only check
build-ids for executable files in non-debuginfo packages. This moves the
isDbg check up so the is executeble check can be done when the file is
part of the main package.

This fixes the build of qemu and uboot-tools in fedora. Both ship
non-executable ELF bios files in architecture specific packages.
https://bugzilla.redhat.com/show_bug.cgi?id=1433837

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 19 ++-
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/build/files.c b/build/files.c
index 6021643..afa01cd 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1711,6 +1711,19 @@ static int generateBuildIDs(FileList fl)
 for (i = 0, flp = fl->files.recs; i < fl->files.used; i++, flp++) {
struct stat sbuf;
if (lstat(flp->diskPath, ) == 0 && S_ISREG (sbuf.st_mode)) {
+   /* We determine whether this is a main or
+  debug ELF based on path.  */
+   #define DEBUGPATH "/usr/lib/debug/"
+   int isDbg = strncmp (flp->cpioPath,
+DEBUGPATH, strlen (DEBUGPATH)) == 0;
+
+   /* For the main package files mimic what find-debuginfo.sh does.
+  Only check build-ids for executable files. Debug files are
+  always non-executable. */
+   if (!isDbg
+   && (sbuf.st_mode & (S_IXUSR | S_IXGRP | S_IXOTH)) == 0)
+ continue;
+
int fd = open (flp->diskPath, O_RDONLY);
if (fd >= 0) {
/* Only real ELF files, that are ET_EXEC, ET_DYN or
@@ -1732,12 +1745,8 @@ static int generateBuildIDs(FileList fl)
   is 128 bits) and 64 bytes (largest sha3 is 512
   bits), common is 20 bytes (sha1 is 160 bits). */
if (len >= 16 && len <= 64) {
-   /* We determine whether this is a main or
-  debug ELF based on path.  */
-   #define DEBUGPATH "/usr/lib/debug/"
int addid = 0;
-   if (strncmp (flp->cpioPath,
-DEBUGPATH, strlen (DEBUGPATH)) == 0) {
+   if (isDbg) {
needDbg = 1;
addid = 1;
}
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] tests/rpmbuildid.at: Make file sed regexp more strict to extract BuildID.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

Like commit f0a5819 for rpmbuild.at. In the case of rpmbuildid.at the
sed expression looked to work, but only matched by accident. Make the sed
regexp more strict by only matching a hex-string. And properly "escape"
[ and ] which inside an AT_CHECK should be [[ and ]].

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/rpmbuildid.at | 44 ++--
 1 file changed, 22 insertions(+), 22 deletions(-)

diff --git a/tests/rpmbuildid.at b/tests/rpmbuildid.at
index 15c0620..1c06ca1 100644
--- a/tests/rpmbuildid.at
+++ b/tests/rpmbuildid.at
@@ -97,7 +97,7 @@ main_file=./usr/local/bin/hello
 test -f "${main_file}" || echo "No main file ${main_file}"
 
 # Extract the build-id from the main file
-id_main=$(file $main_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_main=$(file $main_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 id_main_file="./usr/lib/debug/.build-id/${id_main:0:2}/${id_main:2}"
 test -L "$id_main_file" || echo "No build-id file $id_main_file"
@@ -120,7 +120,7 @@ debug_file=./usr/lib/debug/usr/local/bin/hello.debug
 test -f ${debug_file} || echo "No debug file ${debug_file}"
 
 # Extract the build-id from the .debug file
-id_debug=$(file $debug_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_debug=$(file $debug_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test ${id_main} = ${id_debug} || echo "unequal main and debug id"
 
@@ -190,7 +190,7 @@ main_file=./usr/local/bin/hello
 test -f "${main_file}" || echo "No main file ${main_file}"
 
 # Extract the build-id from the main file
-id_main=$(file $main_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_main=$(file $main_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 id_main_file="./usr/lib/debug/.build-id/${id_main:0:2}/${id_main:2}"
 test -L "$id_main_file" || echo "No build-id file $id_main_file"
@@ -213,7 +213,7 @@ debug_file=./usr/lib/debug/usr/local/bin/hello-*.debug
 test -f ${debug_file} || echo "No debug file ${debug_file}"
 
 # Extract the build-id from the .debug file
-id_debug=$(file $debug_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_debug=$(file $debug_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test ${id_main} = ${id_debug} || echo "unequal main and debug id"
 
@@ -283,7 +283,7 @@ main_file=./usr/local/bin/hello
 test -f "${main_file}" || echo "No main file ${main_file}"
 
 # Extract the build-id from the main file
-id_main=$(file $main_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_main=$(file $main_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 id_main_file="./usr/lib/.build-id/${id_main:0:2}/${id_main:2}"
 test -L "$id_main_file" || echo "No build-id file $id_main_file"
@@ -306,7 +306,7 @@ debug_file=./usr/lib/debug/usr/local/bin/hello.debug
 test -f ${debug_file} || echo "No debug file ${debug_file}"
 
 # Extract the build-id from the .debug file
-id_debug=$(file $debug_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_debug=$(file $debug_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test ${id_main} = ${id_debug} || echo "unequal main and debug id"
 
@@ -375,7 +375,7 @@ main_file=./usr/local/bin/hello
 test -f "${main_file}" || echo "No main file ${main_file}"
 
 # Extract the build-id from the main file
-id_main=$(file $main_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_main=$(file $main_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 id_main_file="./usr/lib/.build-id/${id_main:0:2}/${id_main:2}"
 test -L "$id_main_file" || echo "No build-id file $id_main_file"
@@ -398,7 +398,7 @@ debug_file=./usr/lib/debug/usr/local/bin/hello-*.debug
 test -f ${debug_file} || echo "No debug file ${debug_file}"
 
 # Extract the build-id from the .debug file
-id_debug=$(file $debug_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_debug=$(file $debug_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test ${id_main} = ${id_debug} || echo "unequal main and debug id"
 
@@ -469,7 +469,7 @@ main_file=./usr/local/bin/hello
 test -f "${main_file}" || echo "No main file ${main_file}"
 
 # Extract the build-id from the main file
-id_main=$(file $main_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id_main=$(file $main_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 id_main_file="./usr/lib/.build-id/${id_main:0:2}/${id_main:2}"
 test -L "$id_main_file" || echo "No build-id file $id_main_file"
@@ -492,7 +492,7 @@ debug_file=./usr/lib/debug/usr/local/bin/hello.debug
 test -f ${debug_file} || echo "No debug file ${debug_file}"
 
 # Extract the build-id from the .debug file
-id_d

[Rpm-maint] [PATCH] debugedit: Fix edit_dwarf2_line replace_dirs -> replace_files typo.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

We wouldn't replace the changed file names if replace_dirs was false,
but replace_files was true. This could overrun the new debug_line data
buffer if the original file name was larger than the replacement. It
wasn't found before because often when we need to replace files we
also would have to replace dirs.

This fixes the kubernetes build in fedora.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tools/debugedit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/debugedit.c b/tools/debugedit.c
index 4798c63..87a423f 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -1185,7 +1185,7 @@ edit_dwarf2_line (DSO *dso)
{
  const char *file = (const char *) optr;
  const char *file_path = NULL;
- if (t->replace_dirs)
+ if (t->replace_files)
{
  file_path = skip_dir_prefix (file, base_dir);
  if (file_path != NULL)
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] build/files.c (processPackageFiles): Don't call generateBuildIDs for noarch.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

We don't want to do build-id processing for noarch packages. It might be
that noarch packages do contain architecture depended files, but those are
already handled by processBinaryFiles.

This fixes the building of openbios in fedora.
https://bugzilla.redhat.com/show_bug.cgi?id=1433129

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/build/files.c b/build/files.c
index 6021643..35b2dd0 100644
--- a/build/files.c
+++ b/build/files.c
@@ -2384,10 +2384,14 @@ static rpmRC processPackageFiles(rpmSpec spec, 
rpmBuildPkgFlags pkgFlags,
goto exit;
 
 #if HAVE_LIBDW
-if (generateBuildIDs () != 0) {
-   rpmlog(RPMLOG_ERR, _("Generating build-id links failed\n"));
-   fl.processingFailed = 1;
-   goto exit;
+/* Check build-ids and add build-ids links for files to package list. */
+const char *arch = headerGetString(pkg->header, RPMTAG_ARCH);
+if (!rstreq(arch, "noarch")) {
+   if (generateBuildIDs () != 0) {
+   rpmlog(RPMLOG_ERR, _("Generating build-id links failed\n"));
+   fl.processingFailed = 1;
+   goto exit;
+   }
 }
 #endif
 
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Make sure to reset file attributes for generated build-id directories.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

[Note this patch is currently being tested in Fedora. See bug below.]

When creating the build-id directories we should reset the file attributes
to the defaults. Otherwise if the file list contained an %attr or %defattr
the directories would come out with the wrong mode.

Includes a testcase based on a spec by Igor Gnatenko that fails before
and Check that build-id directories are created with the right permissions
even if the spec file sets attrs explicitly.

https://bugzilla.redhat.com/show_bug.cgi?id=1432372

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c| 21 +++
 tests/Makefile.am|  2 ++
 tests/data/SOURCES/hello.c   |  8 +++
 tests/data/SPECS/hello-attr-buildid.spec | 27 
 tests/rpmbuildid.at  | 36 
 5 files changed, 94 insertions(+)
 create mode 100644 tests/data/SOURCES/hello.c
 create mode 100644 tests/data/SPECS/hello-attr-buildid.spec

diff --git a/build/files.c b/build/files.c
index 6021643..9479e6e 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1797,6 +1797,7 @@ static int generateBuildIDs(FileList fl)
char *mainiddir = NULL;
char *debugiddir = NULL;
if (rc == 0) {
+   char *attrstr;
/* Add .build-id directories to hold the subdirs/symlinks.  */
 #define BUILD_ID_DIR "/usr/lib/.build-id"
 #define DEBUG_ID_DIR "/usr/lib/debug/.build-id"
@@ -1804,6 +1805,18 @@ static int generateBuildIDs(FileList fl)
mainiddir = rpmGetPath(fl->buildRoot, BUILD_ID_DIR, NULL);
debugiddir = rpmGetPath(fl->buildRoot, DEBUG_ID_DIR, NULL);
 
+   /* Make sure to reset all file flags to defaults.
+  Uses parseForAttr to reset ar, arFlags, and specdFlags.
+  Note that parseForAttr pokes at the attrstr, so we cannot
+  just pass a static string. */
+   fl->def.verifyFlags = RPMVERIFY_ALL;
+   fl->cur.verifyFlags = RPMVERIFY_ALL;
+   fl->def.specdFlags |= SPECD_VERIFY;
+   fl->cur.specdFlags |= SPECD_VERIFY;
+   attrstr = xstrdup ("%defattr(-,root,root)");
+   parseForAttr(fl->pool, attrstr, 1, >def);
+   free (attrstr);
+
/* Supported, but questionable.  */
if (needMain && needDbg)
rpmlog(RPMLOG_WARNING,
@@ -1813,8 +1826,12 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(mainiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, mainiddir);
} else {
+   rasprintf (, "%s %s", "%attr(-,root,root) ",
+  mainiddir);
+   parseForAttr(fl->pool, attrstr, 0, >cur);
fl->cur.isDir = 1;
rc = addFile(fl, mainiddir, NULL);
+   free (attrstr);
}
}
 
@@ -1822,8 +1839,12 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(debugiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, debugiddir);
} else {
+   rasprintf (, "%s %s", "%attr(-,root,root) ",
+  debugiddir);
+   parseForAttr(fl->pool, attrstr, 0, >cur);
fl->cur.isDir = 1;
rc = addFile(fl, debugiddir, NULL);
+   free (attrstr);
}
}
}
diff --git a/tests/Makefile.am b/tests/Makefile.am
index 815a390..ad41957 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -69,6 +69,8 @@ EXTRA_DIST += data/RPMS/hello-2.0-1.i686.rpm
 EXTRA_DIST += data/RPMS/hello-2.0-1.x86_64.rpm
 EXTRA_DIST += data/SRPMS/foo-1.0-1.src.rpm
 EXTRA_DIST += data/SRPMS/hello-1.0-1.src.rpm
+EXTRA_DIST += data/SOURCES/hello.c
+EXTRA_DIST += data/SPECS/hello-attr-buildid.spec
 
 # testsuite voodoo
 AUTOTEST = $(AUTOM4TE) --language=autotest
diff --git a/tests/data/SOURCES/hello.c b/tests/data/SOURCES/hello.c
new file mode 100644
index 000..020484e
--- /dev/null
+++ b/tests/data/SOURCES/hello.c
@@ -0,0 +1,8 @@
+#include 
+
+int
+main (void)
+{
+  printf ("Hello, world!\n");
+  return 0;
+}
diff --git a/tests/data/SPECS/hello-attr-buildid.spec 
b/tests/data/SPECS/hello-attr-buildid.spec
new file mode 100644
index 000..cdab95f
--- /dev/null
+++ b/tests/data/SPECS/hello-attr-buildid.spec
@@ -0,0 +1,27 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%prep
+%autosetup -c -D -T
+cp -

Re: [Rpm-maint] [rpm-software-management/rpm] error: Missing build-id in /builddir/build/BUILDROOT/... (#177)

[Mark Wielaard]

On Wed, 2017-03-15 at 10:10 -0700, John Dulaney wrote:
> When scratch building a source rpm (0) in koji (1), I receive the following 
> error in the build log:
> 
> error: Missing build-id in 
> /builddir/build/BUILDROOT/guile-2.1.8-1.fc27.x86_64/usr/lib64/guile/2.2/ccache/language/elisp/spec.go
> error: Missing build-id in 
> /builddir/build/BUILDROOT/guile-2.1.8-1.fc27.x86_64/usr/lib64/guile/2.2/ccache/language/elisp/falias.go
> error: Missing build-id in 
> /builddir/build/BUILDROOT/guile-2.1.8-1.fc27.x86_64/usr/lib64/guile/2.2/ccache/language/elisp/runtime/function-slot.go
> 
> I will attempt to reproduce by rebuilding the same srpm (2).
> 
> (0)  https://st.jdulaney.com/guile-2.1.8-1.fc25.src.rpm
> (1)  https://kojipkgs.fedoraproject.org//work/tasks/1877/18391877/build.log
> (2)  https://koji.fedoraproject.org/koji/taskinfo?taskID=18400786

Thanks for the report.
I couldn't replicate this with the current fedora guile-2.0.14-1
package. There the .go files are not actual ELF files. So rpm will not
complain about them not having a build-id. If that has changed in with
guile 2.1.8 and the .go files are ELF binaries now and it is intended
that they don't have a build-id then you could work around it by
undefining %_missing_build_ids_terminate_build.

I'll try with the src.rpm you provided.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Make rpmsign tests work for builddir != srcdir.

[Mark Wielaard]

The gpg HOME is in the builddir testing directory.
But the keys to import are in the srcdir data/keys directory.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 6adc709..5fa06eb 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -110,7 +110,7 @@ EXTRA_DIST += atlocal.in
 # Hack: Abusing testing$(bindir)/rpmbuild as stamp file
 # The chmod is needed when copying from read-only sources (eg in distcheck)
 testing$(bindir)/rpmbuild: ../rpmbuild
-   HOME=$(abs_srcdir)/testing gpg-connect-agent --no-autostart killagent 
bye ||:
+   HOME=$(abs_builddir)/testing gpg-connect-agent --no-autostart killagent 
bye ||:
rm -rf testing
mkdir -p testing/$(bindir)
ln -s ./$(bindir) testing/bin
@@ -126,7 +126,7 @@ testing$(bindir)/rpmbuild: ../rpmbuild
for prog in gzip cat patch tar sh ln chmod rm mkdir uname grep sed find 
file ionice mktemp nice cut sort diff touch; do p=`which $${prog}`; ln -s $${p} 
testing/$(bindir)/; done
for d in /proc /sys /selinux /etc/selinux; do if [ -d $${d} ]; then ln 
-s $${d} testing/$${d}; fi; done
(cd testing/magic && file -C)
-   HOME=$(abs_srcdir)/testing gpg2 --import data/keys/*.secret || 
HOME=$(abs_srcdir)/testing gpg --import data/keys/*.secret
+   HOME=$(abs_builddir)/testing gpg2 --import 
${abs_srcdir}/data/keys/*.secret || HOME=$(abs_builddir)/testing gpg --import 
${abs_srcdir}/data/keys/*.secret
 
 check_DATA = atconfig atlocal $(TESTSUITE)
 check_DATA += testing$(bindir)/rpmbuild
-- 
2.9.3

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] debugedit: Only output comp_dir under build dir (once).

[Mark Wielaard]

The fix for rhbz#444310 (commit c1a5eb - Include empty CU current dirs)
was a little greedy. It would also include comp_dirs outside the build
root. Those are unnecessary and we don't have a good way to store them.
Such dirs (e.g. /tmp) would then show up at the root of /usr/src/debug.

Fix this by including only comp_dirs under base_dir. Also only output
all dirs once (during phase zero) and don't output empty dirs (which
was harmless but would produce a warning from cpio).

This still includes all empty dirs from the original rhbz#444310
nodir testcase and it is an alternative fix for rhbz#641022
(commit c707ab).

Both fixes are necessary in case of an unexpected mode for a directory
actually in the build root that we want to include in the source list.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tools/debugedit.c | 39 ---
 1 file changed, 16 insertions(+), 23 deletions(-)

diff --git a/tools/debugedit.c b/tools/debugedit.c
index 8444e03..bf11513 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -1680,30 +1680,23 @@ edit_attributes (DSO *dso, unsigned char *ptr, struct 
abbrev_tag *t, int phase)
   /* Ensure the CU current directory will exist even if only empty.  Source
  filenames possibly located in its parent directories refer relatively to
  it and the debugger (GDB) cannot safely optimize out the missing
- CU current dir subdirectories.  */
-  if (comp_dir && list_file_fd != -1)
+ CU current dir subdirectories.  Only do this once in phase one. And
+ only do this for dirs under our build/base_dir.  Don't output the
+ empty string (in case the comp_dir == base_dir).  */
+  if (phase == 0 && base_dir && comp_dir && list_file_fd != -1)
 {
-  const char *p = NULL;
-  size_t size;
-
-  if (base_dir)
-   {
- p = skip_dir_prefix (comp_dir, base_dir);
- if (p == NULL && dest_dir != NULL)
-   p = skip_dir_prefix (comp_dir, dest_dir);
-   }
-
-  if (p == NULL)
-   p = comp_dir;
-
-  size = strlen (p) + 1;
-  while (size > 0)
-   {
- ssize_t ret = write (list_file_fd, p, size);
- if (ret == -1)
-   break;
- size -= ret;
- p += ret;
+  const char *p = skip_dir_prefix (comp_dir, base_dir);
+  if (p != NULL && p[0] != '\0')
+{
+ size_t size = strlen (p) + 1;
+ while (size > 0)
+   {
+ ssize_t ret = write (list_file_fd, p, size);
+ if (ret == -1)
+   break;
+ size -= ret;
+ p += ret;
+   }
}
 }
 
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] debugedit: Add -n, --no-recompute-build-id.

[Mark Wielaard]

Some packages depend on the build-ids as generated during the build
and cannot handle rpmbuild recomputing them before generating the
package file list. Add -n, --no-recompute-build-id to debugedit and
add -n to find-debuginfo.sh set by defining the %_no_recompute_build_ids
macro for such packages. %_no_recompute_build_ids can not be used together
with %_unique_build_ids.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 macros.in |   7 ++-
 scripts/find-debuginfo.sh |  20 +++-
 tests/rpmbuildid.at   | 122 ++
 tools/debugedit.c |   6 ++-
 4 files changed, 151 insertions(+), 4 deletions(-)

diff --git a/macros.in b/macros.in
index f7d16de..cf22628 100644
--- a/macros.in
+++ b/macros.in
@@ -172,7 +172,7 @@
 #  the script.  See the script for details.
 #
 %__debug_install_post   \
-   %{_rpmconfigdir}/find-debuginfo.sh %{?_smp_mflags} 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i} 
%{?_unique_build_ids:--ver-rel "%{VERSION}-%{RELEASE}"} 
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"} 
%{?_unique_debug_srcs:--unique-debug-src-base "%{name}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
+   %{_rpmconfigdir}/find-debuginfo.sh %{?_smp_mflags} 
%{?_missing_build_ids_terminate_build:--strict-build-id} 
%{?_no_recompute_build_ids:-n} %{?_include_minidebuginfo:-m} 
%{?_include_gdb_index:-i} %{?_unique_build_ids:--ver-rel 
"%{VERSION}-%{RELEASE}"} %{?_unique_debug_names:--unique-debug-arch "%{_arch}"} 
%{?_unique_debug_srcs:--unique-debug-src-base "%{name}"} 
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts} 
"%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
 #  Template for debug information sub-package.
@@ -498,6 +498,11 @@ package or when debugging this package.\
 # onto debugedit --build-id-seed to be used to prime the build-id note hash.
 %_unique_build_ids 1
 
+# Do not recompute build-ids but keep whatever is in the ELF file already.
+# Cannot be used together with _unique_build_ids (which forces recomputation).
+# Defaults to undefined (unset).
+#%_no_recompute_build_ids 1
+
 # Whether .debug files should be made unique between package version,
 # release and architecture. If set to 1 this will pass
 # --unique-debug-arch "%{_arch}" to find-debuginfo.sh to create
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 39bb0b7..8effac8 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -2,7 +2,7 @@
 #find-debuginfo.sh - automagically generate debug info and file list
 #for inclusion in an rpm spec file.
 #
-# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m] [-i]
+# Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m] [-i] [-n]
 # [-o debugfiles.list]
 # [--run-dwz] [--dwz-low-mem-die-limit N]
 # [--dwz-max-die-limit N]
@@ -16,6 +16,7 @@
 # The -r flag says to use eu-strip --reloc-debug-sections.
 # The -m flag says to include a .gnu_debugdata section in the main binary.
 # The -i flag says to include a .gdb_index section in the .debug file.
+# The -n flag says to not recompute the build-id.
 #
 # A single -o switch before any -l or -p switches simply renames
 # the primary output file from debugfiles.list to something else.
@@ -56,6 +57,9 @@ include_gdb_index=false
 # Barf on missing build IDs.
 strict=false
 
+# Do not recompute build IDs.
+no_recompute_build_id=false
+
 # DWZ parameters.
 run_dwz=false
 dwz_low_mem_die_limit=
@@ -110,6 +114,9 @@ while [ $# -gt 0 ]; do
   -m)
 include_minidebug=true
 ;;
+  -n)
+no_recompute_build_id=true
+;;
   -i)
 include_gdb_index=true
 ;;
@@ -159,6 +166,11 @@ if test -z "$unique_debug_arch" -a -n 
"$unique_debug_src_base"; then
   exit 2
 fi
 
+if test -n "$ver_rel" -a "$no_recompute_build_id" = "true"; then
+  echo >&2 "*** ERROR: --ver-rel (unique build-ids) and -n (do not recompute 
build-id cannot be used together"
+  exit 2
+fi
+
 i=0
 while ((i < nout)); do
   outs[$i]="$BUILDDIR/${outs[$i]}"
@@ -324,8 +336,12 @@ do_file()
 debug_base_name="$BUILDDIR"
 
debug_dest_name="/usr/src/debug/${unique_debug_src_base}-${ver_rel}.${unique_debug_arch}"
   fi
+  no_recompute=
+  if [ "$no_recompute_build_id" = "true" ]; then
+no_recompute="-n"
+  fi
   id=$(${lib_rpm_dir}/debugedit -b $debug_base_name -d $debug_dest_name \
- -i $build_id_seed -l "$SOURCEFILE" "$f") || exit
+   $no_recompute -i $build_id_seed -l "$SOURCEFILE" "$f") || exit
   if [ -z "$id" ]; then
 echo >&

[Rpm-maint] [PATCH] find-debuginfo.sh: Only add minisymtab for executables or shared libraries.

[Mark Wielaard]

It only makes sense to add a minisymtab for executables and shared
libraries. Other executable ELF files (like kernel modules) don't need it.
Since those don't have a dynsym section trying to add it will fail and
produce confusing errors from nm.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 6f38e19..39bb0b7 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -358,7 +358,16 @@ do_file()
   fi
 
   # strip -g implies we have full symtab, don't add mini symtab in that case.
-  $strip_g || ($include_minidebug && add_minidebug "${debugfn}" "$f")
+  # It only makes sense to add a minisymtab for executables and shared
+  # libraries. Other executable ELF files (like kernel modules) don't need it.
+  if [ "$include_minidebug" = "true" -a "$strip_g" = "false" ]; then
+skip_mini=true
+case "$(file -bi "$f")" in
+  application/x-sharedlib*) skip_mini=false ;;
+  application/x-executable*) skip_mini=false ;;
+esac
+$skip_mini || add_minidebug "${debugfn}" "$f"
+  fi
 
   echo "./${f#$RPM_BUILD_ROOT}" >> "$ELFBINSFILE"
 
-- 
2.9.3

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] build/files.c: Unset __debug_package implies missing build-ids aren't fatal.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

Historically we have only checked build_ids when __debug_package was
defined. So don't terminate the build if __debug_package is unset, even
when _missing_build_ids_terminate_build is. Only warn.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/build/files.c b/build/files.c
index 26f53bd..8b516c6 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1699,7 +1699,11 @@ static int generateBuildIDs(FileList fl)
 if (build_id_links == BUILD_IDS_NONE || rc != 0)
return rc;
 
-int terminate = rpmExpandNumeric("%{?_missing_build_ids_terminate_build}");
+/* Historically we have only checked build_ids when __debug_package
+   was defined. So don't terminate the build if __debug_package is
+   unset, even when _missing_build_ids_terminate_build is. */
+int terminate = (rpmExpandNumeric("%{?_missing_build_ids_terminate_build}")
+&& rpmExpandNumeric("%{?__debug_package}"));
 
 /* Collect and check all build-ids for ELF files in this package.  */
 int needMain = 0;
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] generateBuildIDs: Don't warn or error for object files without build-id.

[Mark Wielaard]

On Thu, 2017-03-09 at 12:22 +0200, Panu Matilainen wrote:
> On 03/09/2017 10:34 AM, Mark Wielaard wrote:
> > From: Mark Wielaard <m...@klomp.org>
> >
> > Only loadable ELF images (executables, shared libraries, kernel modules)
> > should have build-ids. So don't warn or error out when an object file is
> > found without one.
> >
> > Signed-off-by: Mark Wielaard <m...@klomp.org>
> 
> Applied, thanks.

Thanks, but unfortunately my patch was not correct.
It suppressed the warning/error, but it still failed the build.
Now without giving any clear message. And because we also failed to set
processingFailed the error message if generateBuildIDs failed for any
reason the error message the user got was incomprehensible. The attached
patch fixes both these issues.
From 0591eed98d5cf258f6ea7675128837d49498f84c Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@klomp.org>
Date: Thu, 9 Mar 2017 22:13:01 +0100
Subject: [PATCH] generateBuildIDs: Fix error handling.

commit e6bdf7 made it so that we don't give a warning or error message
for non-kernel ET_REL object files with missing or bad build-ids. But
we still (unintentionally) failed generateBuildIDs which made us skip
generating the cpioList causing an obscure failure message.

Move the sanity check earlier so we don't process such object files at
all. And if there is any real error from generateBuildIDs give a clear
error message and explicitly set processingFailed.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 18 +++---
 1 file changed, 11 insertions(+), 7 deletions(-)

diff --git a/build/files.c b/build/files.c
index 8b516c6..6021643 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1713,10 +1713,16 @@ static int generateBuildIDs(FileList fl)
 	if (lstat(flp->diskPath, ) == 0 && S_ISREG (sbuf.st_mode)) {
 	int fd = open (flp->diskPath, O_RDONLY);
 	if (fd >= 0) {
+		/* Only real ELF files, that are ET_EXEC, ET_DYN or
+		   kernel modules (ET_REL files with names ending in .ko)
+		   should have build-ids. */
 		GElf_Ehdr ehdr;
 		Elf *elf = elf_begin (fd, ELF_C_READ, NULL);
 		if (elf != NULL && elf_kind(elf) == ELF_K_ELF
-		&& gelf_getehdr(elf, ) != NULL) {
+		&& gelf_getehdr(elf, ) != NULL
+		&& (ehdr.e_type == ET_EXEC || ehdr.e_type == ET_DYN
+			|| (ehdr.e_type == ET_REL
+			&& rpmFileHasSuffix (flp->diskPath, ".ko" {
 		const void *build_id;
 		ssize_t len = dwelf_elf_gnu_build_id (elf, _id);
 		/* len == -1 means error. Zero means no
@@ -1765,11 +1771,6 @@ static int generateBuildIDs(FileList fl)
    _("error reading build-id in %s: %s\n"),
    flp->diskPath, elf_errmsg (-1));
 			} else if (len == 0) {
-			/* Only ET_EXEC, ET_DYN or kernel modules
-			   have build-ids. */
-			if (ehdr.e_type == ET_EXEC || ehdr.e_type == ET_DYN
-|| (ehdr.e_type == ET_REL
-&& rpmFileHasSuffix (flp->diskPath, ".ko")))
 			  rpmlog(terminate ? RPMLOG_ERR : RPMLOG_WARNING,
  _("Missing build-id in %s\n"),
  flp->diskPath);
@@ -2383,8 +2384,11 @@ static rpmRC processPackageFiles(rpmSpec spec, rpmBuildPkgFlags pkgFlags,
 	goto exit;
 
 #if HAVE_LIBDW
-if (generateBuildIDs () != 0)
+if (generateBuildIDs () != 0) {
+	rpmlog(RPMLOG_ERR, _("Generating build-id links failed\n"));
+	fl.processingFailed = 1;
 	goto exit;
+}
 #endif
 
 /* Verify that file attributes scope over hardlinks correctly. */
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] tests/tpmbuild.at: Make file sed regexp more strict to extract BuildID.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

In some testcases we extract the BuildID with the file command.
Unfortunately the file command output changed slightly between versions.
Make the sed regexp more strict by only matching a hex-string.
Also properly "escape" [ and ] which inside an AT_CHECK should be [[ and ]].

Tested against file versions 5.11, 5.29 and 5.30.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/rpmbuild.at | 20 ++--
 1 file changed, 10 insertions(+), 10 deletions(-)

diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 0a2c01e..1c18740 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -422,11 +422,11 @@ hello2_file=./usr/local/bin/hello2
 test -f $hello_file || echo "no hello file: $hello_file"
 test -f $hello2_file || echo "no hello2 file: $hello2_file"
 
-id1=$(file $hello_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id2=$(file $hello2_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id1debug=$(file $hello_file_debug | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id2debug=$(file $hello2_file_debug | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-idmulti=$(file $hello_multi_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id1=$(file $hello_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id2=$(file $hello2_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id1debug=$(file $hello_file_debug | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id2debug=$(file $hello2_file_debug | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+idmulti=$(file $hello_multi_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test "$id1" = "$id1debug" || echo "id1: $id1 != id1debug: $id1debug"
 test "$id2" = "$id2debug" || echo "id2: $id2 != id2debug: $id2debug"
@@ -515,11 +515,11 @@ hello2_file=./usr/local/bin/hello2
 test -f $hello_file || echo "no hello file: $hello_file"
 test -f $hello2_file || echo "no hello2 file: $hello2_file"
 
-id1=$(file $hello_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id2=$(file $hello2_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id1debug=$(file $hello_file_debug | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-id2debug=$(file $hello2_file_debug | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
-idmulti=$(file $hello_multi_file | sed 's/.*, BuildID[.*]=\(.*\),.*/\1/')
+id1=$(file $hello_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id2=$(file $hello2_file | sed 's/.*, BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id1debug=$(file $hello_file_debug | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+id2debug=$(file $hello2_file_debug | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
+idmulti=$(file $hello_multi_file | sed 's/.*, 
BuildID\[[.*\]]=\([[0-9a-f]]*\),.*/\1/')
 
 test "$id1" = "$id1debug" || echo "id1: $id1 != id1debug: $id1debug"
 test "$id2" = "$id2debug" || echo "id2: $id2 != id2debug: $id2debug"
-- 
2.9.3

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] generateBuildIDs: Don't warn or error for object files without build-id.

[Mark Wielaard]

From: Mark Wielaard <m...@klomp.org>

Only loadable ELF images (executables, shared libraries, kernel modules)
should have build-ids. So don't warn or error out when an object file is
found without one.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/build/files.c b/build/files.c
index cbcc0a9..26f53bd 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1709,8 +1709,10 @@ static int generateBuildIDs(FileList fl)
if (lstat(flp->diskPath, ) == 0 && S_ISREG (sbuf.st_mode)) {
int fd = open (flp->diskPath, O_RDONLY);
if (fd >= 0) {
+   GElf_Ehdr ehdr;
Elf *elf = elf_begin (fd, ELF_C_READ, NULL);
-   if (elf != NULL && elf_kind(elf) == ELF_K_ELF) {
+   if (elf != NULL && elf_kind(elf) == ELF_K_ELF
+   && gelf_getehdr(elf, ) != NULL) {
const void *build_id;
ssize_t len = dwelf_elf_gnu_build_id (elf, _id);
/* len == -1 means error. Zero means no
@@ -1759,9 +1761,14 @@ static int generateBuildIDs(FileList fl)
   _("error reading build-id in %s: %s\n"),
   flp->diskPath, elf_errmsg (-1));
} else if (len == 0) {
-   rpmlog(terminate ? RPMLOG_ERR : RPMLOG_WARNING,
-  _("Missing build-id in %s\n"),
-  flp->diskPath);
+   /* Only ET_EXEC, ET_DYN or kernel modules
+  have build-ids. */
+   if (ehdr.e_type == ET_EXEC || ehdr.e_type == ET_DYN
+   || (ehdr.e_type == ET_REL
+   && rpmFileHasSuffix (flp->diskPath, ".ko")))
+ rpmlog(terminate ? RPMLOG_ERR : RPMLOG_WARNING,
+_("Missing build-id in %s\n"),
+flp->diskPath);
} else {
rpmlog(terminate ? RPMLOG_ERR : RPMLOG_WARNING,
   (len < 16
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] debug edit build generation is endian dependent when cross-compiling? (#171)

[Mark Wielaard]

On Sun, 2017-03-05 at 09:48 -0800, Jeff Johnson wrote:
> For rpm portability with older versions of elfutils, this harmless sanity 
> check might be useful:
> 
> (excuse the different hash functions in use in the patch snippet below)
> @@ -1533,40 +1493,12 @@
>/* Now format the build ID bits in hex to print out.  */
>{   
>  const uint8_t * id = (uint8_t *)build_id->d_buf + build_id_offset;
> -char hex[build_id_size * 2 + 1];
> -int n = snprintf (hex, 3, "%02" PRIx8, id[0]);
> -assert (n == 2);
> -for (i = 1; i < (int)build_id_size; ++i)
> -  {
> -   n = snprintf ([i * 2], 3, "%02" PRIx8, id[i]);
> -   assert (n == 2);
> -  }
> +char *hex = pgpHexStr(id, build_id_size);
>  puts (hex);
> +free(hex);
>}   
>  }

I don't understand this hunk. What is it attempting to do?

Also I assume you posted the reverse patch?

> -/* It avoided the segment fault while file's bss offset have a large number.
> -   See https://bugzilla.redhat.com/show_bug.cgi?id=1019707
> -   https://bugzilla.redhat.com/show_bug.cgi?id=1020842 for detail. */

This is indeed a bug in elfutils, fixed in elfutils 0.162.
But also only occurs for bogus/bad ELF files in the first place.

> -void valid_file(int fd)
> -{
> -  Elf *elf = elf_begin (fd, ELF_C_RDWR, NULL);
> -  if (elf == NULL)
> -  {
> -error (1, 0, "elf_begin: %s", elf_errmsg (-1));
> -return;
> -  }
> -
> -  elf_flagelf (elf, ELF_C_SET, ELF_F_LAYOUT);
> -
> -  if (elf_update (elf, ELF_C_WRITE) < 0)
> -error (1, 0, "elf_update: %s", elf_errmsg (-1));
> -
> -  elf_end (elf);
> -
> -  return;
> -}

So this opens and closes the ELF file and tries to write it out to disk,
but since nothing changed all this does is make libelf do a layout check
of all headers. Where it would notice nothing changed, so ELF_C_WRITE
would not actually write something out. Except if it does see a
discrepancy and might "correct" a header field. It sounds slightly safer
to do an ELF_C_UPDATE so that nothing might accidentally be written out.

Although it might indeed catch the issue with the bogus ELF file earlier
(and so prevent a crash with a buggy elfutils) I am not sure it is
really the task of rpm/debugedit to make sure the user doesn't have
bogus/bad ELF files and/or an old/buggy libelf implementation. But if
you don't mind opening/closing/reading the ELF file twice this might
indeed not be that expensive (compared to all the other work debugedit
does). It is somewhat bogus and obscure though.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] debugedit: Support String/Line table rewriting for larger/smaller paths.

[Mark Wielaard]

On Sat, 2017-03-04 at 11:28 +0200, Panu Matilainen wrote:
> I dont feel qualified to really review this, and perhaps others are 
> feeling the same way since it's been out there for a week now with no 
> comments at all.
> 
> Because this seems quite awesome (even if also a bit scary), to avoid 
> stalling forever with no-one to review it: if there are no objections 
> raised by Monday I'm going to just apply it (and the couple of other 
> recent debuginfo patches). So anybody having doubts, speak up now.

Thanks. Obviously I think the patch is awesome and should be integrated
sooner rather than later :) But being slightly worried this means "you
touched it last, you own it now", let me do at least my own review.

Questions I would ask about this patch if someone else would have
submitted it:

- Why depend on elfutils libebl/libdw for the .debug_str string table
  reconstruction? Isn't a string table just a simple concatenation of
  strings?

It is indeed a simple concatenation of strings, but to pack them
efficiently substrings can be reused/shared. e.g. if the table contains
the string "easy_and_slow" at index 16, then it also contains the string
"slow" at index 25. The elfutils strtab code already handles this
efficiently, so I wanted to depend on that instead of open coding it.

- Why introduce those string storage allocation blocks? Can't you just
  malloc whenever you need a new string?

To make sure we don't leak too much memory we need to track our mallocs
already. strings come either from the existing .debug_str table or from
newly created (replacement) strings. I felt it was a good idea to pack
those strings as closely together because they might be compared often.
So keep them close together helps data cache locality. But this was
probably premature optimisation since we end up at the strings
indirectly through the original indexes and we compare those first (if
the original indexes are equal then the (replacement) strings would be
the same too (and so we end up not having to compare that many strings
anyway). But we needed a structure to keep track of our allocated
strings anyway. So I kept it as is.

- To compare/find existing string table indexes the code uses a binary
  search tree (tsearch), but for line table indexes the code uses a
  simple (sorted) array (bsearch). Why different mechanisms for the same
  kind of lookup?

There are much, much more string table indexes than line table offsets
(e.g. libstdc++.so has 125 line tables, and 10331 full strings [ignoring
shared "tails"]). So for line tables we don't mind if we sometimes have
to do a quick linear search (in phase 0). But for the string indexes it
is important to always be able to search/insert them efficiently. So for
the string table indexes we always use the binary search tree balanced.
For line tables we simply sort the array when we collected all offsets.
The binary search tree does have a bit more memory overhead, but it was
worth it for the fast lookups [*].

- Does the switch from mmap and rawdata to simple read/write and
  elf_data cause much more memory usage?

Yes and no. When mmapping the file the memory is still allocated, but
might not really be used. We do in most cases end up using most of it
because for recalculating the build-id we need to mmap/read in all
section data. The reading in of the ELF data through elf_getdata is
slightly less efficient that elf_rawdata because it might have to do
endian conversion if the ELF file is not native. But in practice it
always is (we just build it). We can probably do better by being smarter
about when/how we update the build-ids. I didn't try to do that yet.

- The relocation rewriting seems scary.

Yes it is :{ Luckily this is only needed for object files (ET_REL) and
normally they aren't packaged. The big exceptions are kernel modules
(why those are ET_REL files and not simply ET_DYN is beyond me) and
static ar archives. If we would have to handle all possible
(architecture specific) relocations this would be really impossible.
Luckily for the indexes into .debug sections only simple relocation
types are used (basically it provides an extra offset into the section).
The line table relocations pointing into code sections are also simple
because we don't care about the actual relocation type, just where in
the section they are. So we can simply move the relocation offset.
We don't handle static ar archives at all. Which is something we might
want to look at at some later point.

- The code ends up touching allocated sections! Isn't that a no-no?

Yes. Anything allocated really shouldn't be touched. Only the
non-allocated (debug) sections should be touched/written out. Sadly a
bug in elfutils means we do end up having to mark the allocated sections
dirty to make sure the whole ELF file ends up correct on disk. This is
really unfortunate and might make things a bit slower. But no
(allocated) data is really changed at all. And once the bug is fixed (in
elfutils 0.169) this 

Re: [Rpm-maint] [rpm-software-management/rpm] debug edit build generation is endian dependent when cross-compiling? (#171)

[Mark Wielaard]

On Sun, 2017-03-05 at 09:58 -0800, Jeff Johnson wrote:
> Portability of debugedit.c on systems that do not support #include
>  (solaris, *BSD) can be achieved by duplicating certain DWARF
> standard constants:

I think you reversed the patch you are proposing again.
I'll leave it to others to say whether it makes sense or if rpm should
just require up to date elf.h and dwarf.h headers be present.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [rpm-software-management/rpm] debug edit build generation is endian dependent when cross-compiling? (#171)

[Mark Wielaard]

On Sun, 2017-03-05 at 09:09 -0800, Jeff Johnson wrote:
> rpm5.org carries this patch from Yocto:
>   https://patchwork.openembedded.org/patch/46887
> which likely should be added when updating debug edit.c soon.

It is slightly hard to review a patch to a patch.
Could you extract the precise patch you suggest would be helpful and
post it to rpm-maint for review?

Also I suspect that the latest patch that moved away from using
ELF_C_RDWR_MMAP makes the original problem also go away.

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Warn and create empty debugsource package if there are no sources.

[Mark Wielaard]

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 13 +
 tests/rpmbuild.at | 33 +
 2 files changed, 46 insertions(+)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index d8725ad..9b3815a 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -573,6 +573,19 @@ if [ -n "$srcout" ]; then
  find src/debug -mindepth 1 -maxdepth 1
 ) | sed 's,^,/usr/,' >> "$srcout"
   fi
+  if [ ! -s "$srcout" ]; then
+echo >&2 "*** WARNING: No source files found.  Creating empty debugsource 
package"
+# Create the empty directory.
+# See also debugedit invocation. Directories must match up.
+debug_base_name="$RPM_BUILD_DIR"
+debug_dest_name="/usr/src/debug"
+if [ ! -z "$unique_debug_src_base" ]; then
+  debug_base_name="$BUILDDIR"
+  debug_dest_name="/usr/src/debug/${unique_debug_src_base}"
+fi
+mkdir -p "${RPM_BUILD_ROOT}${debug_dest_name}"
+echo "$debug_dest_name" > "$srcout"
+  fi
 fi
 
 # Append to $1 only the lines from stdin not already in the file.
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 3f41652..2359c63 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -900,3 +900,36 @@ ls ./usr/src/debug/test-1.0*/
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that defining _debugsource_packages creates -debugsource package
+# doesn't break when there are no source files (found).
+AT_SETUP([rpmbuild debugsource no source files])
+AT_KEYWORDS([build] [debuginfo] [debugsource])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello-1.0.tar.gz 
"${abs_srcdir}"/data/SOURCES/hello-1.0-modernize.patch ${TOPDIR}/SOURCES
+
+# Build a package that doesn't use -g.
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define "_debugsource_packages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/hello.spec
+
+# Unpack the debugsource rpm so we can check the sources are (not) there.
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello-debugsource-1.0-1.*.rpm \
+  | cpio -diu --quiet
+
+# There should be just one empty directory (strip off nvra).
+find ./usr/src/debug/* | cut -f1 -d-
+
+],
+[0],
+[./usr/src/debug/hello
+],
+[ignore])
+AT_CLEANUP
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Add two testcases for generating debugsource packages.

[Mark Wielaard]

These tests define _debugsource_packages and check a debugsource package
is created that contains the source of the program created.

Without the fix to make sure that debugsourcefiles.list is generated in
the build dir the second testcase will fail with:
error: Could not open %files file

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am  |  1 +
 tests/data/SPECS/hello-cd.spec | 29 
 tests/rpmbuild.at  | 62 ++
 3 files changed, 92 insertions(+)
 create mode 100644 tests/data/SPECS/hello-cd.spec

diff --git a/tests/Makefile.am b/tests/Makefile.am
index b962e4a..09239ba 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -78,6 +78,7 @@ EXTRA_DIST += data/SRPMS/hello-1.0-1.src.rpm
 EXTRA_DIST += data/SOURCES/hello.c
 EXTRA_DIST += data/SPECS/hello-attr-buildid.spec
 EXTRA_DIST += data/SPECS/hello-config-buildid.spec
+EXTRA_DIST += data/SPECS/hello-cd.spec
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.pub
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.secret
 
diff --git a/tests/data/SPECS/hello-cd.spec b/tests/data/SPECS/hello-cd.spec
new file mode 100644
index 000..ab1e300
--- /dev/null
+++ b/tests/data/SPECS/hello-cd.spec
@@ -0,0 +1,29 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+
+%install
+mkdir -p %{buildroot}/bin
+install -D -p -m 0755 -t %{buildroot}/bin hello
+# Pretend we need to do something in /tmp
+cd /tmp
+
+%files
+%attr(644,root,root) /bin/hello
+
+%changelog
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 3200fb2..3f41652 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -838,3 +838,65 @@ hello-1.0
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that defining _debugsource_packages creates -debugsource package
+AT_SETUP([rpmbuild debugsource])
+AT_KEYWORDS([build] [debuginfo] [debugsource])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Build a package that has some debuginfo
+cp "${abs_srcdir}"/data/SOURCES/hello-1.0.tar.gz 
"${abs_srcdir}"/data/SOURCES/hello-1.0-modernize.patch ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define "_debugsource_packages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/hello2.spec
+
+# Unpack the debuginfo rpms so we can check the sources are there.
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/hello2-debugsource-1.0-1.*.rpm \
+  | cpio -diu --quiet
+
+# Check that hello.c is there.
+ls ./usr/src/debug/hello2-1.0*/
+],
+[0],
+[hello.c
+],
+[ignore])
+AT_CLEANUP
+
+# --
+# Check that defining _debugsource_packages creates -debugsource package
+# even if %install changes the working directory (debugsourcefiles.list
+# should be in expected build dir).
+AT_SETUP([rpmbuild debugsource debugsourcefiles.list path])
+AT_KEYWORDS([build] [debuginfo] [debugsource])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define "_debugsource_packages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/hello-cd.spec
+
+# Unpack the debuginfo rpms so we can check the sources are there.
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-debugsource-1.0-1.*.rpm \
+  | cpio -diu --quiet
+
+# Check that hello.c is there.
+ls ./usr/src/debug/test-1.0*/
+],
+[0],
+[hello.c
+],
+[ignore])
+AT_CLEANUP
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH 2/2] Test split debuginfo packages with RemovePathPostfixes.

[Mark Wielaard]

On Fri, 2017-07-28 at 23:24 +0200, Mark Wielaard wrote:
> If split debuginfo code doesn't know about RemovePathPostfixes it fails:
> 
> error: Installed (but unpackaged) file(s) found:
>/usr/lib/debug/bin/hello.foobar-1.0-1.x86_64.debug

This one also now passes with Igor's fixes.

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH 1/2] Test split debuginfo packages with excluded files.

[Mark Wielaard]

On Fri, 2017-07-28 at 23:24 +0200, Mark Wielaard wrote:
> If split debuginfo code doesn't know about excluded files it fails:
> 
> error: Installed (but unpackaged) file(s) found:
>/usr/lib/debug/bin/hello3-1.0-1.x86_64.debug

And with Igor's fixes this test now passes.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Add rpmbuild debuginfo subpackages tests.

[Mark Wielaard]

On Fri, 2017-07-28 at 21:55 +0200, Mark Wielaard wrote:
> This adds various tests for making sure multiple subpackages are build
> correctly. Without debuginfo subpackages, with subpackages, subpackages
> with unique debug file and source dir paths and with separate debugsources.

Ping. These are simply tests for existing functionality that didn't have
tests before.
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Warn and create empty debugsource package if there are no sources.

[Mark Wielaard]

Hi,

On Fri, 2017-07-28 at 15:29 +0200, Mark Wielaard wrote:
> +  if [ ! -s "$srcout" ]; then
> +echo >&2 "*** WARNING: No source files found.  Creating empty 
> debugsource package"
> +# Create the empty directory.
> +# See also debugedit invocation. Directories must match up.
> +debug_base_name="$RPM_BUILD_DIR"
> +debug_dest_name="/usr/src/debug"
> +if [ ! -z "$unique_debug_src_base" ]; then
> +  debug_base_name="$BUILDDIR"
> +  debug_dest_name="/usr/src/debug/${unique_debug_src_base}"
> +fi
> +mkdir -p "${RPM_BUILD_ROOT}${debug_dest_name}"
> +echo "$debug_dest_name" > "$srcout"
> +  fi
>  fi

Do we want to be nicer to packages that don't include sources like this?
Or should we just declare that this is probably a packaging issue (badly
produced debuginfo) and improve the error message?

Alternatively we might just skip creation of the whole debugsource
package. But I couldn't figure out how to do that.

Cheers,

Mark

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] Warn and create empty debugsource package if there are no sources.

[Mark Wielaard]

On Wed, 2017-08-02 at 17:43 +0200, Igor Gnatenko wrote:
> On Wed, 2017-08-02 at 13:09 +0200, Mark Wielaard wrote:
> > Do we want to be nicer to packages that don't include sources like
> > this?
> > Or should we just declare that this is probably a packaging issue
> > (badly produced debuginfo) and improve the error message?
>
> We do this today with debuginfo packages, I don't think debugsource
> should be treated differently. If there will be use-case, then we can
> figure something out.

What precisely does "this" refer to? It currently is treated
differently, that is what the patch tries to fix. Although I am
certainly open to alternatives.

If there are no sources, currently the debuginfo package is just created
without any sources, just the .debug files. But when
_debugsource_packages is defined we error out (with a somewhat cryptic
error message IMHO). It would be nice to make this behavior consistent.
What option do you suggest we should implement?

> > Alternatively we might just skip creation of the whole debugsource
> > package. But I couldn't figure out how to do that.

Cheers,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Add --keep-section and --remove-section for eu-strip.

[Mark Wielaard]

Use --keep-section SECTION or --remove-section SECTION to explicitly
keep a (non-allocated) section in the main executable or explicitly
remove it into the .debug file. SECTION is an extended wildcard pattern.
Both options can be given more than once.

https://bugzilla.redhat.com/show_bug.cgi?id=1465997

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 21 +++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 5613391..e19ce9b 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -3,6 +3,7 @@
 #for inclusion in an rpm spec file.
 #
 # Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m] [-i] [-n]
+# [--keep-section SECTION] [--remove-section SECTION]
 # [-j N]
 # [-o debugfiles.list]
 # [-S debugsourcefiles.list]
@@ -15,9 +16,14 @@
 # [builddir]
 #
 # The -g flag says to use strip -g instead of full strip on DSOs or EXEs.
+# The -r flag says to use eu-strip --reloc-debug-sections.
+# Use --keep-section SECTION or --remove-section SECTION to explicitly
+# keep a (non-allocated) section in the main executable or explicitly
+# remove it into the .debug file. SECTION is an extended wildcard pattern.
+# Both options can be given more than once.
+#
 # The --strict-build-id flag says to exit with failure status if
 # any ELF binary processed fails to contain a build-id note.
-# The -r flag says to use eu-strip --reloc-debug-sections.
 # The -m flag says to include a .gnu_debugdata section in the main binary.
 # The -i flag says to include a .gdb_index section in the .debug file.
 # The -n flag says to not recompute the build-id.
@@ -65,6 +71,9 @@ strip_g=false
 # with -r arg, pass --reloc-debug-sections to eu-strip.
 strip_r=false
 
+# keep or remove arguments to eu-strip.
+keep_remove_args=
+
 # with -m arg, add minimal debuginfo to binary.
 include_minidebug=false
 
@@ -158,6 +167,14 @@ while [ $# -gt 0 ]; do
   -r)
 strip_r=true
 ;;
+  --keep-section)
+keep_remove_args="${keep_remove_args} --keep-section $2"
+shift
+;;
+  --remove-section)
+keep_remove_args="${keep_remove_args} --remove-section $2"
+shift
+;;
   -j)
 n_jobs=$2
 shift
@@ -215,7 +232,7 @@ strip_to_debug()
   application/x-sharedlib*) g=-g ;;
   application/x-executable*) g=-g ;;
   esac
-  eu-strip --remove-comment $r $g -f "$1" "$2" || exit
+  eu-strip --remove-comment $r $g ${keep_remove_args} -f "$1" "$2" || exit
   chmod 444 "$1" || exit
 }
 
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] config: Detect major/minor warnings and include the correct system header.

[Mark Wielaard]

Hi,

On Thu, 2017-07-20 at 17:06 +0300, Dmitry V. Levin wrote:
> I'm not sure what sys/mkdev.h does, but glibc's sys/sysmacros.h certainly
> undefines major, minor, and makedev prior to defining its own versions of
> these macros.
> 
> My guess is that these undefs are not needed.

You are right. This makes the patch simpler. Now we only need to fix the
AC_HEADER_MAJOR check. Cleaned up patch attached. Retested on rhel7 and
fedora26.

sys/mkdev.h is a solaris/bsd thing. I checked and they also undef the
macros before (re)defining them. Although there they might not be
defined in another header in the first place.

Thanks,

Mark
From 77a745abd88f278b4c29aadefb2f6c5d47bde5e6 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@klomp.org>
Date: Wed, 19 Jul 2017 14:43:28 +0200
Subject: [PATCH] config: Detect major/minor warnings and include the correct
 system header.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

glibc 2.25 introduced (really long and annoying) warnings for each use
of the major/minor macros from the wrong header:

lib/cpio.c: In function ‘rpmcpioHeaderWrite’:
lib/cpio.c:245:13: warning: In the GNU C Library, "major" is defined
 by . For historical compatibility, it is
 currently defined by  as well, but we plan to
 remove this soon. To use "major", include 
 directly. If you did not intend to use a system-defined macro
 "major", you should undefine it after including .
 dev = major(st->st_dev); SET_NUM_FIELD(hdr->devMajor, dev, field);
 ^~

Adjust the configure check to correctly detect the header to include
that doesn't produce those warning producing macros.

Tested against RHEL7 (glibc 2.17) and Fedora 26 (glibc 2.25).

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 configure.ac | 8 
 1 file changed, 8 insertions(+)

diff --git a/configure.ac b/configure.ac
index cc657ec..017a908 100644
--- a/configure.ac
+++ b/configure.ac
@@ -561,7 +561,15 @@ AM_ICONV
 
 dnl Checks for header files we can live without.
 AC_HEADER_STDC
+dnl glibc and autoconf don't really play well together.
+dnl glibc will produce a warning when including the wrong header.
+dnl but still define major and minor. Causing us to include the header
+dnl that produces a giant warning for each major/minor use.
+dnl Use -Werror to work around that.
+old_CFLAGS=$CFLAGS
+CFLAGS="$CFLAGS -Werror"
 AC_HEADER_MAJOR
+CFLAGS=$old_CFLAGS
 AC_STRUCT_DIRENT_D_TYPE
 
 AC_CHECK_HEADERS(limits.h)
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Remove non-allocated NOBITS sections from minisymtab.

[Mark Wielaard]

In the minisymtab section (the .gnu_debugdata embedded ELF image) we
do not need unallocated sections (except for the SYMTAB and STRTAB
sections we are creating). We already remove PROGBITS and NOTES. Also
remove NOBITS sections. They should not really take up much (any) space
but they still add to the section tables. These sections might be created
with the new --keep-section support (which puts the actual section in
the main ELF binary, and a NOBITS variant in the .debug file).

Also binutils objcopy seems to sometimes add them anyway filled with
zeros instead of marking them NOBITS.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 8 ++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index e19ce9b..2fa95e8 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -247,12 +247,16 @@ add_minidebug()
   local mini_debuginfo=`mktemp`
 
   # In the minisymtab we don't need the .debug_ sections (already removed
-  # by -S) but also not any other non-allocated PROGBITS or NOTE sections.
+  # by -S) but also not other non-allocated PROGBITS, NOTE or NOBITS sections.
   # List and remove them explicitly. We do want to keep the allocated,
   # symbol and NOBITS sections so cannot use --keep-only because that is
   # too aggressive. Field $2 is the section name, $3 is the section type
   # and $8 are the section flags.
-  local remove_sections=`readelf -W -S "$debuginfo" | awk '{ if 
(index($2,".debug_") != 1 && ($3 == "PROGBITS" || $3 == "NOTE") && 
index($8,"A") == 0) printf "--remove-section "$2" " }'`
+  local remove_sections=`readelf -W -S "$debuginfo" \
+   | awk '{ if (index($2,".debug_") != 1 \
+&& ($3 == "PROGBITS" || $3 == "NOTE" || $3 == "NOBITS") \
+&& index($8,"A") == 0) \
+  printf "--remove-section "$2" " }'`
 
   # Extract the dynamic symbols from the main binary, there is no need to also 
have these
   # in the normal symbol table
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] config: Detect major/minor warnings and include the correct system header.

[Mark Wielaard]

glibc 2.25 introduced (really long and annoying) warnings for each use
of the major/minor macros from the wrong header:

lib/cpio.c: In function ‘rpmcpioHeaderWrite’:
lib/cpio.c:245:13: warning: In the GNU C Library, "major" is defined
 by . For historical compatibility, it is
 currently defined by  as well, but we plan to
 remove this soon. To use "major", include 
 directly. If you did not intend to use a system-defined macro
 "major", you should undefine it after including .
 dev = major(st->st_dev); SET_NUM_FIELD(hdr->devMajor, dev, field);
 ^~

Adjust the configure check and undef the warning producing macros to get
rid of the wrong definitions and use the macros from the right include.

Tested against RHEL7 (glibc 2.17) and Fedora 26 (glibc 2.25).

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 configure.ac | 8 
 lib/cpio.c   | 6 ++
 2 files changed, 14 insertions(+)

diff --git a/configure.ac b/configure.ac
index cc657ec..017a908 100644
--- a/configure.ac
+++ b/configure.ac
@@ -561,7 +561,15 @@ AM_ICONV
 
 dnl Checks for header files we can live without.
 AC_HEADER_STDC
+dnl glibc and autoconf don't really play well together.
+dnl glibc will produce a warning when including the wrong header.
+dnl but still define major and minor. Causing us to include the header
+dnl that produces a giant warning for each major/minor use.
+dnl Use -Werror to work around that.
+old_CFLAGS=$CFLAGS
+CFLAGS="$CFLAGS -Werror"
 AC_HEADER_MAJOR
+CFLAGS=$old_CFLAGS
 AC_STRUCT_DIRENT_D_TYPE
 
 AC_CHECK_HEADERS(limits.h)
diff --git a/lib/cpio.c b/lib/cpio.c
index 57c9592..b7ba27d 100644
--- a/lib/cpio.c
+++ b/lib/cpio.c
@@ -9,9 +9,15 @@
 
 #include "system.h"
 
+/* system.h might already have included the wrong header, undef major
+   and minor and get the real definition from one of the correct headers. */
 #if MAJOR_IN_MKDEV
+#undef major
+#undef minor
 #include 
 #elif MAJOR_IN_SYSMACROS
+#undef major
+#undef minor
 #include 
 #else
 #include  /* already included from system.h */
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] replaceSigDigests is only used with IMAEVM.

[Mark Wielaard]

The replaceSigDigests function is only used in includeFileSignatures
when WITH_IMAEVM is defined. If not warning is generated.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 sign/rpmgensig.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/sign/rpmgensig.c b/sign/rpmgensig.c
index 0731363..d29c178 100644
--- a/sign/rpmgensig.c
+++ b/sign/rpmgensig.c
@@ -469,6 +469,7 @@ static void unloadImmutableRegion(Header *hdrp, rpmTagVal 
tag)
 }
 }
 
+#ifdef WITH_IMAEVM
 static rpmRC replaceSigDigests(FD_t fd, const char *rpm, Header *sigp,
   off_t sigStart, off_t sigTargetSize,
   char *SHA256, char *SHA1, uint8_t *MD5)
@@ -516,6 +517,7 @@ static rpmRC replaceSigDigests(FD_t fd, const char *rpm, 
Header *sigp,
 exit:
 return rc;
 }
+#endif
 
 static rpmRC includeFileSignatures(FD_t fd, const char *rpm,
   Header *sigp, Header *hdrp,
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] debugedit: skip_dir_prefix should check for dir separator.

[Mark Wielaard]

To count as a real directory prefix the string matched should either
be equal to the given prefix or start with the prefix plus '/'.

skip_dir_prefix is always used with base_dir or dest_dir which don't
end with a slash themselves.

This really only is an issue if a package would put a directory named
similar to the package source dir (which cargo on fedora does, by adding
a directory named cargo-vendor in the builddir itself).

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tools/debugedit.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/tools/debugedit.c b/tools/debugedit.c
index abd2ca4..a271b91 100644
--- a/tools/debugedit.c
+++ b/tools/debugedit.c
@@ -662,7 +662,8 @@ canonicalize_path (const char *s, char *d)
 /* Returns the rest of PATH if it starts with DIR_PREFIX, skipping any
/ path separators, or NULL if PATH doesn't start with
DIR_PREFIX. Might return the empty string if PATH equals DIR_PREFIX
-   (modulo trailing slashes). Never returns path starting with '/'. */
+   (modulo trailing slashes). Never returns path starting with '/'.
+   Note that DIR_PREFIX itself should NOT end with a '/'.  */
 static const char *
 skip_dir_prefix (const char *path, const char *dir_prefix)
 {
@@ -670,12 +671,17 @@ skip_dir_prefix (const char *path, const char *dir_prefix)
   if (strncmp (path, dir_prefix, prefix_len) == 0)
 {
   path += prefix_len;
+  /* Unless path == dir_prefix there should be at least one '/'
+in the path (which we will skip).  Otherwise the path has
+a different (longer) directory prefix.  */
+  if (*path != '\0' && !IS_DIR_SEPARATOR (*path))
+   return NULL;
   while (IS_DIR_SEPARATOR (path[0]))
path++;
   return path;
 }
 
-  return 0;
+  return NULL;
 }
 
 /* Most strings will be in the existing debug string table. But to
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Filter out all like fake file names.

[Mark Wielaard]

There is no official way to mark an instruction range as being not
part of some actual source code, but as part of a compiler built-in
construct in DWARF. So different compilers have come up with fake
source file names like  or <__thread_local_inner macros>.
We already filtered out the strings "" and "".
Just filter out all '(^|/)<[a-z _-]+>$'. They are fake files!

This is mainly to appease the rustc compiler which generates lots of
different variants to encode some instruction sequence is part of an
compiler generated macro expansion.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 0233d92..555e7b8 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -502,7 +502,11 @@ if [ -s "$SOURCEFILE" ]; then
   fi
 
   mkdir -p "${RPM_BUILD_ROOT}${debug_dest_name}"
-  LC_ALL=C sort -z -u "$SOURCEFILE" | grep -E -v -z '(|)$' 
|
+  # Filter out anything compiler generated which isn't a source file.
+  # e.g. , , <__thread_local_inner macros>.
+  # Some compilers generate them as if they are part of the working
+  # directory (which is why we match against ^ or /).
+  LC_ALL=C sort -z -u "$SOURCEFILE" | grep -E -v -z '(^|/)<[a-z _-]+>$' |
   (cd "${debug_base_name}"; cpio -pd0mL "${RPM_BUILD_ROOT}${debug_dest_name}")
   # stupid cpio creates new directories in mode 0700,
   # and non-standard modes may be inherented from original directories, fixup
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Use 'return', not 'continue', to break out do_file().

[Mark Wielaard]

commit 038bfe "Split directory traversal and debuginfo extraction"
put the core of a while loop into its own function 'do_file()'.
That means that instead of using 'continue' to break out early it now
needs to use 'return'. Otherwise the script will give errors like:

  continue: only meaningful in a `for', `while', or `until' loop

https://bugzilla.redhat.com/show_bug.cgi?id=1465170

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 360982d..59a4ff3 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -366,7 +366,7 @@ do_file()
   # just has its file names collected and adjusted.
   case "$dn" in
   /usr/lib/debug/*)
-continue ;;
+return ;;
   esac
 
   mkdir -p "${debugdn}"
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 2/3] Use a file list to add build-id files to pkgList.

[Mark Wielaard]

Change the generation of build-id files to a file list using ARGV_t.
First go through the current package list and generate a files list.
Then add those files as if they were part of the original package file
list using the new resetPackageFilesDefaults() and addPackageFileList().

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 74 ++-
 1 file changed, 33 insertions(+), 41 deletions(-)

diff --git a/build/files.c b/build/files.c
index c7debdc..cb7def6 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1614,6 +1614,15 @@ exit:
 return rc;
 }
 
+/* add a directory to the file list */
+static void argvAddDir(ARGV_t *filesp, const char *dir)
+{
+char *line = NULL;
+rasprintf(, "%%dir %s", dir);
+argvAdd(filesp, line);
+_free(line);
+}
+
 #if HAVE_LIBDW
 /* How build id links are generated.  See macros.in for description.  */
 #define BUILD_IDS_NONE 0
@@ -1621,7 +1630,7 @@ exit:
 #define BUILD_IDS_SEPARATE 2
 #define BUILD_IDS_COMPAT   3
 
-static int addNewIDSymlink(FileList fl,
+static int addNewIDSymlink(ARGV_t *files,
   char *targetpath, char *idlinkpath,
   int isDbg, int *dups)
 {
@@ -1670,8 +1679,7 @@ static int addNewIDSymlink(FileList fl,
rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
   linkerr, linkpath, targetpath);
 } else {
-   fl->cur.isDir = 0;
-   rc = addFile(fl, linkpath, NULL);
+   rc = argvAdd(files, linkpath);
 }
 
 if (nr > 0) {
@@ -1709,7 +1717,7 @@ static int addNewIDSymlink(FileList fl,
 return rc;
 }
 
-static int generateBuildIDs(FileList fl)
+static int generateBuildIDs(FileList fl, ARGV_t *files)
 {
 int rc = 0;
 int i;
@@ -1858,18 +1866,9 @@ static int generateBuildIDs(FileList fl)
mainiddir = rpmGetPath(fl->buildRoot, BUILD_ID_DIR, NULL);
debugiddir = rpmGetPath(fl->buildRoot, DEBUG_ID_DIR, NULL);
 
-   /* Make sure to reset all file flags to defaults.
-  Uses parseForAttr to reset ar, arFlags, and specdFlags.
-  Note that parseForAttr pokes at the attrstr, so we cannot
-  just pass a static string. */
-   fl->cur.attrFlags = 0;
-   fl->def.attrFlags = 0;
-   fl->def.verifyFlags = RPMVERIFY_ALL;
-   fl->cur.verifyFlags = RPMVERIFY_ALL;
-   fl->def.specdFlags |= SPECD_VERIFY;
-   fl->cur.specdFlags |= SPECD_VERIFY;
+   /* Make sure to reset all file flags to defaults.  */
attrstr = mkattr(NULL);
-   parseForAttr(fl->pool, attrstr, 1, >def);
+   argvAdd(files, attrstr);
free (attrstr);
 
/* Supported, but questionable.  */
@@ -1881,11 +1880,7 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(mainiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, mainiddir);
} else {
-   attrstr = mkattr(mainiddir);
-   parseForAttr(fl->pool, attrstr, 0, >cur);
-   fl->cur.isDir = 1;
-   rc = addFile(fl, mainiddir, NULL);
-   free (attrstr);
+   argvAddDir(files, mainiddir);
}
}
 
@@ -1893,11 +1888,7 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(debugiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, debugiddir);
} else {
-   attrstr = mkattr(debugiddir);
-   parseForAttr(fl->pool, attrstr, 0, >cur);
-   fl->cur.isDir = 1;
-   rc = addFile(fl, debugiddir, NULL);
-   free (attrstr);
+   argvAddDir(files, debugiddir);
}
}
}
@@ -1936,9 +1927,9 @@ static int generateBuildIDs(FileList fl)
&& (rc = rpmioMkpath(buildidsubdir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, buildidsubdir);
} else {
-   fl->cur.isDir = 1;
-   if (!addsubdir
-   || (rc = addFile(fl, buildidsubdir, NULL)) == 0) {
+   if (addsubdir)
+  argvAddDir (files, buildidsubdir);
+   if (rc == 0) {
char *linkpattern, *targetpattern;
char *linkpath, *targetpath;
int dups = 0;
@@ -1952,7 +1943,7 @@ static int generateBuildIDs(FileList fl)
rasprintf(, linkpattern,
  buildidsubdir, [i][2]);
rasprintf(, targetpattern, paths[i]);
-   rc = addNewIDSymlink(fl, targetpath, linkpath,

[Rpm-maint] [PATCH 3/3] Change mkattr to always create a %defattr with explicitly set modes.

[Mark Wielaard]

mkattr used "-" as default mode which would pick up the mode for files
as they were on disk. This could cause files generated by rpmbuild to
use a "non-standard" mode if umask was set by the user. Explitictly
use 755 for directories and 644 for files to make builds independent
of any umask settings.

This works as is for both files and directories, so no file argument
is necessary anymore.

https://bugzilla.redhat.com/show_bug.cgi?id=1452893
https://bugzilla.redhat.com/show_bug.cgi?id=1458839

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 17 -
 1 file changed, 8 insertions(+), 9 deletions(-)

diff --git a/build/files.c b/build/files.c
index cb7def6..dcc772e 100644
--- a/build/files.c
+++ b/build/files.c
@@ -203,13 +203,12 @@ static void dupAttrRec(const AttrRec oar, AttrRec nar)
 *nar = *oar; /* struct assignment */
 }
 
-static char *mkattr(const char *fn)
+/* Creates a default $defattr string. Can be used with argvAdd().
+   Caller owns the new string which needs to be freed when done.  */
+static char *mkattr(void)
 {
 char *s = NULL;
-if (fn)
-   rasprintf(, "%s(-,%s,%s) %s", "%attr", UID_0_USER, GID_0_GROUP, fn);
-else
-   rasprintf(, "%s(-,%s,%s)", "%defattr", UID_0_USER, GID_0_GROUP);
+rasprintf(, "%s(644,%s,%s,755)", "%defattr", UID_0_USER, GID_0_GROUP);
 return s;
 }
 
@@ -1867,7 +1866,7 @@ static int generateBuildIDs(FileList fl, ARGV_t *files)
debugiddir = rpmGetPath(fl->buildRoot, DEBUG_ID_DIR, NULL);
 
/* Make sure to reset all file flags to defaults.  */
-   attrstr = mkattr(NULL);
+   attrstr = mkattr();
argvAdd(files, attrstr);
free (attrstr);
 
@@ -2825,7 +2824,7 @@ static void filterDebuginfoPackage(rpmSpec spec, Package 
pkg,
if (access(path, F_OK) == 0) {
/* Append the file list preamble */
if (!files) {
-   char *attr = mkattr(NULL);
+   char *attr = mkattr();
argvAdd(, attr);
argvAddDir(, DEBUG_LIB_DIR);
free(attr);
@@ -2885,7 +2884,7 @@ static int addDebugDwz(Package pkg, char *buildroot)
 rasprintf(, "%s%s", buildroot, DEBUG_DWZ_DIR);
 if (lstat(path, ) == 0 && S_ISDIR(sbuf.st_mode)) {
if (!pkg->fileList) {
-   char *attr = mkattr(NULL);
+   char *attr = mkattr();
argvAdd(>fileList, attr);
argvAddDir(>fileList, DEBUG_LIB_DIR);
free(attr);
@@ -2919,7 +2918,7 @@ static int addDebugSrc(Package pkg, char *buildroot)
continue;
rasprintf(, "%s/%s", DEBUG_SRC_DIR, de->d_name);
if (!pkg->fileList) {
-   char *attr = mkattr(NULL);
+   char *attr = mkattr();
argvAdd(>fileList, attr);
free(attr);
}
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 1/3] Extract package file list processing in separate functions.

[Mark Wielaard]

Extract two functions resetPackageFilesDefaults() and addPackageFileList()
from processPackageFiles(). This will make it possible to add multiple
(generated) file lists to a package later.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 170 --
 1 file changed, 107 insertions(+), 63 deletions(-)

diff --git a/build/files.c b/build/files.c
index 6504b8c..c7debdc 100644
--- a/build/files.c
+++ b/build/files.c
@@ -2349,45 +2349,35 @@ static void processSpecialDir(rpmSpec spec, Package 
pkg, FileList fl,
 freeStringBuf(docScript);
 free(mkdocdir);
 }
-   
 
-static rpmRC processPackageFiles(rpmSpec spec, rpmBuildPkgFlags pkgFlags,
-Package pkg, int didInstall, int test)
+
+/* Resets the default settings for files in the package list.
+   Used in processPackageFiles whenever a new set of files is added. */
+static void resetPackageFilesDefaults (struct FileList_s *fl,
+  rpmBuildPkgFlags pkgFlags)
 {
 struct AttrRec_s root_ar = { 0, 0, 0, 0, 0, 0 };
-struct FileList_s fl;
-ARGV_t fileNames = NULL;
-specialDir specialDoc = NULL;
-specialDir specialLic = NULL;
 
-pkg->cpioList = NULL;
+root_ar.ar_user = rpmstrPoolId(fl->pool, UID_0_USER, 1);
+root_ar.ar_group = rpmstrPoolId(fl->pool, GID_0_GROUP, 1);
+dupAttrRec(_ar, >def.ar); /* XXX assume %defattr(-,root,root) */
 
-for (ARGV_const_t fp = pkg->fileFile; fp && *fp != NULL; fp++) {
-   if (readFilesManifest(spec, pkg, *fp))
-   return RPMRC_FAIL;
-}
-/* Init the file list structure */
-memset(, 0, sizeof(fl));
-
-fl.pool = rpmstrPoolLink(spec->pool);
-/* XXX spec->buildRoot == NULL, then xstrdup("") is returned */
-fl.buildRoot = rpmGenPath(spec->rootDir, spec->buildRoot, NULL);
-fl.buildRootLen = strlen(fl.buildRoot);
+fl->def.verifyFlags = RPMVERIFY_ALL;
 
-root_ar.ar_user = rpmstrPoolId(fl.pool, UID_0_USER, 1);
-root_ar.ar_group = rpmstrPoolId(fl.pool, GID_0_GROUP, 1);
-dupAttrRec(_ar, );  /* XXX assume %defattr(-,root,root) */
-
-fl.def.verifyFlags = RPMVERIFY_ALL;
-
-fl.pkgFlags = pkgFlags;
+fl->pkgFlags = pkgFlags;
+}
 
-{  char *docs = rpmGetPath("%{?__docdir_path}", NULL);
-   argvSplit(, docs, ":");
-   free(docs);
-}
-
-for (ARGV_const_t fp = pkg->fileList; *fp != NULL; fp++) {
+/* Adds the given fileList to the package. If fromSpecFileList is not zero
+   then the specialDirs are also filled in and the files are sanitized
+   through processBinaryFile(). Otherwise no special files are processed
+   and the files are added directly through addFile().  */
+static void addPackageFileList (struct FileList_s *fl, Package pkg,
+   ARGV_t *fileList,
+   specialDir *specialDoc, specialDir *specialLic,
+   int fromSpecFileList)
+{
+ARGV_t fileNames = NULL;
+for (ARGV_const_t fp = *fileList; *fp != NULL; fp++) {
char buf[strlen(*fp) + 1];
const char *s = *fp;
SKIPSPACE(s);
@@ -2397,41 +2387,63 @@ static rpmRC processPackageFiles(rpmSpec spec, 
rpmBuildPkgFlags pkgFlags,
rstrlcpy(buf, s, sizeof(buf));

/* Reset for a new line in %files */
-   FileEntryFree();
+   FileEntryFree(>cur);
 
/* turn explicit flags into %def'd ones (gosh this is hacky...) */
-   fl.cur.specdFlags = ((unsigned)fl.def.specdFlags) >> 8;
-   fl.cur.verifyFlags = fl.def.verifyFlags;
-
-   if (parseForVerify(buf, 0, ) ||
-   parseForVerify(buf, 1, ) ||
-   parseForAttr(fl.pool, buf, 0, ) ||
-   parseForAttr(fl.pool, buf, 1, ) ||
-   parseForDev(buf, ) ||
-   parseForConfig(buf, ) ||
-   parseForLang(buf, ) ||
-   parseForCaps(buf, ) ||
-   parseForSimple(buf, , ))
+   fl->cur.specdFlags = ((unsigned)fl->def.specdFlags) >> 8;
+   fl->cur.verifyFlags = fl->def.verifyFlags;
+
+   if (parseForVerify(buf, 0, >cur) ||
+   parseForVerify(buf, 1, >def) ||
+   parseForAttr(fl->pool, buf, 0, >cur) ||
+   parseForAttr(fl->pool, buf, 1, >def) ||
+   parseForDev(buf, >cur) ||
+   parseForConfig(buf, >cur) ||
+   parseForLang(buf, >cur) ||
+   parseForCaps(buf, >cur) ||
+   parseForSimple(buf, >cur, ))
{
-   fl.processingFailed = 1;
+   fl->processingFailed = 1;
continue;
}
 
for (ARGV_const_t fn = fileNames; fn && *fn; fn++) {
-   if (fl.cur.attrFlags & RPMFILE_SPECIALDIR) {
-   rpmFlags oattrs = (fl.cur.attrFlags & ~RPMFILE_SPECIALDIR);
+
+   /* For file lists 

[Rpm-maint] [PATCH 1/2] Test split debuginfo packages with excluded files.

[Mark Wielaard]

If split debuginfo code doesn't know about excluded files it fails:

error: Installed (but unpackaged) file(s) found:
   /usr/lib/debug/bin/hello3-1.0-1.x86_64.debug

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am  |  1 +
 tests/data/SPECS/test-subpackages-exclude.spec | 42 
 tests/rpmbuild.at  | 66 ++
 3 files changed, 109 insertions(+)
 create mode 100644 tests/data/SPECS/test-subpackages-exclude.spec

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 7b69ba0..687992f 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -65,6 +65,7 @@ EXTRA_DIST += data/SPECS/prefixtest.spec
 EXTRA_DIST += data/SPECS/testdoc.spec
 EXTRA_DIST += data/SPECS/sigpipe.spec
 EXTRA_DIST += data/SPECS/test-subpackages.spec
+EXTRA_DIST += data/SPECS/test-subpackages-exclude.spec
 EXTRA_DIST += data/SOURCES/hello-1.0-modernize.patch
 EXTRA_DIST += data/SOURCES/hello-1.0.tar.gz
 EXTRA_DIST += data/SOURCES/hello-2.0.tar.gz
diff --git a/tests/data/SPECS/test-subpackages-exclude.spec 
b/tests/data/SPECS/test-subpackages-exclude.spec
new file mode 100644
index 000..e93b10b
--- /dev/null
+++ b/tests/data/SPECS/test-subpackages-exclude.spec
@@ -0,0 +1,42 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%package test2
+Summary: Test2.
+%description test2
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+cp hello.c hello2.c
+gcc -g hello2.c -o hello2
+cp hello.c hello3.c
+gcc -g hello3.c -o hello3
+
+%install
+mkdir -p %{buildroot}/bin
+install -D -p -m 0755 -t %{buildroot}/bin hello
+install -D -p -m 0755 -t %{buildroot}/bin hello2
+# Install it, but then exclude it...
+install -D -p -m 0755 -t %{buildroot}/bin hello3
+
+%files
+/bin/hello
+
+%files test2
+/bin/hello2
+%exclude /bin/hello3
+
+%changelog
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index e2008da..cbe653d 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -1166,3 +1166,69 @@ hello3 debug exists
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that defining _debuginfo_subpackages works with excluded files.
+AT_SETUP([rpmbuild debuginfo subpackages multiple excluded])
+AT_KEYWORDS([build] [debuginfo] [debugsubpackage] [debugsource])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define "_unique_debug_names 1" \
+  --define "_unique_debug_srcs 1" \
+  --define "_debugsource_packages 1" \
+  --define "_debuginfo_subpackages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/test-subpackages-exclude.spec
+
+# Check that there are 2 debuginfo packages.
+ls ${abs_builddir}/testing/build/RPMS/*/*debuginfo*rpm | wc --lines
+
+# First contains hello.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section, first string)
+debug_name=$(readelf -p .gnu_debuglink ./bin/hello | grep hello | cut -c13-)
+
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-debuginfo-1.0-1.*.rpm \
+  | cpio -diu --quiet
+if test -f ./usr/lib/debug/bin/$debug_name; then
+  echo "hello debug exists"
+else
+  echo "No hello: $debug_name"
+fi
+
+# Second contains hello2.debug but NOT hello3.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-test2-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section, first string)
+debug_name=$(readelf -p .gnu_debuglink ./bin/hello2 | grep hello | cut -c13-)
+
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-test2-debuginfo-1.0-1.*.rpm 
\
+  | cpio -diu --quiet
+if test -f ./usr/lib/debug/bin/$debug_name; then
+  echo "hello2 debug exists"
+else
+  echo "No hello2: $debug_name"
+fi
+
+if test -f ./usr/lib/debug/bin/hello3*; then
+  echo "hello3 debug exists"
+else
+  echo "No hello3 debug"
+fi
+],
+[0],
+[2
+hello debug exists
+hello2 debug exists
+No hello3 debug
+],
+[ignore])
+AT_CLEANUP
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH 2/2] Test split debuginfo packages with RemovePathPostfixes.

[Mark Wielaard]

If split debuginfo code doesn't know about RemovePathPostfixes it fails:

error: Installed (but unpackaged) file(s) found:
   /usr/lib/debug/bin/hello.foobar-1.0-1.x86_64.debug

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am  |  1 +
 .../data/SPECS/test-subpackages-pathpostfixes.spec | 39 
 tests/rpmbuild.at  | 71 ++
 3 files changed, 111 insertions(+)
 create mode 100644 tests/data/SPECS/test-subpackages-pathpostfixes.spec

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 687992f..03cdea9 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -66,6 +66,7 @@ EXTRA_DIST += data/SPECS/testdoc.spec
 EXTRA_DIST += data/SPECS/sigpipe.spec
 EXTRA_DIST += data/SPECS/test-subpackages.spec
 EXTRA_DIST += data/SPECS/test-subpackages-exclude.spec
+EXTRA_DIST += data/SPECS/test-subpackages-pathpostfixes.spec
 EXTRA_DIST += data/SOURCES/hello-1.0-modernize.patch
 EXTRA_DIST += data/SOURCES/hello-1.0.tar.gz
 EXTRA_DIST += data/SOURCES/hello-2.0.tar.gz
diff --git a/tests/data/SPECS/test-subpackages-pathpostfixes.spec 
b/tests/data/SPECS/test-subpackages-pathpostfixes.spec
new file mode 100644
index 000..19b3b9e
--- /dev/null
+++ b/tests/data/SPECS/test-subpackages-pathpostfixes.spec
@@ -0,0 +1,39 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%package test2
+RemovePathPostfixes: .foobar
+Summary: Test2.
+%description test2
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+cp hello.c hello2.c
+gcc -g hello2.c -o hello.foobar
+
+%install
+mkdir -p %{buildroot}/bin
+install -D -p -m 0755 -t %{buildroot}/bin hello
+# Install as hello.foobar, but we want the postfix removed in the package...
+install -D -p -m 0755 -t %{buildroot}/bin hello.foobar
+
+%files
+/bin/hello
+
+%files test2
+/bin/hello.foobar
+
+%changelog
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index cbe653d..7864477 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -1232,3 +1232,74 @@ No hello3 debug
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that defining _debuginfo_subpackages works with RemovePathPostfixes.
+AT_SETUP([rpmbuild debuginfo subpackages multiple excluded])
+AT_KEYWORDS([build] [debuginfo] [debugsubpackage] [debugsource])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define "_unique_debug_names 1" \
+  --define "_unique_debug_srcs 1" \
+  --define "_debugsource_packages 1" \
+  --define "_debuginfo_subpackages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/test-subpackages-pathpostfixes.spec
+
+# Check that there are 2 debuginfo packages.
+ls ${abs_builddir}/testing/build/RPMS/*/*debuginfo*rpm | wc --lines
+
+# First contains hello.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section, first string)
+debug_name=$(readelf -p .gnu_debuglink ./bin/hello | grep hello | cut -c13-)
+
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-debuginfo-1.0-1.*.rpm \
+  | cpio -diu --quiet
+if test -f ./usr/lib/debug/bin/$debug_name; then
+  echo "hello debug exists"
+else
+  echo "No hello: $debug_name"
+fi
+
+# remove it, we are going to check the other debuginfo package.
+rm ./bin/hello
+rm ./usr/lib/debug/bin/$debug_name
+orig_debugname=$debugname
+
+# Second contains hello.foobar.debug but NOT hello.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-test2-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section, first string)
+debug_name=$(readelf -p .gnu_debuglink ./bin/hello | grep hello | cut -c13-)
+
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-test2-debuginfo-1.0-1.*.rpm 
\
+  | cpio -diu --quiet
+if test -f ./usr/lib/debug/bin/$debug_name; then
+  echo "hello.foobar debug exists"
+else
+  echo "No hello2: $debug_name"
+fi
+
+if test -f ./usr/lib/debug/bin/$orig_debugname; then
+  echo "$orig_debugname exists"
+else
+  echo "No hello.debug"
+fi
+],
+[0],
+[2
+hello debug exists
+hello.foobar debug exists
+No hello.debug
+],
+[ignore])
+AT_CLEANUP
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Add rpmbuild debuginfo subpackages tests.

[Mark Wielaard]

This adds various tests for making sure multiple subpackages are build
correctly. Without debuginfo subpackages, with subpackages, subpackages
with unique debug file and source dir paths and with separate debugsources.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 tests/Makefile.am  |   1 +
 tests/data/SPECS/test-subpackages.spec |  47 ++
 tests/rpmbuild.at  | 266 +
 3 files changed, 314 insertions(+)
 create mode 100644 tests/data/SPECS/test-subpackages.spec

diff --git a/tests/Makefile.am b/tests/Makefile.am
index 09239ba..7b69ba0 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -64,6 +64,7 @@ EXTRA_DIST += data/SPECS/filetriggers.spec
 EXTRA_DIST += data/SPECS/prefixtest.spec
 EXTRA_DIST += data/SPECS/testdoc.spec
 EXTRA_DIST += data/SPECS/sigpipe.spec
+EXTRA_DIST += data/SPECS/test-subpackages.spec
 EXTRA_DIST += data/SOURCES/hello-1.0-modernize.patch
 EXTRA_DIST += data/SOURCES/hello-1.0.tar.gz
 EXTRA_DIST += data/SOURCES/hello-2.0.tar.gz
diff --git a/tests/data/SPECS/test-subpackages.spec 
b/tests/data/SPECS/test-subpackages.spec
new file mode 100644
index 000..8306600
--- /dev/null
+++ b/tests/data/SPECS/test-subpackages.spec
@@ -0,0 +1,47 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%package test2
+Summary: Test2.
+%description test2
+
+%package test3
+Summary: Test3.
+%description test3
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+cp hello.c hello2.c
+gcc -g hello2.c -o hello2
+cp hello.c hello3.c
+gcc -g hello3.c -o hello3
+
+%install
+mkdir -p %{buildroot}/bin
+install -D -p -m 0755 -t %{buildroot}/bin hello
+install -D -p -m 0755 -t %{buildroot}/bin hello2
+install -D -p -m 0755 -t %{buildroot}/bin hello3
+
+%files
+/bin/hello
+
+%files test2
+/bin/hello2
+
+%files test3
+/bin/hello3
+
+%changelog
diff --git a/tests/rpmbuild.at b/tests/rpmbuild.at
index 3f41652..e2008da 100644
--- a/tests/rpmbuild.at
+++ b/tests/rpmbuild.at
@@ -900,3 +900,269 @@ ls ./usr/src/debug/test-1.0*/
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that undefining _debuginfo_subpackages creates one single -debuginfo.
+AT_SETUP([rpmbuild debuginfo subpackages single])
+AT_KEYWORDS([build] [debuginfo] [debugsubpackage])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --undefine "_unique_debug_names" \
+  --undefine "_unique_debug_srcs" \
+  --undefine "_debugsource_packages" \
+  --undefine "_debuginfo_subpackages" \
+  -ba "${abs_srcdir}"/data/SPECS/test-subpackages.spec
+
+# Check that there are is just one debuginfo packages.
+ls ${abs_builddir}/testing/build/RPMS/*/*debuginfo*rpm | wc --lines
+
+# Which contains hello.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section, first string)
+debug_name=$(readelf -p .gnu_debuglink ./bin/hello | grep hello | cut -c13-)
+
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-debuginfo-1.0-1.*.rpm \
+  | cpio -diu --quiet
+if test -f ./usr/lib/debug/bin/$debug_name; then
+  echo "hello debug exists"
+else
+  echo "No hello: $debug_name"
+fi
+],
+[0],
+[1
+hello debug exists
+],
+[ignore])
+AT_CLEANUP
+
+# --
+# Check that defining _debuginfo_subpackages creates multiple -debuginfos.
+AT_SETUP([rpmbuild debuginfo subpackages multiple])
+AT_KEYWORDS([build] [debuginfo] [debugsubpackage])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+run rpmbuild --quiet \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --undefine "_unique_debug_names" \
+  --undefine "_unique_debug_srcs" \
+  --undefine "_debugsource_packages" \
+  --define "_debuginfo_subpackages 1" \
+  -ba "${abs_srcdir}"/data/SPECS/test-subpackages.spec
+
+# Check that there are 3 debuginfo packages.
+ls ${abs_builddir}/testing/build/RPMS/*/*debuginfo*rpm | wc --lines
+
+# First contains hello.debug
+rpm2cpio ${abs_builddir}/testing/build/RPMS/*/test-1.0-1.*.rpm \
+  | cpio -diu --quiet
+# Extract the debug name from the exe (.gnu_debuglink section,

[Rpm-maint] [PATCH] Make sure that debugsourcefiles.list is generated in the build dir.

[Mark Wielaard]

The %_debugsource_template expects the debugsourcefiles.list file
to be in the (current) build dir. Make sure that is always the case
even if find-debuginfo.sh was invoked in a different (sub) directory
by adding the build dir path as explicit argument to -S.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 macros.in | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/macros.in b/macros.in
index 8518228..147db83 100644
--- a/macros.in
+++ b/macros.in
@@ -185,7 +185,7 @@
 %{?_unique_debug_srcs:--unique-debug-src-base 
"%{name}-%{VERSION}-%{RELEASE}.%{_arch}"} \\\
 %{?_find_debuginfo_dwz_opts} \\\
 %{?_find_debuginfo_opts} \\\
-%{?_debugsource_packages:-S debugsourcefiles.list} \\\
+%{?_debugsource_packages:-S 
"%{_builddir}/%{?buildsubdir}/debugsourcefiles.list"} \\\
 "%{_builddir}/%{?buildsubdir}"\
 %{nil}
 
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] find-debuginfo.sh: Don't create dwz multi file if there is only one .debug.

[Mark Wielaard]

dwz -m multi only works when there are multiple .debug input files.
With just one .debug file it doesn't really make sense to extract
the shared debug info into a separate file and dwz will complain:

  dwz: Too few files for multifile optimization.

So only add -m multi if there is more than one .debug file.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 scripts/find-debuginfo.sh | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index aaf4c75..7ab39a2 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -442,8 +442,8 @@ fi
 # Invoke the DWARF Compressor utility.
 if $run_dwz \
&& [ -d "${RPM_BUILD_ROOT}/usr/lib/debug" ]; then
-  dwz_files="`cd "${RPM_BUILD_ROOT}/usr/lib/debug"; find -type f -name 
\*.debug`"
-  if [ -n "${dwz_files}" ]; then
+  readarray dwz_files < <(cd "${RPM_BUILD_ROOT}/usr/lib/debug"; find -type f 
-name \*.debug)
+  if [ ${#dwz_files[@]} -gt 0 ]; then
 
dwz_multifile_name="${RPM_PACKAGE_NAME}-${RPM_PACKAGE_VERSION}-${RPM_PACKAGE_RELEASE}.${RPM_ARCH}"
 dwz_multifile_suffix=
 dwz_multifile_idx=0
@@ -452,14 +452,16 @@ if $run_dwz \
   dwz_multifile_suffix=".${dwz_multifile_idx}"
 done
 dwz_multfile_name="${dwz_multifile_name}${dwz_multifile_suffix}"
-dwz_opts="-h -q -r -m .dwz/${dwz_multifile_name}"
+dwz_opts="-h -q -r"
+[ ${#dwz_files[@]} -gt 1 ] \
+  && dwz_opts="${dwz_opts} -m .dwz/${dwz_multifile_name}"
 mkdir -p "${RPM_BUILD_ROOT}/usr/lib/debug/.dwz"
 [ -n "${dwz_low_mem_die_limit}" ] \
   && dwz_opts="${dwz_opts} -l ${dwz_low_mem_die_limit}"
 [ -n "${dwz_max_die_limit}" ] \
   && dwz_opts="${dwz_opts} -L ${dwz_max_die_limit}"
 if type dwz >/dev/null 2>&1; then
-  ( cd "${RPM_BUILD_ROOT}/usr/lib/debug" && dwz $dwz_opts $dwz_files )
+  ( cd "${RPM_BUILD_ROOT}/usr/lib/debug" && dwz $dwz_opts ${dwz_files[@]} )
 else
   echo >&2 "*** ERROR: DWARF compression requested, but no dwz installed"
   exit 2
-- 
1.8.3.1

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Update find-debuginfo.sh options and macros documentation.

[Mark Wielaard]

This adds some missing documentation for rpm macros and find-debuginfo.sh
options that were recently added (or renamed). -j N, --build-id-seed SEED,
--unique-debug-suffix SUFFIX and --unique-debug-src-base BASE.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 macros.in | 14 --
 scripts/find-debuginfo.sh | 28 ++--
 2 files changed, 30 insertions(+), 12 deletions(-)

diff --git a/macros.in b/macros.in
index a2c15ad..78352a8 100644
--- a/macros.in
+++ b/macros.in
@@ -522,8 +522,9 @@ package or when debugging this package.\
 
 # Whether build-ids should be made unique between package version/releases
 # when generating debuginfo packages. If set to 1 this will pass
-# --ver-rel "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will pass it
-# onto debugedit --build-id-seed to be used to prime the build-id note hash.
+# --build-id-seed "%{VERSION}-%{RELEASE}" to find-debuginfo.sh which will
+# pass it onto debugedit --build-id-seed to be used to prime the build-id
+# note hash.
 %_unique_build_ids 1
 
 # Do not recompute build-ids but keep whatever is in the ELF file already.
@@ -533,15 +534,16 @@ package or when debugging this package.\
 
 # Whether .debug files should be made unique between package version,
 # release and architecture. If set to 1 this will pass
-# --unique-debug-arch "%{_arch}" to find-debuginfo.sh to create
-# debuginfo files which end in --..debug
+# --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch} find-debuginfo.sh
+# to create debuginfo files which end in --..debug
 # Requires _unique_build_ids.
 %_unique_debug_names   1
 
 # Whether the /usr/debug/src/ directories should be unique between
 # package version, release and architecture. If set to 1 this will pass
-# --unique-debug-src-base "%{name}" to find-debuginfo.sh to name the
-# directory under /usr/debug/src as --.
+# --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}" to
+# find-debuginfo.sh to name the directory under /usr/debug/src as
+# --..
 %_unique_debug_srcs1
 
 # Whether rpm should put debug source files into its own subpackage
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index aaf4c75..0511777 100755
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -3,11 +3,14 @@
 #for inclusion in an rpm spec file.
 #
 # Usage: find-debuginfo.sh [--strict-build-id] [-g] [-r] [-m] [-i] [-n]
+# [-j N]
 # [-o debugfiles.list]
 # [-S debugsourcefiles.list]
 # [--run-dwz] [--dwz-low-mem-die-limit N]
 # [--dwz-max-die-limit N]
-# [--build-id-seed VERSION-RELEASE]
+# [--build-id-seed SEED]
+# [--unique-debug-suffix SUFFIX]
+# [--unique-debug-src-base BASE]
 # [[-l filelist]... [-p 'pattern'] -o debuginfo.list]
 # [builddir]
 #
@@ -19,6 +22,9 @@
 # The -i flag says to include a .gdb_index section in the .debug file.
 # The -n flag says to not recompute the build-id.
 #
+# The -j N option will spawn N processes to do the debuginfo extraction
+# in parallel.
+#
 # A single -o switch before any -l or -p switches simply renames
 # the primary output file from debugfiles.list to something else.
 # A -o switch that follows a -p switch or some -l switches produces
@@ -31,11 +37,21 @@
 # if available, and --dwz-low-mem-die-limit and --dwz-max-die-limit
 # provide detailed limits.  See dwz(1) -l and -L option for details.
 #
-# If --build-id-seed VERSION-RELEASE is given then debugedit is called to
-# update the build-ids it finds adding the VERSION-RELEASE string as
-# seed to recalculate the build-id hash.  This makes sure the
-# build-ids in the ELF files are unique between versions and releases
-# of the same package.
+# If --build-id-seed SEED is given then debugedit is called to
+# update the build-ids it finds adding the SEED as seed to recalculate
+# the build-id hash.  This makes sure the build-ids in the ELF files
+# are unique between versions and releases of the same package.
+# (Use --build-id-seed "%{VERSION}-%{RELEASE}".)
+#
+# If --unique-debug-suffix SUFFIX is given then the debug files created
+# for  will be named -.debug.  This makes sure .debug
+# are unique between package version, release and architecture.
+# (Use --unique-debug-suffix "-%{VERSION}-%{RELEASE}.%{_arch}".)
+#
+# If --unique-debug-src-base BASE is given then the source directory
+# will be called /usr/debug/src/.  This makes sure the debug source
+# directories are unique between package version, release and architecture.
+# (Use --unique-debug-src-base "%{name}-%{VERSION}-%{RELEASE}.%{_arch}".)
 #
 # All file names in switches are relative to builddir (. if not given).
 #
-- 
1.8.3.1

Re: [Rpm-maint] [PATCH] Use a file list to add build-id files to pkgList and explicitly set attrs.

[Mark Wielaard]

Hi,

Panu sadi on irc he didn't like the duplication of code that parsed the
spec file lists. So this updated patch extracts the setup and parsing
loop in their own function and just calls them twice. I also reformatted
the patch a little so the whitespace differences are minimal.

Cheers,

Mark

From 739796798ac854f80ae2f0d677f74bca734055f7 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@klomp.org>
Date: Wed, 21 Jun 2017 16:57:13 +0200
Subject: [PATCH] Use a file list to add build-id files to pkgList and
 explicitly set attrs.

mkattr used "-" as default mode which would pick up the mode for files
as they were on disk. This could cause files generated by rpmbuild to
use a "non-standard" mode if umask was set by the user. Explitictly
use 755 for directories and 644 for files to make builds independent
of any umask settings.

Change the generation of build-id files to a file list using ARGV_t.
First go through the current package list and generate a files list.
Then add those files using the defaults mode/attr settings as if they
were part of the original package file list.

https://bugzilla.redhat.com/show_bug.cgi?id=1452893
https://bugzilla.redhat.com/show_bug.cgi?id=1458839

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 244 +-
 1 file changed, 138 insertions(+), 106 deletions(-)

diff --git a/build/files.c b/build/files.c
index 4911162d1..a90af3bee 100644
--- a/build/files.c
+++ b/build/files.c
@@ -207,9 +207,9 @@ static char *mkattr(const char *fn)
 {
 char *s = NULL;
 if (fn)
-	rasprintf(, "%s(-,%s,%s) %s", "%attr", UID_0_USER, GID_0_GROUP, fn);
+	rasprintf(, "%s(755,%s,%s) %s", "%attr", UID_0_USER, GID_0_GROUP, fn);
 else
-	rasprintf(, "%s(-,%s,%s)", "%defattr", UID_0_USER, GID_0_GROUP);
+	rasprintf(, "%s(644,%s,%s,755)", "%defattr", UID_0_USER, GID_0_GROUP);
 return s;
 }
 
@@ -1614,6 +1614,15 @@ exit:
 return rc;
 }
 
+/* add a directory to the file list */
+static void argvAddDir(ARGV_t *filesp, const char *dir)
+{
+char *line = NULL;
+rasprintf(, "%%dir %s", dir);
+argvAdd(filesp, line);
+_free(line);
+}
+
 #if HAVE_LIBDW
 /* How build id links are generated.  See macros.in for description.  */
 #define BUILD_IDS_NONE 0
@@ -1621,7 +1630,7 @@ exit:
 #define BUILD_IDS_SEPARATE 2
 #define BUILD_IDS_COMPAT   3
 
-static int addNewIDSymlink(FileList fl,
+static int addNewIDSymlink(ARGV_t *files,
 			   char *targetpath, char *idlinkpath,
 			   int isDbg, int *dups)
 {
@@ -1670,8 +1679,7 @@ static int addNewIDSymlink(FileList fl,
 	rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
 	   linkerr, linkpath, targetpath);
 } else {
-	fl->cur.isDir = 0;
-	rc = addFile(fl, linkpath, NULL);
+	rc = argvAdd(files, linkpath);
 }
 
 if (nr > 0) {
@@ -1709,7 +1717,7 @@ static int addNewIDSymlink(FileList fl,
 return rc;
 }
 
-static int generateBuildIDs(FileList fl)
+static int generateBuildIDs(FileList fl, ARGV_t *files)
 {
 int rc = 0;
 int i;
@@ -1858,18 +1866,9 @@ static int generateBuildIDs(FileList fl)
 	mainiddir = rpmGetPath(fl->buildRoot, BUILD_ID_DIR, NULL);
 	debugiddir = rpmGetPath(fl->buildRoot, DEBUG_ID_DIR, NULL);
 
-	/* Make sure to reset all file flags to defaults.
-	   Uses parseForAttr to reset ar, arFlags, and specdFlags.
-	   Note that parseForAttr pokes at the attrstr, so we cannot
-	   just pass a static string. */
-	fl->cur.attrFlags = 0;
-	fl->def.attrFlags = 0;
-	fl->def.verifyFlags = RPMVERIFY_ALL;
-	fl->cur.verifyFlags = RPMVERIFY_ALL;
-	fl->def.specdFlags |= SPECD_VERIFY;
-	fl->cur.specdFlags |= SPECD_VERIFY;
+	/* Make sure to reset all file flags to defaults.  */
 	attrstr = mkattr(NULL);
-	parseForAttr(fl->pool, attrstr, 1, >def);
+	argvAdd(files, attrstr);
 	free (attrstr);
 
 	/* Supported, but questionable.  */
@@ -1881,11 +1880,7 @@ static int generateBuildIDs(FileList fl)
 		if ((rc = rpmioMkpath(mainiddir, 0755, -1, -1)) != 0) {
 		rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, mainiddir);
 		} else {
-		attrstr = mkattr(mainiddir);
-		parseForAttr(fl->pool, attrstr, 0, >cur);
-		fl->cur.isDir = 1;
-		rc = addFile(fl, mainiddir, NULL);
-		free (attrstr);
+		argvAddDir(files, mainiddir);
 		}
 	}
 
@@ -1893,11 +1888,7 @@ static int generateBuildIDs(FileList fl)
 		if ((rc = rpmioMkpath(debugiddir, 0755, -1, -1)) != 0) {
 		rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, debugiddir);
 		} else {
-		attrstr = mkattr(debugiddir);
-		parseForAttr(fl->pool, attrstr, 0, >cur);
-		fl->cur.isDir = 1;
-		rc = addFile(fl, debugiddir, NULL);
-		free (attrstr);
+		argvAddDir(files, debugiddir);
 		}
 	}
 	}
@@ -1936,9 +1927,

[Rpm-maint] [PATCH] rpmbuild: Reset attrFlags in generateBuildIDs.

[Mark Wielaard]

Debuginfo directories and files could be marked as configuration files
if the file list ended with a config file.

Patch by Panu Matilainen. Testcase by me.

https://bugzilla.redhat.com/show_bug.cgi?id=1449732

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c  |  2 ++
 tests/Makefile.am  |  1 +
 tests/data/SPECS/hello-config-buildid.spec | 30 
 tests/rpmbuildid.at| 32 ++
 4 files changed, 65 insertions(+)
 create mode 100644 tests/data/SPECS/hello-config-buildid.spec

diff --git a/build/files.c b/build/files.c
index 791bbe2..86c601b 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1860,6 +1860,8 @@ static int generateBuildIDs(FileList fl)
   Uses parseForAttr to reset ar, arFlags, and specdFlags.
   Note that parseForAttr pokes at the attrstr, so we cannot
   just pass a static string. */
+   fl->cur.attrFlags = 0;
+   fl->def.attrFlags = 0;
fl->def.verifyFlags = RPMVERIFY_ALL;
fl->cur.verifyFlags = RPMVERIFY_ALL;
fl->def.specdFlags |= SPECD_VERIFY;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a778af1..8670dcf 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -76,6 +76,7 @@ EXTRA_DIST += data/SRPMS/foo-1.0-1.src.rpm
 EXTRA_DIST += data/SRPMS/hello-1.0-1.src.rpm
 EXTRA_DIST += data/SOURCES/hello.c
 EXTRA_DIST += data/SPECS/hello-attr-buildid.spec
+EXTRA_DIST += data/SPECS/hello-config-buildid.spec
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.pub
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.secret
 
diff --git a/tests/data/SPECS/hello-config-buildid.spec 
b/tests/data/SPECS/hello-config-buildid.spec
new file mode 100644
index 000..346f266
--- /dev/null
+++ b/tests/data/SPECS/hello-config-buildid.spec
@@ -0,0 +1,30 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+
+%install
+mkdir -p %{buildroot}%{_bindir}
+install -D -p -m 0755 -t %{buildroot}%{_bindir} hello
+mkdir -p %{buildroot}%{_sysconfdir}
+echo "settings" > %{buildroot}%{_sysconfdir}/config.file
+
+%files
+%attr(644,root,root) %{_bindir}/hello
+%config(noreplace) %{_sysconfdir}/config.file
+
+%changelog
diff --git a/tests/rpmbuildid.at b/tests/rpmbuildid.at
index 88ce226..3511673 100644
--- a/tests/rpmbuildid.at
+++ b/tests/rpmbuildid.at
@@ -1324,3 +1324,35 @@ run rpm -qp --qf "[[%{filenames} %{filemodes:perms}\n]]" 
\
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that build-id directories are created with the right attributes
+# even if the spec file sets config explicitly.
+AT_SETUP([rpmbuild buildid config attrs])
+AT_KEYWORDS([build] [debuginfo] [buildid])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+# Build, contains one ELF which should have a buildid.
+run rpmbuild \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define="_build_id_links compat" \
+  --define "_unique_debug_names 1" \
+  --define "_unique_debug_srcs 1" \
+  --quiet -ba "${abs_srcdir}"/data/SPECS/hello-config-buildid.spec
+
+# Should contain one config file.
+run rpm -c -qp ${abs_builddir}/testing/build/RPMS/*/test-1.0-1*rpm
+# Should not contain config files.
+run rpm -c -qp ${abs_builddir}/testing/build/RPMS/*/test-debuginfo-1.0-1*rpm
+],
+[0],
+[/usr/local/etc/config.file
+],
+[ignore])
+AT_CLEANUP
-- 
2.9.3

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

Re: [Rpm-maint] [PATCH] rpmbuild: Reset attrFlags in generateBuildIDs.

[Mark Wielaard]

On Tue, May 23, 2017 at 02:01:46PM +0300, Panu Matilainen wrote:
> Hmm, I'm getting FAILED on the test:
> 
> --- - 2017-05-23 10:52:13.874228079 +
> +++ /home/pmatilai/repos/rpm/tests/rpmtests.dir/at-groups/144/stdout
> 2017-05-23 10:52:13.870950157 +
> @@ -1,2 +1,2 @@
> -/usr/local/etc/config.file
> +/etc/config.file
> 
> ...so probably because of different ./configure options to rpm itself that
> are inherited by the macro. %{_bindir}, %{_sysconfdir} etc are best avoided
> in the test-suite to avoid these issues.

O right of course. What about the attached which just uses /bin and /etc.
The actual path used doesn't really matter for the test and this should
work whatever --prefix you happen to configure with.

Thanks,

Mark
>From 396a52016dfde76a6d3c89b15d998a30341517b5 Mon Sep 17 00:00:00 2001
From: Mark Wielaard <m...@klomp.org>
Date: Fri, 19 May 2017 23:11:39 +0200
Subject: [PATCH] rpmbuild: Reset attrFlags in generateBuildIDs.

Debuginfo directories and files could be marked as configuration files
if the file list ended with a config file.

Patch by Panu Matilainen. Testcase by me.

https://bugzilla.redhat.com/show_bug.cgi?id=1449732

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c  |  2 ++
 tests/Makefile.am  |  1 +
 tests/data/SPECS/hello-config-buildid.spec | 30 
 tests/rpmbuildid.at| 32 ++
 4 files changed, 65 insertions(+)
 create mode 100644 tests/data/SPECS/hello-config-buildid.spec

diff --git a/build/files.c b/build/files.c
index 791bbe2..86c601b 100644
--- a/build/files.c
+++ b/build/files.c
@@ -1860,6 +1860,8 @@ static int generateBuildIDs(FileList fl)
   Uses parseForAttr to reset ar, arFlags, and specdFlags.
   Note that parseForAttr pokes at the attrstr, so we cannot
   just pass a static string. */
+   fl->cur.attrFlags = 0;
+   fl->def.attrFlags = 0;
fl->def.verifyFlags = RPMVERIFY_ALL;
fl->cur.verifyFlags = RPMVERIFY_ALL;
fl->def.specdFlags |= SPECD_VERIFY;
diff --git a/tests/Makefile.am b/tests/Makefile.am
index a778af1..8670dcf 100644
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -76,6 +76,7 @@ EXTRA_DIST += data/SRPMS/foo-1.0-1.src.rpm
 EXTRA_DIST += data/SRPMS/hello-1.0-1.src.rpm
 EXTRA_DIST += data/SOURCES/hello.c
 EXTRA_DIST += data/SPECS/hello-attr-buildid.spec
+EXTRA_DIST += data/SPECS/hello-config-buildid.spec
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.pub
 EXTRA_DIST += data/keys/rpm.org-rsa-2048-test.secret
 
diff --git a/tests/data/SPECS/hello-config-buildid.spec 
b/tests/data/SPECS/hello-config-buildid.spec
new file mode 100644
index 000..ca2d30d
--- /dev/null
+++ b/tests/data/SPECS/hello-config-buildid.spec
@@ -0,0 +1,30 @@
+Name:   test
+Version:1.0
+Release:1
+Summary:Test
+
+License:Public Domain
+URL:https://fedoraproject.org
+Source: hello.c
+
+%description
+%{summary}.
+
+%prep
+%autosetup -c -D -T
+cp -a %{S:0} .
+
+%build
+gcc -g hello.c -o hello
+
+%install
+mkdir -p %{buildroot}/bin
+install -D -p -m 0755 -t %{buildroot}/bin hello
+mkdir -p %{buildroot}/etc
+echo "settings" > %{buildroot}/etc/config.file
+
+%files
+%attr(644,root,root) /bin/hello
+%config(noreplace) /etc/config.file
+
+%changelog
diff --git a/tests/rpmbuildid.at b/tests/rpmbuildid.at
index 88ce226..4fab3d5 100644
--- a/tests/rpmbuildid.at
+++ b/tests/rpmbuildid.at
@@ -1324,3 +1324,35 @@ run rpm -qp --qf "[[%{filenames} %{filemodes:perms}\n]]" 
\
 ],
 [ignore])
 AT_CLEANUP
+
+# --
+# Check that build-id directories are created with the right attributes
+# even if the spec file sets config explicitly.
+AT_SETUP([rpmbuild buildid config attrs])
+AT_KEYWORDS([build] [debuginfo] [buildid])
+AT_CHECK([
+rm -rf ${TOPDIR}
+AS_MKDIR_P(${TOPDIR}/SOURCES)
+
+# Setup sources
+cp "${abs_srcdir}"/data/SOURCES/hello.c ${TOPDIR}/SOURCES
+
+# Build, contains one ELF which should have a buildid.
+run rpmbuild \
+  
--macros=${abs_top_builddir}/macros:${abs_top_builddir}/tests/testing/usr/local/lib/rpm/platform/%{_target_cpu}-%{_target_os}/macros:${top_srcdir}/macros.debug
 \
+  --rcfile=${abs_top_builddir}/rpmrc \
+  --define="_build_id_links compat" \
+  --define "_unique_debug_names 1" \
+  --define "_unique_debug_srcs 1" \
+  --quiet -ba "${abs_srcdir}"/data/SPECS/hello-config-buildid.spec
+
+# Should contain one config file.
+run rpm -c -qp ${abs_builddir}/testing/build/RPMS/*/test-1.0-1*rpm
+# Should not contain config files.
+run rpm -c -qp ${abs_builddir}/testing/build/RPMS/*/test-debuginfo-1.0-1*rpm
+],
+[0],
+[/etc/config.file
+],
+[ignore])
+AT_CLEANUP
-- 
2.9.4

___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint

[Rpm-maint] [PATCH] Use a file list to add build-id files to pkgList and explicitly set attrs.

[Mark Wielaard]

When setting attributes with %attr or %defattr we want to be explicit
about the mode and not just use - to pick up the mode from the file
on disk.

Change the generation of build-id files to a file list using ARGV_t.
First go through the current package list and generate a files list.
Then add those files using the defaults mode/attr settings as if they
were part of the original package file list.

https://bugzilla.redhat.com/show_bug.cgi?id=1452893
https://bugzilla.redhat.com/show_bug.cgi?id=1458839

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 310 +++---
 1 file changed, 167 insertions(+), 143 deletions(-)

diff --git a/build/files.c b/build/files.c
index 4911162..be72f0e 100644
--- a/build/files.c
+++ b/build/files.c
@@ -207,9 +207,9 @@ static char *mkattr(const char *fn)
 {
 char *s = NULL;
 if (fn)
-   rasprintf(, "%s(-,%s,%s) %s", "%attr", UID_0_USER, GID_0_GROUP, fn);
+   rasprintf(, "%s(755,%s,%s) %s", "%attr", UID_0_USER, GID_0_GROUP, fn);
 else
-   rasprintf(, "%s(-,%s,%s)", "%defattr", UID_0_USER, GID_0_GROUP);
+   rasprintf(, "%s(644,%s,%s,755)", "%defattr", UID_0_USER, GID_0_GROUP);
 return s;
 }
 
@@ -1614,6 +1614,15 @@ exit:
 return rc;
 }
 
+/* add a directory to the file list */
+static void argvAddDir(ARGV_t *filesp, const char *dir)
+{
+char *line = NULL;
+rasprintf(, "%%dir %s", dir);
+argvAdd(filesp, line);
+_free(line);
+}
+
 #if HAVE_LIBDW
 /* How build id links are generated.  See macros.in for description.  */
 #define BUILD_IDS_NONE 0
@@ -1621,7 +1630,7 @@ exit:
 #define BUILD_IDS_SEPARATE 2
 #define BUILD_IDS_COMPAT   3
 
-static int addNewIDSymlink(FileList fl,
+static int addNewIDSymlink(ARGV_t *files,
   char *targetpath, char *idlinkpath,
   int isDbg, int *dups)
 {
@@ -1670,8 +1679,7 @@ static int addNewIDSymlink(FileList fl,
rpmlog(RPMLOG_ERR, "%s: %s -> %s: %m\n",
   linkerr, linkpath, targetpath);
 } else {
-   fl->cur.isDir = 0;
-   rc = addFile(fl, linkpath, NULL);
+   rc = argvAdd(files, linkpath);
 }
 
 if (nr > 0) {
@@ -1709,7 +1717,7 @@ static int addNewIDSymlink(FileList fl,
 return rc;
 }
 
-static int generateBuildIDs(FileList fl)
+static int generateBuildIDs(FileList fl, ARGV_t *files)
 {
 int rc = 0;
 int i;
@@ -1858,18 +1866,9 @@ static int generateBuildIDs(FileList fl)
mainiddir = rpmGetPath(fl->buildRoot, BUILD_ID_DIR, NULL);
debugiddir = rpmGetPath(fl->buildRoot, DEBUG_ID_DIR, NULL);
 
-   /* Make sure to reset all file flags to defaults.
-  Uses parseForAttr to reset ar, arFlags, and specdFlags.
-  Note that parseForAttr pokes at the attrstr, so we cannot
-  just pass a static string. */
-   fl->cur.attrFlags = 0;
-   fl->def.attrFlags = 0;
-   fl->def.verifyFlags = RPMVERIFY_ALL;
-   fl->cur.verifyFlags = RPMVERIFY_ALL;
-   fl->def.specdFlags |= SPECD_VERIFY;
-   fl->cur.specdFlags |= SPECD_VERIFY;
+   /* Make sure to reset all file flags to defaults.  */
attrstr = mkattr(NULL);
-   parseForAttr(fl->pool, attrstr, 1, >def);
+   argvAdd(files, attrstr);
free (attrstr);
 
/* Supported, but questionable.  */
@@ -1881,11 +1880,7 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(mainiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, mainiddir);
} else {
-   attrstr = mkattr(mainiddir);
-   parseForAttr(fl->pool, attrstr, 0, >cur);
-   fl->cur.isDir = 1;
-   rc = addFile(fl, mainiddir, NULL);
-   free (attrstr);
+   argvAddDir(files, mainiddir);
}
}
 
@@ -1893,11 +1888,7 @@ static int generateBuildIDs(FileList fl)
if ((rc = rpmioMkpath(debugiddir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, debugiddir);
} else {
-   attrstr = mkattr(debugiddir);
-   parseForAttr(fl->pool, attrstr, 0, >cur);
-   fl->cur.isDir = 1;
-   rc = addFile(fl, debugiddir, NULL);
-   free (attrstr);
+   argvAddDir(files, debugiddir);
}
}
}
@@ -1936,114 +1927,113 @@ static int generateBuildIDs(FileList fl)
&& (rc = rpmioMkpath(buildidsubdir, 0755, -1, -1)) != 0) {
rpmlog(RPMLOG_ERR, "%s %s: %m\n", errdir, buildidsubdir);
  

[Rpm-maint] [PATCH] Add debugsource recommends to debuginfo packages.

[Mark Wielaard]

Debuginfo packages are useful without debugsource files. But it is often
useful to also have the debugsourc files. So make debuginfo packages that
don't contain sources recommend the debugsource package (or the main
debuginfo one if the sources are not in a separate debugsource package).

Rename addPackageRequires to addPackageRequiresRecommends with an argument
to indicate whether the package is required or recommended. Move up so it
can be used with filterDebuginfoPackage. Add Package dbgsrc as argument to
filterDebuginfoPackage so it can be added as recommendation. Add a new
function findDebugsourcePackage. Use it to add a requires to the main
debuginfo file and/or the debuginfo subpackages.

Extend the various rpmbuild.at tests that create debugsource and/or
debuginfo subpackages to check the debugsource (or main debuginfo)
package is recommended.

Signed-off-by: Mark Wielaard <m...@klomp.org>
---
 build/files.c | 59 ---
 tests/rpmbuild.at | 46 ++-
 2 files changed, 88 insertions(+), 17 deletions(-)

diff --git a/build/files.c b/build/files.c
index 5e84532..86ec80a 100644
--- a/build/files.c
+++ b/build/files.c
@@ -2773,6 +2773,21 @@ static void patchDebugPackageString(Package dbg, rpmTag 
tag, Package pkg, Packag
 _free(newsubst);
 }
 
+/* add a requires or recommends for package "to" into package "from". */
+static void addPackageRequiresRecommends(Package from, Package to,
+ bool recommends)
+{
+const char *name;
+char *evr, *isaprov;
+enum rpmTag_e tag = recommends ? RPMTAG_RECOMMENDNAME : RPMTAG_REQUIRENAME;
+name = headerGetString(to->header, RPMTAG_NAME);
+evr = headerGetAsString(to->header, RPMTAG_EVR);
+isaprov = rpmExpand(name, "%{?_isa}", NULL);
+addReqProv(from, tag, isaprov, evr, RPMSENSE_EQUAL, 0);
+free(isaprov);
+free(evr);
+}
+
 /* create a new debuginfo subpackage for package pkg from the
  * main debuginfo package */
 static Package cloneDebuginfoPackage(rpmSpec spec, Package pkg, Package 
maindbg)
@@ -2805,7 +2820,8 @@ static Package cloneDebuginfoPackage(rpmSpec spec, 
Package pkg, Package maindbg)
 /* collect the debug files for package pkg and put them into
  * a (possibly new) debuginfo subpackage */
 static void filterDebuginfoPackage(rpmSpec spec, Package pkg,
-   Package maindbg, char *buildroot, char *uniquearch)
+  Package maindbg, Package dbgsrc,
+  char *buildroot, char *uniquearch)
 {
 rpmfi fi;
 ARGV_t files = NULL;
@@ -2914,6 +2930,8 @@ static void filterDebuginfoPackage(rpmSpec spec, Package 
pkg,
else {
Package dbg = cloneDebuginfoPackage(spec, pkg, maindbg);
dbg->fileList = files;
+   /* Recommend the debugsource package (or the main debuginfo).  */
+   addPackageRequiresRecommends(dbg, dbgsrc ?: maindbg, true);
}
 }
 }
@@ -2976,6 +2994,16 @@ static int addDebugSrc(Package pkg, char *buildroot)
 return ret;
 }
 
+/* find the debugsource package, if it has been created.
+ * We do this simply by searching for a package with the right name. */
+static Package findDebugsourcePackage(rpmSpec spec)
+{
+Package pkg = NULL;
+if (lookupPackage(spec, "debugsource", PART_SUBNAME|PART_QUIET, ))
+   return NULL;
+return pkg && pkg->fileList ? pkg : NULL;
+}
+
 /* find the main debuginfo package. We do this simply by
  * searching for a package with the right name. */
 static Package findDebuginfoPackage(rpmSpec spec)
@@ -2986,19 +3014,6 @@ static Package findDebuginfoPackage(rpmSpec spec)
 return pkg && pkg->fileList ? pkg : NULL;
 }
 
-/* add a requires for package "to" into package "from". */
-static void addPackageRequires(Package from, Package to)
-{
-const char *name;
-char *evr, *isaprov;
-name = headerGetString(to->header, RPMTAG_NAME);
-evr = headerGetAsString(to->header, RPMTAG_EVR);
-isaprov = rpmExpand(name, "%{?_isa}", NULL);
-addReqProv(from, RPMTAG_REQUIRENAME, isaprov, evr, RPMSENSE_EQUAL, 0);
-free(isaprov);
-free(evr);
-}
-
 rpmRC processBinaryFiles(rpmSpec spec, rpmBuildPkgFlags pkgFlags,
int didInstall, int test)
 {
@@ -3008,6 +3023,9 @@ rpmRC processBinaryFiles(rpmSpec spec, rpmBuildPkgFlags 
pkgFlags,
 char *uniquearch = NULL;
 Package maindbg = NULL;/* the (existing) main debuginfo 
package */
 Package deplink = NULL;/* create requires to this package */
+/* The debugsource package, if it exists, that the debuginfo package(s)
+   should Recommend.  */
+Package dbgsrcpkg = findDebugsourcePackage(spec);
 
 #if HAVE_LIBDW
 elf_version (EV_CURRENT);
@@ -3038,6 +3056,12 @@ rpmRC proc

Re: [Rpm-maint] [PATCH] Add debugsource recommends to debuginfo packages.

[Mark Wielaard]

On Thu, 2017-09-21 at 10:41 +0300, Panu Matilainen wrote:
> Moving an entire function while also changing it is a bit of a no-
> no, because it's makes it hard to see what actually changed. Moving
> things around also obfuscates git history, which is why I prefer
> adding a prototype to the top of the file instead. But if you need to
> move stuff around (such as different file entirely), do so in a
> separate commit that doesn't change anything and also state this in
> the commit message.
>
> Also please split refactoring changes like this to separate commits:
> one commit that does the necessary enhancements/changes to 
> addPackageRequires() and updates existing callers, and another commit
> to add the new functionality. It just makes things the diffs so much
> more obvious and nicer to review.

OK. Patch split in two and added an early prototype instead of moving
the function around.

> > @@ -2914,6 +2930,8 @@ static void filterDebuginfoPackage(rpmSpec spec, 
> > Package pkg,
> >     else {
> >     Package dbg = cloneDebuginfoPackage(spec, pkg, maindbg);
> >     dbg->fileList = files;
> > +   /* Recommend the debugsource package (or the main debuginfo).  */
> > +   addPackageRequiresRecommends(dbg, dbgsrc ?: maindbg, true);
> 
> Omitting middle operand of conditional expressions is a gcc
> extension, those are best avoided in general and especially when
> there's no actual need to use one.

I am surprised it is a GNU extension. IMHO it should be standard C. I
like it, because it is more concise and precise than duplicating the
condition. But OK. Changed to the long form.

> [...]
> > +   addPackageRequiresRecommends(pkg, deplink, false);
> 
> Hmm, I think this would be both more obvious and generally useful if
> you just call the function addPackageDeps() and pass the relevant
> dependency tag instead of true/false for require/recommend, eg
> 
> addPackageDeps(pkg, deplink, RPMTAG_REQUIRENAME);
> addPackageDeps(extradbg, dbgsrcpkg, RPMTAG_RECOMMENDNAME);
> 
> vs
> 
> addPackageRequiresRecommends(pkg, deplink, false);
> addPackageRequiresRecommends(extradbg, dbgsrcpkg, true);
> 
> That way we wont end up with 
> addPackageRequiresRecommendsConflictsObsoltetes() one day :)

Makes sense. Changed the function name (and argument).

Thanks,

Mark
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint