Re: [Rpm-maint] [PATCH v3 1/2] find-debuginfo.sh: Split directory traversal and debuginfo extraction
On 2016-09-16 17:43, Mark Wielaard wrote: > On Fri, 2016-09-16 at 13:53 +0200, Michal Marek wrote: >> Dne 15.9.2016 v 16:41 Mark Wielaard napsal(a): >>> On Sat, 2016-09-10 at 23:13 +0200, Michal Marek wrote: >>>> This siplifies the handling of hardlinks a bit and allows a later patch >>>> to parallelize the debuginfo extraction. >>> >>> I had to read it three times to understand the shell variable voodoo :) >> >> Sorry about that. If you like the eval more, I can change that part back. > > No worries. Bash scripts are just not the easiest things to read. > Your changes were fine. Just slightly hard to grok for someone like me > who you doesn't remember all the variable expansion tricks. > And Florian already pushed both patches to git master. That's great, thanks! Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [PATCH v3 1/2] find-debuginfo.sh: Split directory traversal and debuginfo extraction
Dne 15.9.2016 v 16:41 Mark Wielaard napsal(a): > On Sat, 2016-09-10 at 23:13 +0200, Michal Marek wrote: >> This siplifies the handling of hardlinks a bit and allows a later patch >> to parallelize the debuginfo extraction. > > I had to read it three times to understand the shell variable voodoo :) Sorry about that. If you like the eval more, I can change that part back. > But it looks fine to me and seems to pass with the current testsuite. Great. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [PATCH v3 2/2] find-debuginfo.sh: Process files in parallel
Dne 10.9.2016 v 23:13 Michal Marek napsal(a): > Add a -j option, which, when used, will spawn processes to do the > debuginfo extraction in parallel. A named pipe is used to dispatch the > files among the processes. I just noticed that the changelog is not accurate since v2: s/named pipe/pipe/. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH v3 2/2] find-debuginfo.sh: Process files in parallel
Add a -j option, which, when used, will spawn processes to do the
debuginfo extraction in parallel. A named pipe is used to dispatch the
files among the processes.
Signed-off-by: Michal Marek <mma...@suse.com>
---
v3:
- Rebased onto current master
- Build debugsources.list (and now also elfbins.list) from per-job fragments
to avoid races.
v2: Use a regular pipe, because a named pipe leads to races.
macros.in | 2 +-
scripts/find-debuginfo.sh | 63 ---
2 files changed, 61 insertions(+), 4 deletions(-)
diff --git a/macros.in b/macros.in
index b03c5a9e95fa..8bde2d77a581 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
# the script. See the script for details.
#
%__debug_install_post \
- %{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id}
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i}
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"}
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"}
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts}
"%{_builddir}/%{?buildsubdir}"\
+ %{_rpmconfigdir}/find-debuginfo.sh %{?_smp_mflags}
%{?_missing_build_ids_terminate_build:--strict-build-id}
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i}
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"}
%{?_unique_debug_names:--unique-debug-arch "%{_arch}"}
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts}
"%{_builddir}/%{?buildsubdir}"\
%{nil}
# Template for debug information sub-package.
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 6dcd5a46f700..2016222e7bf0 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -67,6 +67,9 @@ ver_rel=
# Arch given by --unique-debug-arch
unique_debug_arch=
+# Number of parallel jobs to spawn
+n_jobs=1
+
BUILDDIR=.
out=debugfiles.list
nout=0
@@ -123,6 +126,13 @@ while [ $# -gt 0 ]; do
-r)
strip_r=true
;;
+ -j)
+n_jobs=$2
+shift
+;;
+ -j*)
+n_jobs=${1#-j}
+;;
*)
BUILDDIR=$1
shift
@@ -334,9 +344,56 @@ do_file()
fi
}
-while read nlinks inum f; do
- do_file "$nlinks" "$inum" "$f"
-done <"$temp/primary"
+# 16^6 - 1 or about 16 milion files
+FILENUM_DIGITS=6
+run_job()
+{
+ local jobid=$1 filenum
+ local SOURCEFILE=$temp/debugsources.$jobid ELFBINSFILE=$temp/elfbins.$jobid
+
+ >"$SOURCEFILE"
+ >"$ELFBINSFILE"
+ # can't use read -n , because it reads bytes one by one, allowing for
+ # races
+ while :; do
+filenum=$(dd bs=$(( FILENUM_DIGITS + 1 )) count=1 status=none)
+if test -z "$filenum"; then
+ break
+fi
+do_file $(sed -n "$(( 0x$filenum )) p" "$temp/primary")
+ done
+ echo 0 >"$temp/res.$jobid"
+}
+
+n_files=$(wc -l <"$temp/primary")
+if [ $n_jobs -gt $n_files ]; then
+ n_jobs=$n_files
+fi
+if [ $n_jobs -le 1 ]; then
+ while read nlinks inum f; do
+do_file "$nlinks" "$inum" "$f"
+ done <"$temp/primary"
+else
+ for ((i = 1; i <= n_files; i++)); do
+printf "%0${FILENUM_DIGITS}x\\n" $i
+ done | (
+exec 3<&0
+for ((i = 0; i < n_jobs; i++)); do
+ # The shell redirects stdin to /dev/null for background jobs. Work
+ # around this by duplicating fd 0
+ run_job $i <&3 &
+done
+wait
+ )
+ for f in "$temp"/res.*; do
+res=$(< "$f")
+if [ "$res" != "0" ]; then
+ exit 1
+fi
+ done
+ cat "$temp"/debugsources.* >"$SOURCEFILE"
+ cat "$temp"/elfbins.* >"$ELFBINSFILE"
+fi
# Invoke the DWARF Compressor utility.
if $run_dwz \
--
2.6.6
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH v3 1/2] find-debuginfo.sh: Split directory traversal and debuginfo extraction
This siplifies the handling of hardlinks a bit and allows a later patch
to parallelize the debuginfo extraction.
Signed-off-by: Michal Marek <mma...@suse.com>
---
v3: Rebased onto current master
v2: Fix for packages with no ELF files.
scripts/find-debuginfo.sh | 53 ++-
1 file changed, 34 insertions(+), 19 deletions(-)
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index d83c3e2c47f3..6dcd5a46f700 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -248,32 +248,36 @@ set -o pipefail
strict_error=ERROR
$strict || strict_error=WARNING
-# Strip ELF binaries
+temp=$(mktemp -d ${TMPDIR:-/tmp}/find-debuginfo.XX)
+trap 'rm -rf "$temp"' EXIT
+
+# Build a list of unstripped ELF files and their hardlinks
+touch "$temp/primary"
find "$RPM_BUILD_ROOT" ! -path "${debugdir}/*.debug" -type f \
\( -perm -0100 -or -perm -0010 -or -perm -0001 \) \
-print |
file -N -f - | sed -n -e 's/^\(.*\):[ ]*.*ELF.*, not stripped.*/\1/p' |
xargs --no-run-if-empty stat -c '%h %D_%i %n' |
while read nlinks inum f; do
- get_debugfn "$f"
- [ -f "${debugfn}" ] && continue
-
- # If this file has multiple links, keep track and make
- # the corresponding .debug files all links to one file too.
if [ $nlinks -gt 1 ]; then
-eval linked=\$linked_$inum
-if [ -n "$linked" ]; then
- eval id=\$linkedid_$inum
- link=$debugfn
- get_debugfn "$linked"
- echo "hard linked $link to $debugfn"
- mkdir -p "$(dirname "$link")" && ln -nf "$debugfn" "$link"
+var=seen_$inum
+if test -n "${!var}"; then
+ echo "$inum $f" >>"$temp/linked"
continue
else
- eval linked_$inum=\$f
- echo "file $f has $[$nlinks - 1] other hard links"
+ read "$var" < <(echo 1)
fi
fi
+ echo "$nlinks $inum $f" >>"$temp/primary"
+done
+
+# Strip ELF binaries
+do_file()
+{
+ local nlinks=$1 inum=$2 f=$3 id link linked
+
+ get_debugfn "$f"
+ [ -f "${debugfn}" ] && return
echo "extracting debug info from $f"
build_id_seed=
@@ -282,9 +286,6 @@ while read nlinks inum f; do
fi
id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
-i $build_id_seed -l "$SOURCEFILE" "$f") || exit
- if [ $nlinks -gt 1 ]; then
-eval linkedid_$inum=\$id
- fi
if [ -z "$id" ]; then
echo >&2 "*** ${strict_error}: No build ID note found in $f"
$strict && exit 2
@@ -321,7 +322,21 @@ while read nlinks inum f; do
echo "./${f#$RPM_BUILD_ROOT}" >> "$ELFBINSFILE"
-done || exit
+ # If this file has multiple links, make the corresponding .debug files
+ # all links to one file too.
+ if [ $nlinks -gt 1 ]; then
+grep "^$inum " "$temp/linked" | while read inum linked; do
+ link=$debugfn
+ get_debugfn "$linked"
+ echo "hard linked $link to $debugfn"
+ mkdir -p "$(dirname "$debugfn")" && ln -nf "$link" "$debugfn"
+done
+ fi
+}
+
+while read nlinks inum f; do
+ do_file "$nlinks" "$inum" "$f"
+done <"$temp/primary"
# Invoke the DWARF Compressor utility.
if $run_dwz \
--
2.6.6
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [PATCH] Merge with build-id rework
Dne 2.8.2016 v 16:57 Florian Festi napsal(a): > I just merged the Mark Wielaars' patches upstream. Even with this merge > patch things do not apply cleanly. Can you just rebase on top of master, > please? Will do. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH] Merge with build-id rework
Mark asked me to check his changes at
http://lists.rpm.org/pipermail/rpm-maint/2016-June/004365.html with this
series. The conflict resolution is more or less obvious, only the
elfbins.list handing needs to be done per-job.
Conflicts:
macros.in
scripts/find-debuginfo.sh
diff --cc macros.in
index 205a1f3c6247,eca9f4011e66..fc66936514fa
--- a/macros.in
+++ b/macros.in
@@@ -178,7 -178,7 +178,7 @@@
# the script. See the script for details.
#
%__debug_install_post \
-%{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id}
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i}
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"}
%{?_find_debuginfo_dwz_opts} %{?_find_debuginfo_opts}
"%{_builddir}/%{?buildsubdir}"\
- %{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id} %{?_smp_mflags}
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
++ %{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id}
%{?_include_minidebuginfo:-m} %{?_include_gdb_index:-i}
%{?_unique_build_ids:--ver-rel "%{version}-%{release}"}
%{?_find_debuginfo_dwz_opts} %{?_smp_mflags} %{?_find_debuginfo_opts}
"%{_builddir}/%{?buildsubdir}"\
%{nil}
# Template for debug information sub-package.
diff --cc scripts/find-debuginfo.sh
index ef0ca923f3dd,64e4ccb31f6b..02e502499ef6
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@@ -56,14 -32,9 +56,17 @@@ include_gdb_index=fals
# Barf on missing build IDs.
strict=false
+# DWZ parameters.
+run_dwz=false
+dwz_low_mem_die_limit=
+dwz_max_die_limit=
+
+# Version and release of the spec. Given by --ver-rel
+ver_rel=
+
+ # Number of parallel jobs to spawn
+ n_jobs=1
+
BUILDDIR=.
out=debugfiles.list
nout=0
@@@ -237,38 -221,31 +251,37 @@@ find "$RPM_BUILD_ROOT" ! -path "${debug
\( -perm -0100 -or -perm -0010 -or -perm -0001 \) \
-print |
file -N -f - | sed -n -e 's/^\(.*\):[ ]*.*ELF.*, not stripped.*/\1/p'
|
- xargs --no-run-if-empty stat -c '%h %D_%i %n' |
+ xargs --no-run-if-empty stat -c '%h %D_%i %n' | sort -k3 |
while read nlinks inum f; do
- get_debugfn "$f"
- [ -f "${debugfn}" ] && continue
-
- # If this file has multiple links, keep track and make
- # the corresponding .debug files all links to one file too.
if [ $nlinks -gt 1 ]; then
- eval linked=\$linked_$inum
- if [ -n "$linked" ]; then
- eval id=\$linkedid_$inum
- link=$debugfn
- get_debugfn "$linked"
- echo "hard linked $link to $debugfn"
- mkdir -p "$(dirname "$link")" && ln -nf "$debugfn" "$link"
+ var=seen_$inum
+ if test -n "${!var}"; then
+ echo "$inum $f" >>"$temp/linked"
continue
else
- eval linked_$inum=\$f
- echo "file $f has $[$nlinks - 1] other hard links"
+ read "$var" < <(echo 1)
fi
fi
+ echo "$nlinks $inum $f" >>"$temp/primary"
+ done
+
+ # Strip ELF binaries
+ do_file()
+ {
+ local nlinks=$1 inum=$2 f=$3 id link linked
+
+ get_debugfn "$f"
+ [ -f "${debugfn}" ] && return
echo "extracting debug info from $f"
+ build_id_seed=
+ if [ ! -z "$ver_rel" ]; then
+build_id_seed="--build-id-seed=$ver_rel"
+ fi
+ id=$(${lib_rpm_dir}/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
+-i $build_id_seed -l "$SOURCEFILE" "$f") || exit
- if [ $nlinks -gt 1 ]; then
- eval linkedid_$inum=\$id
- fi
+ id=$(/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
+ -i -l "$SOURCEFILE" "$f") || exit
if [ -z "$id" ]; then
echo >&2 "*** ${strict_error}: No build ID note found in $f"
$strict && exit 2
@@@ -300,50 -270,69 +313,107 @@@
chmod u-w "$f"
fi
- if [ -n "$id" ]; then
-make_id_link "$id" "$dn/$(basename $f)"
-make_id_link "$id" "/usr/lib/debug$dn/$bn" .debug
- fi
+ # strip -g implies we have full symtab, don't add mini symtab in that case.
+ $strip_g || ($include_minidebug && add_minidebug "${debugfn}" "$f")
+
+ echo "./${f#$RPM_BUILD_ROOT}" >> "$ELFBINSFILE"
- done || exit
+ # If this file has multiple links, make the corresponding .debug files
+ # all links to one file too.
+ if [ $nlinks -gt 1 ]; then
+ grep "^$inum " "$temp/linked" | while read inum linked; do
- make_id_dup_link "$id" "$dn/$(basename $f)"
- make_id_dup_link "$id" "/usr/lib/debug$dn/$bn" .debug
+ link=$debugfn
+ get_debugfn "$linked"
+ echo "hard linked $link to $debugfn"
+ mkdir -p "$(dirname "$debugfn")" && ln -nf "$link" "$debugfn"
+ done
+ fi
+ }
+
+ # 16^6 - 1 or about 16 milion files
+ FILENUM_DIGITS=6
+ run_job()
+ {
- local jobid=$1 filenum
++ local jobid=$1 filenum ELFBINSFILE="$temp/elfbins.$jobid"
+
+ # can't use read -n , because it reads bytes one by one, allowing for
+ # races
+
[Rpm-maint] [PATCH v2 0/3] find-debuginfo.sh speedup
Hi, this series allows find-debuginfo.sh to run in parallel. It also makes the duplicate build-id links predictable. Changes in v2: I found that using a named pipe is racy, because a process might open it too late, after the main process has written all the ids into it and the other processes have read them. This version uses a regular pipe, which gives EOF as soon as the last writer has closed it. Also, I fixed a bug with packages that do not contain any ELF files. Michal Michal Marek (3): find-debuginfo.sh: Split directory traversal and debuginfo extraction find-debuginfo.sh: Make the duplicate id links deterministic find-debuginfo.sh: Process files in parallel macros.in | 2 +- scripts/find-debuginfo.sh | 112 +- 2 files changed, 91 insertions(+), 23 deletions(-) -- 2.6.2 ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [PATCH 0/3] find-debuginfo.sh speedup
Dne 2.7.2016 v 16:28 Mark Wielaard napsal(a): > Hi, > > On Fri, 2016-07-01 at 22:43 +0200, Michal Marek wrote: >> this series allows find-debuginfo.sh to run in parallel. It also makes the >> duplicate build-id links predictable. > > The build-id link code was completely rewritten in one of the patches I > wrote some weeks ago: > http://lists.rpm.org/pipermail/rpm-maint/2016-June/004365.html > > Could you look at how your changes interact with that? I think your "3/3 > find-debuginfo.sh: Process files in parallel" patch should work as is > with that cleanup. OK, I will have a look. > And do you have any testcases for this or could you check with the > testcases in the above patch set to see if it has the right behavior? > It adds a rpmbuildid.at that explicitly checks generating build-id > symlinks for binaries that are hard linked and/or have duplicate > build-ids. I do not have any systematic testcases, but in my random testing I found a race in patch 3/3: Using a named pipe was not a good idea, because a job might open it too late and hang forever. So I will be posting a v2 soon. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH 0/3] find-debuginfo.sh speedup
Hi, this series allows find-debuginfo.sh to run in parallel. It also makes the duplicate build-id links predictable. Michal Michal Marek (3): find-debuginfo.sh: Split directory traversal and debuginfo extraction find-debuginfo.sh: Make the duplicate id links deterministic find-debuginfo.sh: Process files in parallel macros.in | 2 +- scripts/find-debuginfo.sh | 108 -- 2 files changed, 87 insertions(+), 23 deletions(-) -- 2.6.2 ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH 3/3] find-debuginfo.sh: Process files in parallel
Add a -j option, which, when used, will spawn processes to do the
debuginfo extraction in parallel. A named pipe is used to dispatch the
files among the processes.
Signed-off-by: Michal Marek <mma...@suse.com>
---
macros.in | 2 +-
scripts/find-debuginfo.sh | 55 ---
2 files changed, 53 insertions(+), 4 deletions(-)
diff --git a/macros.in b/macros.in
index 9e0cdea75154..eca9f4011e66 100644
--- a/macros.in
+++ b/macros.in
@@ -178,7 +178,7 @@
# the script. See the script for details.
#
%__debug_install_post \
- %{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id}
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
+ %{_rpmconfigdir}/find-debuginfo.sh
%{?_missing_build_ids_terminate_build:--strict-build-id} %{?_smp_mflags}
%{?_find_debuginfo_opts} "%{_builddir}/%{?buildsubdir}"\
%{nil}
# Template for debug information sub-package.
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index f3a3e45d122f..aa70e1eb14ca 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -32,6 +32,9 @@ strip_r=false
# Barf on missing build IDs.
strict=false
+# Number of parallel jobs to spawn
+n_jobs=1
+
BUILDDIR=.
out=debugfiles.list
nout=0
@@ -63,6 +66,13 @@ while [ $# -gt 0 ]; do
-r)
strip_r=true
;;
+ -j)
+n_jobs=$2
+shift
+;;
+ -j*)
+n_jobs=${1#-j}
+;;
*)
BUILDDIR=$1
shift
@@ -278,9 +288,48 @@ do_file()
fi
}
-while read nlinks inum f; do
- do_file "$nlinks" "$inum" "$f"
-done <"$temp/primary"
+# 16^6 - 1 or about 16 milion files
+FILENUM_DIGITS=6
+run_job()
+{
+ local jobid=$1 filenum
+
+ # can't use read -n , because it reads bytes one by one, allowing for
+ # races
+ while :; do
+filenum=$(dd bs=$(( FILENUM_DIGITS + 1 )) count=1 status=none)
+if test -z "$filenum"; then
+ break
+fi
+do_file $(sed -n "$(( 0x$filenum )) p" "$temp/primary")
+ done <"$temp/pipe"
+ echo 0 >"$temp/res.$jobid"
+}
+
+n_files=$(wc -l <"$temp/primary")
+if [ $n_jobs -gt $n_files ]; then
+ n_jobs=$n_files
+fi
+if [ $n_jobs -le 1 ]; then
+ while read nlinks inum f; do
+do_file "$nlinks" "$inum" "$f"
+ done <"$temp/primary"
+else
+ mkfifo "$temp/pipe"
+ for ((i = 0; i < n_jobs; i++)); do
+run_job $i &
+ done
+ for ((i = 1; i <= n_files; i++)); do
+printf "%0${FILENUM_DIGITS}x\\n" $i
+ done >"$temp/pipe"
+ wait
+ for f in "$temp"/res.*; do
+res=$(< "$f")
+if [ "$res" != "0" ]; then
+ exit 1
+fi
+ done
+fi
# For each symlink whose target has a .debug file,
# make a .debug symlink to that file.
--
2.6.2
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [PATCH 1/3] find-debuginfo.sh: Split directory traversal and debuginfo extraction
This siplifies the handling of hardlinks a bit and allows a later patch
to parallelize the debuginfo extraction.
Signed-off-by: Michal Marek <mma...@suse.com>
---
scripts/find-debuginfo.sh | 57 ++-
1 file changed, 36 insertions(+), 21 deletions(-)
diff --git a/scripts/find-debuginfo.sh b/scripts/find-debuginfo.sh
index 17522e038f67..3a84aad6a60b 100644
--- a/scripts/find-debuginfo.sh
+++ b/scripts/find-debuginfo.sh
@@ -202,41 +202,39 @@ set -o pipefail
strict_error=ERROR
$strict || strict_error=WARNING
-# Strip ELF binaries
+temp=$(mktemp -d ${TMPDIR:-/tmp}/find-debuginfo.XX)
+trap 'rm -rf "$temp"' EXIT
+
+# Build a list of unstripped ELF files and their hardlinks
find "$RPM_BUILD_ROOT" ! -path "${debugdir}/*.debug" -type f \
\( -perm -0100 -or -perm -0010 -or -perm -0001 \) \
-print |
file -N -f - | sed -n -e 's/^\(.*\):[ ]*.*ELF.*, not stripped.*/\1/p' |
xargs --no-run-if-empty stat -c '%h %D_%i %n' |
while read nlinks inum f; do
- get_debugfn "$f"
- [ -f "${debugfn}" ] && continue
-
- # If this file has multiple links, keep track and make
- # the corresponding .debug files all links to one file too.
if [ $nlinks -gt 1 ]; then
-eval linked=\$linked_$inum
-if [ -n "$linked" ]; then
- eval id=\$linkedid_$inum
- make_id_dup_link "$id" "$dn/$(basename $f)"
- make_id_dup_link "$id" "/usr/lib/debug$dn/$bn" .debug
- link=$debugfn
- get_debugfn "$linked"
- echo "hard linked $link to $debugfn"
- mkdir -p "$(dirname "$link")" && ln -nf "$debugfn" "$link"
+var=seen_$inum
+if test -n "${!var}"; then
+ echo "$inum $f" >>"$temp/linked"
continue
else
- eval linked_$inum=\$f
- echo "file $f has $[$nlinks - 1] other hard links"
+ read "$var" < <(echo 1)
fi
fi
+ echo "$nlinks $inum $f" >>"$temp/primary"
+done
+
+# Strip ELF binaries
+do_file()
+{
+ local nlinks=$1 inum=$2 f=$3 id link linked
+
+ get_debugfn "$f"
+ [ -f "${debugfn}" ] && return
echo "extracting debug info from $f"
id=$(/usr/lib/rpm/debugedit -b "$RPM_BUILD_DIR" -d /usr/src/debug \
-i -l "$SOURCEFILE" "$f") || exit
- if [ $nlinks -gt 1 ]; then
-eval linkedid_$inum=\$id
- fi
if [ -z "$id" ]; then
echo >&2 "*** ${strict_error}: No build ID note found in $f"
$strict && exit 2
@@ -265,7 +263,24 @@ while read nlinks inum f; do
make_id_link "$id" "$dn/$(basename $f)"
make_id_link "$id" "/usr/lib/debug$dn/$bn" .debug
fi
-done || exit
+
+ # If this file has multiple links, make the corresponding .debug files
+ # all links to one file too.
+ if [ $nlinks -gt 1 ]; then
+grep "^$inum " "$temp/linked" | while read inum linked; do
+ make_id_dup_link "$id" "$dn/$(basename $f)"
+ make_id_dup_link "$id" "/usr/lib/debug$dn/$bn" .debug
+ link=$debugfn
+ get_debugfn "$linked"
+ echo "hard linked $link to $debugfn"
+ mkdir -p "$(dirname "$debugfn")" && ln -nf "$link" "$debugfn"
+done
+ fi
+}
+
+while read nlinks inum f; do
+ do_file "$nlinks" "$inum" "$f"
+done <"$temp/primary"
# For each symlink whose target has a .debug file,
# make a .debug symlink to that file.
--
2.6.2
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
http://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [patch] noarch subpackages
Pixel wrote: http://www.mail-archive.com/[EMAIL PROTECTED]/msg02055.html http://www.mail-archive.com/[EMAIL PROTECTED]/msg00211.html Thanks for the links. for the record, instead of %_noarch_packages, we could use BuildArch and still be backward compatible (as far as i have tested...): - force BuildArch: %_target_cpu in main package - add BuildArch: noarch to sub-packages which should be noarch since currently rpm takes the first BuildArch (tested on rpm 4.4.2.3 and 4.4.8) Indeed that seems to work (i.e. is ignored) and looks a bit cleaner. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org https://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [patch] noarch subpackages
Pixel wrote: http://www.mail-archive.com/[EMAIL PROTECTED]/msg02055.html http://www.mail-archive.com/[EMAIL PROTECTED]/msg00211.html Thanks for the links. for the record, instead of %_noarch_packages, we could use BuildArch and still be backward compatible (as far as i have tested...): - force BuildArch: %_target_cpu in main package - add BuildArch: noarch to sub-packages which should be noarch since currently rpm takes the first BuildArch (tested on rpm 4.4.2.3 and 4.4.8) Indeed that seems to work (i.e. is ignored) and looks a bit cleaner. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org https://lists.rpm.org/mailman/listinfo/rpm-maint
[Rpm-maint] [patch] noarch subpackages
Hi,
I'm wondering - doesn't RPM support creating anyarch + noarch
subpackages in one build simply because it has never had the feature in
first place and no-one implemented it so far (for backwards
compatibility reasons perhaps)? Or are there good reasons not to allow this?
A wrote a short patch that allows creating noarch subpackages in a
backwards-compatible way: The spec file can define a
%_noarch_subpackages macro containing a space-separated list of
subpackages that should be noarch. E.g.
%define _noarch_subpackages %name-doc %name-examples
makes the doc and examples subpackages noarch. Such spec file will still
build fine with any rpm version, it'll just duplicate examples and
documentation for every architecture.
Comments?
Michal
diff -r 5165a84e8400 build/parseSpec.c
--- a/build/parseSpec.c Mon Feb 11 20:47:03 2008 +0200
+++ b/build/parseSpec.c Wed Feb 13 11:21:37 2008 +0100
@@ -586,22 +586,45 @@ int parseSpec(rpmts ts, const char *spec
char *platform = rpmExpand(%{_target_platform}, NULL);
char *arch = rpmExpand(%{_target_cpu}, NULL);
char *os = rpmExpand(%{_target_os}, NULL);
+char *noarch_packages = rpmExpand(%{?_noarch_packages}, NULL);
+char *noarchPlatform;
+/* hack: temporarily set _target_cpu to noarch to get
+ * noarch-%{_vendor}-%{_target_os} in noarchPlatform */
+addMacro(NULL, _target_cpu, NULL, noarch, RMIL_RPMRC);
+noarchPlatform = rpmExpand(%{_target_platform}, NULL);
+addMacro(NULL, _target_cpu, NULL, arch, RMIL_RPMRC);
for (pkg = spec-packages; pkg != NULL; pkg = pkg-next) {
+const char *p, *name, *pkgArch = arch, *pkgPlatform = platform;
+size_t nameLen;
+
+(void) headerNVR(pkg-header, name, NULL, NULL);
if (!headerIsEntry(pkg-header, RPMTAG_DESCRIPTION)) {
- const char * name;
- (void) headerNVR(pkg-header, name, NULL, NULL);
rpmlog(RPMLOG_ERR, _(Package has no %%description: %s\n),
name);
spec = freeSpec(spec);
return RPMRC_FAIL;
}
+ nameLen = strlen(name);
+ for (p = noarch_packages; ; p += nameLen) {
+ p = strstr(p, name);
+ if (!p) {
+ break;
+ }
+ /* check if we found a whole word */
+ if ((p == noarch_packages || p[-1] == ' ' || p[-1] == '\t')
+ (!p[nameLen] || p[nameLen] == ' ' || p[nameLen] == '\t')) {
+ pkgArch = noarch;
+ pkgPlatform = noarchPlatform;
+ break;
+ }
+ }
(void) headerAddEntry(pkg-header, RPMTAG_OS, RPM_STRING_TYPE, os, 1);
(void) headerAddEntry(pkg-header, RPMTAG_ARCH,
- RPM_STRING_TYPE, arch, 1);
+ RPM_STRING_TYPE, pkgArch, 1);
(void) headerAddEntry(pkg-header, RPMTAG_PLATFORM,
- RPM_STRING_TYPE, platform, 1);
+ RPM_STRING_TYPE, pkgPlatform, 1);
pkg-ds = rpmdsThis(pkg-header, RPMTAG_REQUIRENAME, RPMSENSE_EQUAL);
@@ -610,6 +633,8 @@ int parseSpec(rpmts ts, const char *spec
platform = _free(platform);
arch = _free(arch);
os = _free(os);
+noarch_packages = _free(noarch_packages);
+noarchPlatform = _free(noarchPlatform);
}
closeSpec(spec);
___
Rpm-maint mailing list
Rpm-maint@lists.rpm.org
https://lists.rpm.org/mailman/listinfo/rpm-maint
Re: [Rpm-maint] [patch] warnings for rpmExpand(), rpmlog(), rpmGetPath()
Panu Matilainen wrote: I also have little doubt that there are a number of other gcc extensions too that would be useful. So instead of littering the headers with #if defined(__GNUC__) __GNUC__ = someversion #endif everywhere, might as well add macros for them centrally someplace, Sure, why not. similarly to what glib's gmacros.h does: #if __GNUC__ = 4 #define G_GNUC_NULL_TERMINATED __attribute__((__sentinel__)) #else #define G_GNUC_NULL_TERMINATED #endif ..etc. Maybe just grab a copy of gmacros.h instead of reinventing the wheel... Well, adding a (verbatim) copy to the rpmlib API is maybe not a good idea - you could end up with two different gmacros.h versions included in your app (I don't know how rapidly does this header file change, but still...). I think adding glib2 as a build-time requirement would be cleaner and still quite non-intrusive. Michal ___ Rpm-maint mailing list Rpm-maint@lists.rpm.org https://lists.rpm.org/mailman/listinfo/rpm-maint
