Bug#956418: src:glibc: Please provide optimized builds for ARMv8.1

2020-04-21 Thread Noah Meyerhans
On Sun, Apr 12, 2020 at 12:18:35PM +0200, Aurelien Jarno wrote:
> > Significant performance impact has also been observed in less contrived
> > cases (MariaDB and Postgres), but I don't have a repro to share.
> But indeed what counts is number on real workloads. It would be nice to
> get numbers when those software are run against a rebuilt glibc. As
> those software are using a lot of atomics directly, it would be also
> interesting to have numbers with those software also rebuilt to use
> those new instructions.

Agreed.  I don't have specific examples of real world impact at the
moment.  AIUI, the most significant impact comes in the usage of atomics
in pthread_mutex_lock().  When there are multiple threads contending for
a lock, one thread will (approximately) always obtain the lock, while
the others will starve.  With atomics support in place, the probability
of obtaining the lock is roughly evenly distributed among all the
threads.  So any workload in which multiple threads may contend for a
lock should be a candidate to demonstrate this problem in the real

> It would also be nice to have numbers to see the impact on non-ARMv8.1
> CPU on real workloads. As pointed out by Florian, and if the impact is
> negligible, it might be a good idea to enable -moutline-atomics
> globally at the GCC level so that all software can benefit from it, and
> instead of only glibc. That could be either upstream or only in Debian,
> that's probably a separate discussion. Otherwise we will likely end up
> using this non-default GCC option on all packages that runs faster with
> it.


> To be honest from a glibc maintenance point of view it's something I
> would like to avoid. We haven't been actively trying to remove the
> remaining optimized libraries (on i386, hurd and alpha), but we have
> tried to avoid adding new ones. The problem is not building a second
> optimized glibc, but rather providing a safe upgrade as the optimized
> and the non-optimized package have to be at the same version or one of
> them has to be disabled. This has caused many system breakages overall.

Understood, that makes sense.  I wonder if it's worth it to investigate
techniques to improve the situation around optimized libraries.  Do you
have any thoughts on what such an improvement might look like?

> Also note that the mechanism allowing a safe upgrade *does* incur a 
> runtime overhead as every binary now has to test for the presence of
> /etc/ld.so.nohwcap to detect a possible upgrade of the glibc in
> progress. That's why we have disabled it on architecture not providing
> an optimized library [1].

Thanks for the pointer, it's interesting to see data on that.  This also
suggests that it might be worthwhile to investigate a better mechanism
for identifying the availability of hardware features.

> > I've tested both options and found them to be acceptable on v8.1a (Neoverse
> > N1) and v8a (Cortex A72) CPUs.  I can provide bulk test run data of the
> > various different configuration permutations if you'd like to see additional
> > data.
> As said above I think we would need more numbers on real workload to
> take a decision. Don't get me wrong I do not oppose on improving atomics
> on ARMv8.1, but I would like that we chose the best option. Also if we
> go with the -moutline-atomics option, I believe it rather has to be a
> ARM porters decision than a glibc maintainers decision (hence the Cc:).

I'll see what I can come up with.

Do the arm porters have any opinions on this matter?


Bug#91815: Patch to add memusage and memusagestat

2020-04-21 Thread Aurelien Jarno

On 2020-04-21 20:58, Stephen Kitt wrote:
> Control: tag -1 + patch
> Hi,

Thanks for trying to fix one of the oldest glibc bugs ;-)

> The attached patch adds memusage and memusagestat to the libc-bin package.
> This does mean that the latter becomes dependent on libgd3, so it might be
> better to add a new memusage package; I can take care of that if the
> maintainers think it’s better.

I am not sure we want to add a new dependency for libc-bin, I am sure
people running embedded systems won't appreciate. Any reason for not
shipping it in libc-dev-bin instead? For me that looks more like a tool
to be used for "development". At least the memusagestat is similar to
the mtrace one that is in libc-dev-bin.

We also have to make sure that this new build-dependency doesn't break
bootstrapping. I have added Helmut in Cc so that he can have a look.


Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net

Description: PGP signature

Processed: Patch to add memusage and memusagestat

2020-04-21 Thread Debian Bug Tracking System
Processing control commands:

> tag -1 + patch
Bug #91815 [src:glibc] Build memusage and memusagestat ?
Bug #214257 [src:glibc] Include memusage in distribution
Bug #264039 [src:glibc] libc6-dev: memusage and libmemusage missing
Added tag(s) patch.
Added tag(s) patch.
Added tag(s) patch.

214257: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=214257
264039: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=264039
91815: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=91815
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems

Bug#91815: Patch to add memusage and memusagestat

2020-04-21 Thread Stephen Kitt
Control: tag -1 + patch


The attached patch adds memusage and memusagestat to the libc-bin package.
This does mean that the latter becomes dependent on libgd3, so it might be
better to add a new memusage package; I can take care of that if the
maintainers think it’s better.


From c2bc49abc1dfba93609544378f3f65dec1b98a7c Mon Sep 17 00:00:00 2001
From: Stephen Kitt 
Date: Tue, 21 Apr 2020 20:55:53 +0200
Subject: [PATCH] Build and package memusage*

This builds memusage and memusagestat in the libc pass, and ships them
in libc-bin. This involves adding a build-dependency on
libgd-dev (outside stage1) and libc-bin acquires a runtime dependency
on the corresponding library package.

Closes: #91815
Signed-off-by: Stephen Kitt 
 debian/control.in/main | 4 +++-
 debian/debhelper.in/libc-bin.install   | 2 ++
 debian/debhelper.in/libc-bin.lintian-overrides | 2 ++
 debian/rules.d/build.mk| 6 +-
 4 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/debian/control.in/main b/debian/control.in/main
index 659267bd..3ff82664 100644
--- a/debian/control.in/main
+++ b/debian/control.in/main
@@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>= 1.17.14), xz-utils, file,
  g++-9, g++-9-multilib [amd64 i386 kfreebsd-amd64 mips mipsel mipsn32 mipsn32el mips64 mips64el mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el powerpc ppc64 s390x sparc sparc64 x32] ,
  libidn2-0 (>= 2.0.5~) ,
- libc-bin (>= @GLIBC_VERSION@) 
+ libc-bin (>= @GLIBC_VERSION@) ,
+ libgd-dev 
 Build-Depends-Indep: perl, po-debconf (>= 1.0)
 Maintainer: GNU Libc Maintainers 
 Uploaders: Clint Adams , Aurelien Jarno , Adam Conrad , Samuel Thibault 
@@ -39,6 +40,7 @@ Description: GNU C Library: Binaries
   * iconv, iconvconfig: convert between character encodings
   * ldd, ldconfig: print/configure shared library dependencies
   * locale, localedef: show/generate locale definitions
+  * memusage, memusagestat: measure memory usage
   * tzselect, zdump, zic: select/dump/compile time zones
 Package: libc-dev-bin
diff --git a/debian/debhelper.in/libc-bin.install b/debian/debhelper.in/libc-bin.install
index dfab166b..a76d191f 100644
--- a/debian/debhelper.in/libc-bin.install
+++ b/debian/debhelper.in/libc-bin.install
@@ -12,6 +12,8 @@ debian/tmp-libc/usr/bin/iconv usr/bin
 debian/tmp-libc/usr/bin/ldd usr/bin
 debian/tmp-libc/usr/bin/localedef usr/bin
 debian/tmp-libc/usr/bin/locale usr/bin
+debian/tmp-libc/usr/bin/memusage usr/bin
+debian/tmp-libc/usr/bin/memusagestat usr/bin
 debian/tmp-libc/usr/bin/pldd usr/bin
 debian/tmp-libc/usr/bin/tzselect usr/bin
 debian/tmp-libc/usr/lib/pt_chown usr/lib
diff --git a/debian/debhelper.in/libc-bin.lintian-overrides b/debian/debhelper.in/libc-bin.lintian-overrides
index 01eb456c..ba411883 100644
--- a/debian/debhelper.in/libc-bin.lintian-overrides
+++ b/debian/debhelper.in/libc-bin.lintian-overrides
@@ -17,6 +17,8 @@ libc-bin: binary-without-manpage usr/bin/iconv
 libc-bin: binary-without-manpage usr/bin/ldd
 libc-bin: binary-without-manpage usr/bin/locale
 libc-bin: binary-without-manpage usr/bin/localedef
+libc-bin: binary-without-manpage usr/bin/memusage
+libc-bin: binary-without-manpage usr/bin/memusagestat
 libc-bin: binary-without-manpage usr/bin/pldd
 libc-bin: binary-without-manpage usr/bin/zdump
 libc-bin: binary-without-manpage usr/sbin/iconvconfig
diff --git a/debian/rules.d/build.mk b/debian/rules.d/build.mk
index 0d03116a..3f664316 100644
--- a/debian/rules.d/build.mk
+++ b/debian/rules.d/build.mk
@@ -37,7 +37,11 @@ $(stamp)configure_%: $(stamp)config_sub_guess $(stamp)patch $(KERNEL_HEADER_DIR)
 	echo "BASH := /bin/bash"  >> $(DEB_BUILDDIR)/configparms
 	echo "KSH := /bin/bash"   >> $(DEB_BUILDDIR)/configparms
 	echo "SHELL := /bin/bash" >> $(DEB_BUILDDIR)/configparms
-	echo "LIBGD = no" >> $(DEB_BUILDDIR)/configparms
+	if [ "$(curpass)" = "libc" ]; then \
+	  echo "LIBGD = yes"  >> $(DEB_BUILDDIR)/configparms; \
+	else \
+	  echo "LIBGD = no"   >> $(DEB_BUILDDIR)/configparms; \
+	fi
 	echo "bindir = $(bindir)" >> $(DEB_BUILDDIR)/configparms
 	echo "datadir = $(datadir)"   >> $(DEB_BUILDDIR)/configparms
 	echo "complocaledir = $(complocaledir)"   >> $(DEB_BUILDDIR)/configparms

Description: OpenPGP digital signature