Bug#91815: Patch to add memusage and memusagestat

2020-07-08 Thread Stephen Kitt
Hi Helmut,

On Wed, 8 Jul 2020 18:49:42 +0200, Helmut Grohne  wrote:
[...]
> > diff --git a/debian/control.in/main b/debian/control.in/main
> > index 659267bd..c513a01a 100644
> > --- a/debian/control.in/main
> > +++ b/debian/control.in/main
> > @@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>=
> > 1.17.14), xz-utils, file, g++-9, g++-9-multilib [amd64 i386
> > kfreebsd-amd64 mips mipsel mipsn32 mipsn32el mips64 mips64el mipsr6
> > mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el powerpc ppc64 s390x
> > sparc sparc64 x32] , python3:native, libidn2-0 (>= 2.0.5~)
> > ,
> > - libc-bin (>= @GLIBC_VERSION@) 
> > + libc-bin (>= @GLIBC_VERSION@) ,
> > + libgd-dev
> 
> I suggest adding a trailing comma to make the diff for the next change
> smaller.
> 
> >  Build-Depends-Indep: perl, po-debconf (>= 1.0)
> >  Maintainer: GNU Libc Maintainers 
> >  Uploaders: Clint Adams , Aurelien Jarno
> > , Adam Conrad , Samuel Thibault
> >  @@ -47,11 +48,31 @@ Section: libdevel Priority:
> > optional Multi-Arch: foreign
> >  Depends: ${shlibs:Depends}, ${misc:Depends}
> > -Recommends: manpages, manpages-dev
> > +Recommends: libc-devtools (>> @GLIBC_VERSION@)
> >  Build-Profiles: 
> >  Description: GNU C Library: Development binaries
> >   This package contains utility programs related to the GNU C Library
> >   development package.
> > + .
> > +  * gencat: generate message catalogs
> > +  * rpcgen: compile RPC protocols to C
> > +
> > +Package: libc-devtools
> > +Architecture: any
> > +Section: devel
> > +Priority: optional
> > +Multi-Arch: foreign  
> 
> I doubt that memusage is eligible for Multi-Arch: foreign.
> 
> > +Depends: ${shlibs:Depends}, ${misc:Depends}
> > +Recommends: manpages, manpages-dev
> > +Build-Profiles:
> 
> You need Breaks + Replaces here. With the current patch, piuparts should
> be unhappy.

Thanks for the careful review, I’ve fixed all the above in the attached patch
(using ${binary:Version} for Breaks/Replaces for now).

Regards,

Stephen
From c0322308d2be0a5e54fa39ae1437f0fad7901540 Mon Sep 17 00:00:00 2001
From: Stephen Kitt 
Date: Tue, 21 Apr 2020 20:55:53 +0200
Subject: [PATCH] Build and package memusage*

This builds memusage and memusagestat in the libc pass, and ships them
in a new package, libc-devtools (short for "libc-provided developer
tools", and not libc-dev-tools to avoid making it seem to
closely-related to libc-dev-bin). This involves adding a
build-dependency on libgd-dev (outside stage1 and stage2). Other tools
which are not used to build *with* libc, but useful for development in
general, are moved to libc-devtools: mtrace, sotruss, sprof.

libc-dev-bin recommends libc-devtools to provide a simple
transition (see #91815 for the discussion).

Closes: #91815
Signed-off-by: Stephen Kitt 
---
 debian/control.in/main| 27 +--
 debian/debhelper.in/libc-dev-bin.install  |  3 ---
 debian/debhelper.in/libc-dev-bin.manpages |  1 -
 debian/debhelper.in/libc-devtools.install |  5 
 ...rrides => libc-devtools.lintian-overrides} |  2 ++
 debian/debhelper.in/libc-devtools.manpages|  1 +
 debian/rules  |  3 +++
 debian/rules.d/build.mk   |  8 ++
 8 files changed, 44 insertions(+), 6 deletions(-)
 create mode 100644 debian/debhelper.in/libc-devtools.install
 rename debian/debhelper.in/{libc-dev-bin.lintian-overrides => libc-devtools.lintian-overrides} (58%)
 create mode 100644 debian/debhelper.in/libc-devtools.manpages

diff --git a/debian/control.in/main b/debian/control.in/main
index 659267bd..2a2fb5a8 100644
--- a/debian/control.in/main
+++ b/debian/control.in/main
@@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>= 1.17.14), xz-utils, file,
  g++-9, g++-9-multilib [amd64 i386 kfreebsd-amd64 mips mipsel mipsn32 mipsn32el mips64 mips64el mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el powerpc ppc64 s390x sparc sparc64 x32] ,
  python3:native,
  libidn2-0 (>= 2.0.5~) ,
- libc-bin (>= @GLIBC_VERSION@) 
+ libc-bin (>= @GLIBC_VERSION@) ,
+ libgd-dev  ,
 Build-Depends-Indep: perl, po-debconf (>= 1.0)
 Maintainer: GNU Libc Maintainers 
 Uploaders: Clint Adams , Aurelien Jarno , Adam Conrad , Samuel Thibault 
@@ -47,11 +48,33 @@ Section: libdevel
 Priority: optional
 Multi-Arch: foreign
 Depends: ${shlibs:Depends}, ${misc:Depends}
-Recommends: manpages, manpages-dev
+Recommends: libc-devtools (>> @GLIBC_VERSION@)
 Build-Profiles: 
 Description: GNU C Library: Development binaries
  This package contains utility programs related to the GNU C Library
  development package.
+ .
+  * gencat: generate message catalogs
+  * rpcgen: compile RPC protocols to C
+
+Package: libc-devtools
+Architecture: any
+Section: devel
+Priority: optional
+Multi-Arch: no
+Depends: ${shlibs:Depends}, ${misc:Depends}
+Breaks: libc-dev-bin (<< ${binary:Version})
+Replaces: libc-dev-bin (<< ${binary:Version})
+Recommends: manpages, manpages-dev

Bug#91815: Patch to add memusage and memusagestat

2020-07-08 Thread Helmut Grohne
Hi Stephen,

On Wed, Jul 08, 2020 at 05:38:46PM +0200, Stephen Kitt wrote:
> I don’t mind, it’s not as if this is an urgent bug ;-).

Thank you for your patience and work on this.

> I went for libc-devtools to avoid making it too closely-tied to libc-dev-bin,
> on purpose; it’s not only useful for people needing libc-dev.

Sounds good to me.

> I’ve implemented this, see the attached patch. I’m not sure the upgrade
> scenario is ideal though: recommends are ignored on upgrades. This means
> that, if libc-dev-bin recommends libc-devtools, upgrades won’t install the
> latter, but new installations of libc-dev-bin will.

There is little to add here.

> Pushing memusage* to a separate package would result in that being installed
> on upgrade: libc-dev-bin would depend on libc-devtools (following the
> transition rules strictly), so that would get pulled in automatically on
> upgrade, and since it’s a new package, it would count as installation, and
> thus pull in its recommendations.

I don't have an opinion on this and leave it to the glibc maintainers.

> diff --git a/debian/control.in/main b/debian/control.in/main
> index 659267bd..c513a01a 100644
> --- a/debian/control.in/main
> +++ b/debian/control.in/main
> @@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>= 
> 1.17.14), xz-utils, file,
>   g++-9, g++-9-multilib [amd64 i386 kfreebsd-amd64 mips mipsel mipsn32 
> mipsn32el mips64 mips64el mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 
> mips64r6el powerpc ppc64 s390x sparc sparc64 x32] ,
>   python3:native,
>   libidn2-0 (>= 2.0.5~) ,
> - libc-bin (>= @GLIBC_VERSION@) 
> + libc-bin (>= @GLIBC_VERSION@) ,
> + libgd-dev  

I suggest adding a trailing comma to make the diff for the next change
smaller.

>  Build-Depends-Indep: perl, po-debconf (>= 1.0)
>  Maintainer: GNU Libc Maintainers 
>  Uploaders: Clint Adams , Aurelien Jarno 
> , Adam Conrad , Samuel Thibault 
> 
> @@ -47,11 +48,31 @@ Section: libdevel
>  Priority: optional
>  Multi-Arch: foreign
>  Depends: ${shlibs:Depends}, ${misc:Depends}
> -Recommends: manpages, manpages-dev
> +Recommends: libc-devtools (>> @GLIBC_VERSION@)
>  Build-Profiles: 
>  Description: GNU C Library: Development binaries
>   This package contains utility programs related to the GNU C Library
>   development package.
> + .
> +  * gencat: generate message catalogs
> +  * rpcgen: compile RPC protocols to C
> +
> +Package: libc-devtools
> +Architecture: any
> +Section: devel
> +Priority: optional
> +Multi-Arch: foreign

I doubt that memusage is eligible for Multi-Arch: foreign.

> +Depends: ${shlibs:Depends}, ${misc:Depends}
> +Recommends: manpages, manpages-dev
> +Build-Profiles:  

You need Breaks + Replaces here. With the current patch, piuparts should
be unhappy.

> +Description: GNU C Library: Development tools
> + This package contains development tools shipped by the GNU C
> + Library.
> + .
> +  * memusage, memusagestat: profile a program's memory usage
> +  * mtrace: interpret the malloc trace log
> +  * sotruss: trace shared library calls
> +  * sprof: display shared object profiling data
>  
>  Package: libc-l10n
>  Architecture: all
> diff --git a/debian/debhelper.in/libc-dev-bin.install 
> b/debian/debhelper.in/libc-dev-bin.install
> index 0d760a7e..902029b5 100644
> --- a/debian/debhelper.in/libc-dev-bin.install
> +++ b/debian/debhelper.in/libc-dev-bin.install
> @@ -1,5 +1,2 @@
>  debian/tmp-libc/usr/bin/gencat usr/bin
> -debian/tmp-libc/usr/bin/mtrace usr/bin
>  debian/tmp-libc/usr/bin/rpcgen usr/bin
> -debian/tmp-libc/usr/bin/sotruss usr/bin
> -debian/tmp-libc/usr/bin/sprof usr/bin
> diff --git a/debian/debhelper.in/libc-dev-bin.manpages 
> b/debian/debhelper.in/libc-dev-bin.manpages
> index c33123a0..576ec52c 100644
> --- a/debian/debhelper.in/libc-dev-bin.manpages
> +++ b/debian/debhelper.in/libc-dev-bin.manpages
> @@ -1,3 +1,2 @@
>  debian/local/manpages/gencat.1
>  debian/local/manpages/rpcgen.1 
> -debian/local/manpages/sotruss.1
> diff --git a/debian/debhelper.in/libc-devtools.install 
> b/debian/debhelper.in/libc-devtools.install
> new file mode 100644
> index ..7a0024da
> --- /dev/null
> +++ b/debian/debhelper.in/libc-devtools.install
> @@ -0,0 +1,5 @@
> +debian/tmp-libc/usr/bin/memusage usr/bin
> +debian/tmp-libc/usr/bin/memusagestat usr/bin
> +debian/tmp-libc/usr/bin/mtrace usr/bin
> +debian/tmp-libc/usr/bin/sotruss usr/bin
> +debian/tmp-libc/usr/bin/sprof usr/bin
> diff --git a/debian/debhelper.in/libc-dev-bin.lintian-overrides 
> b/debian/debhelper.in/libc-devtools.lintian-overrides
> similarity index 58%
> rename from debian/debhelper.in/libc-dev-bin.lintian-overrides
> rename to debian/debhelper.in/libc-devtools.lintian-overrides
> index eedd7cbd..1031449a 100644
> --- a/debian/debhelper.in/libc-dev-bin.lintian-overrides
> +++ b/debian/debhelper.in/libc-devtools.lintian-overrides
> @@ -1,3 +1,5 @@
>  # these manpages are provided by the manpages package
> +libc-dev-bin: binary-without-manpage usr/bin/memusage
> 

Bug#91815: Patch to add memusage and memusagestat

2020-07-08 Thread Stephen Kitt
Hi,

On Sat, 16 May 2020 13:35:02 +0200, Aurelien Jarno  wrote:
> On 2020-05-09 12:27, Helmut Grohne wrote:
> > Thank you for not dropping the ball after my initial "it's not that
> > easy" reply.
> > 
> > On Sat, May 09, 2020 at 10:53:10AM +0200, Stephen Kitt wrote:  
> > > There’s another part of the transition which bothers me: if we add
> > > memusage to a package which is depended upon (albeit temporarily) by
> > > libc-dev-bin, it becomes part of build-essential, and adding memusage
> > > means adding libgd3, libfontconfig1, libfreetype6, etc. to all builds...
> > > 
> > > It occurred to me that there is however a way to add memusage while
> > > transitioning cleanly, over a few years. It seems to me that the
> > > binaries in libc-dev-bin can be split into two categories: binaries
> > > that are expected as build tools (gencat and rpcgen), and binaries that
> > > are useful as tools for understanding programs but not building them
> > > (memusage, memusagestat, mtrace, sotruss, sprof). How about the
> > > following?
> > > 
> > > * We add a new package, say libc-devtools, containing the second type of
> > >   tools (mtrace, sotruss, sprof). For transition purposes, libc-dev-bin
> > >   depends on that in Bullseye.
> > > * We add another package, memusage, containing memusage and
> > > memusagestat. That’s the package which ends up with all the annoying
> > > extra dependencies, but nothing depends on it.
> > > * In Bookworm, libc-dev-bin can drop the dependency on libc-devtools,
> > > and we can merge memusage into libc-devtools.
> > > 
> > > Does that make sense?  
> 
> Yes, with the points raised by Helmut, I think it makes sense.
> Unfortunately many changes in glibc requires transitions lasting many
> years...

I don’t mind, it’s not as if this is an urgent bug ;-).

> I just wonder if we should call it libc-devtools or libc-dev-tools to
> match the pattern from libc-dev-bin. But that's a minor detail over the
> whole plan.

I went for libc-devtools to avoid making it too closely-tied to libc-dev-bin,
on purpose; it’s not only useful for people needing libc-dev.

> > All of what you write here makes very much sense to me and the
> > trade-offs seem good to me. What you propose will result in making the
> > bootstrapping/profile stuff simpler/better. That said, I am not a glibc
> > maintainer. I don't see the full picture.
> > 
> > I have two minor remarks:
> >  * Should libc-devtools maybe recommend memusage already?  
> 
> It's something we can do, I think it mostly depends if we basically want
> libc-dev-bin to recommends memusage.
> 
> >  * We cannot simply have libc-devtools absorb memusage in bookworm.
> >Doing so would break upgrades when someone has memusage, but not
> >libc-devtools installed. Therefore memusage likely needs to be a
> >transitional dummy package in bookworm and can only be removed one
> >release later. I'm wondering whether this could be avoided if we had
> >memusage depend on libc-devtools.  
> 
> I agree with that.
> 
> > Neither of these touch the core of your thoughts.  
> 
> Another faster alternative that came to my mind is to rely n the fact
> that recommends are enabled by default, but not used to resolve
> build-dependencies. In that case we can already create libc-devtools
> with memstatusage and also move mtrace, sotruss, sprof there. Those 3
> binaries are very unlikely to be used to build packages, so I don't
> expect breakages. From the user point of view, it's just like if (part
> of) a dependency has been demoted to a recommends. It's something that
> is often done for other packages, and it seems we accept that even if it
> causes breakages (latest example that comes to my mind is util-linux
> dropping the dependency on fdisk). I guess adding an entry in NEWS would
> be necessary though.

I’ve implemented this, see the attached patch. I’m not sure the upgrade
scenario is ideal though: recommends are ignored on upgrades. This means
that, if libc-dev-bin recommends libc-devtools, upgrades won’t install the
latter, but new installations of libc-dev-bin will.

Pushing memusage* to a separate package would result in that being installed
on upgrade: libc-dev-bin would depend on libc-devtools (following the
transition rules strictly), so that would get pulled in automatically on
upgrade, and since it’s a new package, it would count as installation, and
thus pull in its recommendations.

> I don't know if it's something that's acceptable. What do you think?
> Maybe we should ask the release team?

That would be useful! I need to check the upgrades scenarios again, I’ll
reach out to them in a couple of weeks’ time, unless you want to do it
directly.

Regards,

Stephen
From 95f482efe7c2ae6d9a87d0a47cd5619cb26a4f55 Mon Sep 17 00:00:00 2001
From: Stephen Kitt 
Date: Tue, 21 Apr 2020 20:55:53 +0200
Subject: [PATCH] Build and package memusage*

This builds memusage and memusagestat in the libc pass, and ships them
in a new package, libc-devtools 

Bug#91815: Patch to add memusage and memusagestat

2020-05-16 Thread Aurelien Jarno
Hi Stephen, hi Helmut,

On 2020-05-09 12:27, Helmut Grohne wrote:
> Hi Stephen,
> 
> Thank you for not dropping the ball after my initial "it's not that
> easy" reply.
> 
> On Sat, May 09, 2020 at 10:53:10AM +0200, Stephen Kitt wrote:
> > There’s another part of the transition which bothers me: if we add memusage
> > to a package which is depended upon (albeit temporarily) by libc-dev-bin, it
> > becomes part of build-essential, and adding memusage means adding libgd3,
> > libfontconfig1, libfreetype6, etc. to all builds...
> > 
> > It occurred to me that there is however a way to add memusage while
> > transitioning cleanly, over a few years. It seems to me that the binaries in
> > libc-dev-bin can be split into two categories: binaries that are expected as
> > build tools (gencat and rpcgen), and binaries that are useful as tools for
> > understanding programs but not building them (memusage, memusagestat, 
> > mtrace,
> > sotruss, sprof). How about the following?
> > 
> > * We add a new package, say libc-devtools, containing the second type of
> >   tools (mtrace, sotruss, sprof). For transition purposes, libc-dev-bin
> >   depends on that in Bullseye.
> > * We add another package, memusage, containing memusage and memusagestat.
> >   That’s the package which ends up with all the annoying extra dependencies,
> >   but nothing depends on it.
> > * In Bookworm, libc-dev-bin can drop the dependency on libc-devtools, and we
> >   can merge memusage into libc-devtools.
> > 
> > Does that make sense?

Yes, with the points raised by Helmut, I think it makes sense.
Unfortunately many changes in glibc requires transitions lasting many
years...

I just wonder if we should call it libc-devtools or libc-dev-tools to
match the pattern from libc-dev-bin. But that's a minor detail over the
whole plan.

> All of what you write here makes very much sense to me and the
> trade-offs seem good to me. What you propose will result in making the
> bootstrapping/profile stuff simpler/better. That said, I am not a glibc
> maintainer. I don't see the full picture.
> 
> I have two minor remarks:
>  * Should libc-devtools maybe recommend memusage already?

It's something we can do, I think it mostly depends if we basically want
libc-dev-bin to recommends memusage.

>  * We cannot simply have libc-devtools absorb memusage in bookworm.
>Doing so would break upgrades when someone has memusage, but not
>libc-devtools installed. Therefore memusage likely needs to be a
>transitional dummy package in bookworm and can only be removed one
>release later. I'm wondering whether this could be avoided if we had
>memusage depend on libc-devtools.

I agree with that.

> Neither of these touch the core of your thoughts.

Another faster alternative that came to my mind is to rely n the fact
that recommends are enabled by default, but not used to resolve
build-dependencies. In that case we can already create libc-devtools
with memstatusage and also move mtrace, sotruss, sprof there. Those 3
binaries are very unlikely to be used to build packages, so I don't
expect breakages. From the user point of view, it's just like if (part
of) a dependency has been demoted to a recommends. It's something that
is often done for other packages, and it seems we accept that even if it
causes breakages (latest example that comes to my mind is util-linux
dropping the dependency on fdisk). I guess adding an entry in NEWS would
be necessary though.

I don't know if it's something that's acceptable. What do you think?
Maybe we should ask the release team?

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net



Bug#91815: Patch to add memusage and memusagestat

2020-05-09 Thread Helmut Grohne
Hi Stephen,

Thank you for not dropping the ball after my initial "it's not that
easy" reply.

On Sat, May 09, 2020 at 10:53:10AM +0200, Stephen Kitt wrote:
> There’s another part of the transition which bothers me: if we add memusage
> to a package which is depended upon (albeit temporarily) by libc-dev-bin, it
> becomes part of build-essential, and adding memusage means adding libgd3,
> libfontconfig1, libfreetype6, etc. to all builds...
> 
> It occurred to me that there is however a way to add memusage while
> transitioning cleanly, over a few years. It seems to me that the binaries in
> libc-dev-bin can be split into two categories: binaries that are expected as
> build tools (gencat and rpcgen), and binaries that are useful as tools for
> understanding programs but not building them (memusage, memusagestat, mtrace,
> sotruss, sprof). How about the following?
> 
> * We add a new package, say libc-devtools, containing the second type of
>   tools (mtrace, sotruss, sprof). For transition purposes, libc-dev-bin
>   depends on that in Bullseye.
> * We add another package, memusage, containing memusage and memusagestat.
>   That’s the package which ends up with all the annoying extra dependencies,
>   but nothing depends on it.
> * In Bookworm, libc-dev-bin can drop the dependency on libc-devtools, and we
>   can merge memusage into libc-devtools.
> 
> Does that make sense?

All of what you write here makes very much sense to me and the
trade-offs seem good to me. What you propose will result in making the
bootstrapping/profile stuff simpler/better. That said, I am not a glibc
maintainer. I don't see the full picture.

I have two minor remarks:
 * Should libc-devtools maybe recommend memusage already?
 * We cannot simply have libc-devtools absorb memusage in bookworm.
   Doing so would break upgrades when someone has memusage, but not
   libc-devtools installed. Therefore memusage likely needs to be a
   transitional dummy package in bookworm and can only be removed one
   release later. I'm wondering whether this could be avoided if we had
   memusage depend on libc-devtools.

Neither of these touch the core of your thoughts.

I'm happy to review patches to make this happen. Please continue to Cc
me if you want my review.

Helmut



Bug#91815: Patch to add memusage and memusagestat

2020-05-09 Thread Stephen Kitt
Hi Aurélien, hi Helmut,

On Mon, 4 May 2020 07:11:37 +0200, Helmut Grohne  wrote:
> On Mon, May 04, 2020 at 12:00:28AM +0200, Aurelien Jarno wrote:
> > Thanks for the detailed explanations. It looks to me that it's better to
> > add a different package. And while mtrace looks a good candidate to join
> > this package, I am not sure we can handle the transition easily. It
> > would mean that libc6-dev-bin need to depends on that new package at
> > least for one release cycle. And we actually don't want that for stage
> > 2, which means producing different packages for stage2...  
> 
> Let me point out that a transitional dependency is not a huge cost to
> reproducibility. Making the dependency optional to stage2 is reasonable.
> libc packages are not yet reproducible across stages. That's more of a
> vision. In diffoscope, such a dependency difference is very light.

There’s another part of the transition which bothers me: if we add memusage
to a package which is depended upon (albeit temporarily) by libc-dev-bin, it
becomes part of build-essential, and adding memusage means adding libgd3,
libfontconfig1, libfreetype6, etc. to all builds...

It occurred to me that there is however a way to add memusage while
transitioning cleanly, over a few years. It seems to me that the binaries in
libc-dev-bin can be split into two categories: binaries that are expected as
build tools (gencat and rpcgen), and binaries that are useful as tools for
understanding programs but not building them (memusage, memusagestat, mtrace,
sotruss, sprof). How about the following?

* We add a new package, say libc-devtools, containing the second type of
  tools (mtrace, sotruss, sprof). For transition purposes, libc-dev-bin
  depends on that in Bullseye.
* We add another package, memusage, containing memusage and memusagestat.
  That’s the package which ends up with all the annoying extra dependencies,
  but nothing depends on it.
* In Bookworm, libc-dev-bin can drop the dependency on libc-devtools, and we
  can merge memusage into libc-devtools.

Does that make sense?

Regards,

Stephen


pgpCZZtK9enyl.pgp
Description: OpenPGP digital signature


Bug#91815: Patch to add memusage and memusagestat

2020-05-03 Thread Helmut Grohne
Hi Aurelien and Stephen,

On Mon, May 04, 2020 at 12:00:28AM +0200, Aurelien Jarno wrote:
> It's something we can fix. I found strange nobody notice so far. Does it
> mean the bootstrap is done ignoring the build-dependencies?

Yes, I still ignore build-depends entirely. glibc depends on a specific
version of the compiler and such dependencies are not compatible with
cross compilation until we fix #666743 aka gcc-for-host. That bug has a
patch, but nobody wants to review it and communication isn't working
best on the issue either.

If you really want to make a dent now, you need to do like src:linux does:

Build-Depends:
 g++-9 ,
 g++-9-aarch64-linux-gnu [arm64] ,
 g++-9-arm-linux-gnueabi [armel] ,
 ...

Do you want that?

Doing so still means me hacking up the package during bootstrap, because
I always force the compiler version regardless of whether glibc supports
that.

> Thanks for the detailed explanations. It looks to me that it's better to
> add a different package. And while mtrace looks a good candidate to join
> this package, I am not sure we can handle the transition easily. It
> would mean that libc6-dev-bin need to depends on that new package at
> least for one release cycle. And we actually don't want that for stage
> 2, which means producing different packages for stage2...

Let me point out that a transitional dependency is not a huge cost to
reproducibility. Making the dependency optional to stage2 is reasonable.
libc packages are not yet reproducible across stages. That's more of a
vision. In diffoscope, such a dependency difference is very light.

I agree with everything else you said around moving mtrace though. So
judge it on its own knowing that it won't negatively impact bootstrap.

When thinking about bootstrap, it helps to always think of any kind of
transition as atomic (i.e. completed once started). It might be
temporarily broken, but since you always rebuild everything from source,
you can simply fix up anyth broken rdeps immediately.

Helmut



Bug#91815: Patch to add memusage and memusagestat

2020-05-03 Thread Aurelien Jarno
Hi,

On 2020-04-22 06:53, Helmut Grohne wrote:
> Hi Aurelien and Stephen,
> 
> On Wed, Apr 22, 2020 at 12:04:32AM +0200, Aurelien Jarno wrote:
> > > The attached patch adds memusage and memusagestat to the libc-bin package.
> > > This does mean that the latter becomes dependent on libgd3, so it might be
> > > better to add a new memusage package; I can take care of that if the
> > > maintainers think it’s better.
> > 
> > I am not sure we want to add a new dependency for libc-bin, I am sure
> > people running embedded systems won't appreciate. Any reason for not
> > shipping it in libc-dev-bin instead? For me that looks more like a tool
> > to be used for "development". At least the memusagestat is similar to
> > the mtrace one that is in libc-dev-bin.
> > 
> > We also have to make sure that this new build-dependency doesn't break
> > bootstrapping. I have added Helmut in Cc so that he can have a look.
> 
> Thank you for notifying me. Indeed, the patch doesn't work for
> bootstrapping as is. A stage2 build of glibc would start requiring
> libgd-dev -> libgd3 -> libc6, but the stage1 libc6 will not be
> sufficient to build src:libgd2. It must be possible to build stage2
> without that dependency.
> 
> (Note that this is going to become more confusing as there is ongoing
> work on removing stage1 and maybe renaming stage2, but let's stick to
> the current names for now.)
> 
> >From a bootstrapping pov, the libgd-dev dependency is similar to the
> libselinux-dev dependency, but I notice just now that it is not
> correctly annotated with  in Build-Depends!

It's something we can fix. I found strange nobody notice so far. Does it
mean the bootstrap is done ignoring the build-dependencies?

> >From a build profile pov, it is very undesirable to have package
> contents change with profiles. Doing so makes it impossible to validate
> them using reproducible builds. However, since we need libc6-dev from
> stage2 and it depends on libc-dev-bin we cannot skip generating
> libc-dev-bin in stage2. I wonder whether it would make sense to add
> another binary package for development tools that are not normally
> needed for building software. Then we could skip generating that
> package. mtrace would be a good candidate to join that package indeed.

Thanks for the detailed explanations. It looks to me that it's better to
add a different package. And while mtrace looks a good candidate to join
this package, I am not sure we can handle the transition easily. It
would mean that libc6-dev-bin need to depends on that new package at
least for one release cycle. And we actually don't want that for stage
2, which means producing different packages for stage2...

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net



Bug#91815: Patch to add memusage and memusagestat

2020-04-22 Thread Helmut Grohne
Hi Aurelien and Stephen,

On Wed, Apr 22, 2020 at 12:04:32AM +0200, Aurelien Jarno wrote:
> > The attached patch adds memusage and memusagestat to the libc-bin package.
> > This does mean that the latter becomes dependent on libgd3, so it might be
> > better to add a new memusage package; I can take care of that if the
> > maintainers think it’s better.
> 
> I am not sure we want to add a new dependency for libc-bin, I am sure
> people running embedded systems won't appreciate. Any reason for not
> shipping it in libc-dev-bin instead? For me that looks more like a tool
> to be used for "development". At least the memusagestat is similar to
> the mtrace one that is in libc-dev-bin.
> 
> We also have to make sure that this new build-dependency doesn't break
> bootstrapping. I have added Helmut in Cc so that he can have a look.

Thank you for notifying me. Indeed, the patch doesn't work for
bootstrapping as is. A stage2 build of glibc would start requiring
libgd-dev -> libgd3 -> libc6, but the stage1 libc6 will not be
sufficient to build src:libgd2. It must be possible to build stage2
without that dependency.

(Note that this is going to become more confusing as there is ongoing
work on removing stage1 and maybe renaming stage2, but let's stick to
the current names for now.)

>From a bootstrapping pov, the libgd-dev dependency is similar to the
libselinux-dev dependency, but I notice just now that it is not
correctly annotated with  in Build-Depends!

>From a build profile pov, it is very undesirable to have package
contents change with profiles. Doing so makes it impossible to validate
them using reproducible builds. However, since we need libc6-dev from
stage2 and it depends on libc-dev-bin we cannot skip generating
libc-dev-bin in stage2. I wonder whether it would make sense to add
another binary package for development tools that are not normally
needed for building software. Then we could skip generating that
package. mtrace would be a good candidate to join that package indeed.

Failing that, we'll have to cope with changing package contents for
stage2 and disable the new dependency for stage2 (or some other
profile).

Helmut



Bug#91815: Patch to add memusage and memusagestat

2020-04-22 Thread Stephen Kitt
Hi,

On Wed, 22 Apr 2020 00:04:32 +0200, Aurelien Jarno  wrote:
> On 2020-04-21 20:58, Stephen Kitt wrote:
> Thanks for trying to fix one of the oldest glibc bugs ;-)

You’re welcome!

> > The attached patch adds memusage and memusagestat to the libc-bin package.
> > This does mean that the latter becomes dependent on libgd3, so it might be
> > better to add a new memusage package; I can take care of that if the
> > maintainers think it’s better.  
> 
> I am not sure we want to add a new dependency for libc-bin, I am sure
> people running embedded systems won't appreciate. Any reason for not
> shipping it in libc-dev-bin instead? For me that looks more like a tool
> to be used for "development". At least the memusagestat is similar to
> the mtrace one that is in libc-dev-bin.

Indeed, especially since libgd3 is a rather heavy-weight dependency. I’m
attaching a patch which adds the tools to libc-dev-bin instead.

> We also have to make sure that this new build-dependency doesn't break
> bootstrapping. I have added Helmut in Cc so that he can have a look.

Isn’t non-cross bootstrapping handled by the stage1 build profile? If this
causes issues with cross bootstrapping then shipping a separate memusage
package ( ) should take care of things.

Regards,

Stephen
From bd3f09edb6296b8a8e2987aefd3169d48903625f Mon Sep 17 00:00:00 2001
From: Stephen Kitt 
Date: Tue, 21 Apr 2020 20:55:53 +0200
Subject: [PATCH] Build and package memusage*

This builds memusage and memusagestat in the libc pass, and ships them
in libc-dev-bin. This involves adding a build-dependency on
libgd-dev (outside stage1) and libc-dev-bin acquires a runtime
dependency on the corresponding library package.

Closes: #91815
Signed-off-by: Stephen Kitt 
---
 debian/control.in/main | 3 ++-
 debian/debhelper.in/libc-dev-bin.install   | 2 ++
 debian/debhelper.in/libc-dev-bin.lintian-overrides | 2 ++
 debian/rules.d/build.mk| 6 +-
 4 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/debian/control.in/main b/debian/control.in/main
index 659267bd..3537b751 100644
--- a/debian/control.in/main
+++ b/debian/control.in/main
@@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>= 1.17.14), xz-utils, file,
  g++-9, g++-9-multilib [amd64 i386 kfreebsd-amd64 mips mipsel mipsn32 mipsn32el mips64 mips64el mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el powerpc ppc64 s390x sparc sparc64 x32] ,
  python3:native,
  libidn2-0 (>= 2.0.5~) ,
- libc-bin (>= @GLIBC_VERSION@) 
+ libc-bin (>= @GLIBC_VERSION@) ,
+ libgd-dev 
 Build-Depends-Indep: perl, po-debconf (>= 1.0)
 Maintainer: GNU Libc Maintainers 
 Uploaders: Clint Adams , Aurelien Jarno , Adam Conrad , Samuel Thibault 
diff --git a/debian/debhelper.in/libc-dev-bin.install b/debian/debhelper.in/libc-dev-bin.install
index 0d760a7e..8ced4378 100644
--- a/debian/debhelper.in/libc-dev-bin.install
+++ b/debian/debhelper.in/libc-dev-bin.install
@@ -1,4 +1,6 @@
 debian/tmp-libc/usr/bin/gencat usr/bin
+debian/tmp-libc/usr/bin/memusage usr/bin
+debian/tmp-libc/usr/bin/memusagestat usr/bin
 debian/tmp-libc/usr/bin/mtrace usr/bin
 debian/tmp-libc/usr/bin/rpcgen usr/bin
 debian/tmp-libc/usr/bin/sotruss usr/bin
diff --git a/debian/debhelper.in/libc-dev-bin.lintian-overrides b/debian/debhelper.in/libc-dev-bin.lintian-overrides
index eedd7cbd..1031449a 100644
--- a/debian/debhelper.in/libc-dev-bin.lintian-overrides
+++ b/debian/debhelper.in/libc-dev-bin.lintian-overrides
@@ -1,3 +1,5 @@
 # these manpages are provided by the manpages package
+libc-dev-bin: binary-without-manpage usr/bin/memusage
+libc-dev-bin: binary-without-manpage usr/bin/memusagestat
 libc-dev-bin: binary-without-manpage usr/bin/mtrace
 libc-dev-bin: binary-without-manpage usr/bin/sprof
diff --git a/debian/rules.d/build.mk b/debian/rules.d/build.mk
index 0d03116a..3f664316 100644
--- a/debian/rules.d/build.mk
+++ b/debian/rules.d/build.mk
@@ -37,7 +37,11 @@ $(stamp)configure_%: $(stamp)config_sub_guess $(stamp)patch $(KERNEL_HEADER_DIR)
 	echo "BASH := /bin/bash"  >> $(DEB_BUILDDIR)/configparms
 	echo "KSH := /bin/bash"   >> $(DEB_BUILDDIR)/configparms
 	echo "SHELL := /bin/bash" >> $(DEB_BUILDDIR)/configparms
-	echo "LIBGD = no" >> $(DEB_BUILDDIR)/configparms
+	if [ "$(curpass)" = "libc" ]; then \
+	  echo "LIBGD = yes"  >> $(DEB_BUILDDIR)/configparms; \
+	else \
+	  echo "LIBGD = no"   >> $(DEB_BUILDDIR)/configparms; \
+	fi
 	echo "bindir = $(bindir)" >> $(DEB_BUILDDIR)/configparms
 	echo "datadir = $(datadir)"   >> $(DEB_BUILDDIR)/configparms
 	echo "complocaledir = $(complocaledir)"   >> $(DEB_BUILDDIR)/configparms
-- 
2.20.1



pgpEALW2r4ByK.pgp
Description: OpenPGP digital signature


Bug#91815: Patch to add memusage and memusagestat

2020-04-21 Thread Aurelien Jarno
Hi,

On 2020-04-21 20:58, Stephen Kitt wrote:
> Control: tag -1 + patch
> 
> Hi,

Thanks for trying to fix one of the oldest glibc bugs ;-)

> The attached patch adds memusage and memusagestat to the libc-bin package.
> This does mean that the latter becomes dependent on libgd3, so it might be
> better to add a new memusage package; I can take care of that if the
> maintainers think it’s better.

I am not sure we want to add a new dependency for libc-bin, I am sure
people running embedded systems won't appreciate. Any reason for not
shipping it in libc-dev-bin instead? For me that looks more like a tool
to be used for "development". At least the memusagestat is similar to
the mtrace one that is in libc-dev-bin.

We also have to make sure that this new build-dependency doesn't break
bootstrapping. I have added Helmut in Cc so that he can have a look.

Thanks,
Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature


Bug#91815: Patch to add memusage and memusagestat

2020-04-21 Thread Stephen Kitt
Control: tag -1 + patch

Hi,

The attached patch adds memusage and memusagestat to the libc-bin package.
This does mean that the latter becomes dependent on libgd3, so it might be
better to add a new memusage package; I can take care of that if the
maintainers think it’s better.

Regards,

Stephen
From c2bc49abc1dfba93609544378f3f65dec1b98a7c Mon Sep 17 00:00:00 2001
From: Stephen Kitt 
Date: Tue, 21 Apr 2020 20:55:53 +0200
Subject: [PATCH] Build and package memusage*

This builds memusage and memusagestat in the libc pass, and ships them
in libc-bin. This involves adding a build-dependency on
libgd-dev (outside stage1) and libc-bin acquires a runtime dependency
on the corresponding library package.

Closes: #91815
Signed-off-by: Stephen Kitt 
---
 debian/control.in/main | 4 +++-
 debian/debhelper.in/libc-bin.install   | 2 ++
 debian/debhelper.in/libc-bin.lintian-overrides | 2 ++
 debian/rules.d/build.mk| 6 +-
 4 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/debian/control.in/main b/debian/control.in/main
index 659267bd..3ff82664 100644
--- a/debian/control.in/main
+++ b/debian/control.in/main
@@ -12,7 +12,8 @@ Build-Depends: gettext, dpkg (>= 1.18.7), dpkg-dev (>= 1.17.14), xz-utils, file,
  g++-9, g++-9-multilib [amd64 i386 kfreebsd-amd64 mips mipsel mipsn32 mipsn32el mips64 mips64el mipsr6 mipsr6el mipsn32r6 mipsn32r6el mips64r6 mips64r6el powerpc ppc64 s390x sparc sparc64 x32] ,
  python3:native,
  libidn2-0 (>= 2.0.5~) ,
- libc-bin (>= @GLIBC_VERSION@) 
+ libc-bin (>= @GLIBC_VERSION@) ,
+ libgd-dev 
 Build-Depends-Indep: perl, po-debconf (>= 1.0)
 Maintainer: GNU Libc Maintainers 
 Uploaders: Clint Adams , Aurelien Jarno , Adam Conrad , Samuel Thibault 
@@ -39,6 +40,7 @@ Description: GNU C Library: Binaries
   * iconv, iconvconfig: convert between character encodings
   * ldd, ldconfig: print/configure shared library dependencies
   * locale, localedef: show/generate locale definitions
+  * memusage, memusagestat: measure memory usage
   * tzselect, zdump, zic: select/dump/compile time zones
 
 Package: libc-dev-bin
diff --git a/debian/debhelper.in/libc-bin.install b/debian/debhelper.in/libc-bin.install
index dfab166b..a76d191f 100644
--- a/debian/debhelper.in/libc-bin.install
+++ b/debian/debhelper.in/libc-bin.install
@@ -12,6 +12,8 @@ debian/tmp-libc/usr/bin/iconv usr/bin
 debian/tmp-libc/usr/bin/ldd usr/bin
 debian/tmp-libc/usr/bin/localedef usr/bin
 debian/tmp-libc/usr/bin/locale usr/bin
+debian/tmp-libc/usr/bin/memusage usr/bin
+debian/tmp-libc/usr/bin/memusagestat usr/bin
 debian/tmp-libc/usr/bin/pldd usr/bin
 debian/tmp-libc/usr/bin/tzselect usr/bin
 debian/tmp-libc/usr/lib/pt_chown usr/lib
diff --git a/debian/debhelper.in/libc-bin.lintian-overrides b/debian/debhelper.in/libc-bin.lintian-overrides
index 01eb456c..ba411883 100644
--- a/debian/debhelper.in/libc-bin.lintian-overrides
+++ b/debian/debhelper.in/libc-bin.lintian-overrides
@@ -17,6 +17,8 @@ libc-bin: binary-without-manpage usr/bin/iconv
 libc-bin: binary-without-manpage usr/bin/ldd
 libc-bin: binary-without-manpage usr/bin/locale
 libc-bin: binary-without-manpage usr/bin/localedef
+libc-bin: binary-without-manpage usr/bin/memusage
+libc-bin: binary-without-manpage usr/bin/memusagestat
 libc-bin: binary-without-manpage usr/bin/pldd
 libc-bin: binary-without-manpage usr/bin/zdump
 libc-bin: binary-without-manpage usr/sbin/iconvconfig
diff --git a/debian/rules.d/build.mk b/debian/rules.d/build.mk
index 0d03116a..3f664316 100644
--- a/debian/rules.d/build.mk
+++ b/debian/rules.d/build.mk
@@ -37,7 +37,11 @@ $(stamp)configure_%: $(stamp)config_sub_guess $(stamp)patch $(KERNEL_HEADER_DIR)
 	echo "BASH := /bin/bash"  >> $(DEB_BUILDDIR)/configparms
 	echo "KSH := /bin/bash"   >> $(DEB_BUILDDIR)/configparms
 	echo "SHELL := /bin/bash" >> $(DEB_BUILDDIR)/configparms
-	echo "LIBGD = no" >> $(DEB_BUILDDIR)/configparms
+	if [ "$(curpass)" = "libc" ]; then \
+	  echo "LIBGD = yes"  >> $(DEB_BUILDDIR)/configparms; \
+	else \
+	  echo "LIBGD = no"   >> $(DEB_BUILDDIR)/configparms; \
+	fi
 	echo "bindir = $(bindir)" >> $(DEB_BUILDDIR)/configparms
 	echo "datadir = $(datadir)"   >> $(DEB_BUILDDIR)/configparms
 	echo "complocaledir = $(complocaledir)"   >> $(DEB_BUILDDIR)/configparms
-- 
2.20.1



pgpe_6n3ge4HH.pgp
Description: OpenPGP digital signature