Re: Bug#843073: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

On Mon, 2016-11-21 at 17:23:19 +0100, Marco d'Itri wrote:
> On Nov 21, Guillem Jover  wrote:
> > Oh, and forgot to mention, this issue has been known for over 8
> > months, and now there's this need to be pushy and rush things, etc.
> > I certainly do not appreciate that.

> No, not really: it was not clear (e.g. I could never reproduce it) until 
> very recently.

Well, it seems it was reproducible enough, that dpkg in Tanglu got a
workaround for it:

  


Regards,
Guillem

Re: Debian Installer Stretch Alpha 8 release

[Raphael Hertzog]

On Mon, 21 Nov 2016, Guillem Jover wrote:
> Oh, and forgot to mention, this issue has been known for over 8
> months, and now there's this need to be pushy and rush things, etc.
> I certainly do not appreciate that.

I have not been involved in this project so I don't know its history
but #843073 is not 8 months old and I only discovered the problem
when I read about in on debian-devel and yet I have followed former
discussions on debian-devel about merged-/usr.

I can understand that this project was of no interest to you and that
while you were aware of the problem for 8 months, you did not want to fix
it by yourself. But then I'm also sympathetic to people who want
to get their project moving forward and I'm always happy to assist them
when it touches code that I wrote in dpkg. Feel free to direct such people
towards me when you are not interested yourself.

In this specific case, the merged-/usr was mostly done and ready before
the freeze but then this problem (which was possibly underrated by Marco
or whoever else was aware of it) came into light. I would not like that we
end up with this feature reverted at freeze time because you did not find
the time to merge the patch. Hence my query.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Re: Bug#843073: Debian Installer Stretch Alpha 8 release

[Marco d'Itri]

On Nov 21, Guillem Jover  wrote:

> Oh, and forgot to mention, this issue has been known for over 8
> months, and now there's this need to be pushy and rush things, etc.
> I certainly do not appreciate that.
No, not really: it was not clear (e.g. I could never reproduce it) until 
very recently.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

On Mon, 2016-11-21 at 17:09:50 +0100, Guillem Jover wrote:
> On Mon, 2016-11-21 at 16:38:07 +0100, Raphael Hertzog wrote:
> > I can offer to upload dpkg 1.18.15.1 to sid with just those patches if
> > it relieves you from handling an intermediary upload until you
> > release 1.18.16 on your usual schedule.
> 
> No, thanks. That's not the blocker. Also the release intervals will be
> shorter for a while, at least until deeper freeze stages.

Oh, and forgot to mention, this issue has been known for over 8
months, and now there's this need to be pushy and rush things, etc.
I certainly do not appreciate that.

Regards,
Guillem

Re: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

Just a brief reply.

On Mon, 2016-11-21 at 16:38:07 +0100, Raphael Hertzog wrote:
> On Mon, 14 Nov 2016, Michael Biebl wrote:
> > Just for the record: I can confirm it fixes the problem in dpkg-shlibdeps.
> [...]
> > Guillem, it would be great if you can upload a fixed dpkg soon.
> 
> A full week went by already. What's your plan?

First I have to go over a list of queued pending items and then I'll
get to this during this week. I have not yet reviewed the patches (in
part because I didn't do much Debian stuff last week due to lack of
motivation after an unpleasant interaction precisely due to this
issue, and TBH it has become one of those energey drainers), there's
a pending run in rebootstrap by Helmut (which is now waiting for a
gixed gcc), and then I need to write a mail summary of the current
situation and implications.

> I can offer to upload dpkg 1.18.15.1 to sid with just those patches if
> it relieves you from handling an intermediary upload until you
> release 1.18.16 on your usual schedule.

No, thanks. That's not the blocker. Also the release intervals will be
shorter for a while, at least until deeper freeze stages.

Regards,
Guillem

Re: Debian Installer Stretch Alpha 8 release

[Raphael Hertzog]

Hello Guillem,

On Mon, 14 Nov 2016, Michael Biebl wrote:
> Just for the record: I can confirm it fixes the problem in dpkg-shlibdeps.
[...]
> Guillem, it would be great if you can upload a fixed dpkg soon.

A full week went by already. What's your plan?

I can offer to upload dpkg 1.18.15.1 to sid with just those patches if
it relieves you from handling an intermediary upload until you
release 1.18.16 on your usual schedule.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

Re: Debian Installer Stretch Alpha 8 release

[James Clarke]

On Sat, Nov 12, 2016 at 07:58:33PM +, Mattia Rizzolo wrote:
> On Sat, Nov 12, 2016 at 08:55:43PM +0100, Michael Biebl wrote:
> > Am 12.11.2016 um 19:17 schrieb Guillem Jover:
> > > Control: severity 843073 important
> >
> > > Ok, Mattia Rizzolo tells me (thanks!) that it's actually twice per
> > > week, and that the debootstrap used is from jessie
> >
> > Is this really so for all buildds?
> > See #843433, the sparc64 buildds apparently do use a merged-usr chroot.
>
> It's for all buildds running a stable distribution.
>
> ports don't have stable releases, so they of course can't run stable,
> so they run on sid, basically.

In fact we have explicitly disabled it on sparc64 and powerpcspe. For
m68k, sh4 and x32 the machine architecture is amd64, so it's
running stable.

Regards,
James

Re: Debian Installer Stretch Alpha 8 release

[Michael Biebl]

Am 14.11.2016 um 12:22 schrieb Raphael Hertzog:

> Please find two patches attached.
> 
> I checked that the command below was failing with the current dpkg-dev
> and it did no longer fail with the updated one.
> 
> $ sbuild -d sid --add-depends=usrmerge --chroot-setup-commands="sed -i 
> 

Re: Debian Installer Stretch Alpha 8 release

[Raphael Hertzog]

Control: tag -1 + patch
Control: severity -1 serious

Hi Guillem,

On Sun, 13 Nov 2016, Guillem Jover wrote:
> > The /usr merge violates core assumptions in dpkg-shlibdeps. The reason
> > that amd64 isn't broken is sheer luck.
> > /etc/ld.so.conf.d/x86_64-linux-gnu.conf lists /lib before /usr/lib, so
> > dpkg-shlibdeps considers that first. Swapping them or simply removing
> > /lib (as seems reasonable on a merged /usr), breaks almost any build on
> > amd64 (e.g. dash). The breakage on amd64 is simply hidden.
> 
> Right. I'm happy to assist people who want to fix this to try to find
> a proper solution in dpkg-dev, but I'm not planning on spending time
> on this on my own.

Please find two patches attached.

I checked that the command below was failing with the current dpkg-dev
and it did no longer fail with the updated one.

$ sbuild -d sid --add-depends=usrmerge --chroot-setup-commands="sed -i 
's#^/usr##;t;s#^/lib#/usr

Re: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

Control: clone 843073 -1
Control: reassign -1 debootstrap 1.0.85
Control: retitle -1 debootstrap: Please revert merged-/usr by default as it 
breaks builds
Control: severity -1 serious
Control: affects -1 dpkg-dev
Control: severity 843073 wishlist
Control: block 810499 by 843073
Control: severity 810499 serious
Control: affects 810499 dpkg-dev

Hi!

On Sun, 2016-11-13 at 08:38:53 +0100, Helmut Grohne wrote:
> On Sat, Nov 12, 2016 at 08:55:43PM +0100, Michael Biebl wrote:
> > Is this really so for all buildds?
> > See #843433, the sparc64 buildds apparently do use a merged-usr chroot.
> 
> The issue depends on the loader path of the architecture. Although I do
> not understand why, it seems that /lib64 is less prone to exposing it
> than /lib is. We have people saying:
>  * not reproducible on amd64 /lib64 (Marco)
>  * not reproducible on sparc64 /lib64 (Michael)
>  * reproducible on i386 /lib (#810499)
>  * reproducible on armhf /lib (Uwe)
> 
> So the expectation I have is that it'll break:
>  * armel
>  * armhf
>  * i386
>  * mips
>  * mipsel
>  * s390x
> 
> Plus a number of non-release architectures.

Thanks for taking a look!

> > Hm, I would still consider it release critical, i.e. something which
> > needs to be fixed before we can release stretch. Otherwise this will
> > bite us later.
> 
> I agree.

Ok fine, this looks pretty worse than it looked before. So I'm rearranging
the bugs to where they belong.

> The /usr merge violates core assumptions in dpkg-shlibdeps. The reason
> that amd64 isn't broken is sheer luck.
> /etc/ld.so.conf.d/x86_64-linux-gnu.conf lists /lib before /usr/lib, so
> dpkg-shlibdeps considers that first. Swapping them or simply removing
> /lib (as seems reasonable on a merged /usr), breaks almost any build on
> amd64 (e.g. dash). The breakage on amd64 is simply hidden.

Right. I'm happy to assist people who want to fix this to try to find
a proper solution in dpkg-dev, but I'm not planning on spending time
on this on my own.

On Sun, 2016-11-13 at 12:04:07 +0100, Marco d'Itri wrote:
> On Nov 13, Helmut Grohne  wrote:
> > Thus I think that debootstrap should revert to unmerged /usr until
> > dpkg-shlibdeps has been fixed. Fixing is non-trivial and likely requires
> > an archive rebuild on several architectures.

> Not really: dpkg-shlibdeps just needs to be fixed to search for 
> libraries in $directory and /usr/$directory, and then everything will 
> work again as usual.
> And yes, hacks like this one are a side effect of maintaining support 
> for non-merged systems.

Err, well exactly because usemerge is a major hack, and I'm actually
surprised we are deploying systems by default with that. As an
interesting thing to install and try out on ones system that's
perfectly fine though. But IMO if the merge needs to be done it should
be done by installing things into their final desired destination on
each package. Of course that makes split installations not possible,
but oh well.

Thanks,
Guillem

Re: Debian Installer Stretch Alpha 8 release

[Samuel Thibault]

Marco d'Itri, on Sun 13 Nov 2016 12:04:07 +0100, wrote:
> On Nov 13, Helmut Grohne  wrote:
> > Thus I think that debootstrap should revert to unmerged /usr until
> > dpkg-shlibdeps has been fixed. Fixing is non-trivial and likely requires
> > an archive rebuild on several architectures.
> Not really: dpkg-shlibdeps just needs to be fixed to search for 
> libraries in $directory and /usr/$directory, and then everything will 
> work again as usual.

Then that should be fixed before enabling "merged-usr by default".

> And yes, hacks like this one are a side effect of maintaining support 
> for non-merged systems.

Rather, a side effect of taking into account the whole lot of existing
packages, whose file list show libraries either in /lib or /usr/lib.

Samuel

Re: Debian Installer Stretch Alpha 8 release

[Marco d'Itri]

On Nov 13, Helmut Grohne  wrote:

> Thus I think that debootstrap should revert to unmerged /usr until
> dpkg-shlibdeps has been fixed. Fixing is non-trivial and likely requires
> an archive rebuild on several architectures.
Not really: dpkg-shlibdeps just needs to be fixed to search for 
libraries in $directory and /usr/$directory, and then everything will 
work again as usual.
And yes, hacks like this one are a side effect of maintaining support 
for non-merged systems.

-- 
ciao,
Marco


signature.asc
Description: PGP signature

Re: Debian Installer Stretch Alpha 8 release

[Helmut Grohne]

On Sat, Nov 12, 2016 at 08:55:43PM +0100, Michael Biebl wrote:
> Is this really so for all buildds?
> See #843433, the sparc64 buildds apparently do use a merged-usr chroot.

The issue depends on the loader path of the architecture. Although I do
not understand why, it seems that /lib64 is less prone to exposing it
than /lib is. We have people saying:
 * not reproducible on amd64 /lib64 (Marco)
 * not reproducible on sparc64 /lib64 (Michael)
 * reproducible on i386 /lib (#810499)
 * reproducible on armhf /lib (Uwe)

So the expectation I have is that it'll break:
 * armel
 * armhf
 * i386
 * mips
 * mipsel
 * s390x

Plus a number of non-release architectures.

> Hm, I would still consider it release critical, i.e. something which
> needs to be fixed before we can release stretch. Otherwise this will
> bite us later.

I agree.

The /usr merge violates core assumptions in dpkg-shlibdeps. The reason
that amd64 isn't broken is sheer luck.
/etc/ld.so.conf.d/x86_64-linux-gnu.conf lists /lib before /usr/lib, so
dpkg-shlibdeps considers that first. Swapping them or simply removing
/lib (as seems reasonable on a merged /usr), breaks almost any build on
amd64 (e.g. dash). The breakage on amd64 is simply hidden.

Thus I think that debootstrap should revert to unmerged /usr until
dpkg-shlibdeps has been fixed. Fixing is non-trivial and likely requires
an archive rebuild on several architectures. The problem has been known
for some 8 months and hasn't been fixed.

Helmut

Re: Debian Installer Stretch Alpha 8 release

[Mattia Rizzolo]

On Sat, Nov 12, 2016 at 08:55:43PM +0100, Michael Biebl wrote:
> Am 12.11.2016 um 19:17 schrieb Guillem Jover:
> > Control: severity 843073 important
> 
> > Ok, Mattia Rizzolo tells me (thanks!) that it's actually twice per
> > week, and that the debootstrap used is from jessie 
> 
> Is this really so for all buildds?
> See #843433, the sparc64 buildds apparently do use a merged-usr chroot.

It's for all buildds running a stable distribution.

ports don't have stable releases, so they of course can't run stable,
so they run on sid, basically.

> so this should not
> > be such a big deal for the Debian buildds. I'm thus lowering the
> > severity.
> 
> Hm, I would still consider it release critical, i.e. something which
> needs to be fixed before we can release stretch. Otherwise this will
> bite us later.

I agree.

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Debian Installer Stretch Alpha 8 release

[Michael Biebl]

Am 12.11.2016 um 19:17 schrieb Guillem Jover:
> Control: severity 843073 important

> Ok, Mattia Rizzolo tells me (thanks!) that it's actually twice per
> week, and that the debootstrap used is from jessie 

Is this really so for all buildds?
See #843433, the sparc64 buildds apparently do use a merged-usr chroot.

so this should not
> be such a big deal for the Debian buildds. I'm thus lowering the
> severity.

Hm, I would still consider it release critical, i.e. something which
needs to be fixed before we can release stretch. Otherwise this will
bite us later.

Michael


-- 
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?



signature.asc
Description: OpenPGP digital signature

Re: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

Control: severity 843073 important

On Sat, 2016-11-12 at 18:37:52 +0100, Guillem Jover wrote:
> On Sat, 2016-11-12 at 16:24:32 +0100, Cyril Brulebois wrote:
> > Important change in this release of the installer
> > =
> > 
> >  * debootstrap now defaults to merged-/usr, that is with /bin, /sbin,
> >/lib* being symlinks to their counterpart in /usr (more details on:
> >https://lists.debian.org/debian-devel/2016/09/msg00269.html).
> 
> This has caused build issues with dpkg-shlibdeps (#843073) on some
> architectures, which was a known problem (#810499) at the time of
> the default switch. :(
> 
> As I'm told the buildds regenerate the chroots every couple of weeks (?),
> which means several of those buildds will most probably start failing,
> I'd appreciate if the proponents of this change could either help
> Felipe Sateler (many thanks to him for working on this!) to track down
> and find an acceptable solution for dpkg-shlibdeps, or revert the change
> in debootstrap.

Ok, Mattia Rizzolo tells me (thanks!) that it's actually twice per
week, and that the debootstrap used is from jessie so this should not
be such a big deal for the Debian buildds. I'm thus lowering the
severity. All other comments still apply though.

> Otherwise I guess I might just reassign the dpkg-shlibdeps bug to both
> usrmerge and debootstrap.

Thanks,
Guillem

Re: Debian Installer Stretch Alpha 8 release

[Guillem Jover]

Hi,

On Sat, 2016-11-12 at 16:24:32 +0100, Cyril Brulebois wrote:
> Important change in this release of the installer
> =
> 
>  * debootstrap now defaults to merged-/usr, that is with /bin, /sbin,
>/lib* being symlinks to their counterpart in /usr (more details on:
>https://lists.debian.org/debian-devel/2016/09/msg00269.html).

This has caused build issues with dpkg-shlibdeps (#843073) on some
architectures, which was a known problem (#810499) at the time of
the default switch. :(

As I'm told the buildds regenerate the chroots every couple of weeks (?),
which means several of those buildds will most probably start failing,
I'd appreciate if the proponents of this change could either help
Felipe Sateler (many thanks to him for working on this!) to track down
and find an acceptable solution for dpkg-shlibdeps, or revert the change
in debootstrap.

Otherwise I guess I might just reassign the dpkg-shlibdeps bug to both
usrmerge and debootstrap.

Thanks,
Guillem