Re: Bug#851790: installation-reports: DNS not working

[Aurelien Jarno]

On 2017-01-23 17:30, Wookey wrote:
> On 2017-01-19 11:04 +0100, Cyril Brulebois wrote:
> > Steve McIntyre  (2017-01-19):
> > > On Thu, Jan 19, 2017 at 08:57:54AM +0100, Aurelien Jarno wrote:
> > > >
> > > >The workaround are to make sure the chroots are up-to-date (which should
> > > >be the case now on the build daemons). An other alternative would be to
> > > >avoid copying a library in mklibs if it is already present in the image.
> > > >That might break if some very strict dependencies are used, though
> > > >I guess the way the udebs are downloaded, they should always have the
> > > >same or a newer version than in the chroot.
> > > 
> > > Thanks for the explanation - it's appreciated!
> > 
> > Yeah, thanks for the confirmation.
> 
> OK. I tested today's image (2017-01-23 04:56) and the install went
> through OK, so we are back in sync and this issue is gone for now. It should
> probably be retitled to something about library sync/using host libs
> and left open until it's fixed propoerly.

I have pushed a patch a few days ago that should fix the issue. Well I
don't know if it should be considered as a fix or a hack, but at least
it looks less a hack than the existing code...

The longterm solution is clearly to fully get rid of mklibs. That should
wait for after stretch though, as it requires new udebs from some
packages and thus some coordination.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature

Re: Bug#851790: installation-reports: DNS not working

[Wookey]

On 2017-01-19 11:04 +0100, Cyril Brulebois wrote:
> Steve McIntyre  (2017-01-19):
> > On Thu, Jan 19, 2017 at 08:57:54AM +0100, Aurelien Jarno wrote:
> > >
> > >The workaround are to make sure the chroots are up-to-date (which should
> > >be the case now on the build daemons). An other alternative would be to
> > >avoid copying a library in mklibs if it is already present in the image.
> > >That might break if some very strict dependencies are used, though
> > >I guess the way the udebs are downloaded, they should always have the
> > >same or a newer version than in the chroot.
> > 
> > Thanks for the explanation - it's appreciated!
> 
> Yeah, thanks for the confirmation.

OK. I tested today's image (2017-01-23 04:56) and the install went
through OK, so we are back in sync and this issue is gone for now. It should
probably be retitled to something about library sync/using host libs
and left open until it's fixed propoerly.

Wookey
-- 
Principal hats:  Linaro, Debian, Wookware, ARM
http://wookware.org/


signature.asc
Description: Digital signature

Re: Bug#851790: installation-reports: DNS not working

[Cyril Brulebois]

Steve McIntyre  (2017-01-19):
> On Thu, Jan 19, 2017 at 08:57:54AM +0100, Aurelien Jarno wrote:
> >On 2017-01-19 01:53, Cyril Brulebois wrote:
> >
> >> It's been a while since I last looked at/understood mklibs stuff though,
> >> feel free to fix my suspicions/conclusions.
> >
> >The long term solution is to package all the libraries into udeb
> >packages. That way we can simply get rid of the mklibs pass.
> >
> >The workaround are to make sure the chroots are up-to-date (which should
> >be the case now on the build daemons). An other alternative would be to
> >avoid copying a library in mklibs if it is already present in the image.
> >That might break if some very strict dependencies are used, though
> >I guess the way the udebs are downloaded, they should always have the
> >same or a newer version than in the chroot.
> 
> Thanks for the explanation - it's appreciated!

Yeah, thanks for the confirmation.

> Is there anything we could do to fail the build if versions are out of
> sync, rather than let a broken build through?

Well, I think Aurélien mentioned it: ensure chroots are up-to-date.
Tweaking the buildscript might do the trick, I suppose. AFAIUI, the
build isn't broken every time there's a divergence in versions anyway;
you're sometimes (un)lucky.

Can't really devote time right now to investigating the “let's not copy
stuff over if it's already present” suggestion…


KiBi.


signature.asc
Description: Digital signature

Re: Bug#851790: installation-reports: DNS not working

[Steve McIntyre]

On Thu, Jan 19, 2017 at 08:57:54AM +0100, Aurelien Jarno wrote:
>On 2017-01-19 01:53, Cyril Brulebois wrote:
>
>> It's been a while since I last looked at/understood mklibs stuff though,
>> feel free to fix my suspicions/conclusions.
>
>The long term solution is to package all the libraries into udeb
>packages. That way we can simply get rid of the mklibs pass.
>
>The workaround are to make sure the chroots are up-to-date (which should
>be the case now on the build daemons). An other alternative would be to
>avoid copying a library in mklibs if it is already present in the image.
>That might break if some very strict dependencies are used, though
>I guess the way the udebs are downloaded, they should always have the
>same or a newer version than in the chroot.

Thanks for the explanation - it's appreciated!

Is there anything we could do to fail the build if versions are out of
sync, rather than let a broken build through?

-- 
Steve McIntyre, Cambridge, UK.st...@einval.com
< Aardvark> I dislike C++ to start with. C++11 just seems to be
handing rope-creating factories for users to hang multiple
instances of themselves.

Re: Bug#851790: installation-reports: DNS not working

[Aurelien Jarno]

On 2017-01-19 01:53, Cyril Brulebois wrote:
> Cyril Brulebois  (2017-01-19):
> > Summing up things from IRC:
> >  - in an uptodate sid chroot (both development version and minimal one
> >with daily-build script): no DNS issues with the generated mini.iso
> >(amd64, tested within stable's kvm on amd64). My image was also
> >tested successfully by Steve, so not a setup issue.
> > 
> >  - I can reproduce the issue with dailies from 2017-01-18, and from
> >2017-01-19 (I picked it in advance during its build on barriere).
> > 
> >  - I can reproduce the issue in the above mentioned sid chroot if
> >I downgrade these packages to testing's version (-9 → -8):
> >  sudo apt-get install libc6:amd64=2.24-8 libc6-dev:amd64=2.24-8 
> > libc-dev-bin=2.24-8
> > 
> >  - The older set of libc* packages is installed in barriere's current
> >amd64 sid chroot, too.
> > 
> >  - Steve could only produce broken images when building locally, until
> >he upgraded his libc* packages as well.
> 
> And since Steve was wondering how we could release something for Stretch
> RC 1 with that… I've just confirmed that using -8 .deb with a -8 .udeb
> (reversioned as -9+hack so that its being dropped under localudebs makes
> it take precedence over sid's .udeb) leads to an image that's working
> fine. And AFAICT that's the combination we had at the time.
> 
> I suspect the implementation change through the following patch in -9:
> glibc-2.24/debian/patches/any/cvs-resolv-internal-qtype.diff

Indeed, I haven't done any test, but looking at the code, it shows that
the changes assume that libresolv.so.2 and libnss_dns.so.2 stay in sync.

> plus 283e7a294275a7da53258600deaaafbbec6b96c1 in debian-installer.git is
> what is triggering the issue? (Explaining why host's libc .deb have an
> impact on the built images.)

Indeed this has been done to avoid crashes when libc6-udeb is
re-installed at the beginning if the installer process, often causing
crashes in case of version mismatch (because the libnss* libraries were
in a different package).

Now libc6-udeb is unpacked while building the image, but given we still
have half a dozen of libraries without the corresponding udeb,
mklibs copy them on the image, as well as the dependencies, which
presumably includes libresolv.so.2.

> It's been a while since I last looked at/understood mklibs stuff though,
> feel free to fix my suspicions/conclusions.

The long term solution is to package all the libraries into udeb
packages. That way we can simply get rid of the mklibs pass.

The workaround are to make sure the chroots are up-to-date (which should
be the case now on the build daemons). An other alternative would be to
avoid copying a library in mklibs if it is already present in the image.
That might break if some very strict dependencies are used, though
I guess the way the udebs are downloaded, they should always have the
same or a newer version than in the chroot.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature

Re: Bug#851790: installation-reports: DNS not working

[Cyril Brulebois]

Cyril Brulebois  (2017-01-19):
> Summing up things from IRC:
>  - in an uptodate sid chroot (both development version and minimal one
>with daily-build script): no DNS issues with the generated mini.iso
>(amd64, tested within stable's kvm on amd64). My image was also
>tested successfully by Steve, so not a setup issue.
> 
>  - I can reproduce the issue with dailies from 2017-01-18, and from
>2017-01-19 (I picked it in advance during its build on barriere).
> 
>  - I can reproduce the issue in the above mentioned sid chroot if
>I downgrade these packages to testing's version (-9 → -8):
>  sudo apt-get install libc6:amd64=2.24-8 libc6-dev:amd64=2.24-8 
> libc-dev-bin=2.24-8
> 
>  - The older set of libc* packages is installed in barriere's current
>amd64 sid chroot, too.
> 
>  - Steve could only produce broken images when building locally, until
>he upgraded his libc* packages as well.

And since Steve was wondering how we could release something for Stretch
RC 1 with that… I've just confirmed that using -8 .deb with a -8 .udeb
(reversioned as -9+hack so that its being dropped under localudebs makes
it take precedence over sid's .udeb) leads to an image that's working
fine. And AFAICT that's the combination we had at the time.

I suspect the implementation change through the following patch in -9:
glibc-2.24/debian/patches/any/cvs-resolv-internal-qtype.diff

plus 283e7a294275a7da53258600deaaafbbec6b96c1 in debian-installer.git is
what is triggering the issue? (Explaining why host's libc .deb have an
impact on the built images.)

It's been a while since I last looked at/understood mklibs stuff though,
feel free to fix my suspicions/conclusions.


KiBi.


signature.asc
Description: Digital signature

Re: Bug#851790: installation-reports: DNS not working

[Cyril Brulebois]

Cyril Brulebois  (2017-01-19):
> You can try downgrading the kernel, but I'd usually look at glibc first
> for DNS related issues.

Summing up things from IRC:
 - in an uptodate sid chroot (both development version and minimal one
   with daily-build script): no DNS issues with the generated mini.iso
   (amd64, tested within stable's kvm on amd64). My image was also
   tested successfully by Steve, so not a setup issue.

 - I can reproduce the issue with dailies from 2017-01-18, and from
   2017-01-19 (I picked it in advance during its build on barriere).

 - I can reproduce the issue in the above mentioned sid chroot if
   I downgrade these packages to testing's version (-9 → -8):
 sudo apt-get install libc6:amd64=2.24-8 libc6-dev:amd64=2.24-8 
libc-dev-bin=2.24-8

 - The older set of libc* packages is installed in barriere's current
   amd64 sid chroot, too.

 - Steve could only produce broken images when building locally, until
   he upgraded his libc* packages as well.


→ Copying glibc@packages for the time being, even I suspect this will
  likely disappear entirely once -9 propagates…


KiBi.


signature.asc
Description: Digital signature