Re: Bug#856152: python-apt: FTBFS: Testsuite failure

[Aurelien Jarno]

On 2017-07-31 15:11, Mattia Rizzolo wrote:
> On Sun, Apr 23, 2017 at 10:07:08PM +0200, Julian Andres Klode wrote:
> > On Wed, Mar 01, 2017 at 03:33:55PM +, James Clarke wrote:
> > > So the issue here is that, by default, pbuilder runs the build in a
> > > separate network namespace with only a loopback interface configured.
> > > The loopback interface works, you can bind and connect as normal
> > > (otherwise this would have been found a long time ago), but getaddrinfo
> > > has a slightly interesting deviation from POSIX. POSIX states[1]:
> > 
> > It also fails on my system now since I switched from dnsmasq to 
> > systemd-resolved. That's really annoying.
> 
> Just a quick note that it's probably the same thing that is tracked in
> glibc's #844420 (glibc FTBFS in pbuilder due a test failing to resolve
> localhost).

I doubt it. The bug in #844420 is that glibc fails to resolve entries
from /etc/hosts when they are fully qualified. It means with the
standard /etc/hosts, it fails to resolv "localhost." but succeed in
resolving "localhost".
In the python-apt bug, it seems that the entry to be resolved is
"localhost".

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature

Re: Bug#856152: python-apt: FTBFS: Testsuite failure

[Mattia Rizzolo]

On Sun, Apr 23, 2017 at 10:07:08PM +0200, Julian Andres Klode wrote:
> On Wed, Mar 01, 2017 at 03:33:55PM +, James Clarke wrote:
> > So the issue here is that, by default, pbuilder runs the build in a
> > separate network namespace with only a loopback interface configured.
> > The loopback interface works, you can bind and connect as normal
> > (otherwise this would have been found a long time ago), but getaddrinfo
> > has a slightly interesting deviation from POSIX. POSIX states[1]:
> 
> It also fails on my system now since I switched from dnsmasq to 
> systemd-resolved. That's really annoying.

Just a quick note that it's probably the same thing that is tracked in
glibc's #844420 (glibc FTBFS in pbuilder due a test failing to resolve
localhost).

-- 
regards,
Mattia Rizzolo

GPG Key: 66AE 2B4A FCCF 3F52 DA18  4D18 4B04 3FCD B944 4540  .''`.
more about me:  https://mapreri.org : :'  :
Launchpad user: https://launchpad.net/~mapreri  `. `'`
Debian QA page: https://qa.debian.org/developer.php?login=mattia  `-


signature.asc
Description: PGP signature

Re: Bug#856152: python-apt: FTBFS: Testsuite failure

[Julian Andres Klode]

Control: tags -1 - moreinfo

(jrtc forgot to untag this...)

On Wed, Mar 01, 2017 at 03:33:55PM +, James Clarke wrote:
> Control: reassign -1 dirmngr 2.1.18-6
> Control: retitle -1 dirmngr: Fails to resolve localhost and loopback 
> addresses when only a loopback interface is available
> 
> On Wed, Mar 01, 2017 at 01:31:40AM +0100, Julian Andres Klode wrote:
> > Control: severity 856152 important
> >
> > On Wed, Mar 01, 2017 at 06:39:37AM +0800, Chris Lamb wrote:
> > > retitle 856152 python-apt: FTBFS: AptKeyError: recv from 
> > > 'hkp://localhost:19191' failed for 
> > > '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
> > > thanks
> > >
> > > Julian Andres Klode wrote:
> > >
> > > > Retry it. Maybe it timed out or something.
> > >
> > > I don't think this is a timeout issue, but if it is, surely the package
> > > build should be a little more reliable? :)
> >
> > Well, it's some GPG issue, we can't figure out every GPG thing.
> >
> > This works fine with an up-to-date sid chroot in sbuild, so I don't
> > really care, or well, can't reproduce it. Seems more like a pbuilder
> > related issue.
> 
> So the issue here is that, by default, pbuilder runs the build in a
> separate network namespace with only a loopback interface configured.
> The loopback interface works, you can bind and connect as normal
> (otherwise this would have been found a long time ago), but getaddrinfo
> has a slightly interesting deviation from POSIX. POSIX states[1]:

It also fails on my system now since I switched from dnsmasq to 
systemd-resolved. That's really annoying.

-- 
Debian Developer - deb.li/jak | jak-linux.org - free software dev
  |  Ubuntu Core Developer |
When replying, only quote what is necessary, and write each reply
directly below the part(s) it pertains to ('inline').  Thank you.

Re: Bug#856152: python-apt: FTBFS: Testsuite failure

[James Clarke]

Control: reassign -1 dirmngr 2.1.18-6
Control: retitle -1 dirmngr: Fails to resolve localhost and loopback addresses 
when only a loopback interface is available

On Wed, Mar 01, 2017 at 01:31:40AM +0100, Julian Andres Klode wrote:
> Control: severity 856152 important
>
> On Wed, Mar 01, 2017 at 06:39:37AM +0800, Chris Lamb wrote:
> > retitle 856152 python-apt: FTBFS: AptKeyError: recv from 
> > 'hkp://localhost:19191' failed for 
> > '0xa1bD8E9D78F7FE5C3E65D8AF8B48AD6246925553'
> > thanks
> >
> > Julian Andres Klode wrote:
> >
> > > Retry it. Maybe it timed out or something.
> >
> > I don't think this is a timeout issue, but if it is, surely the package
> > build should be a little more reliable? :)
>
> Well, it's some GPG issue, we can't figure out every GPG thing.
>
> This works fine with an up-to-date sid chroot in sbuild, so I don't
> really care, or well, can't reproduce it. Seems more like a pbuilder
> related issue.

So the issue here is that, by default, pbuilder runs the build in a
separate network namespace with only a loopback interface configured.
The loopback interface works, you can bind and connect as normal
(otherwise this would have been found a long time ago), but getaddrinfo
has a slightly interesting deviation from POSIX. POSIX states[1]:

> If the AI_ADDRCONFIG flag is specified, IPv4 addresses shall be
> returned only if an IPv4 address is configured on the local system,
> and IPv6 addresses shall be returned only if an IPv6 address is
> configured on the local system.

However, glibc's manpage states this:

> If hints.ai_flags includes the AI_ADDRCONFIG flag, then IPv4 addresses
> are returned in the list pointed to by res only if the local system
> has at least one IPv4 address configured, and IPv6 addresses are
> returned only if the local system has at least one IPv6 address
> configured. The loopback address is not considered for this case as
> valid as a configured address. This flag is useful on, for example,
> IPv4-only systems, to ensure that getaddrinfo() does not return IPv6
> socket addresses that would always fail in connect(2) or bind(2).

Note that in fact glibc is in agreement with RFC 3493, which has the
following modified version of POSIX's definition (thanks to Olly Betts
for finding this):

> If the AI_ADDRCONFIG flag is specified, IPv4 addresses shall be
> returned only if an IPv4 address is configured on the local system,
> and IPv6 addresses shall be returned only if an IPv6 address is
> configured on the local system. The loopback address is not considered
> for this case as valid as a configured address.

Now, GPG is using getaddrinfo with AI_ADDRCONFIG[2]. I have yet to
confirm that this is the issue here, but #853107 is a very similar bug,
and removing the AI_ADDRCONFIG flag makes the build succeed.

I don't know what the solution should be. Glibc's implementation seems
sensible in the general case, but horribly broken when resolving
localhost/127.0.0.1 with just a loopback interface up, which should be a
perfectly valid thing to do. I believe that trying to build on a machine
without a non-loopback interface up would also run into this bug, which
is a rare situation these days, but not completely out of the question.
In my opinion, glibc should not be ignoring the loopback address in this
case, but I don't know whether that's easily possible in general and
whether it would break existing programs. For pbuilder I guess we could
work around it by bringing up a separate virtual interface, but I really
don't like that idea, and it doesn't solve the problem of building on
machines without a non-loopback interface.

Here is a reproduction of the issue with plain GPG. I have a local SKS
instance bound to 127.0.0.1:11371 which contains a single key. I had to
kill dirmngr in between because it seems to cache lookups.

> debian:~ james% ifconfig
> eth0: flags=4163  mtu 1500
> inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255
> inet6 fe80::a00:27ff:fe01:add2  prefixlen 64  scopeid 0x20
> ether 08:00:27:01:ad:d2  txqueuelen 1000  (Ethernet)
> RX packets 245463  bytes 341360397 (325.5 MiB)
> RX errors 0  dropped 0  overruns 0  frame 0
> TX packets 89176  bytes 5170918 (4.9 MiB)
> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> lo: flags=73  mtu 65536
> inet 127.0.0.1  netmask 255.0.0.0
> inet6 ::1  prefixlen 128  scopeid 0x10
> loop  txqueuelen 1  (Local Loopback)
> RX packets 822  bytes 166056 (162.1 KiB)
> RX errors 0  dropped 0  overruns 0  frame 0
> TX packets 822  bytes 166056 (162.1 KiB)
> TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
>
> debian:~ james% gpg --no-default-keyring --no-options --homedir 
> /tmp/test-gnupg --secret-keyring /tmp/test-gnupg/secring.gpg --keyring 
> /tmp/test-gnupg/pubring.gpg --keyserver hkp://127.0.0.1:11371 --recv 186A1C7D
> gpg: