Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
Control: tag -1 + fixed-upstream patch On Mon, Mar 13, 2017 at 04:24:01PM +1300, Olly Betts wrote: > I think this is worthwhile addressing in xapian-core. I've pushed a fix for this upstream: https://trac.xapian.org/changeset/9ed4dec1cb51c241cf7eb0b927f0e3122c2676a2/git Once the release is out I'll push this to unstable. Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
Control: tag -1 -moreinfo Control: tag -1 +upstream This came up on #debian-devel and jrtc27 (James Clarke) spotted the underlying cause - from "man getaddrinfo": | If hints.ai_flags includes the AI_ADDRCONFIG flag, then IPv4 addresses | are returned in the list pointed to by res only if the local system has | at least one IPv4 address configured, and IPv6 addresses are returned | only if the local system has at least one IPv6 address configured. The | loopback address is not considered for this case as valid as a config‐ | ured address. The last sentence about loopback isn't in POSIX, but is in RFC3493. And this is the problem here - inside the chroot, pbuilder has removed all but loopback, so getaddrinfo() won't return an IPv4 address as there's no IPv4 address configured (not considering loopback). By similar logic, it won't return an IPv6 address either. Practically, you wouldn't actually use Xapian's remote backend on a box with only loopback - you'd have to run the server on the same box, which makes little sense. But this setup is likely to be seen in testsuites for both xapian-core, and for things which build on it and use the remote backend, and disabling networking except localhost is sane for autobuilders, so I think this is worthwhile addressing in xapian-core. Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Thu, Feb 16, 2017 at 03:21:01PM -0800, Daniel Schepler wrote: > On Thu, Feb 16, 2017 at 1:37 PM, Olly Bettswrote: > > One thought - is /etc/gai.conf present in the chroot with any uncommented > > content? > > Nope, it's just the normal template /etc/gai.conf file with everything > commented. > > I think pbuilder does blank out /etc/resolv.conf in the chroot to > prevent that from confusing things. Is it possible that for some > reason, the code now requires a DNS server to be available even to > bind to a numeric IP address? I just tried temporarily emptying /etc/resolv.conf on my dev box and "./apitest -bremotetcp_glass" still works (using HEAD of RELEASE/1.4 branch, which hasn't changed in any relevant way since 1.4.3). Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Thu, Feb 16, 2017 at 1:37 PM, Olly Bettswrote: > One thought - is /etc/gai.conf present in the chroot with any uncommented > content? Nope, it's just the normal template /etc/gai.conf file with everything commented. I think pbuilder does blank out /etc/resolv.conf in the chroot to prevent that from confusing things. Is it possible that for some reason, the code now requires a DNS server to be available even to bind to a numeric IP address? -- Daniel
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Thu, Feb 16, 2017 at 09:19:35AM -0800, Daniel Schepler wrote: > > Daniel: What version of pbuilder are you using? If it's more than a > > year old, then upgrading will probably fix this. If not, then either > > that fix has regressed in pbuilder regression, or else something more > > complex is going on. > > I was still able to reproduce it with the latest pbuilder version > (0.228.4); and even though my previous pbuilder version was somewhat > out of date, it was still new enough to have the code setting up an > "lo" interface. So, as you said, something else must be going on > here. OK, thanks for checking with the latest. > Incidentally, the build did work until recently - the last version I > have with a successful build is 1.2.23-1, and I think it started > failing with the 1.4.* series. As I said above: | The reproducible build used to work but it looks like it regressed in | 1.3.4-1: | | https://tests.reproducible-builds.org/debian/history/xapian-core.html | | That's very suspicious as that's the first version packaged for Debian | which used getaddrinfo() (1.2.x used gethostbyname(), etc), and the | history shows that 1.2.x continued to build OK at the same time. So | that seems to point firmly to this not being a pbuilder regression. | | But I've no idea what the cause is. My only thoughts are that perhaps | there's only IPv6 loopback (but then 1.2.x wouldn't work), or that | getaddrinfo() is interacting badly with the restricted networking. One thought - is /etc/gai.conf present in the chroot with any uncommented content? Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Sun, Jan 29, 2017 at 9:06 PM, Olly Bettswrote: > On Sun, Jan 29, 2017 at 06:00:06PM -0400, David Bremner wrote: >> Daniel Schepler writes: >> >> > Source: xapian-core >> > Version: 1.4.3-1 >> > Severity: normal >> >> I'm not the xapian maintainer, but I consider this not a bug. Using >> localhost networking for testing is common, and not forbidden by Debian >> policy afaik. > > Policy's wording isn't entirely clear here - from §4.9: > > | For packages in the main archive, no required targets may attempt > | network access. > > My reading (and I assume what is intended) is that this is not meant to > include localhost, but localhost is the loopback *network* interface, so > I can see how people could interpret it to exclude that too. > > But I think having localhost available is valuable, and if the latter > interpretation was being argued for, I think policy needs clarifying. > As David points out, having localhost available allows testsuites to test > network functionality. If we can't run such tests we could miss > Debian-specific patches introducing regressions which the testsuite would > have caught if we'd been able to run it. > > But pbuilder with USENETWORK=no should actually still provide lo. > It didn't originally but that was reported and addressed: > > https://bugs.debian.org/753944 > > Daniel: What version of pbuilder are you using? If it's more than a > year old, then upgrading will probably fix this. If not, then either > that fix has regressed in pbuilder regression, or else something more > complex is going on. I was still able to reproduce it with the latest pbuilder version (0.228.4); and even though my previous pbuilder version was somewhat out of date, it was still new enough to have the code setting up an "lo" interface. So, as you said, something else must be going on here. Incidentally, the build did work until recently - the last version I have with a successful build is 1.2.23-1, and I think it started failing with the 1.4.* series. -- Daniel
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Tue, Jan 31, 2017 at 10:09:57PM +0100, Georg Faerber wrote: > > Daniel: What version of pbuilder are you using? If it's more than a > > year old, then upgrading will probably fix this. If not, then either > > that fix has regressed in pbuilder regression, or else something more > > complex is going on. > > This is happening here [1] as well, which is a recent pbuilder version, > I guess. I'm wondering if this is a bug in pbuilder or a wrapper around. > > Cheers, > Georg > (who debugged for some hours a similar problem over here [2]...) Did you reach any useful conclusions? FWIW, the code that appears to fail is: struct addrinfo hints; memset(, 0, sizeof(struct addrinfo)); hints.ai_family = AF_INET; hints.ai_socktype = SOCK_STREAM; hints.ai_flags = AI_PASSIVE | AI_ADDRCONFIG | AI_NUMERICSERV; hints.ai_protocol = 0; hints.ai_canonname = NULL; hints.ai_addr = NULL; hints.ai_next = NULL; const char * node = host.empty() ? NULL : host.c_str(); struct addrinfo *result; int s = getaddrinfo(node, str(port).c_str(), , ); if (s != 0) { throw Xapian::NetworkError("Couldn't resolve host " + host, eai_to_xapian(s)); } Where host is "127.0.0.1" and port is "1239". It seems the error is: EAI_NONAME The node or service is not known; or both node and service are NULL; or AI_NUMERICSERV was specified in hints.ai_flags and ser‐ vice was not a numeric port-number string. The reproducible build used to work but it looks like it regressed in 1.3.4-1: https://tests.reproducible-builds.org/debian/history/xapian-core.html That's very suspicious as that's the first version packaged for Debian which used getaddrinfo() (1.2.x used gethostbyname(), etc), and the history shows that 1.2.x continued to build OK at the same time. So that seems to point firmly to this not being a pbuilder regression. But I've no idea what the cause is. My only thoughts are that perhaps there's only IPv6 loopback (but then 1.2.x wouldn't work), or that getaddrinfo() is interacting badly with the restricted networking. Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
Hi Olly, > Daniel: What version of pbuilder are you using? If it's more than a > year old, then upgrading will probably fix this. If not, then either > that fix has regressed in pbuilder regression, or else something more > complex is going on. This is happening here [1] as well, which is a recent pbuilder version, I guess. I'm wondering if this is a bug in pbuilder or a wrapper around. Cheers, Georg (who debugged for some hours a similar problem over here [2]...) [1] https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/xapian-core_1.4.3-1.rbuild.log [2] https://tests.reproducible-builds.org/debian/rbuild/unstable/amd64/schleuder_3.0.1-1.rbuild.log signature.asc Description: Digital signature
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
On Sun, Jan 29, 2017 at 06:00:06PM -0400, David Bremner wrote: > Daniel Scheplerwrites: > > > Source: xapian-core > > Version: 1.4.3-1 > > Severity: normal > > I'm not the xapian maintainer, but I consider this not a bug. Using > localhost networking for testing is common, and not forbidden by Debian > policy afaik. Policy's wording isn't entirely clear here - from §4.9: | For packages in the main archive, no required targets may attempt | network access. My reading (and I assume what is intended) is that this is not meant to include localhost, but localhost is the loopback *network* interface, so I can see how people could interpret it to exclude that too. But I think having localhost available is valuable, and if the latter interpretation was being argued for, I think policy needs clarifying. As David points out, having localhost available allows testsuites to test network functionality. If we can't run such tests we could miss Debian-specific patches introducing regressions which the testsuite would have caught if we'd been able to run it. But pbuilder with USENETWORK=no should actually still provide lo. It didn't originally but that was reported and addressed: https://bugs.debian.org/753944 Daniel: What version of pbuilder are you using? If it's more than a year old, then upgrading will probably fix this. If not, then either that fix has regressed in pbuilder regression, or else something more complex is going on. Cheers, Olly
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
Daniel Scheplerwrites: > Source: xapian-core > Version: 1.4.3-1 > Severity: normal > I'm not the xapian maintainer, but I consider this not a bug. Using localhost networking for testing is common, and not forbidden by Debian policy afaik. d
Bug#853107: xapian-core: FTBFS under pbuilder with USENETWORK=no
Source: xapian-core Version: 1.4.3-1 Severity: normal >From my pbuilder build log: ... Running test: qpmemoryleak1... EXCEPTION std::string Failed to get 'Listening...' from command '../bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 -t30 --writable .chert/dbw' (output: Starting writable serv er on host 127.0.0.1, port 1239 NetworkError: Couldn't resolve host 127.0.0.1 (Name or service not known) ) Running test: closedb4... EXCEPTION std::string Failed to get 'Listening...' from command '../bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 -t30 --writable .chert/dbw' (output: Starting writable server on host 127.0.0.1, port 1239 NetworkError: Couldn't resolve host 127.0.0.1 (Name or service not known) ) Running test: closedb10... EXCEPTION std::string Failed to get 'Listening...' from command '../bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 -t30 --writable .chert/dbw' (output: Starting writable server o n host 127.0.0.1, port 1239 NetworkError: Couldn't resolve host 127.0.0.1 (Name or service not known) ) Running test: writeread1... EXCEPTION std::string Failed to get 'Listening...' from command '../bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 -t30 --writable .chert/dbw' (output: Starting writable server on host 127.0.0.1, port 1239 NetworkError: Couldn't resolve host 127.0.0.1 (Name or service not known) ) Running test: valuestats1... EXCEPTION std::string Failed to get 'Listening...' from command '../bin/xapian-tcpsrv --one-shot --interface 127.0.0.1 --port 1239 -t30 --writable .chert/dbw' (output: Starting writable server on host 127.0.0.1, port 1239 NetworkError: Couldn't resolve host 127.0.0.1 (Name or service not known) ) Running test: valueweightsource5... SKIPPED ./apitest backend remotetcp_chert: 1 tests passed, 286 failed, 2 skipped. ./apitest total: 2441 tests passed, 572 failed, 41 skipped. FAIL: apitest Running test: exception1... ok Running test: refcnt1... ok Running test: refcnt2... ok ... PASS: unittest 1 of 4 tests failed Please report to https://xapian.org/bugs Makefile:1003: recipe for target 'check-TESTS' failed make[5]: *** [check-TESTS] Error 1 make[5]: Leaving directory '/build/xapian-core-1.4.3/build/tests' Makefile:1126: recipe for target 'check-am' failed make[4]: *** [check-am] Error 2 make[4]: Leaving directory '/build/xapian-core-1.4.3/build/tests' Makefile:1129: recipe for target 'check' failed make[3]: *** [check] Error 2 make[3]: Leaving directory '/build/xapian-core-1.4.3/build/tests' Makefile:3209: recipe for target 'check-recursive' failed make[2]: *** [check-recursive] Error 1 make[2]: Leaving directory '/build/xapian-core-1.4.3/build' Makefile:3497: recipe for target 'check' failed make[1]: *** [check] Error 2 make[1]: Leaving directory '/build/xapian-core-1.4.3/build' debian/rules:167: recipe for target 'build-stamp' failed make: *** [build-stamp] Error 2 dpkg-buildpackage: error: debian/rules build gave error exit status 2 -- Daniel Schepler