Re: Bug#910560: [choose-mirror] fails to build when parallel build is activated
On Mon, Oct 08, 2018 at 10:30:06AM +0200, Philipp Kern wrote: > > > (can also be seen at jenkins: > > > https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ > > > where I found it initially) > dpkg-buildpackage -j is like the worst option to ever have been introduced > and not removed. Try -J instead. :( thanks, I've now changed the jenkins scripts to use -J instead of -j (*) and indeed https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/148/console finally succeeded again. -- cheers, Holger (*) so now they either use -J$(nproc) or set DEB_BUILD_OPTIONS="parallel=$(nproc)" signature.asc Description: PGP signature
Bug#910560: [choose-mirror] fails to build when parallel build is activated
On 10/8/18 5:33 PM, Philipp Kern wrote: >> Controversial opinion: It should use sbuild instead of pbuilder. sbuild >> is more actively maintained and more reliable in my experience. sbuild >> is also what the buildds are using. > > sbuild doesn't solve this particular problem either. You need to pass in > DEB_BUILD_OPTIONS=parallel=n rather than setting --jobs. The latter is mapped > to -j, which breaks (because it's put into MAKEFLAGS) and the former maps to > -J. Ok, I've never used "-j" though which is probably why I have never run into this problem. Sounds like an obscure option to me after reading the manpage. > Julien is right in that there is a bug here that's worth fixing but the > default build environment which is incredibly hard to discover does not > expose them. What exactly is this particular bug then? Does the rules file of choose-mirror anything special in this regard? Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#910560: [choose-mirror] fails to build when parallel build is activated
On 2018-10-08 12:42, John Paul Adrian Glaubitz wrote: On 10/8/18 12:38 PM, Holger Wansing wrote: Ok, so we have no package problem at all? Well, it builds fine on the buildds and it also does not show build issues on reproducible-builds.org: https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/choose-mirror.html Should that bug be reassigned to jenkins.debian.org then, to make jenkins happy on that topic, too? jenkins seems to not having parallel builds activated? Should probably use similar settings as the buildds, to give comparable results? Controversial opinion: It should use sbuild instead of pbuilder. sbuild is more actively maintained and more reliable in my experience. sbuild is also what the buildds are using. sbuild doesn't solve this particular problem either. You need to pass in DEB_BUILD_OPTIONS=parallel=n rather than setting --jobs. The latter is mapped to -j, which breaks (because it's put into MAKEFLAGS) and the former maps to -J. Julien is right in that there is a bug here that's worth fixing but the default build environment which is incredibly hard to discover does not expose them. Kind regards Philipp Kern
Bug#910560: [choose-mirror] fails to build when parallel build is activated
There's a bug in choose-mirror. It's just not serious. Julien On October 8, 2018 12:38:40 PM GMT+02:00, Holger Wansing wrote: >Hi, > >Philipp Kern wrote: >> On 2018-10-08 09:08, John Paul Adrian Glaubitz wrote: >> > On 10/8/18 7:51 AM, Holger Wansing wrote: >> >> Since version 2.92, choose-mirror fails to build with >> >> "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be >> >> removed by >> >> error: >> >> >> >> (can also be seen at jenkins: >> >> >https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ >> >> where I found it initially) >> > It builds fine here on my machine using sbuild and also fine on the > >> > buildds >> > which are building with sbuild and "parallel=N" with N >= 2 [1]. >> > >> > You are building in an unclean build environment unless you are >> > building with >> > something like sbuild and pbuilder, so your build results can have >> > unexpected >> > results. >> > >> > Please create a local sbuild setup and try again. >> > >> > Adrian >> > >> >> [1] >> >> >https://buildd.debian.org/status/package.php?p=choose-mirror=unstable >> >> dpkg-buildpackage -j is like the worst option to ever have been >> introduced and not removed. Try -J instead. :( > >Ok, so we have no package problem at all? > >Should that bug be reassigned to jenkins.debian.org then, to make >jenkins >happy on that topic, too? >jenkins seems to not having parallel builds activated? >Should probably use similar settings as the buildds, to give comparable >results? > > >Holger > > >-- >Holger Wansing >PGP-Finterprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Bug#910560: [choose-mirror] fails to build when parallel build is activated
On 10/8/18 12:38 PM, Holger Wansing wrote: > Ok, so we have no package problem at all? Well, it builds fine on the buildds and it also does not show build issues on reproducible-builds.org: > https://tests.reproducible-builds.org/debian/rb-pkg/unstable/amd64/choose-mirror.html > Should that bug be reassigned to jenkins.debian.org then, to make jenkins > happy on that topic, too? > jenkins seems to not having parallel builds activated? > Should probably use similar settings as the buildds, to give comparable > results? Controversial opinion: It should use sbuild instead of pbuilder. sbuild is more actively maintained and more reliable in my experience. sbuild is also what the buildds are using. Adrian -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#910560: [choose-mirror] fails to build when parallel build is activated
Hi, Philipp Kern wrote: > On 2018-10-08 09:08, John Paul Adrian Glaubitz wrote: > > On 10/8/18 7:51 AM, Holger Wansing wrote: > >> Since version 2.92, choose-mirror fails to build with > >> "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be > >> removed by > >> error: > >> > >> (can also be seen at jenkins: > >> https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ > >> where I found it initially) > > It builds fine here on my machine using sbuild and also fine on the > > buildds > > which are building with sbuild and "parallel=N" with N >= 2 [1]. > > > > You are building in an unclean build environment unless you are > > building with > > something like sbuild and pbuilder, so your build results can have > > unexpected > > results. > > > > Please create a local sbuild setup and try again. > > > > Adrian > > > >> [1] > >> https://buildd.debian.org/status/package.php?p=choose-mirror=unstable > > dpkg-buildpackage -j is like the worst option to ever have been > introduced and not removed. Try -J instead. :( Ok, so we have no package problem at all? Should that bug be reassigned to jenkins.debian.org then, to make jenkins happy on that topic, too? jenkins seems to not having parallel builds activated? Should probably use similar settings as the buildds, to give comparable results? Holger -- Holger Wansing PGP-Finterprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
Processed: Re: Bug#910560: [choose-mirror] fails to build when parallel build is activated
Processing control commands: > severity -1 wishlist Bug #910560 [choose-mirror] [choose-mirror] fails to build when parallel build is activated Severity set to 'wishlist' from 'serious' -- 910560: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=910560 Debian Bug Tracking System Contact ow...@bugs.debian.org with problems
Bug#910560: [choose-mirror] fails to build when parallel build is activated
Control: severity -1 wishlist On 10/08/2018 07:51 AM, Holger Wansing wrote: > Package: choose-mirror > Severity: serious > Version: 2.92 > > Since version 2.92, choose-mirror fails to build with > "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be removed by > error: > > (can also be seen at jenkins: > https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ > where I found it initially) > > Reducing severity, dpkg-buildpackage -j is broken. Cheers, Julien
Re: Bug#910560: [choose-mirror] fails to build when parallel build is activated
On 2018-10-08 09:08, John Paul Adrian Glaubitz wrote: On 10/8/18 7:51 AM, Holger Wansing wrote: Since version 2.92, choose-mirror fails to build with "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be removed by error: (can also be seen at jenkins: https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ where I found it initially) It builds fine here on my machine using sbuild and also fine on the buildds which are building with sbuild and "parallel=N" with N >= 2 [1]. You are building in an unclean build environment unless you are building with something like sbuild and pbuilder, so your build results can have unexpected results. Please create a local sbuild setup and try again. Adrian [1] https://buildd.debian.org/status/package.php?p=choose-mirror=unstable dpkg-buildpackage -j is like the worst option to ever have been introduced and not removed. Try -J instead. :( Kind regards Philipp Kern
Re: Bug#910560: [choose-mirror] fails to build when parallel build is activated
On 10/8/18 7:51 AM, Holger Wansing wrote: > Since version 2.92, choose-mirror fails to build with > "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be removed by > error: > > (can also be seen at jenkins: > https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ > where I found it initially) It builds fine here on my machine using sbuild and also fine on the buildds which are building with sbuild and "parallel=N" with N >= 2 [1]. You are building in an unclean build environment unless you are building with something like sbuild and pbuilder, so your build results can have unexpected results. Please create a local sbuild setup and try again. Adrian > [1] > https://buildd.debian.org/status/package.php?p=choose-mirror=unstable -- .''`. John Paul Adrian Glaubitz : :' : Debian Developer - glaub...@debian.org `. `' Freie Universitaet Berlin - glaub...@physik.fu-berlin.de `-GPG: 62FF 8A75 84E0 2956 9546 0006 7426 3B37 F5B5 F913
Bug#910560: [choose-mirror] fails to build when parallel build is activated
Package: choose-mirror Severity: serious Version: 2.92 Since version 2.92, choose-mirror fails to build with "dpkg-buildpackage -j", the debian/iso_3166.tab file seems to be removed by error: (can also be seen at jenkins: https://jenkins.debian.net/view/d-i_packages/job/d-i_build_choose-mirror/ where I found it initially) holgerw@t520:~/uploading-d-i/d-i/packages/choose-mirror$ dpkg-buildpackage -j dpkg-buildpackage: info: source package choose-mirror dpkg-buildpackage: info: source version 2.94 dpkg-buildpackage: info: source distribution unstable dpkg-buildpackage: info: source changed by Holger Wansing dpkg-buildpackage: info: host architecture amd64 dpkg-source --before-build choose-mirror fakeroot debian/rules clean dh clean dh_testdir debian/rules override_dh_auto_clean make[1]: Entering directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' rm -rf debian/locales debian/sort-tmp /usr/bin/make clean check-masterlist make[2]: Entering directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' rm -f choose-mirror choose-mirror.o *~ mirrors_*.h rm -f debian/templates-countries debian/httplist-countries debian/httpslist-countries debian/ftplist-countries rm -f demo demo.templates rm -rf debian/iso-codes/ debian/pobuild*/ rm -f debian/iso_3166.tab *** WARNING: Mirrors.masterlist was last committed more *** than 2 weeks ago, maybe it needs an update? You can try the following command to run a sync, and use git diff/git commit: make Mirrors.masterlist make[2]: Leaving directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' make[1]: Leaving directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' dh_clean dpkg-source -b choose-mirror dpkg-source: info: using source format '3.0 (native)' dpkg-source: info: building choose-mirror in choose-mirror_2.94.tar.xz dpkg-source: info: building choose-mirror in choose-mirror_2.94.dsc dpkg-source: warning: missing information for output field Standards-Version debian/rules build dh build dh_testdir dh_update_autotools_config dh_auto_configure debian/rules override_dh_auto_build make[1]: Entering directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' # Don't try to sync the mirror masterlist during the build: /usr/bin/make small ONLINE=n make[2]: Entering directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' rm -f choose-mirror choose-mirror.o *~ mirrors_*.h if [ "$ONLINE" != n ]; then \ if wget -nv 'https://salsa.debian.org/mirror-team/masterlist/raw/master/Mirrors.masterlist' -O - > Mirrors.masterlist.new && \ test -s Mirrors.masterlist.new; then \ mv Mirrors.masterlist.new Mirrors.masterlist; \ else \ rm -f Mirrors.masterlist.new; \ fi; \ fi isoquery -c | cut -f 1,4 | sort >debian/iso_3166.tab rm -f debian/templates-countries debian/httplist-countries debian/httpslist-countries debian/ftplist-countries rm -f demo demo.templates rm -rf debian/iso-codes/ debian/pobuild*/ rm -f debian/iso_3166.tab if [ "$USE_HTTP" ]; then ./mirrorlist http Mirrors.masterlist debian/iso_3166.tab; fi if [ "$USE_HTTPS" ]; then ./mirrorlist https Mirrors.masterlist debian/iso_3166.tab; fi if [ "$USE_FTP" ]; then ./mirrorlist ftp Mirrors.masterlist debian/iso_3166.tab; fi ./mirrorlist httplist Mirrors.masterlist debian/iso_3166.tab ./mirrorlist httpslist Mirrors.masterlist debian/iso_3166.tab ./mirrorlist ftplist Mirrors.masterlist debian/iso_3166.tab Unable to read debian/iso_3166.tab at ./mirrorlist line 23. Unable to read debian/iso_3166.tab at ./mirrorlist line 23. Unable to read debian/iso_3166.tab at ./mirrorlist line 23. Makefile:81: recipe for target 'debian/httplist-countries' failed make[2]: *** [debian/httplist-countries] Error 2 make[2]: *** Waiting for unfinished jobs Makefile:99: recipe for target 'mirrors_https.h' failed make[2]: *** [mirrors_https.h] Error 2 Makefile:84: recipe for target 'debian/httpslist-countries' failed make[2]: *** [debian/httpslist-countries] Error 2 Unable to read debian/iso_3166.tab at ./mirrorlist line 23. Makefile:87: recipe for target 'debian/ftplist-countries' failed make[2]: *** [debian/ftplist-countries] Error 2 Unable to read debian/iso_3166.tab at ./mirrorlist line 23. Makefile:96: recipe for target 'mirrors_http.h' failed make[2]: *** [mirrors_http.h] Error 2 make[2]: Leaving directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' debian/rules:15: recipe for target 'override_dh_auto_build' failed make[1]: *** [override_dh_auto_build] Error 2 make[1]: Leaving directory '/home/holgerw/uploading-d-i/d-i/packages/choose-mirror' debian/rules:3: recipe for target 'build' failed make: *** [build] Error 2 dpkg-buildpackage: error: debian/rules build gave error exit status 2 -- Holger Wansing PGP-Finterprint: 496A C6E8 1442 4B34 8508 3529 59F1 87CA 156E B076
