Re: build-time testing of pure arch:all packages
On Fri, Jun 22, 2012 at 12:29:48AM -0400, Yaroslav Halchenko wrote:
I was thinking about a bit more automated way... ideally (in the long
run) even that FTBFS (e.g. due to failed tests or some other arch
specific quirks) would forbid automatic migration to wheezy etc --
kinda full blown benefits from the Debian infrastructure without
much of work from my side ;)
Did you check if the QA rebuilds try to, or could be asked to, build
arch:all packages? If so, it'd be a good start to routinely do
archive-wide rebuilds of arch:all packages as we routinely do rebuilds
of arch:any packages.
(Cc:-ing Lucas, for his great work on QA rebuilds.)
Cheers.
--
Stefano Zacchiroli zack@{upsilon.cc,pps.jussieu.fr,debian.org} . o .
Maître de conférences .. http://upsilon.cc/zack .. . . o
Debian Project Leader... @zack on identi.ca ...o o o
« the first rule of tautology club is the first rule of tautology club »
signature.asc
Description: Digital signature
Re: build-time testing of pure arch:all packages
Stefano Zacchiroli z...@debian.org writes: On Fri, Jun 22, 2012 at 12:29:48AM -0400, Yaroslav Halchenko wrote: I was thinking about a bit more automated way... ideally (in the long run) even that FTBFS (e.g. due to failed tests or some other arch specific quirks) would forbid automatic migration to wheezy etc -- kinda full blown benefits from the Debian infrastructure without much of work from my side ;) You can do this in a not so nice way like this: 1) if you have no Architecture: any package then add a package-test-logs package. 2) build-arch depends on build-indep (i.e. always compile the arch:all stuff) 3) in build-arch run the tests and include the log in the Architecture:any package This adds a stupid dummy package to the archive that nobody but you need or fattens up an existing one with a logfile nobody but you need. That is the not so nice part. But it would do what you want now. Note: Consider carefully if it wouldn't make more sense to help build a real automatic testing infrastructure (which could reuse idle buildd). How urgently do you need those tests to be run? Did you check if the QA rebuilds try to, or could be asked to, build arch:all packages? If so, it'd be a good start to routinely do archive-wide rebuilds of arch:all packages as we routinely do rebuilds of arch:any packages. (Cc:-ing Lucas, for his great work on QA rebuilds.) Are archive wide rebuilds done on anythiong but i386/amd64? MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87bokblo6o.fsf@frosties.localnet
Re: build-time testing of pure arch:all packages
Hi, On 22/06/12 at 09:37 +0200, Stefano Zacchiroli wrote: On Fri, Jun 22, 2012 at 12:29:48AM -0400, Yaroslav Halchenko wrote: I was thinking about a bit more automated way... ideally (in the long run) even that FTBFS (e.g. due to failed tests or some other arch specific quirks) would forbid automatic migration to wheezy etc -- kinda full blown benefits from the Debian infrastructure without much of work from my side ;) Did you check if the QA rebuilds try to, or could be asked to, build arch:all packages? If so, it'd be a good start to routinely do archive-wide rebuilds of arch:all packages as we routinely do rebuilds of arch:any packages. (Cc:-ing Lucas, for his great work on QA rebuilds.) In my archive rebuilds, I rebuild arch:all packages and file bugs when they fail to build (see the large amount of bugs filed against perl modules due to test suites failures). Those rebuilds happen once or twice a month on average. Lucas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120622111957.ga32...@xanadu.blop.info
Re: build-time testing of pure arch:all packages
On 22/06/12 at 10:56 +0200, Goswin von Brederlow wrote: Are archive wide rebuilds done on anythiong but i386/amd64? A long time ago, I played with archive rebuilds inside qemu for mips or mipsel. It is probably doable, but needs someone to do the work. Lucas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120622112121.gb32...@xanadu.blop.info
Re: build-time testing of pure arch:all packages
On Fri, 22 Jun 2012, Goswin von Brederlow wrote: I was thinking about a bit more automated way... ideally (in the long run) even that FTBFS (e.g. due to failed tests or some other arch specific quirks) would forbid automatic migration to wheezy etc -- kinda full blown benefits from the Debian infrastructure without much of work from my side ;) You can do this in a not so nice way like this: 1) if you have no Architecture: any package then add a package-test-logs package. 2) build-arch depends on build-indep (i.e. always compile the arch:all stuff) 3) in build-arch run the tests and include the log in the Architecture:any package This adds a stupid dummy package to the archive that nobody but you need or fattens up an existing one with a logfile nobody but you need. That is the not so nice part. But it would do what you want now. Thank you Goswin for detailing my evil non-archive-friendly option ;-) Note: Consider carefully if it wouldn't make more sense to help build a real automatic testing infrastructure (which could reuse idle buildd). How urgently do you need those tests to be run? Thanks for asking -- there is no urgency per se. One of the issues in nibabel was triggered by nipy build (failure is still there [1]) so we exercised it and upstream has it already fixed (release/upload will follow shortly). But my email came out of wondering how many of other arch:all packages we have in the archive without such a build-time QA, thus possibly broken on exotic ports. [1] https://buildd.debian.org/status/package.php?p=nipy archive-wide rebuilds of arch:all packages as we routinely do rebuilds of arch:any packages. (Cc:-ing Lucas, for his great work on QA rebuilds.) Are archive wide rebuilds done on anythiong but i386/amd64? IMHO if archive-wide rebuild could be carried on at least one representative big-endian architecture (e.g. sparc) -- it might already be quite useful. Do we also have some time share on one of the top3 HPC boxes [2] + Lucas#2 to take care about it? ;-) [2] http://en.wikipedia.org/wiki/TOP500 -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120622151951.gf5...@onerussian.com
Re: build-time testing of pure arch:all packages
Yaroslav Halchenko deb...@onerussian.com writes: On Fri, 22 Jun 2012, Goswin von Brederlow wrote: archive-wide rebuilds of arch:all packages as we routinely do rebuilds of arch:any packages. (Cc:-ing Lucas, for his great work on QA rebuilds.) Are archive wide rebuilds done on anythiong but i386/amd64? IMHO if archive-wide rebuild could be carried on at least one representative big-endian architecture (e.g. sparc) -- it might already be quite useful. Do we also have some time share on one of the top3 HPC boxes [2] + Lucas#2 to take care about it? ;-) [2] http://en.wikipedia.org/wiki/TOP500 My boss recently came back from a conference and he brought back a flyer about 2U arm server with up to 192 cores and 192 GB ram (on 12 planes a 4 quad core cpus and 4x 4GB ram each) with 10 Gbit networking. I wonder: How long would an archive wide build take with 192 armel buildds? MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/877guzurwh.fsf@frosties.localnet
Re: build-time testing of pure arch:all packages
On 22/06/12 19:23, Goswin von Brederlow wrote: Yaroslav Halchenko deb...@onerussian.com writes: On Fri, 22 Jun 2012, Goswin von Brederlow wrote: archive-wide rebuilds of arch:all packages as we routinely do rebuilds of arch:any packages. (Cc:-ing Lucas, for his great work on QA rebuilds.) Are archive wide rebuilds done on anythiong but i386/amd64? IMHO if archive-wide rebuild could be carried on at least one representative big-endian architecture (e.g. sparc) -- it might already be quite useful. Do we also have some time share on one of the top3 HPC boxes [2] + Lucas#2 to take care about it? ;-) [2] http://en.wikipedia.org/wiki/TOP500 My boss recently came back from a conference and he brought back a flyer about 2U arm server with up to 192 cores and 192 GB ram (on 12 planes a 4 quad core cpus and 4x 4GB ram each) with 10 Gbit networking. I wonder: How long would an archive wide build take with 192 armel buildds? 1 day, 5 hours, 48 minutes, 54.3 seconds libreoffice build in ubuntu amrhf https://launchpad.net/ubuntu/+source/libreoffice/1:3.5.3-0ubuntu1/+build/3458035 Regards, Dmitrijs. -- Regards, Dmitrijs. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fe4bbcf.2030...@debian.org
build-time testing of pure arch:all packages
I wonder if we have a way to achieve that. I package a few Python modules and enable build-time testing in them for at least some QA. Some of the packages, although being pure Python (thus architecture all), deal with data I/O thus prone to bugs related to alignment/endianness etc. Unfortunately I cannot spot those until some dependent on them package hits them (e.g. happened quite a few times with nibabel - nipy bundle). Ideally I wish I could somehow instruct build servers to build those source packages with only arch:all binary packages, discarding any result and only keeping the logs. I wonder if that is somehow possible to achieve cleanly ? weak alternative could be to enable testing of such core package within another dependent arch:any package debian/rules (e.g. test nibabel within nipy's debian/rules) but that is ugly for various reasons. So I thought to ask ;-) Cheers, -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120621172100.gd5...@onerussian.com
Re: build-time testing of pure arch:all packages
On Thu, Jun 21, 2012 at 01:21:00PM -0400, Yaroslav Halchenko wrote: I wonder if we have a way to achieve that. I think this is best handled with autopkgtest. See recent efforts to set up Debian infrastructure to use it. -- I wrote a book: http://gtdfh.branchable.com/ signature.asc Description: Digital signature
Re: build-time testing of pure arch:all packages
On Fri, Jun 22, 2012 at 1:21 AM, Yaroslav Halchenko wrote: I package a few Python modules and enable build-time testing in them for at least some QA. Some of the packages, although being pure Python (thus architecture all), deal with data I/O thus prone to bugs related to alignment/endianness etc. Unfortunately I cannot spot those until some dependent on them package hits them (e.g. happened quite a few times with nibabel - nipy bundle). Ideally I wish I could somehow instruct build servers to build those source packages with only arch:all binary packages, discarding any result and only keeping the logs. I wonder if that is somehow possible to achieve cleanly ? You could run the build (with `./setup.py check` enabled as per usual) on all the porter boxes, we have a number of those for different architectures, at least until the autopkgtest infrastructure is created. -- bye, pabs http://wiki.debian.org/PaulWise -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAKTje6G0mN2DZcLfeet=4mltkwomzpwonneng3aer8cmgg1...@mail.gmail.com
Re: build-time testing of pure arch:all packages
On Fri, 22 Jun 2012, Paul Wise wrote: I package a few Python modules and enable build-time testing in them for at least some QA. Some of the packages, although being pure Python (thus architecture all), deal with data I/O thus prone to bugs related to alignment/endianness etc. Unfortunately I cannot spot those until some dependent on them package hits them (e.g. happened quite a few times with nibabel - nipy bundle). Ideally I wish I could somehow instruct build servers to build those source packages with only arch:all binary packages, discarding any result and only keeping the logs. I wonder if that is somehow possible to achieve cleanly ? You could run the build (with `./setup.py check` enabled as per usual) on all the porter boxes, we have a number of those for different architectures, at least until the autopkgtest infrastructure is created. Good lord that I do not have that many of such packages in mind -- but even with a single one it is quite a few boxes to login/run at (thanks Debian!). I was thinking about a bit more automated way... ideally (in the long run) even that FTBFS (e.g. due to failed tests or some other arch specific quirks) would forbid automatic migration to wheezy etc -- kinda full blown benefits from the Debian infrastructure without much of work from my side ;) But I guess the answer is that we do not have such facilities yet. And only solution for a lazy me would be to modularize the package to carry e.g. python-MODULE-test arch:any package to cause the desired benefits... but that would be evil and hopefully rejected by almighty ftpmasters ;-) -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120622042948.ge5...@onerussian.com
