Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
On Mittwoch, 22. Mai 2013, Jakub Wilk wrote: > FWIW, most of the packages don't need anything more than a chroot. Interesting, thanks. signature.asc Description: This is a digitally signed message part.
Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
* Holger Levsen , 2013-05-22, 13:26: it is not fully related to your original question, but do you think that piuparts could support running Autopkgtests as well ? I think we need another setup for this. Autopkgtests may destroy their environment (and might need more than a chroot) so they cannot be fully tested in our current piuparts.d.o setup. FWIW, most of the packages don't need anything more than a chroot. Out of 116 packages that have DEP-8 tests, only 2 declare the breaks-testbed restrictions. (And, AFAICS, even the two with breaks-testbed don't currently need stronger isolation.) -- Jakub Wilk -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130522121418.ga3...@jwilk.net
Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
Hi, On Mittwoch, 22. Mai 2013, Charles Plessy wrote: > it is not fully related to your original question, but do you think that > piuparts could support running Autopkgtests as well ? I think we need another setup for this. Autopkgtests may destroy their environment (and might need more than a chroot) so they cannot be fully tested in our current piuparts.d.o setup. I'd rather do autopkgtests with jenkins.d.n (and be very happy to help anyone working on them). cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: simplifying running piuparts
Hi Charles, On 2013-05-22 06:05, Charles Plessy wrote: > it is not fully related to your original question, but do you think that > piuparts > could support running Autopkgtests as well ? Theoretically yes, but I haven't looked into DEP8 so far ... reading ... Quoting from the autopkgtest specification: > The cwd of each test is guaranteed to be the root of the source > package, which will have been unpacked but not built. HOWEVER note Since piuparts does not know about "source packages", this may be a bit more difficult. But it looks like it could be rather easy to have an --autopkgtest option for pbuilder/sbuild as they already have the matching sources at hand. And it could be done together with the archive wide rebuilds done by Lucas. Andreas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519c7785.80...@debian.org
Re: simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
Le Tue, May 21, 2013 at 11:48:57AM +0200, Andreas Beckmann a écrit : > @all maintainers: How would you like to run piuparts s.t. it easily > integrates into your workflow and allows improving Debian's quality? > This is something we could improve right now. Integrating piuparts into > the ftp-master/buildd side will take a much longer way. Hi Andreas, it is not fully related to your original question, but do you think that piuparts could support running Autopkgtests as well ? Cheers, -- Charles Plessy Debian Med packaging team, http://www.debian.org/devel/debian-med Tsurumi, Kanagawa, Japan -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130522040503.gb21...@falafel.plessy.net
Re: [Piuparts-devel] simplifying running piuparts
On Tue, May 21, 2013 at 05:58:00PM +0200, Wouter Verhelst wrote: > The trouble is that piuparts doesn't manage its own chroot tarball last > I checked; it uses the pbuilder tarball instead. piuparts -s $HOME/piuparts.tar.gz ... # create the tarball piuparts -b $HOME/piuparts.tar.gz ... # use the tarball That should work. -- http://www.cafepress.com/trunktees -- geeky funny T-shirts http://gtdfh.branchable.com/ -- GTD for hackers -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130521162309.ge5...@mavolio.codethink.co.uk
Re: [Piuparts-devel] simplifying running piuparts
On Dienstag, 21. Mai 2013, Wouter Verhelst wrote: > The trouble is... hence my suggestion to have debuild do this optionally. your suggestion of having pdebuild do it by default is a good one though. I guess someone should do something, eg file bugs. :) signature.asc Description: This is a digitally signed message part.
Re: [Piuparts-devel] simplifying running piuparts
On 21-05-13 12:05, Holger Levsen wrote: > Hi, > > On Dienstag, 21. Mai 2013, Andreas Beckmann wrote: >> @all maintainers: How would you like to run piuparts s.t. it easily >> integrates into your workflow and allows improving Debian's quality? > > have an option to run piuparts automatically by debuild, after or before > lintian. The trouble is that piuparts doesn't manage its own chroot tarball last I checked; it uses the pbuilder tarball instead. However, that makes for an easy solution: rather than debuild, have pdebuild do so. Since it already needs to invoke pbuilder for various other things, this should be relatively easy to do. -- This end should point toward the ground if you want to go to space. If it starts pointing toward space you are having a bad problem and you will not go to space today. -- http://xkcd.com/1133/ signature.asc Description: OpenPGP digital signature
Re: [Piuparts-devel] simplifying running piuparts
On 05/21/2013 06:35 PM, Ondřej Surý wrote: > Also integrate it with git-pbuilder/pbuilder/cowbuilder to run > piuparts inside the created clean(ish) chroot, so it's less time > consuming. > > O. This really would be nice, indeed!!! I've been asking for that feature already, and I am happy to see that I'm not the only one who wish it was there. I believe that the piuparts maintainers have that on their todo already (there's a bug open somewhere) and that progress has been made to do it. Insight from the piuparts maintainers would be IMO welcome, so that we know what the status is. Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519b7aa3.6010...@debian.org
Re: [Piuparts-devel] simplifying running piuparts
On Tue, May 21, 2013 at 3:34 PM, Didier 'OdyX' Raboud wrote: >> I consider the package-by-package testing as a much better test than >> installing all the packages at once because it can discover much more >> dependency issues - and it much closer resembles what is being run on >> piuparts.d.o > > Sure. I just tried the --run-piuparts option of sbuild on on of my packages > (lsb) and it failed miserably for that very reason. package-by-package is nice starter, but on the other hand installing all (combinations of) non-conflicting package could discover different type of bugs. O. -- Ondřej Surý -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caljhhg-u8qhzmjs8oger8qnwodzwgeftat6qjq8sknvxl0p...@mail.gmail.com
Re: [Piuparts-devel] simplifying running piuparts
Le mardi, 21 mai 2013 14.58:17, Andreas Beckmann a écrit : > >> On Tue, May 21, 2013 at 12:05 PM, Holger Levsen > > wrote: > >>> have an option to run piuparts automatically by debuild, after or > >>> before lintian. > > that means we need a driver script that accepts a .changes file and > creates a local repository and runs piuparts package by package That sounds like a plan. > >> Also integrate it with git-pbuilder/pbuilder/cowbuilder to run > >> piuparts inside the created clean(ish) chroot, so it's less time > >> consuming. > > does not easily work for upgrade tests from testing or stable Indeed. But testing that the packages install, upgrade, remove and purge correctly on top of the previous versions from unstable is already a good indicator. > > Actually, doing it in sbuild (and hence hypothetically on the buildds) > > would be awesome: > > - deinstall build-depends > > - install previous version (+ dependencies), > > that part might fail ... but it should abort and *not* fail the test (to > fix the broken (uninstallable) package in sid) Indeed, didn't think of that use-case. > > A failure at each of these stages could fail the build. > > > > (The problem becomes tricky with source packages building mutually- > > incompatible binary packages). > > Only for passing the .changes to piuparts. > If we create a driver script that takes a .changes, creates a local > repository and runs piuparts on each built package in turn, this will > work well. (It will require the arch:all packages to be available if > this was a binary-only build) The .changes file doesn't have the arch:all packages, but they might indeed be dependencies of binary packages. I suspect that the fact that buildds have access to incoming.debian.org could facilitate this step though. > I consider the package-by-package testing as a much better test than > installing all the packages at once because it can discover much more > dependency issues - and it much closer resembles what is being run on > piuparts.d.o Sure. I just tried the --run-piuparts option of sbuild on on of my packages (lsb) and it failed miserably for that very reason. OdyX -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201305211534.54361.o...@debian.org
Re: simplifying running piuparts
Hi Andreas, On Dienstag, 21. Mai 2013, Andreas Beckmann wrote: > That allows some basic test (installing all the .debs from the > .changes), but not the per-package testing as its done on piuparts.d.o. > But such fine-granular testing is required to discover some dependency > issues. Sure! But perfect is the enemy of good here, I wanted to show to people how easy it is to run piuparts, which I think I managed. :) And I'm glad you also point out that various more complex setups are doable! ;) "But" I think there is value if maintainers tried^wregularily run at least the simple piuparts runs. cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: [Piuparts-devel] simplifying running piuparts
On 2013-05-21 13:27, Didier 'OdyX' Raboud wrote: > Le mardi, 21 mai 2013 12.35:47, Ondřej Surý a écrit : >> On Tue, May 21, 2013 at 12:05 PM, Holger Levsen > wrote: >>> have an option to run piuparts automatically by debuild, after or before >>> lintian. that means we need a driver script that accepts a .changes file and creates a local repository and runs piuparts package by package >> Also integrate it with git-pbuilder/pbuilder/cowbuilder to run >> piuparts inside the created clean(ish) chroot, so it's less time >> consuming. does not easily work for upgrade tests from testing or stable > Actually, doing it in sbuild (and hence hypothetically on the buildds) would > be awesome: > - deinstall build-depends > - install previous version (+ dependencies), that part might fail ... but it should abort and *not* fail the test (to fix the broken (uninstallable) package in sid) > then upgrade > - remove > - purge > - … As above, the build chroot is not suitable for doing upgrade tests from testing or stable. > A failure at each of these stages could fail the build. > > (The problem becomes tricky with source packages building mutually- > incompatible binary packages). Only for passing the .changes to piuparts. If we create a driver script that takes a .changes, creates a local repository and runs piuparts on each built package in turn, this will work well. (It will require the arch:all packages to be available if this was a binary-only build) I consider the package-by-package testing as a much better test than installing all the packages at once because it can discover much more dependency issues - and it much closer resembles what is being run on piuparts.d.o Andreas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519b6f69.5050...@debian.org
Re: simplifying running piuparts
On 2013-05-21 14:28, Holger Levsen wrote: > On Dienstag, 21. Mai 2013, Simon McVittie wrote: >> Ideally, I would like a guide to setting up piuparts in a simple, >> recommended way, which doesn't assume I already have in-depth knowledge >> of piuparts, and preferably also doesn't assume I already use pbuilder. > > that's really easy, esp. with the last constraint. Just run: > sudo piuparts $changes_file That allows some basic test (installing all the .debs from the .changes), but not the per-package testing as its done on piuparts.d.o. But such fine-granular testing is required to discover some dependency issues. Andreas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519b6b27.9010...@debian.org
Re: simplifying running piuparts
Hi, On Dienstag, 21. Mai 2013, Simon McVittie wrote: > Ideally, I would like a guide to setting up piuparts in a simple, > recommended way, which doesn't assume I already have in-depth knowledge > of piuparts, and preferably also doesn't assume I already use pbuilder. that's really easy, esp. with the last constraint. Just run: sudo piuparts $changes_file Easy, or? And if you use pbuilder, piuparts will just pick up that base.tgz. > Another thing that would be helpful is some sort of idea of what to do > with a failing piuparts log, for instance "which bits are likely to be > significant?", The part at the end where piuparts emits the error. (This is mentioned in the FAQ.) These lines are also marked "E:". > I've seen at least a couple of conversations in piuparts-generated RC Me too. :/ So specific bug reports are definitly welcome, after years of using piuparts it's all clear to me :-) > OK, I made a minimal squeeze chroot, installed package X and > upgraded to wheezy. It worked. Now what? What was piuparts doing > differently? Reply this to the bug, piuparts does not much differently than that actually. cheers, Holger signature.asc Description: This is a digitally signed message part.
Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
Le mardi, 21 mai 2013 12.35:47, Ondřej Surý a écrit : > On Tue, May 21, 2013 at 12:05 PM, Holger Levsen wrote: > > have an option to run piuparts automatically by debuild, after or before > > lintian. > > Also integrate it with git-pbuilder/pbuilder/cowbuilder to run > piuparts inside the created clean(ish) chroot, so it's less time > consuming. Actually, doing it in sbuild (and hence hypothetically on the buildds) would be awesome: - deinstall build-depends - install previous version (+ dependencies), then upgrade - remove - purge - … A failure at each of these stages could fail the build. (The problem becomes tricky with source packages building mutually- incompatible binary packages). Cheers, OdyX P.S. I'd be interested in getting something along those lines working but am not sure to have enough time on my hands, unfortunately. signature.asc Description: This is a digitally signed message part.
Re: simplifying running piuparts
On Tue, May 21, 2013 at 12:17:38PM +0100, Simon McVittie wrote: > On 21/05/13 10:48, Andreas Beckmann wrote: > > @all maintainers: How would you like to run piuparts s.t. it easily > > integrates into your workflow and allows improving Debian's quality? > > Ideally, I would like a guide to setting up piuparts in a simple, > recommended way, which doesn't assume I already have in-depth knowledge > of piuparts, and preferably also doesn't assume I already use pbuilder. > piuparts.debian.org clearly has some sort of setup for "the official" > piuparts test; I would like to be able to reproduce that without needing > to know the fine details of how piuparts works. > > (I generate source packages from (git|svn)-buildpackage or debuild and > feed them to "sbuild -As" running in a disposable schroot snapshot, to > get the same deterministic dependency-resolution as the real buildds - > so I don't mind one of the steps being "run these commands that use > pbuilder to get sid, jessie, wheezy and squeeze base tarballs", but I > don't have such tarballs at the moment, and don't immediately know how > to make them.) sbuild already supports running piuparts at the end of a build, just as it supports running lintian. However, AIUI it does require it to be set up beforehand. It would be really nice if it could re-use the build chroot for its tests, or we could snapshot a second test chroot for it. It would be great if you could just add --run-puiparts and have it just work with no prior setup required. If we do require piuparts to pass in order for packages to be accepted into unstable, and we do have a pre-unstable staging of uploads before sanity-checking them, then we can get one or more of the buildds to run puiparts as a standard part of the build, and we can make it fail the build if puiparts fails. This wouldn't be too hard to implement. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linuxhttp://people.debian.org/~rleigh/ `. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools `-GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130521112919.gi31...@codelibre.net
Re: simplifying running piuparts
On 21/05/13 10:48, Andreas Beckmann wrote: > @all maintainers: How would you like to run piuparts s.t. it easily > integrates into your workflow and allows improving Debian's quality? Ideally, I would like a guide to setting up piuparts in a simple, recommended way, which doesn't assume I already have in-depth knowledge of piuparts, and preferably also doesn't assume I already use pbuilder. piuparts.debian.org clearly has some sort of setup for "the official" piuparts test; I would like to be able to reproduce that without needing to know the fine details of how piuparts works. (I generate source packages from (git|svn)-buildpackage or debuild and feed them to "sbuild -As" running in a disposable schroot snapshot, to get the same deterministic dependency-resolution as the real buildds - so I don't mind one of the steps being "run these commands that use pbuilder to get sid, jessie, wheezy and squeeze base tarballs", but I don't have such tarballs at the moment, and don't immediately know how to make them.) piuparts seems to have some sort of shortcut UI for "use the pbuilder base tarball", but I don't know what is meant to be in that tarball (sid? stable? oldstable?) and I suspect that a "full" piuparts run actually needs more than one anyway. > Since piuparts is very chatty, you probably want > to have a summary and a list of failures at the end ... > The failing logfiles will need to be analyzed manually. Another thing that would be helpful is some sort of idea of what to do with a failing piuparts log, for instance "which bits are likely to be significant?", "how do I tell apt to show me what it was thinking?", and so on. I've seen at least a couple of conversations in piuparts-generated RC bug logs of this form (paraphrasing to be concise): > squeeze->wheezy testing for package X fails in piuparts OK, I made a minimal squeeze chroot, installed package X and upgraded to wheezy. It worked. Now what? What was piuparts doing differently? Regards, S -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519b57d2.2040...@debian.org
Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
On Tue, May 21, 2013 at 12:05 PM, Holger Levsen wrote: > Hi, > > On Dienstag, 21. Mai 2013, Andreas Beckmann wrote: >> @all maintainers: How would you like to run piuparts s.t. it easily >> integrates into your workflow and allows improving Debian's quality? > > have an option to run piuparts automatically by debuild, after or before > lintian. Also integrate it with git-pbuilder/pbuilder/cowbuilder to run piuparts inside the created clean(ish) chroot, so it's less time consuming. O. -- Ondřej Surý -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caljhhg-zkfs5vxnic+e3_bzdskpwqa7bko+sng0m6p3k_mh...@mail.gmail.com
simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
@all maintainers: How would you like to run piuparts s.t. it easily integrates into your workflow and allows improving Debian's quality? This is something we could improve right now. Integrating piuparts into the ftp-master/buildd side will take a much longer way. On 2013-05-19 18:39, Ondřej Surý wrote: [...] > So apart from the more hands on some packages with high priority, it > would really help me to have some automated tests which would be run > before uploading to testing. > > One thing which immediatelly comes to the mind is the install & upgrade test. > > 1. install every built package > 2. try upgrade from stable for every built package > 3. try upgrade from testing for every built package > 4. try upgrade from unstable for every built package Let me see if we can find a way to simplify running piuparts for all these ... Since you are aiming to test transitions, this will need to cover testing multiple source packages at the same time and a lot of dependencies between them. May I assume you have a local repository of these packages available? (You probably already have one to build updated packages for the transition.) A file:// URL with some .debs and a Packages file will be fine. Or would you prefer to feed a bunch of .changes files to some script? Once you run that yet-to-be-written script, you will get a bunch of logfiles (for k .debs and 4 tests that would be 4*k logfiles), some failed, some success. Since piuparts is very chatty, you probably want to have a summary and a list of failures at the end ... The failing logfiles will need to be analyzed manually. Next question: Do you want to do incremental testing? Assume you identified an incorrectly versioned dependency, fixed that, rebuilt the package, updated the repository. Do you want to * retest a specific failure * retest all failures * retest everything (you probably only want to do this at the end as it may be very time consuming). I would probably go for "test everything that does not have a success log", so you could retest arbitrary subsets by just deleting the corresponding (success-)logs. The good thing is: by now piuparts should have all the needed options to do this, it just needs a new driver script to simplify running it on a bunch of local packages. (And running it on each package from a (set of) .changes file(s) individually, not only installing all at the same time) In http://bugs.debian.org/700849 I posted a script skeleton that (requires manual adjustment to configure it and) allows to quickly run one of the tests listed above for a local (or remote) repository. This yet-to-be-written script should not be specific for uploads to sid, but work for pu, opu or backports as well. Experimental may be a bit more difficult as this is "frequently broken" (as in "requiring a very careful selection of the install set mixing sid and experimental") > Optionally: > 5. do some testing with all r-deps (treeish?) That may take some time to run ... I also don't know how to quickly build the rdep tree. But if you already have the local repository and a list of "interesting" packages (not in the local repo), it should be quite easy to check how they behave (install/upgrade wise) in a sid+prospective environment. That's just my interpretation how this could work ... Andreas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/519b4309.3040...@debian.org
Re: [Piuparts-devel] simplifying running piuparts (was: Re: Debian development and release: always releasable (essay))
Hi, On Dienstag, 21. Mai 2013, Andreas Beckmann wrote: > @all maintainers: How would you like to run piuparts s.t. it easily > integrates into your workflow and allows improving Debian's quality? have an option to run piuparts automatically by debuild, after or before lintian. cheers, Holger signature.asc Description: This is a digitally signed message part.
