Request for Comments: Package testing results analysis, result codes

2006-02-19 Thread Adam Kennedy
I'm starting to get a bit closer (waiting on a test images and some last testing to be done) to finishing the initial PITA test cycle (and thus be able to do an initial release) and so I'm starting to do some prep work now for the next stage, which is to start to assemble some infrastructure

Re: Request for Comments: Package testing results analysis, result codes

2006-02-19 Thread Andreas J. Koenig
On Sun, 19 Feb 2006 22:22:20 +1100, Adam Kennedy [EMAIL PROTECTED] said: 1. Broken or corrupt packaging. A bad tarball, MANIFEST files missing. Make sure you verify that all files in the distro are readable. Reject if the permissions are bogus. Recently we had an increasig number of

Re: Request for Comments: Package testing results analysis, result codes

2006-02-19 Thread Tyler MacDonald
Adam, I have one more edgey case I'd like to see on this list: 13. Tests exist, but fail to be executed. There is tests, but the tests themselves aren't failing. It's the build-process that is failing. 14. Tests run, and some/all tests fail. The normal FAIL case due

Re: Request for Comments: Package testing results analysis, result codes

2006-02-19 Thread Michael Graham
Tyler MacDonald [EMAIL PROTECTED] wrote: Adam, I have one more edgey case I'd like to see on this list: Tests run, but 50% (or maybe 80%?) are skipped. From what I've seen, the most common cause of this is that the package is untestable with the current build

Re: Request for Comments: Package testing results analysis, result codes

2006-02-19 Thread Adam Kennedy
(Andreas J. Koenig) wrote: On Sun, 19 Feb 2006 22:22:20 +1100, Adam Kennedy [EMAIL PROTECTED] said: 1. Broken or corrupt packaging. A bad tarball, MANIFEST files missing. Make sure you verify that all files in the distro are readable. Reject if the permissions are bogus. Recently