#357: Enable meaningful testing of t/native_pbc/*.t
----------------------+-----------------------------------------------------
Reporter: doughera | Owner:
Type: todo | Status: new
Priority: normal | Milestone: 1.0
Component: none | Version:
Severity: medium | Keywords:
Lang: | Patch:
Platform: |
----------------------+-----------------------------------------------------
Version 0.9.1 was released with failing t/native_pbc/*.t tests. If the
tests are to be included in 1.0, it would be nice to have confidence that
they will pass in the released version.
Offhand, I can think of three big problems that contributed to the
failure. I hasten to add that *all* of these were out of the control of
the release manager -- they are inherent in the design of the tests.
1. The tests act differently in "DEVELOPING" versus released directories,
meaning all the tests done in an svn-checkout -- even those done just
prior to release -- weren't necessarily relevant.
2. The release procedure is difficult in practice. It is not easy, on
short notice, to get developers with all the requisite different platforms
to run the appropriate scripts, commit the appropriate files, and re-run
all the appropriate tests. Also, because of item 1, even running all the
appropriate tests does not ensure that the tests will actually pass in the
released version.
3. There was no meaningful code freeze prior to the release, making it
essentially impossible to do the coordination necessary for item 2.
What to do?
First, if the tests are to be kept, they must be designed so that they can
be meaningfully run well in advance of the release. That way, new raw pbc
files can be generated ahead of time, committed, and tested, and failures
can be addressed. If they cannot be redesigned this way, then I think
they are too fragile to waste time on, and they should simply be deleted.
Second, I think the ideas of having a code freeze and issuing release
candidates merit serious consideration prior to 1.0.
--
Ticket URL: <https://trac.parrot.org/parrot/ticket/357>
Parrot <https://trac.parrot.org/parrot/>
Parrot Development
_______________________________________________
parrot-tickets mailing list
[email protected]
http://lists.parrot.org/mailman/listinfo/parrot-tickets
