On Wednesday 19 December 2007 00:24:32 Paul Cochrane wrote: > > [t] Nuked t/src/vtables.t, which pokes into the guts of libparrot using > > functions not documented nor promised in the extension API. Making this > > test work actually correctly in a sane and safe way would require either: > > Between 1/2 and 2/3 of the functions in src/vtables.c is actually > being tested by the test suite (without the vtables.t test); how can > we then exercise the remaining code to ensure that it (a) works or (b) > is even necessary (IIRC Parrot_new_vtable() is never called)?
Coverity can detect never-called functions, I believe, and I think splint can to. I'm not positive about the latter. > I would like to (at some stage) increase the test coverage level of > the C-language parts of Parrot. How is it best to do this? By > testing the C routines directly (my first guess)? Sadly, that won't work reliably for two reasons. First, we don't export all of the necessary symbols from the shared library. We *could*, but then people would use them (even if we didn't document them) and we'd have to support them. This happened with Perl 5 and it's part of the reason that XS is such a mess compared with just about every other dynamic language's extension mechanism. Second (but related), I believe that we'll end up duplicating a lot of Parrot code that sets up the environment just to be able to test this code reliably. Duplication is just about the worst enemy to maintainability; all of a sudden we have two places to update as we refactor and modify the internals of Parrot, and that doesn't end up very well either. I removed this test because it was failing because Jonathan unTODOed it, and after looking at it, it wasn't worth my time to debug why it was failing and then keep it running on the platforms we care about. If vtables were failing such that these tests were failing for reasons unrelated to the previous two reasons, we'd have plenty of other tests failing and we'd be able to debug them appropriately. > Or indirectly via pir or something? That's the best approach. If we absolutely *must* write tests in t/src/ to test something in Parrot that we can't reliably test elsewise, we need to figure out a way to avoid both problems, perhaps by linking against the intermediate .o files and not libparrot, or by pulling in just enough code that we have good entry and exit points without duplicating code across files. My very strong preference is to test the internals of Parrot via PIR as much as possible, and to reserve t/src/ for tests against the documented and supported external interface. -- c
