Re: changes to T::H to enable continuous testing
On Fri, 6 Feb 2004 13:34:01 -0800, Michael G Schwern wrote: > > Test::Harness parses 'ok' and 'not ok' and 'Bail out'... Test::* > modules produce the output Test::Harness parses. So your extra logic > to parse "depends on" would go into your Test::Harness extension, but > the depends_on() function to produce it would go into a seperate > Test::* module. I agree, but I still believe it would be good if Test::Harness laid out syntax rules for extensions. Defining a standard for how test <-> harness communication is done will avoid accidental parsing of test output. Let me summarize what I've done as an example. On the test side of things each of my test files now have one additional line. For example: use Test::More tests => N; use Test::Depends qw(:auto); That :auto import ensures that %INC dependencies are inserted into the test output stream automatically when the test exits. Explicit calls to depends_on(...) are allowed too, but not required. On the harness end of things I used Test::Harness::Straps to create Test::Continuous::reruntests(). The callback function knows what the Test::Depends code will insert into the test output stream and parses it appropriately. The hazard I see is the messages generated by T::Depends and parsed by T::Continuous are completely up to me. A test writer might inadvertently generate their own diagnostic messages that T::Continuous mistakenly parses. With a nod towards SMTP header extensions (e.g. "X-Mail-Software: ...", "X-URL: ...", or "X-Server: ...") I'll suggest "# Test-X-FOO: ..." as a base syntax. The T::H documentation sections "The test script output" or "Anything else" could state that comments starting with "Test-X-\w+:" are used for test extensions. Then my T::Depends extension would use the pattern /^# Test-X-Depends: (.*)/ safely. Comments? Scott
Re: changes to T::H to enable continuous testing
On Fri, Feb 06, 2004 at 11:03:42AM -0600, Scott Bolte wrote: > I'd like to propose an addition to the Test::Harness parsing > rules to support dependency analysis. That, in turn, allows > monitoring for file changes and selective, immediate > re-execution of test files. Is this the right forum for > that discussion? > > Building on the mini_harness.plx example from > Test::Harness::Straps, I added checks for declarations like > the following: > > DEPENDS_ON "file" # implicit test file dependency > "test_file" DEPENDS_ON "module_file" > "test_file" DEPENDS_ON "data_file" In my development copy of Devel::Cover I have code which records which tests trigger the various coverage criteria. Such information should allow these dependencies to be generated automatically, at least as far as code dependencies are concerned, and would also allow more detailed dependencies to be generated if required. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
Re: changes to T::H to enable continuous testing
On Sun, Feb 08, 2004 at 08:41:59AM -0600, Scott Bolte ([EMAIL PROTECTED]) wrote: > I agree, but I still believe it would be good if Test::Harness > laid out syntax rules for extensions. There are no extensions. They're up to whoever wants to. I'm certainly not going to define arbitrary rules based on a sample size of one. I DO want to document the format, however, so that other languages can write to the T::H format reliably, and let T::H become the uber-tester. xoa -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:petdance
Re: changes to T::H to enable continuous testing
On Sun, Feb 08, 2004 at 05:27:02PM -0600, Andy Lester wrote: > On Sun, Feb 08, 2004 at 08:41:59AM -0600, Scott Bolte ([EMAIL PROTECTED]) wrote: > > I agree, but I still believe it would be good if Test::Harness > > laid out syntax rules for extensions. > > There are no extensions. They're up to whoever wants to. I'm certainly > not going to define arbitrary rules based on a sample size of one. Yep, your syntax "rules" are what the existing T::H rules are. Use them as inspiration. > I DO want to document the format, however, so that other languages can > write to the T::H format reliably, and let T::H become the uber-tester. Some sort of BNF, if the syntax can be wedged into one, would be nice. What does need to be done is for me to finish abstracting out the formatter to allow custom harness more easily. -- Michael G Schwern[EMAIL PROTECTED] http://www.pobox.com/~schwern/ If it's stupid, but it works, it isn't stupid.
Re: changes to T::H to enable continuous testing
On Sun, 8 Feb 2004 17:27:02 -0600, Andy Lester wrote: > On Sun, Feb 08, 2004 at 08:41:59AM -0600, Scott Bolte ([EMAIL PROTECTED] > ) wrote: > > I agree, but I still believe it would be good if Test::Harness > > laid out syntax rules for extensions. > > There are no extensions. They're up to whoever wants to. I'm certainly > not going to define arbitrary rules based on a sample size of one. > > I DO want to document the format, however, so that other languages can > write to the T::H format reliably, and let T::H become the uber-tester. Please let me know when you do document the format and make sure to allow for extensions. I urge you do to so before the sample size, and divergent extensions, gets too large. Scott
Re: changes to T::H to enable continuous testing
> Please let me know when you do document the format and make > sure to allow for extensions. I urge you do to so before > the sample size, and divergent extensions, gets too large. Patches are always welcome. Also, I don't see any sample size greater than one on this, unless I'm missing something. -- Andy Lester => [EMAIL PROTECTED] => www.petdance.com => AIM:petdance
