Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Barrie Slaymaker
"Bryan C. Warnock" wrote: > > Is each example to be standalone? ie, I'm thinking of this more as a testing tagset now: =begin testing [] =end testing =for testing [] where the optional indicates the file to append the indicated paragraphs to. This plays well with using =also (for|begin|en

Re: =also for example [was - Re: RFC: Automated Testing Of Code Examples ...]

2000-07-28 Thread Barrie Slaymaker
Michael G Schwern wrote: > > Barrie, you suggested it, so its your babe now! You have the option > to sketch out a new RFC and start working on this or punt the > responsibility to somebody else. I'll take "sketch out a new RFC" for $200, Michael. I'll also tie it in with the =for test stuff.

Re: RFC: Embedded Tests and =test

2000-07-28 Thread Barrie Slaymaker
Michael G Schwern wrote: > > Perl will have to compile all the test routines all the time. This > both slows startup and eats memory. This pretty much kills the > proposal. :( Yup. Brain fart withdrawn. > I don't see how the "sub TEST" thing improves except to keep POD pure > documentation.

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Barrie Slaymaker
Jonathan Scott Duff wrote: > > Just a few changes: > > . Don't rely on both a preamble and a postamble (postambles are > optional) > . all consecutive verbatim paragraphs after a "=end example_preamble" > would be a single set of code to test. > . Better names than "example_preamble" and "ex

Re: RFC: JART - Just Another Regression Test

2000-07-28 Thread Barrie Slaymaker
Michael G Schwern wrote: > > GREAT IDEA! Its yours! Organize a perl-qa FAQ or get someone to do > it for you. Would a perlqa.pod man page be a more effective tool? - Barrie

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Jonathan Scott Duff
On Fri, Jul 28, 2000 at 06:27:55PM -0400, Bryan C.Warnock wrote: > Is each example to be standalone? ie, Yes. Unless/Until we come up with a method of naming test examples such that successive tests with the same name are chained. Just MHO, -Scott -- Jonathan Scott Duff [EMAIL PROTECTED]

Re: =also for example [was - Re: RFC: Automated Testing Of Code Examples ...]

2000-07-28 Thread Jonathan Scott Duff
On Fri, Jul 28, 2000 at 06:25:43PM -0400, Michael G Schwern wrote: > 3) =begin preamble/=end preamble/paragraph to test is compatible, but its > kind of annoying to write (or even explain) and will run into problems > with multi-paragraph examples. However, it is compatible and should >

RE: Coverage analysis

2000-07-28 Thread Tony Payne
> I have no idea what coverage tools are available for C. Someone want > to compile a list? I have also never really done much coverage > analysis. Would someone with time and experience step forward and > take responsiblity for it? There don't seem to be many free/open tools. I was able to

Re: Coverage analysis

2000-07-28 Thread Michael G Schwern
On Fri, Jul 28, 2000 at 03:07:09PM -0700, Tony Payne wrote: > I think Schwern's proposal of a patch for every test, applied appropriately > will do a good job of making sure the Perl test suite has close to 100% > functional coverage of the core. However, we should also consider doing > code cove

Re: On QA/QC [Was: RFC: JART - Just Another Regression Test]

2000-07-28 Thread Michael G Schwern
On Fri, Jul 28, 2000 at 11:12:36AM -0400, Kurt D. Starsinic wrote: > I'm glad you're looking out for Kwalitee, and I have observation about > the RFC's you've offered. None of them have milestones or completion > states. You're right, I've totally forgotten about time! Perhaps its because I

Re: RFC: JART - Just Another Regression Test

2000-07-28 Thread Michael G Schwern
On Thu, Jul 27, 2000 at 09:46:05PM -0400, Barrie Slaymaker wrote: > > > P.S. Why is this perl-qa? Shouldn't it be perl6-qa? > > > > I believe Schwern has already answered that ... > > I think we have here our first FAQ :-). GREAT IDEA! Its yours! Organize a perl-qa FAQ or get someone to do it

Re: RFC: Embedded Tests and =test

2000-07-28 Thread Michael G Schwern
On Thu, Jul 27, 2000 at 03:08:43PM -0400, Barrie Slaymaker wrote: > Michael G Schwern wrote: > > Regression tests should be embedded in the code and documentation near what > > it is they're testing. > > s/embedded/embeddable/ and I'm there. I don't think you can necessarily > embed tests right

Re: RFC: A Test For Every Patch

2000-07-28 Thread Michael G Schwern
On Thu, Jul 27, 2000 at 06:51:46PM -0400, Chaim Frenkel wrote: > I do not believe that it would be a good idea to mix perl5 and perl6 on > any of the lists. If something is useful to perl5 send it on to p5p. It would be artificial to split the two. Most of QA will be applicable to all of Perl.

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Bryan C . Warnock
On Fri, 28 Jul 2000, Barrie Slaymaker wrote: > I think the =also tag approach does that pretty well, but I'm not > hearing anyone else say yay/nay on it. > > Whether the examples get converted to tests could be flagged in > POD at the top of the file, or by using slightly different syntaxs: > >

Re: About the following RFCs

2000-07-28 Thread Michael G Schwern
On Thu, Jul 27, 2000 at 06:23:16PM +0300, Jarkko Hietaniemi wrote: > You may want to think about psychology and wording, though. If > someone writes a code patch but does not supply tests/docs, and he or > she is told that the patch is rejected because of that deficiency, he > or she might not su

Re: About the following RFCs

2000-07-28 Thread Jarkko Hietaniemi
On Wed, Jul 26, 2000 at 04:50:34PM -0400, Michael G Schwern wrote: > While perl6 has not yet defined what an "RFC" really is, what it > should say or what format it should be in, we can't wait. So I'm > going forward with a fistful of things I'm calling RFCs. They are Just for the record, I lov

Re: =also for example [was - Re: RFC: Automated Testing Of Code Examples ...]

2000-07-28 Thread Michael G Schwern
We have an interesting knife edge to walk here. We'd like to automagically test example code with a minimum of work on the POD author's part, and yet avoid false negatives. We also have to keep POD simple to write. On top of that, we'd like to keep it backwards compatible. So we've got a few a

Coverage analysis

2000-07-28 Thread Tony Payne
I think Schwern's proposal of a patch for every test, applied appropriately will do a good job of making sure the Perl test suite has close to 100% functional coverage of the core. However, we should also consider doing code coverage analysis. This can help us to create test cases that accurate

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Jonathan Scott Duff
On Thu, Jul 27, 2000 at 03:21:05PM -0400, Barrie Slaymaker wrote: > One way I can see to work around your dilema would be kludgy: > > =begin example_preamble > >#!perl -w >use LWP::Simple >use Test ; > > =end example_preamble > >$page = get("http://www.perl.org"); > > =begin e

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Jonathan Scott Duff
On Fri, Jul 28, 2000 at 05:07:24PM -0400, Barrie Slaymaker wrote: > > I think the =also tag approach does that pretty well, but I'm not > hearing anyone else say yay/nay on it. > > Whether the examples get converted to tests could be flagged in > POD at the top of the file, or by using slightly

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Barrie Slaymaker
Michael G Schwern wrote: > > tchrist makes a > good point, we can't make it anyting but easy to write POD. I think the =also tag approach does that pretty well, but I'm not hearing anyone else say yay/nay on it. Whether the examples get converted to tests could be flagged in POD at the top of t

Re: RFC: Automated Testing Of Code Examples In Documentation and "=for example"

2000-07-28 Thread Michael G Schwern
On Thu, Jul 27, 2000 at 06:53:23PM -0400, Chaim Frenkel wrote: > How do you know that sample code in the documentation is correct? > That it compiled? Or that it generated the correct result? > > The documentor will need to state the expected result, so that > both the reader and testing scripts

On QA/QC [Was: RFC: JART - Just Another Regression Test]

2000-07-28 Thread Kurt D. Starsinic
Mike, I'm glad you're looking out for Kwalitee, and I have observation about the RFC's you've offered. None of them have milestones or completion states. I think that QC in Perl is very important, and has historically lacked the commitment that you're bringing to it, but I'd like to sug

Example of =also && =for example [Was: =also?]

2000-07-28 Thread Barrie Slaymaker
[[CCed back to perl-qa]] Marek Rouchal DAT CAD HW Tel 25849 wrote: > > I don't understand the benefit of =also in this context. Could you please > elaborate some more on this? The goal is to be able to extract snippets of example code from the POD and test them. The =also solution is kind of a