Barbie wrote:
Hi Dave,
Has anyone done any perl fuzz-testing - that is, written something that
generates valid but silly code in an attempt to find errors in the
interpreter or in a module?
Is Acme::BadExample what you're looking for? The module that spawned a
brilliant and enlightening talk by Jos at both YAPC::Europe and LPW last
year.
[1] http://search.cpan.org/~adamk/Acme-BadExample-1.00/
Nope, Acme::BadExample is a demonstration of a number of things we
already know create problems, and handling of those problems should be
correct.
We're looking for NEW problems.
It seems like a worthwhile thing to do, but I'm struggling to imagine
what it would look like, particularly for testing pure-perl modules.
It was used to test PPI.
Thanks Audrey for this. If was very very useful.
You can see the code in the PPI tests, but all I did was generate
20-1000 character randomly string (from the set of legal perl
characters) and see if the PPI tokenizer crashed.
We never actually got a chance to fuzz-test the lexer properly, what
tended to happen especially for the 1000-char string was it would hit a
single quote or something at about character 20 and then most of the
fuzz would end up in strings.
Audrey suggested (but I never got a chance to implement) a second layer
that would assemble a big collection of legal tokens (from, say, CPAN)
and the regularly with which they occur, and then assemble them in
random order.
Some variations would use more or less whitespace, and some would use
similar density of characters to normal Perl code, and some would not.
And that would hopefully fuzz-test the PPI lexer properly.
I imagine something similar (in terms of two layers) wouldn't be that
hard for Perl, as long as you can properly detect the scenarios you are
looking for.
Adam K
