On 29 Jul 2001, Mama Cass Elliot wrote:
>>
>> Instead, the program behaves differently depending on data and user
>> event input. There's no way to test with every possible input
>> combination.
>
> Nonsense!
>
> It's perfectly possible to design and implement such a test.
>
> You know of all the possible inputs. You know of all the events that a user
> can trigger, you devise a list that has every possible combination, and
> then you go through that list. Yes - it may take more than 5 minutes, but
> it IS *possible*.
Ok, let's see if this is true.
Let's take a very simple and VERY well defined part of the Mozilla
product: The XML Parser.
At the simplest level, the XML Parser takes as input a sequence of UCS2
(two byte) characters, and outputs a data structure. Let us assume that we
will only consider files of exactly 32 bytes (16 characters) in size, for
simplicity (in reality, we would have to consider files of all sizes from
0 bytes up to more than 2 gigabytes, of course).
We will create every possible input combination for the XML Parser, and
then check to see if the output is correct. Now, to create the files is
quite a simple program, so we will do that in an automated fashion.
However, to test the output, we have to examine each one by hand, since
any automation used to test this would be an XML Parser itself, and may
have bugs of its own. Since we have to do this manually, it will take a
little time to check each one. From personal experience (I do this as part
of my job) I know that it takes between 2 and 10 minutes to check a small
DOM for accuracy. So let's say 5 minutes on average (although in practice
a one megabyte DOM would probably take several hours of painstaking care
to check. I'm being idealistic here.)
So how long would it take to do this test?
(65535 ^ 16) * 5 minutes = 5.79e77 minutes = over a billion billion
billion billion billion billion billion billion years.
Now given that the universe has only existed for an estimated 15 billion
years so far, I see a problem.
When do we want to ship Mozilla 1.0, again?
--
Ian Hickson )\ _. - ._.) fL
Netscape, Standards Compliance QA /. `- ' ( `--'
+1 650 937 6593 `- , ) - > ) \
irc.mozilla.org:Hixie _________________________ (.' \) (.' -' __________
