-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
D. Woodill wrote:
> This thread and Dominique?s original messages beg the question, Are
> we again shooting Silver Bullets over The Tower of Babel?
Yours is a witty conversation and makes for an enjoyable reading,
thank you! :-)
> To me, users who write test scripts are techies who enjoy their
> meaningful lives several elevator buttons above non-techies.
That's a bit of elitism I'm afraid (and if there ever was a reason for
towers of Babel to fall down in the realm of software, that would be
it :-)) We do happen to have fairly smart consultants (and users) over
here. But like I said, they cannot be bothered to learn either HTML or
Perl. And developpers cannot be bothered to actually test their web
pages, which is the other facet of my problem :-), let alone place id
attributes in HTML forms.
> Retarded people don?t make you type and spell precisely.
But bureaucracies do. I submit that the world has been a-changing on
this front during the last decades.
> In principal, technically skilled users could (and perhaps, should)
> develop their own test scripts. (Wouldn?t it be fun if users
> could check what you accomplished or didn?t accomplish each
> morning?) On the downside, I?d expect, that as soon as users are
> given a good test tool, they will be faced with difficult questions
> on what to test and how.
In my situation (is this unusual? Definitely not part of XP anyway),
there is a dead-tree embodiment of the acceptance test suite which has
contractual meaning (it's called a "cahier de recette" in french, I
don't know the proper term in english). That's why I don't expect the
users to slap byzantine tests over the head of the dev team: these
things will undergo negotiation. But I definitely require the test
suite to be understandable (and modifiable on a drafting basis) by the
user or someone masquerading as her stakeholder (i.e. one of our
in-house consultants), at the limit *becoming* the contractual
instrumentum by itself. Unambiguous contracts sound like a techie
dreaming out loud, I know, but we can try.
> Will their test click a link or open a page via the href attribute?
>
Click a link, definitely. The only time a URL would need to be input
manually would be during the very first acceptance test.
> If a button has an over event and a mouse out event and it causes a
> form to send request variables to a target with a validate script
> in-between, do you want a non-techie building the tests?
No - But OTOH I don't want to *code* that kind of app in the first
place :-)
> What are valid tests results in varying situations?
I tend to side with the use-case methodology: for each test there
should be a minimal requirement that is fulfilled every time, and a
success requirement that must be attainable. In my very short and
humble experience the minimal requirements tend to be easily
factorable among tests (basically it's something along the lines of
"don't puke the stack trace at the user, display some sweet nonsense
instead" - singing crude lyrics into virgin's ears, you bet!)
> In spite of decades of technical experience, I?m grappling with
> these questions myself. Looks like many tests require judgement
> calls. Acceptance Testing could be straightforward, but, as soon as
> page objects controls multi-page flow, the path mapping complicates
> the results.
Would you please care to elaborate with an example of some functional
requirement that you can successfully transcribe into human text, but
would not trust the user to encode in Selenese?
> "Listen you user, you wrote the test, I passed the test, now ..."
>
This is a matter of politics - in my case, of climate during the
contractual negotiation. Rubbing an acceptance test chart in the face
of a discontent customer is not our long-term best interest (however
much I would like to be able to do that at times). Using it as a
bargaining device ("see? We did what was planned!") for our most, um,
"nonlinear" clients definitely is, though.
The correct mode of operation is of course that customer and dev will
agree naturally on what should go in the acceptance test suite, and
will both be aware that automated testing is a tool not an end. This
boils down to: I shan't worship another god than customer
satisfaction, and Selenium shall be the altar not the idol. (But
thanks for raising that point anyway!)
> Have people written about what to test and how to test earlier on
> the Selenium lists or elsewhere on the web? Links would be
> helpful.
there are loads of CS wisdom at http://www.c2.com/ . I'm pretty sure
they have entire categories about acceptance testing, say starting at
http://xp.c2.com/AcceptanceTest.html
> Dominique, before you develop you idea, you might manually test
> your premise on a few pages that are in development and changing.
> You could have users tell you what they want.
This is of course the voice of wisdom. I'll do that.
> Some of the quicksand around this Tower of Babel is the mess of
> HTML.
I'm well aware of that, hence the heuristics bit in my proposal.
> If you pick only the <BUTTON> tag as standard and let test in your
> new commands fail for <INPUT> button tags, you?ll catch
> non-compliance but you?ll give your developers nervous breakdowns.
I boldly intend to capture all possible HTML form markups - actually
the mess is not that deep (I've been futzing around with X509
"standards" for 4 years now, and I can tell you :-)). There are things
that can be filled out with text, and things that can be clicked.
There are things that are easy to match by their text label (buttons,
<select> lists and <a> links) and things that are not (the rest - but
I have my heuristics ready for them). The rest is just coding.
Regards,
- --
Dominique QUATRAVAUX Ingénieur senior
01 44 42 00 08 IDEALX
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org
iD8DBQFDVnvXMJAKAU3mjcsRArIDAJ4mrtZKjdb8gs16LM+aQIrn3Aq5LwCgrOP1
AUGBmKy7NbtZemRoKHvrmWI=
=+WV8
-----END PGP SIGNATURE-----
_______________________________________________
Selenium-devel mailing list
Selenium-devel@lists.public.thoughtworks.org
http://lists.public.thoughtworks.org/mailman/listinfo/selenium-devel
