Hi,
Martin Hollmichel schrieb:
Andre Schnabel wrote:
Hi,
-------- Original-Nachricht --------
Von: Helge Delfs <[email protected]>
Let me state this and give me a try to help understanding what automated
testing represents.
1. Trust your tools (automated tests, testtool, scripts)
....
With
results created by automated testers at SUN for me it is proven the
tests are running in general and results created are reproducable.
(notice here that this is "at Sun" - QUASTE status clearly shows that
this is not the case "outside SUN"
I even don't think that this is a Sun vs. community issue,
I often get the feedback the the automated test run of the last night
went "red", the QA engineer has to have a look if these are really
errors and in most cases I get the feedback, this was a "false
positive", everything is alright. :-(
There seem to be timing (or similar) problems with some of the test -
which is a bad thing. It just takes time to find and fix these problems.
OTOH there is a set of tests, the Cat0-tests, which should run just fine
and should be reproducable on most if not any system, because these
tests were checked and fixed.
Andre: Did you try to run the Cat0-tests alone and see whether the
results are the same as those from the tests done here in Hamburg?
Developers also might have the expectation that they can use
testautomation in their environment. But this is also not possible for
the same reasons.
For the Cat0-tests this would be possible.
I also have the impression that they are not that many issues filed in
IssueTracker which are detected with the help of the testautomation, but
I may be wrong here since there is no such flag in IssueTracker
indicating that. I'm also not sure if I'd really should expect this, but
I'm wondering about this,
You could probably only find out by introducing such a flag and asking
people to use it. Otherwise AFAIK you can't detect whether an issue was
found by an automated test or manual testing.
Regards,
Jörg
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
