Hi,
I'm late in this thread, but I have to write some words. See my comments
inline.
Martin Hollmichel schrieb:
Andre Schnabel wrote:
Hi,
-------- Original-Nachricht --------
Von: Helge Delfs <[email protected]>
Let me state this and give me a try to help understanding what automated
testing represents.
1. Trust your tools (automated tests, testtool, scripts)
....
With
results created by automated testers at SUN for me it is proven the
tests are running in general and results created are reproducable.
(notice here that this is "at Sun" - QUASTE status clearly shows that
this is not the case "outside SUN"
I even don't think that this is a Sun vs. community issue,
I often get the feedback the the automated test run of the last night
went "red", the QA engineer has to have a look if these are really
errors and in most cases I get the feedback, this was a "false
positive", everything is alright. :-(
The reason is, that every QA engineer inside Sun used his own
environment for automated testing. Therefore often the issues are found
because of different settings in OOo options or different settings in
the system (e.g. accessibility tools are on => worst case).
Developers also might have the expectation that they can use
testautomation in their environment. But this is also not possible for
the same reasons.
Different environments could lead to different results.
=> solution : TestBots for general CWS testing and perhaps for general use
[...}
That said, I think the testautomation is surely a good tool to track
changes in behavior in a well defined environment, but this is surely
not the framework that can be used by developers for doing tests, or can
be used for QA as a general purpose tool for doing broad QA.
Quality Assurance is to check products (software or not) in a defined
environment. When the defined environment is changed, new test cases
are needed to check product in this new environment. Therefore I worked
hard to have TestBots, which can be used for Sun internal and external
CWS. Then it isn't needed for each developer to run the tests in their
environment. Then the tests will start automatically in a defined
environment.
And to get me right. If you changed the environment the VCLTestTool can
identify new or other bugs/problems. But it isn't possible to check a
big product like OOo in all environments on all platforms and all languages.
We discussed this in the ESC for quite some time, that for this purpose
the implementation of Unit tests is important.
Unit tests and Automated Testing with VCLTestTool are different levels
of quality assurance on software. Both are important, because both will
identify problems in the product/code. UNIT tests will identify problems
on code level, which often isn't easy to identify on UI level. And also
UNIT tests can past the tests, but on UI level there are bugs. So both
type of testing is needed.
So stay tuned until mid of the year. I hope we will have a next step
in TestBots.
Thorsten
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
