Hi Martin,
Martin Hollmichel wrote:
Do we have some statistics in which areas we have what amount of
regressions ?
Yes
* for releases and only
* only for show stopper-tasks
All these issues have been introduced in child-workspaces (CWS) and have
been found by automated GUI tests. It would make sense to find such
things on the CWSes to let the master workspace (MWS).
On CWSes we can not do the same - that's the reason why Joerg Jahnke
want to make it automated and at at different place in the development
process:
2.2.0
http://wiki.services.openoffice.org/wiki/OOoRelease1AutomationTestMatrix
#i71529 - Crash while pasting OLE in Calc
#i70517 - Office process does not end after exit (was i71766)
#i71882 - Crash while search into Starsuite Help - Fixed in OOE680m6
#i71891 - Crash when importing extension
2.2.1
http://wiki.services.openoffice.org/wiki/OOoRelease22AutomationTestMatrix
#74125 - (XML file format) ODF and OOo XML file format: Alien attributes
get lost in text style properties (integrated in OOF680m7)
#74334 - (Framework) Crash when loading signed documents (Fix scheduled
for OOF680m8, CWS fwk65)
#74372 - (Framework) Basic IDE Crash (Fixed in OOF680m7)
#74058 - (Framework) Closing Macro-editor crashes office. (integrated in
OOF680m7)
#74523 - (Draw/Impress) Loading html-document crashes office (integrated
in OOF680m8)
For example I would think that regression caused by broken resources
doesn't occur that much any more, are also easy to find by broad
testing. On the other hand I could image that regressions in document
layout do occur much more often and would be reported much more later
than broken resources ?
Okay. If you exact mean a "broken resource", yes. We're nearly save in
that case. What I ment was a broken resource or broken call/use of
dialogs as we could see many of them above. The reason often is
missunderstandings, not enough knowledge of dependencies etc. and our
testing is finding such things with less clicks but if we start it in QA
process we are too late. It makes sense to find such things at the place
where they have been created.
What about Wizards, System Integration, OLE, foreign formats, where do
we have the biggest problems ?
- You find for each section one issue above in the list; so you have
detected the areas where we find regressions.
- System integration is not our part, we're with the type of tests we
use inside OOo. We do regression testing of the functionality of
OOo itselves.
Can we use there more faster tests than we have with the testtool ?
The VCL TestTool is old but it's technology isn't old and is being used
by the most testtools also (QA Wizard Pro, WinRunner, GUIdancer, ...).
The VCL TestTool is being enhanced by Gregor Hartmann that we, the
software test automators, will be able to create faster tests [1]. AFAIK
there is no second test tool for VCL available which is as good as the
VCL TestTool is for it.
Not FASTER is the question here!
EARLIER with different test methods and type of tests:
- _Unit tests_ are the first tests which should be done but they are for
functions etc.
- API tests (which often mentioned) is NOT the level of tests we use
here. They have ALSO to be used but they save the quality of the API
and not the quality of the UI interaction
These types of tests are not part of the suggested enhancement for
'regression testing' and should be discussed in other threads.
- VCL TestTool regression testing is a integration test tool where
the full set of the system under test (SUT) is being tested and it is
the latest step to give the bits away to testing teams or the
the customer.
Cu,
Jogi
[1]
http://wiki.services.openoffice.org/wiki/Steps_towards_regular_automated_regression_testing
http://qa.openoffice.org/qatesttool
http://wiki.services.openoffice.org/wiki/User:Jsi
--
Sun Microsystems GmbH Joerg Sievers
Nagelsweg 55 Quality Assurance Engineer
20097 Hamburg
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
