On 10/31/2011 10:19 AM, Petr Mladek wrote:
What is your opinion, please?
"Hard to tell without trying it out," I'd say. Maybe pick some random
test now done via testtool and see if it can be ported to
subsequenttest's "pure UNO" approach and is better afterwards.
My feeling is that many people do not like testtool because it is:
+ gives random failures:
+ sleeps and waits everywhere in the test code
The root problem here probably is that in some cases appropriate
notification channels are missing, so test code inserts little waits.
Some other places are probably "test didn't work first, inserted a
sleep, test then worked, no idea why" that should rather be analyzed fully.
+ hard to debug:
+ unclear error messaged
+ complex mapping between the elements in the testtool
code and LO code (icons, dialogs)
+ need to run the test several times to understand what
happens there => quite time consuming
+ hard to maintain:
+ menu entries are accessed by the position in menu =>
new item breaks tests
On the other hand, this did find errors in the past. ;) (Some menus or
drop-down lists maintained items in the order as provided by configmgr,
relying on configmgr providing them in a specific order; after the
configmgr rewrite, the items were in a different order, and the actions
associated with the items no longer matched the items' text...)
IMHO, the best solution would be to involve developers into maintaining
the tests. We need some technology that they would like and accept.
Yep. Ideally, such higher level tests should be understandable,
writeable, and maintainable by both QA and Dev folks. testtool is
probably something that neither camp particularly likes.
Stephan
_______________________________________________
List Name: Libreoffice-qa mailing list
Mail address: Libreoffice-qa@lists.freedesktop.org
Change settings: http://lists.freedesktop.org/mailman/listinfo/libreoffice-qa
Problems? http://www.libreoffice.org/get-help/mailing-lists/how-to-unsubscribe/
Posting guidelines + more: http://wiki.documentfoundation.org/Netiquette
List archive: http://lists.freedesktop.org/archives/libreoffice-qa/
