On 16.11.2011 08:21, Raphael Bircher wrote:
Hi Liu
Am 16.11.11 03:29, schrieb Zhe Liu:
Hi all,
Vcl TestTool is used by GUI automated testing for OpenOffice.org, but it
has many drawbacks. Too many errors, difficult to debug, maintain and
execute parallelly. I found many people mentioned this problem.
Libreoffice
has abondoned the tool. But I think GUI automated testing still is
valuable. It can test the product more like the actual users.
The problems with the VCLTestTool are well known. But to write a new one
is not trivial too. It needs a load of work to bring a new tool to the
productiv use. Well, we can maybe use the old testscripts as guide for
new testscripts. But the VCLTestTool as is works fine, and you can find
errors realy quick. Mainly not working dialoges, crashs and freese.
VCLTestTool is not so buggy as many people say. But yes, it needs
aditional work to bring it to a productive tool for the community.
And here's the problem: to work on the testtool, you need the source
code. And the source code is not available, at least not here at Apache.
It is based on a very old version of OOo's source code that was written
years ago. To build it, you will need large parts of the OOo source code
in that old revision - no good idea, moreover, it's totally unclear if
this old revision is covered by Oracle's code grant. At least it would
require and additional IP review.
Of course some testtool source code also exists in the current release,
but it is unused and untested since years. I remember that even the
latest larger code rework in that area (help IDs becoming Strings
instead of integer numbers) was not done for the testtool, only for the
testtool library that is loaded in OOo when tests are executed. Instead
of that larger parts of the testtool source code where just commented
out to please the compiler ("nobody needs that code, why invest time
into it"). Sigh.
Besides that, the available Linux version of the testtool does not run
on at least Ubuntu 64Bit (with 32Bit libs installed), but probably also
on other Linux versions. Sooner or later it won't run on more an more
Linux and probably MacOS versions.
For me the VCLTesttool is dead. We shouldn't invest time there if we had
an alternative GUI testing option with available source code. So we
definitely should give the Java lib a try, if it could be made available
under a suitable license.
Until then we should concentrate on writing integration tests in C++ or
Java. We already have some of them (called "complex tests") and they
have proven to be the most effective tool for bug hunting that we have.
Some reasons why they are more helpful, at least ATM:
- they run much faster than the UI based tests
- they can be executed in parallel (as much as you want)
- the gbuild build system supports them, so they can be automated
without additional tooling (no "test launcher" necessary)
- they are nearer to the code they test, so the root cause of a bug
usually is found faster
- they don't require to learn a strange Basic dialect, developers can
write them in the same language that they use daily
- it's common sense that testing through the UI should be done only if
you want to test the UI itself, but not the code behind it
Regards,
Mathias
