subject:"Re\: \[DISCUSS\]Next steps for automated testing"

Re: [DISCUSS]Next steps for automated testing

2012-06-19 Thread Zhe Liu

2012/6/19 Andrew Rist andrew.r...@oracle.com:

On 6/14/2012 11:31 PM, Zhe Liu wrote:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.
2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

What are the requirements for these tests?
What type of machines? What software needs to be loaded?
Require to install
JDK 1.6, Ant 1.8.2+, JUnit 4.10+.
Testing machines need enable graphics interface. Testing can't be
performed in headless mode.
How long does it take? What do the scripts look like that kick off the
testing?
The default test suite takes about 1 hours. We can change test suite
size. To start testing, only one command is required.
cd testscript ant
I has attached testing code to
https://issues.apache.org/ooo/show_bug.cgi?id=119998
More detail guide has been updated in wiki
http://wiki.services.openoffice.org/wiki/QA/vclauto
What is the output of the testing (single file? logs? web pages?)?
All of logs, HTML and XML are generated. Generally we only need to
check HTML output.

Let's figure out what we need to put this on the buildbots...
A.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

Welcome to comment.

Andrew Rist | Interoperability Architect
OracleCorporate Architecture Group
Redwood Shores, CA | 650.506.9847

--
Best Regards
From aliu...@gmail.com

Re: [DISCUSS]Next steps for automated testing

2012-06-19 Thread Raphael Bircher

Am 19.06.12 09:29, schrieb Zhe Liu:
I

2012/6/19 Andrew Rist andrew.r...@oracle.com:

I propose to do the following tasks next.
1. Migrate the library to our AOO trunk. I has successfully used it to
test AOO 3.4 with some patch.
symphony/trunk/main/testcommon-ooo/trunk/main/testcommon
symphony/trunk/main/testgui-ooo/trunk/main/testgui
symphony/trunk/main/test-ooo/trunk/main/test or
ooo/trunk/main/testoo (Avoid to conflict with the test module that
already exists in AOO)
2. Setup several testing machines to do build verification testing on
daily build. Post the result on somewhere(e.g. wiki, or maillist) .
The testing platforms includes:
Windows XP
Windows 7 32b/64b
Mac os x
Redhat
Suse
Ubuntu
...? (pls suggest)
What are the requirements for these tests?
What type of machines? What software needs to be loaded?
Require to install
JDK 1.6, Ant 1.8.2+, JUnit 4.10+.
Testing machines need enable graphics interface. Testing can't be
performed in headless mode.
How long does it take? What do the scripts look like that kick off the
testing?
The default test suite takes about 1 hours. We can change test suite
size. To start testing, only one command is required.
cd testscript ant
I has attached testing code to
https://issues.apache.org/ooo/show_bug.cgi?id=119998
More detail guide has been updated in wiki
http://wiki.services.openoffice.org/wiki/QA/vclauto
What is the output of the testing (single file? logs? web pages?)?
All of logs, HTML and XML are generated. Generally we only need to
check HTML output.
I will try to run this on a Windows 7 VM

Greetings Raphael

Re: [DISCUSS]Next steps for automated testing

2012-06-18 Thread Andrew Rist

What are the requirements for these tests?
What type of machines? What software needs to be loaded?
How long does it take? What do the scripts look like that kick off the
testing?
What is the output of the testing (single file? logs? web pages?)?

Let's figure out what we need to put this on the buildbots...
A.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

Welcome to comment.

Andrew Rist | Interoperability Architect
OracleCorporate Architecture Group
Redwood Shores, CA | 650.506.9847

Re: [DISCUSS]Next steps for automated testing

2012-06-17 Thread Zhe Liu

Hi Reizinger Zoltán,

2012/6/16 Reizinger Zoltán zreizin...@hdsnet.hu:
2012.06.15. 8:31 keltezéssel, Zhe Liu írta:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.
2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

I'm not a coder, but I run the old VCLtool in past, used for translation
testing, by creating screenshots from each dialog.
This will be possible with new tool?
Yes. Generally, you can think it's a Java version of VCLtesttool.

I try to setup this test configuration on my win7.
I can set up and build test enviroment, and could build it, but can not
run the example test.
Needs some more help on JUnit configuration. Which test class I can use?

Pls wait for two days. I will migrate the newest code from Symphony
code into AOO. Because of the difference between Symphony and AOO, I
also need modify some code.

Regards,
Zoltan

Welcome to comment.

--
Best Regards
From aliu...@gmail.com

Re: [DISCUSS]Next steps for automated testing

2012-06-16 Thread Reizinger Zoltán

2012.06.15. 8:31 keltezéssel, Zhe Liu írta:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.
2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

I'm not a coder, but I run the old VCLtool in past, used for translation
testing, by creating screenshots from each dialog.
This will be possible with new tool?

Welcome to comment.

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread debin lei

Zhe, thank you excellent work and good summary.
I am very interesting in automation test framework.
For the proposal 1, when the codes can be migrated to AOO code base?
For the proposal 3, I am volunteering to do some clean up work.

2012/6/15 Zhe Liu aliu...@gmail.com

Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.
2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

Welcome to comment.

--
Best Regards
From aliu...@gmail.com

--
Best regards
Lei Debin

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Zhe Liu

2012/6/15 debin lei debin@gmail.com:
Zhe, thank you excellent work and good summary.
I am very interesting in automation test framework.
For the proposal 1, when the codes can be migrated to AOO code base?
If there is no objection, I will request committers to do it next week.
For the proposal 3, I am volunteering to do some clean up work.
That's appreciated.

2012/6/15 Zhe Liu aliu...@gmail.com

Welcome to comment.

--
Best Regards
From aliu...@gmail.com

--
Best regards
Lei Debin

--
Best Regards
From aliu...@gmail.com

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Herbert Duerr

Hi Zhe Liu,

On 15.06.2012 08:31, Zhe Liu wrote:
 As mentioned before, I was working on a Java library to perform gui
 testing. [...]

Wonderful! We really need to resurrect automated testing and doing it
with a Java based approach is attractive for a far wider audience than
the older test scripts which were written in Basic.

 I propose to do the following tasks next.
 1. Migrate the library to our AOO trunk. I has successfully used it to
 test AOO 3.4 with some patch.

This sounds like it is almost ready. The sooner we have it the better.

 2. Setup several testing machines to do build verification testing on
 daily build. Post the result on somewhere(e.g. wiki, or maillist) .

I like what the buildbots are doing: sending a mail with the summary and
provide all the details on their automatically updated website.

 The testing platforms includes:
 Windows XP
 Windows 7 32b/64b
 Mac os x
 Redhat
 Suse
 Ubuntu
 ...? (pls suggest)

Both linux 32bit and 64bit should be tested. And maybe we should also
test near our baseline and on recent versions. To avoid multiplying the
hardware requirements for the Linux builds we could e.g. test RHEL5 as
baseline, a recent Suse as 64bit and a recent LTS Ubuntu as 32bit.

Speaking of baseline vs. recent platforms I think we should once in a
while run automated tests on our MacOSX baseline (10.4 as of now).

 3. Continue to clean up the UNO API testing. I tried to run it and
 found there are too many failures and some errors. I think API testing
 is very valuable.Ｉt is essential to revive it.

Absolutetly.

Thanks for your work!
Herbert

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Jürgen Schmidt

On 6/15/12 8:31 AM, Zhe Liu wrote:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.

great news, we can really use such an improved automated test tooling.

Do you oplan to include the performance test as well?

2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

I will take a look on it, thanks for the pointer

you can create sub directories in main/test, I would avoid a new top
level directory. But if it's easier don't hesitate to create one.

2. Setup several testing machines to do build verification testing on
daily build. Post the result on somewhere(e.g. wiki, or maillist) .
The testing platforms includes:
Windows XP
Windows 7 32b/64b
Mac os x
Redhat
Suse
Ubuntu
...? (pls suggest)

When the tests running stable we can think about an integration in the
build bots as well.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

I agree and especially more complex API tests would be useful.

Anyway great work and I am looking forward to see it in action and the
first results.

Juergen

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Zhe Liu

2012/6/15 Jürgen Schmidt jogischm...@googlemail.com:
On 6/15/12 8:31 AM, Zhe Liu wrote:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.

great news, we can really use such an improved automated test tooling.

Do you oplan to include the performance test as well?
Yes!

I will take a look on it, thanks for the pointer

you can create sub directories in main/test, I would avoid a new top
level directory. But if it's easier don't hesitate to create one.
One sub-dir. It sounds better.

When the tests running stable we can think about an integration in the
build bots as well.
To run this kind of testing, graphics env is required. I don't know if
buildbot slaves meet the requirement.
Currently, we can use some external machines, periodically retrieve
builds from buildbot to local and then run testing. That also make us
cover more testing platforms.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

I agree and especially more complex API tests would be useful.

Anyway great work and I am looking forward to see it in action and the
first results.

Juergen

--
Best Regards
From aliu...@gmail.com

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Yi Xuan Liu

We have developed PVT automation script based on this automation framework,
which include Load Show, Load Finish and Save on 12 sample files (2
odt, 2 doc, 2 odp, 2 ppt, 2 ods, and 2 xls). And we've executed this
script on Window XP platform.

We will commit PVT automation script when testing Library is ready.

On Fri, Jun 15, 2012 at 3:27 PM, Jürgen Schmidt
jogischm...@googlemail.comwrote:

great news, we can really use such an improved automated test tooling.

Do you oplan to include the performance test as well?

I will take a look on it, thanks for the pointer

you can create sub directories in main/test, I would avoid a new top
level directory. But if it's easier don't hesitate to create one.

When the tests running stable we can think about an integration in the
build bots as well.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

I agree and especially more complex API tests would be useful.

Anyway great work and I am looking forward to see it in action and the
first results.

Juergen

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Rob Weir

On Fri, Jun 15, 2012 at 2:31 AM, Zhe Liu aliu...@gmail.com wrote:
Hi all,
As mentioned before, I was working on a Java library to perform gui
testing. Actually it has been implemented on Symphony source code. It
involves 3 modules:
1.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.
2.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.
https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

Maybe Windows 8 preview?

-Rob

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

Welcome to comment.

--
Best Regards
From aliu...@gmail.com

Re: [DISCUSS]Next steps for automated testing

2012-06-15 Thread Chao Huang

Wonderful ! Thanks!

2012/6/15 Yi Xuan Liu liuyixuan@gmail.com

We will commit PVT automation script when testing Library is ready.

On Fri, Jun 15, 2012 at 3:27 PM, Jürgen Schmidt
jogischm...@googlemail.comwrote:

https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/test
It contains all testing scripts. Some JUnit testcases have been
written in the package testcase. Smoke testing is re-implemented
based on the lib. We also developed some performance testing script,
but not include in svn.

great news, we can really use such an improved automated test tooling.

Do you oplan to include the performance test as well?

https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testcommon
It contains the low-level implementation to do GUI testing.
3.

https://svn-master.apache.org/repos/test/danielsh/symphony-import/symphony/trunk/main/testgui
It contains the common utilities used by uno api testing and GUI
testing.
I also wrote one wiki page to introduce it.
http://wiki.services.openoffice.org/wiki/QA/vclauto

I will take a look on it, thanks for the pointer

you can create sub directories in main/test, I would avoid a new top
level directory. But if it's easier don't hesitate to create one.

When the tests running stable we can think about an integration in the
build bots as well.

3. Continue to clean up the UNO API testing. I tried to run it and
found there are too many failures and some errors. I think API testing
is very valuable.Ｉt is essential to revive it.

I agree and especially more complex API tests would be useful.

Anyway great work and I am looking forward to see it in action and the
first results.

Juergen

--
Best regards,
Chao Huang

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

Re: [DISCUSS]Next steps for automated testing

13 matches

Site Navigation

Mail list logo

Footer information