[webkit-dev] Cross browser automated tests

[Sylvain Pasche]

Hi,

This is an announcement of a project that may be of interest. The idea 
is to run test cases automatically on some versions of the main 
browsers. Test results should help authors/spec authors/vendors see what 
feature is implemented where and vendors could integrate these tests in 
their regression testing framework.


For now the framework is able to run a subset of WebKit and Mozilla 
automated tests. For the WebKit tests in particular, it runs the 
LayoutTests that use the layoutTestController.dumpAsText() API and 
compare result with a testcase-exepected.txt file.


Results are visible on http://www.browsertests.org. Note that there are 
some features not implemented or limitations that result in tests that 
should pass which fail and reversely. Source code and some documentation 
is available at http://code.google.com/p/browsertests/.


One of the possible future improvements is to run Mozilla tests using 
DumpRenderTree. That means that Mozilla tests could be integrated in 
WebKit's automated testing harness.


If you have questions/suggestions/ideas they're welcome. There's also a 
discussion group on http://groups.google.com/group/browsertests.



Sylvain

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Cross browser automated tests

[Sylvain Pasche]

Hi,

I'm working on an experimental project to build a set of cross browser 
automated tests. The idea would be: having a repository of browser 
independent automated tests. Each browser developer could contribute 
tests to it and use these tests for testing their product. Existing test 
suites that could be run in an automated way could be imported into it 
(something that has already been done by Mozilla/WebKit for the dom 
tests for instance). The technical details of the testing environment 
and API would need to be discussed.

These tests could be run automatically on today's available browsers. 
With the test results published, Web developers could check what feature 
(if covered by tests) are implemented (and to what extent) in a given 
browser.

What do you think about it? Would there be interest in such a project 
from WebKit?

I started already to extract automated tests from Mozilla and WebKit and 
run them on a set of browsers. I first tried to see what tests are not 
WebKit specific. I considered tests which are using 
layoutTestController.dumpAsText(). For these tests, I can use something 
like document.body.nodeValue to get a text serialization of the document 
and compare it against the -expected.txt file.

I ran these tests with a few browsers and published results on 
http://www.browsertests.org/. Note that for WebKit I used the Linux 
GtkLauncher program (some tests made the browser crash). I plan to run 
these tests on Safari Mac/Windows for comparison.

By the way, Mozilla reftest system [1] is interesting in that project 
because it is a browser independent way of testing the layout engine. 
See pages 8-12 on http://www.browsertests.org/test.

I also started a thread on a Mozilla group, visible at [2]


Sylvain

[1] http://mxr.mozilla.org/mozilla/source/layout/tools/reftest/README.txt
[2] 
http://groups.google.com/group/mozilla.dev.quality/browse_frm/thread/b2a959c7547b9877/d5705f4f664bab53

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev

[webkit-dev] Re: Who's working on merging GNUmakefile.am and DerivedSources.make?

[Sylvain Pasche]

Alp Toker wrote:
 * Built object files are scattered throughout the directory hierarchy, 
resulting in svn stat noise.


You can avoid this by using a separate objdir, when doing something like:

mkdir objdir
cd objdir
../autogen.sh
make

This way you get all the .o files inside your objdir.

There a small fix needed to make this possible when using autogen.sh 
(http://bugs.webkit.org/show_bug.cgi?id=16736).



Sylvain

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo/webkit-dev