#4218: Implement CKTester
------------------------+---------------------------------------------------
Reporter: garry.yao | Owner: garry.yao
Type: Bug | Status: assigned
Priority: Normal | Milestone: CKEditor 3.0
Component: QA | Version:
Keywords: Discussion |
------------------------+---------------------------------------------------
Old description:
> CKTester is our internal universal testing forge, it should be capable of
> running tests for different projects
> with different testing tools easily while remain simplicity, below are
> some basic ideas of the project
> based on the previous effort on 'FCKtest'. [[BR]]
>
> === Test Cell ===
> A '''test cell''' is basically a plain html file reside in individual
> project's testing source, in which test suites/test cases
> are written. It'll be executed inside an iframe/popup of the runner page
> so '''full page life cycle'''
> ( from page load to page destroy ) is guaranteed when performing the
> test.[[BR]]
>
> Each cell has only key responsibility of '''reporting''' the
> sucess/failure/time of it's execution to the runner, this is
> done by cross-frame communication with parent runner window.[[BR]]
>
> The cell's testing environment, including testing target, runtime
> libraries, etc. will be all injected by it's belonging
> runner, initiatively or passively, the only required effort in the cell
> is to include the bootstrap file.
> {{{
> <script type="text/javascript" src="{path-to-
> runner}/bootstrap.js"></script>
> }}}
> It could choose to manually pull in resources provided by the runner
> also,
> e.g. A cell want to load an testing target manually, it could have:
> {{{
> <script type="text/javascript" src="../../bootstrap.js"></script>
> <script type="text/javascript">
> CKTester.require( '${myProjectRoot}/myTarget.js' ); // inline
> variable ${myProjectRoot}
> </script>
> }}}
> Note that inline variables like above used in the path are not determined
> by the runner but are defined in '''testing profile''' which know well
> about the
> testing stage( target ) location.
>
> === Profile ===
> A '''tests profile''' is a js file contains information about running a
> bunch of test cells.
> It a function declaration which return a configuration object, contains:
> 1. Cell variables : A object contains key/value pair definition of
> variables which are to be used by individual test cell when communicating
> with runner.
> 1. Cells list : A list of key/values as:
> * Key : The test cell html '''file path''', with each directory within
> the path is resolved as a tag;
> * Value : An array of '''tags'''.
> It will basically looks like:
> {{{
> CKTester.profile = function ()
> {
> return
> {
> variables : [ [ 'ckeditorRoot', 'path-to-
> CKEditor-root'] ],
> cells : [
> [ 'tt/unit/htmldataprocessor/1' , [] ],
> // Implicit tags declaration by file path.
> [ 'ckeditor/3210.html' , [ 'ticket',
> 'unit', 'htmldataprocessor'] ] // Explicit tag declaration.
> ]
> };
> }
> }}}
> The '''tag''' plays an important role as an identifier to denote the
> following aspects of a test cell:
> 1. Dependencies of the cell, e.g. if it has tags 'unit', 'ckeditor' and
> 'domiterator', it denote it's a unit test running by YUITest, the testing
> stage is CKEditor, and the domiterator plugin is required to run the
> test.
> 2. Tagging of the cell, whether the cell should be executed when it's
> been run in a runner with a specific criteria. E.g. A url criteria as
> 'http://t/?tt&plugins' will run only the ticket test of plugins ( with
> 'ticket' and 'plugins' tags).
>
> === Runner ===
> Test runner is bridge between the testing tools/framework ( e.g. Selenium
> ) underneath,
> and the testing resources( our various test cases ), it's resided in
> CKTester project.
>
> The runner is responsible for delegating the running of all the tests to
> the under-laying tools/framework and collect
> the testing result neutrally. It consist of a UI to interact with and
> providing report.
>
> The runner is required to be running with a specified profile, from which
> it will register all the defined tests cells .
> Upon requesting by a specific '''testing criteria''' on running, it
> should filtering down the registered cells
> to only execute the satisfied ones.
>
> Before runner execute each test cell, an extensible '''dependency
> resolver''' is running there to determinate the required/requested
> resources of a specific cell by analysing the '''tags''', guarantee the
> cell always has all the dependencies
> it should receive. These resources are typically
> composed of :
> * The testing target
> * The testing library
>
> === Testing Request ===
> A request is simply a loading of the runner page, it's the beginning of
> the testing life cycle.
> The request could optionally consist of a criteria which is a list of
> tags to filter down the test cells to run, could be
> specified via :
> * A ampersand separated url params;
> * A predefined JSON file denote the criteria;
> * Specified through UI of the test runner page;
New description:
CKTester is our internal universal testing forge, it should be capable of
running tests for different projects
with different testing tools easily while remain simplicity, below are
some basic ideas of the project
based on the previous effort on 'FCKtest'. [[BR]]
=== Test Cell ===
A '''test cell''' is basically a plain html file reside in individual
project's testing source, in which test suites/test cases
are written. It'll be executed inside an iframe/popup of the runner page
so '''full page life cycle'''
( from page load to page destroy ) is guaranteed when performing the
test.[[BR]]
Each cell has only key responsibility of '''reporting''' the
sucess/failure/time of it's execution to the runner, this is
done by cross-frame communication with parent runner window.[[BR]]
The cell's testing environment, including testing target, runtime
libraries, etc. will be all injected by it's belonging
runner, initiatively or passively, the only required effort in the cell is
to include the bootstrap file.
{{{
<script type="text/javascript" src="{path-to-
runner}/bootstrap.js"></script>
}}}
It could choose to manually pull in resources provided by the runner also,
e.g. A cell want to load an testing target manually, it could have:
{{{
<script type="text/javascript" src="../../bootstrap.js"></script>
<script type="text/javascript">
CKTester.require( '${myProjectRoot}/myTarget.js' ); // inline
variable ${myProjectRoot}
</script>
}}}
Note that inline variables like above used in the path are not determined
by the runner but are defined in '''testing profile''' which know well
about the
testing stage( target ) location.
=== Profile ===
A '''tests profile''' is a js file contains information about running a
bunch of test cells.
It a function declaration which return a configuration object, contains:
1. Cell variables : A object contains key/value pair definition of
variables which are to be used by individual test cell when communicating
with runner.
1. Cells list : A list of key/values as:
* Key : The test cell html '''file path''', with each directory within
the path is resolved as a tag;
* Value : An array of '''tags'''.
It will basically looks like:
{{{
CKTester.profile = function ()
{
return
{
variables : [ [ 'ckeditorRoot', 'path-to-CKEditor-
root'] ],
cells : [
[ 'tt/unit/htmldataprocessor/1' ], //
Implicit tags declaration by file path.
[ 'ckeditor/3210.html' , [ 'ticket',
'unit', 'htmldataprocessor'] ] // Explicit tag declaration.
]
};
}
}}}
The '''tag''' plays an important role as an identifier to denote the
following aspects of a test cell:
1. Environments of the cell, e.g. if it has tags 'unit' and 'editor', it
denote a unit test running by YUITest, along with the CKEditor testing
stage is required to run the test.
2. Categories of the cell, when it's been run in a runner with a specific
criteria. E.g. A url criteria as 'http://t/runner?cells=tt,plugins' will
run only the ticket test of plugins ( with 'ticket' and 'plugins' tags ).
=== Runner ===
Test runner is bridge between the testing tools/framework ( e.g. Selenium
) underneath,
and the testing resources( our various test cases ), it's resided in
CKTester project.
The runner is responsible for delegating the running of all the tests to
the under-laying tools/framework and collect
the testing result neutrally. It consist of a UI to interact with and
providing report.
The runner is required to be running with a specified profile, from which
it will register all the defined tests cells .
Upon requesting by a specific '''testing criteria''' on running, it should
filtering down the registered cells
to only execute the satisfied ones.
Before runner execute each test cell, an extensible '''environment
resolver''' is running there to determinate the required/requested
resources of a specific cell by analysing the '''tags''', guarantee the
cell always has all the dependencies
it should receive. These resources are typically
composed of :
* The testing target
* The testing library
* Any resources dedicate the one kind of test
=== Testing Request ===
A request is simply a loading of the runner page, which is the beginning
of the testing life cycle.
1. The request must contains a profile param in the url to specify the
location of profile of this running.
1. The request could optionally consist of a criteria which is a list of
'''tags''' or '''path''' to filter down the test cells to run, could be
specified via :
* A ampersand separated url params;
* A predefined JSON file denote the criteria;
* Specified through UI of the test runner page;
Some example testing requests look like:
* Running only the tickets test of unit test :
http://t/runner.html?profile=../profile.js&cells=tt,unit
* Running a specific test :
http://t/runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/1
* Running two specific tests :
http://t/runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/1,../editor/dd/selenium/visual/1
--
Comment(by garry.yao):
Little adjustments.
--
Ticket URL: <http://dev.fckeditor.net/ticket/4218#comment:2>
FCKeditor <http://www.fckeditor.net/>
The text editor for Internet
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
FCKeditor-Trac mailing list
FCKeditor-Trac@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fckeditor-trac
