#4218: Implement CKTester
------------------------+---------------------------------------------------
Reporter: garry.yao | Owner: garry.yao
Type: Bug | Status: assigned
Priority: Normal | Milestone: CKEditor 3.0
Component: QA | Version:
Keywords: Discussion |
------------------------+---------------------------------------------------
Old description:
> CKTester is our internal universal testing forge, it should be capable of
> running tests for different projects
> with different testing tools easily while remain simplicity, below are
> some basic ideas of the project
> based on the previous effort on 'FCKtest'. [[BR]]
>
> === Test Cell ===
> A '''test cell''' is basically a plain html file reside in individual
> project's testing source, in which test suites/test cases
> are written. It'll be executed inside an iframe/popup of the runner page
> so '''full page life cycle'''
> ( from page load to page destroy ) is guaranteed when performing the
> test.[[BR]]
>
> Each cell has only key responsibility of '''reporting''' the
> sucess/failure/time of it's execution to the runner, this is
> done by cross-frame communication with parent runner window.[[BR]]
>
> The cell's testing environment, including testing target, runtime
> libraries, etc. will be all injected by it's belonging
> runner, initiatively or passively, the only required effort in the cell
> is to include the bootstrap file.
> {{{
> <script type="text/javascript" src="{path-to-
> runner}/bootstrap.js"></script>
> }}}
> It could choose to manually pull in resources provided by the runner
> also,
> e.g. A cell want to load an testing target manually, it could have:
> {{{
> <script type="text/javascript" src="../../bootstrap.js"></script>
> <script type="text/javascript">
> CKTester.require( '${myProjectRoot}/myTarget.js' ); // inline
> variable ${myProjectRoot}
> </script>
> }}}
> Note that inline variables like above used in the path are not determined
> by the runner but are defined in '''testing profile''' which know well
> about the
> testing stage( target ) location.
>
> === Profile ===
> A '''tests profile''' is a js file contains information about running a
> bunch of test cells.
> It a function declaration which return a configuration object, contains:
> 1. Cell variables : A object contains key/value pair definition of
> variables which are to be used by individual test cell when communicating
> with runner.
> 1. Cells list : A list of key/values as:
> * Key : The test cell html '''file path''', with each directory within
> the path is resolved as a tag;
> * Value : An array of '''tags'''.
> It will basically looks like:
> {{{
> CKTester.profile = function ()
> {
> return
> {
> variables : [ [ 'ckeditorRoot', 'path-to-
> CKEditor-root'] ],
> cells : [
> [ 'tt/unit/htmldataprocessor/1' ],
> // Implicit tags declaration by file path.
> [ 'ckeditor/3210.html' , [ 'ticket',
> 'unit', 'htmldataprocessor'] ] // Explicit tag declaration.
> ]
> };
> }
> }}}
> The '''tag''' plays an important role as an identifier to denote the
> following aspects of a test cell:
> 1. Environments of the cell, e.g. if it has tags 'unit' and 'editor', it
> denote a unit test running by YUITest, along with the CKEditor testing
> stage is required to run the test.
> 2. Categories of the cell, when it's been run in a runner with a
> specific criteria. E.g. A url criteria as
> 'http://t/runner?cells=tt,plugins' will run only the ticket test of
> plugins ( with 'ticket' and 'plugins' tags ).
>
> === Runner ===
> Test runner is bridge between the testing tools/framework ( e.g. Selenium
> ) underneath,
> and the testing resources( our various test cases ), it's resided in
> CKTester project.
>
> The runner is responsible for delegating the running of all the tests to
> the under-laying tools/framework and collect
> the testing result neutrally. It consist of a UI to interact with and
> providing report.
>
> The runner is required to be running with a specified profile, from which
> it will register all the defined tests cells .
> Upon requesting by a specific '''testing criteria''' on running, it
> should filtering down the registered cells
> to only execute the satisfied ones.
>
> Before runner execute each test cell, an extensible '''environment
> resolver''' is running there to determinate the required/requested
> resources of a specific cell by analysing the '''tags''', guarantee the
> cell always has all the dependencies
> it should receive. These resources are typically
> composed of :
> * The testing target
> * The testing library
> * Any resources dedicate the one kind of test
>
> === Testing Request ===
> A request is simply a loading of the runner page, which is the beginning
> of the testing life cycle.
> 1. The request must contains a profile param in the url to specify the
> location of profile of this running.
> 1. The request could optionally consist of a criteria which is a list of
> '''tags''' or '''path''' to filter down the test cells to run, could be
> specified via :
> * A ampersand separated url params;
> * A predefined JSON file denote the criteria;
> * Specified through UI of the test runner page;
> Some example testing requests look like:
> * Running only the tickets test of unit test :
> http://t/runner.html?profile=../profile.js&cells=tt,unit
> * Running a specific test :
> http://t/runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/1
> * Running two specific tests :
> http://t/runner.html?profile=../profile.js&cells=../editor/tt/unit/htmldataprocessor/1,../editor/dd/selenium/visual/1
New description:
CKTester is our internal universal testing swarm, it should be capable of
running tests for different projects
with different testing tools easily while remain simplicity, below are
some basic ideas of the project
based on the previous effort on 'FCKtest'. [[BR]]
=== Test Cell ===
A '''test cell''' is basically a plain html file reside in individual
project's testing source, in which test suites/test cases
are written. It'll be executed inside an iframe/popup of the fort so
'''full page life cycle'''
( from page load to page destroy ) is guaranteed when performing the
test.[[BR]]
Each cell has only key responsibility of '''reporting''' the
sucess/failure/time of it's execution to the fort, this is
done by cross-frame communication with parent/opener window.[[BR]]
The cell's testing environment, including testing target, runtime
libraries, etc. will be all injected by the fort,
initiatively or passively, the only required effort in the cell is to
include the bootstrap file.
{{{
<script type="text/javascript"
src="{CKTester_ROOT}/bootstrap.js"></script>
}}}
A cell could be running in '''two modes''':
1. Managed Mode : It's the default mode when it's running in a batch of
cells by the forge, where the cell is running in an iframe.
2. Standalone Mode : This mode is adapted when a cell is running
separately, on the opposite of 'Managed Mode', the cell will be running
inside a popup window, and it doesn't require to report to the fort.
=== Profile ===
A '''tests profile''' is a js file contains information about running a
bunch of test cells.
It a function declaration which return a configuration object, contains:
1. Cell variables : An hash of variable names and their values which are
often embeded in profile paths to indicate location.
1. Cells list : A list of key/values:
* Key : The test cell '''file path''', where all the paths are relative
to the runner;
* Value : An array of '''tags'''.
1. Cell resolvers : An array of function which were used to manipulate
each cell's attributes at run-time, e.g. Inject necessary dependencies for
cells when encounter certain tags. Note that The fort already has a few
default resolvers, usages including : converting the cell path to tags;
calculate resource path for each cell, etc.
A profile will basically looks like:
{{{
CKTester.profile = function ()
{
return
{
variables : { 'CKEDITOR_ROOT' : 'path-to-editor-
root' },
cells : [
[
'editor/tt/unit/plugins/htmldataprocessor/1' ], // Implicit
tag declaration.
[ '3210.html' , [ 'tt', 'unit',
'htmldataprocessor'] ] // Explicit tag declaration.
]
cellResolvers : [ ... ]
};
}
}}}
It's important to note that '''tag''' plays an important role as the
following indicators:
1. Environments( dependencies ) of a cell, e.g. if the cell has tags
'unit' and 'editor', it denote a unit test running by YUITest, along with
the CKEditor testing stage is required to run the test;
2. Categories of the cell, when it's been run in a runner with a specific
criteria. E.g. A url criteria as 'http://t/runner?cells=tt,plugins' will
run only the tests of category 'ticket' and 'plugins' ( with 'ticket' and
'plugins' tags );
3. Any reasonable denotation.
=== Fort ===
The '''Test Fort''' is a bridge between the runner tools/framework ( e.g.
Selenium ) underneath, and the testing resources, so it's above the level
of any TC or TS. It's resided in CKTester project.[[BR]]
The fort is responsible for delegating the running of all the tests to the
under-laying tools/framework and collecting
the testing result neutrally. It consist of a UI to interact with and
providing report.
The runner is required to be running with a specified profile, from which
it will register all the defined tests cells in it.
Upon requesting by a specific '''criteria''' on running, it filtering down
the registered cells
to only execute the satisfied ones.
Generally, before runner execute each test cell, an extensible '''cell
resolver''' is working out the required/requested
resources of a specific cell by analysing the '''tags''', guarantee the
cell always has all the dependencies
it should receive. These resources are typically
composed of :
* The testing target
* The testing library
* Any resources dedicated to the cell.
=== Testing Request ===
A request is simply a page load of the fort, which form the begin of the
testing life cycle.
1. The request must use url param to specify the location of profile file
relative of the fort.
1. The request url could optionally consist of a criteria which is a list
of '''tags''' or '''path''' to filter the test cells to run, it could be
specified in various ways:
* A comma separated param in URL;
* Specified through UI of the test runner page;
Some example testing requests look like ( I've remove the 'profile' param
for clear understanding ):
* Running only the ticket tests of unit test type:
http://t/fort.html?cells=tt,unit
* Running all the '''stable''' tests : http://t/fort.html?cells=stable
* Running two specific tests :
http://t/fort.html?cells=../editor/tt/unit/htmldataprocessor/1,../editor/dd/selenium/visual/1
[[BR]]
Note that since it's not convenient to specify a cell path within URL,
it's a shortcut that open the cell url directly would have it redirect to
correct url back to the forge.
--
Comment(by garry.yao):
Update description according to implementation:
1. Rename 'runner' to 'fort' for disambiguation;
2. Adding 'running mode' concepts of cell;
3. Simplify 'criteria' pattern;
4. Adding 'cell shortcut' concept;
--
Ticket URL: <http://dev.fckeditor.net/ticket/4218#comment:3>
FCKeditor <http://www.fckeditor.net/>
The text editor for Internet
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
FCKeditor-Trac mailing list
FCKeditor-Trac@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fckeditor-trac
