You control individual test cases by the statements in the control file and/or python script for that particular test. You can also use the front end to start/stop and monitor the test case progress (it will grab various logs for you to look at during and after the test runs). The amount of control you have over each individual test case will depend on how you write and manage them. Whatever your top level control file is, you will have control to manage that process.
Pass and Fail is indicated on the dashboard if you are using the front end (Green color signifies pass), or from the command line it will log the results into the run logs. All of your stderr/stdout will be logged, as well as any messages you want to pass to the logs (there are various functions you can use to log messages). There may be some better examples of looking at the results then test.kernel.org, as that seems to be limited to a subset of tests. There are also graphing capabilities in AutoTest that work well with benchmarks, etc. If you have a spare Linux box at home, you may just want to install it and play around with it. kdl On Mon, Mar 1, 2010 at 6:00 AM, shiraz hashim <[email protected] > wrote: > Hello Lucas > > On 3/1/2010 12:13 AM, K.D. Lucas wrote: > > In AutoTest a control file can execute either an individual test case or > > a suite of test cases. You can also create a control file that will > > execute multiple test cases. The control file will usually point to a > > python script that will execute your test case. Each test case is > > usually separated into it's own directory with it's own control file. > > It's very flexible as you can define dependencies. > > OK. And how will I control individual test case. So lets say a control > file executes many test cases, will I be able to select/deselect, pause > or abort individual test case during execution. > Secondly, how is the pass/fail of an individual test case is communicated > to serve and how is the log generated by test cases maintained by the > server. > > > I was looking at STAF long ago (it may have changed in the last 4 years) > > and at that time I concluded it doesn't have near the power or > > flexibility of AutoTest. > > You may be right. But now at first look it seems that STAF/STAX and > autotest > have overlapping goals. STAF is basically communication framework with > number of > possible services. STAX is one of these service to execute and > maintain processes > on local or destination machines (using STAF framework). STAX provides an > XML/python based interface to do this. It also provides a gui job monitor, > has structured logging and message mechanism. > > > Are you just looking for a way to execute your tests, or do you want > > additional system statistics, profiling, and reporting? > > The basic requirement is testing but any thing above it, that is supported > by the framework would be great. > I think related to this, autotest has some good features ( and therfore, I > am confused ;) ) > I did go through test.kernel.org to see report generated by autotest. > I could find the results of the tests for different kernel version. > But I couldnot search for details (in readable form) on individual test > cases their logs etc. > > > Kelly > > regards > Shiraz > > > > > On Sun, Feb 28, 2010 at 12:18 AM, shiraz hashim > > <[email protected]> wrote: > >> > >> Hello, > >> > >> I am looking for an automated test framework for Linux based embedded > >> environment. The objective is to have an automated, less bulky, > >> efficient framework, which can be used often during development and also > >> after for comprehensive/stressed testing. > >> > >> I came across autotest and STAF/STAX which seems to have similar goals > >> and are based on python. > >> > >> While test cases are clearly separable, definable and controllable in > >> STAX, I could not figure out how this is done in autotest. > >> What I understand is that in autotest, one can execute a set of test > cases > >> through control file but how are individual test cases controlled and > >> separated is not known to me. > >> > >> Can you please help me on this. > >> Sorry, If I am asking a very naive question. Actually I am bit confused > >> and not able to figure out what would be good for me. > >> > >> thanks > >> > >> -- > >> regards > >> Shiraz Hashim > >> _______________________________________________ > >> Autotest mailing list > >> [email protected] > >> http://test.kernel.org/cgi-bin/mailman/listinfo/autotest > > > > > > > > -- > > K.D. Lucas > > [email protected] > > > > > > -- > regards > Shiraz Hashim > -- K.D. Lucas [email protected]
_______________________________________________ Autotest mailing list [email protected] http://test.kernel.org/cgi-bin/mailman/listinfo/autotest
