On Thu, Feb 28, 2008 at 11:06:45AM +0530, Sudhir Kumar wrote:
> Hi,
>
> This patch adds the documentation for cpu controller test cases.
Minor nits..
> <add-documentation.patch>
>
> Signed-off-by: Sudhir Kumar <[EMAIL PROTECTED]>
>
> Index: ltp-full-20080131/testcases/kernel/controllers/README
> ===================================================================
> --- ltp-full-20080131.orig/testcases/kernel/controllers/README
> +++ ltp-full-20080131/testcases/kernel/controllers/README
> @@ -28,6 +28,10 @@ cpuctl
> Directory containing the cpu controller testcases. A similar directory will
> be created for
> any new controller.
>
> +libcontrollers
> +--------------
> +This directory contains the library for cpucontroller testing.
> +
> Makefile
> --------
> The usual Makefile to conduct all the tests.
> @@ -35,4 +39,15 @@ The usual Makefile to conduct all the te
> Before starting compilation this file checks if the kernel is supporting the
> cpu controller.
> If yes it enters lower directory and compiles the source files otherwise
> exits warning to user.
> Hence the user has to take care of the fact that the test runs or just
> quiets.
> -(Your kernel needs to be 2.6.24-rc1 onwards with proper config options)
> +(Your kernel needs to be 2.6.24 onwards with proper config options)
> +
> +A snapshot of config file options is given below:
> +CONFIG_CGROUPS=y
> +CONFIG_CGROUP_DEBUG=y
> +CONFIG_CGROUP_NS=y
> +CONFIG_CPUSETS=y
> +CONFIG_GROUP_SCHED=y
> +CONFIG_FAIR_GROUP_SCHED=y
> +CONFIG_RT_GROUP_SCHED=y
> +# CONFIG_USER_SCHED is not set
> +CONFIG_CGROUP_SCHED=y
This is for 2.6.25-rc1 onwards. 2.6.24 will have different options (but
i guess it doesn't really matter)
> Index: ltp-full-20080131/testcases/kernel/controllers/cpuctl/README
> ===================================================================
> --- ltp-full-20080131.orig/testcases/kernel/controllers/cpuctl/README
> +++ ltp-full-20080131/testcases/kernel/controllers/cpuctl/README
> @@ -2,64 +2,82 @@
> TEST SUITE:
>
> The directory cpuctl contains the tests related to the cpu controller.
> -At present there is only 1 testcase but it is the first test case in it's
> -series of test cases and further testcases will be added in the forthcoming
> -release of LTP
> -
> -TEST AIM:
> -
> -The aim of the test is to find any dependency of cpu controller on exact
> value
> -of shares to schedule the tasks. For example if there are two groups running
> -1 task each then they must get the cpu time in the same ratio irrespective of
> -the share values i.e. share ratios 1:2, 10:20, 100:200, 1000:2000 etc should
> -give the same cpu usage for each group.
> +There are total 10 testcases that have been added till date.
a total of
> +More testcases are expected to be added in future.
>
> -FILES DESCRIPTION:
> +TESTS AIM:
> +
> +The aim of the tests is to test cpu controller functionality.
>
> -cpuctl_enabled_check.sh
> ------------------------
> -This file is in 1 directory up and is the threshold to run the test.
> -This script file checks the kernel version and if FAIR_GROUP_SCHEDULER has
> been
> -compiled in the kernel. If not then the test is not run and user is asked if
> -he wants the new kernel be compiled with proper configuration and then run
> the test.
> +FILES DESCRIPTION:
>
> -cpuctl_test01.c
> +cpuctl_testN.c
> ---------------
> -The test case for cpu controller testing. This basicaly runs a task in a
> group
> -under cpu controller. Each task runs for an interval TIME_INTERVAL seconds
> and
> -reports the total time it could run on all cpus(for convinience given in % ).
> +These are the tasks to run for cpu controller testing.
> +The tasks have been automated in the sense that they can assign themselves to
> +the appropriate group, can modify their group shares, can migrate etc.
> +Each task runs for an interval TIME_INTERVAL seconds and eports the total
> time
> +it could run on all cpus in an interval of INTERVAL seconds. (for convinience
> +calculate cpu time is given in % and seconds both).
> +A task can call a library routine from libcontrollers library to calculate
> +total amount of shares of all the groups, total number of tasks in it's
> group etc.
> +And thus a task knows what is the expected cpu time it should get to run.
>
> -After say n TIME_INTERVAL it modifies it's shares and again report the cpu
> +After say n SETS it modifies it's parameters and again report the cpu
reports
> usage.
> +Maximum effort has been used to reuse the code and keep total code size low.
>
> -run_cpuctl_test.sh
> +parameters.sh
> ----------
> -This file contains the script which does setup for the test. It creates a
> +This file contains the functions which do setup for the test. It creates a
> /dev/cpuctl directory, mounts cgroup filesystem on it with cpu. It then
> creates
> -a number(n) of groups in /dev/cpuctl and fires (n) tasks to run at the same
> time.
> +a number(n) of groups in /dev/cpuctl. The cleanup function does a complete
> cleanup
> +of the system.
> +(*However most of the error scenarios ahve been tested for a sane cleanup,
> still if
> +sometime it is unable to do it justt manualy execute the commands written in
> cleanup
just manually
> +function)
> +
> +run_cpuctl_test.sh
> +------------------
> +This script creates different scenarios for cpu controller testing and fires
> (n) tasks
> +in different groups to run at the same time. It waits for the return status
> from
> +tasks and reports test pass/fail accordingly.
>
> Makefile
> --------
>
> The usual makefile for this directory
>
> -README:
> ---------
> -The one you have gone through
> -
> -$LTPROOT/output/cpuctl_results.txt
> +$LTPROOT/output/cpuctl_resultsN.txt
> --------------
> This file will be created to log the results once the test is run. It
> contains the test
> results which are numbers and following is a description which will help to
> understand
> the results.
>
> -The cpu time for each group(task ) is calculated in %. There are two
> outcomes of the test:
> +There are two common major expected outcomes of all the tests:
>
> 1. A group should get cpu time in the same ratio as it's shares.
>
> 2. This time should not change with the changes in share values while the
> ratio in those
> values is same.
>
> -NOTE: In case 1 a variation of 1-2 % is acceptable.
> +The results file are straight forward to read. To keep things very simple
> just look at
> +the two fields calc:- and exp:- in % as below:
> +
> +CPU TIME{calc:- 60.01(s)i.e. 50.01(%) exp:- 50.00(%)}
> +
> +PASS/FAIL CRITERIO:
> +==================
> +A major difference in the two % values is a failure of cpu controller.
> +Also the difference between consecutive runs under similar conditions is
> fail.
> +Because of some feature which is not currently developed in kernel it is not
> +possible to create an ideal scenario and hence the decision of PASS/FAIL is
> +not taken for all the tests at the moment. The statistics is generated in the
> +results file and a quick look on it gives a proper understanding.
>
> +NOTE: In current scenario a variation of 1-3 % is acceptable.
> +
> +README:
> +--------
> +The one you have gone through.
>
> Index: ltp-full-20080131/testcases/kernel/controllers/testplan.txt
> ===================================================================
> --- ltp-full-20080131.orig/testcases/kernel/controllers/testplan.txt
> +++ ltp-full-20080131/testcases/kernel/controllers/testplan.txt
> @@ -1,10 +1,24 @@
> THE RESOURCE MANAGEMENT (CONTROLLERS) TEST PLAN
>
> -This directory "controllers" is created to include all test cases related to
> the resource
> -controllers in linux. The testplan at present includes testing of cpu
> controller and memory
> -controller. This is the first testcase in LTP to test the resource
> controllers (and tests
> -cpu controller only). Future testcases willl be included to test the cpu
> controller further
> -and memory controller. Each controller will have it's own directory to
> contain all test cases.
> +This directory "controllers" is created to include all test cases related to
> +the resource controllers in linux. The testplan at present includes testing
> +of cpu controller and memory controller.
> +At present the test cases are there to test cpu controller only. In future
> +testcases willl be included to test the cpu controller further and memory
> +controller. Each controller will have it's own directory to contain all test
> +cases.
> +A brief description of all the testcases is given in the corresponding
> +controllers directory.
>
> +CPU CONTROLLER:
> +==============
> +
> +Test 01:
> +
> +The aim of the test is to find any dependency of cpu controller on exact
> value
> +of shares to schedule the tasks. For example if there are two groups running
> +1 task each then they must get the cpu time in the same ratio irrespective of
> +the share values i.e. share ratios 1:2, 10:20, 100:200, 1000:2000 etc should
> +give the same cpu usage for each group.
>
> For more information on resource controllers please refer to cgroups.txt in
> kernel documentation.
> Index:
> ltp-full-20080131/testcases/kernel/controllers/cpuctl/cpuctl_testplan.txt
> ===================================================================
> ---
> ltp-full-20080131.orig/testcases/kernel/controllers/cpuctl/cpuctl_testplan.txt
> +++ ltp-full-20080131/testcases/kernel/controllers/cpuctl/cpuctl_testplan.txt
> @@ -1,25 +1,11 @@
> -The cpucontroller testplan includes a complete set of testcases that test
> the cpu controller
> -in different scenarios. So this testcase is the first in it's series.
> +The cpucontroller testplan includes a complete set of testcases that test the
> +cpu controller in different scenarios.
>
> -**This testcase tests the cpu controller under flat hierarchy.**
> +**These testcases test the cpu controller under flat hierarchy.**
>
> -
> -The test plan for this testcase is as below:
> -
> -First of all mount the cpu controller on /dev/cpuctl and create n groups.
> The number of
> -groups should be > the number of cpus for checking scheduling fairness(as we
> will run 1
> -task per group). then we create say n groups. By default each group is
> assigned 1024
> -shares. The cpu controller schedules the tasks in different groups on the
> basis of the
> -shares assigned to that group. So the cpu usage of a task depends on the
> amount of shares
> -it's group has out of the total number of shares(no upper limit). Let us say
> that we have
> -3 groups(1 task each) A,B,C each having 1, 2, 3 shares respectively. Hence
> if the tasks are
> -running infinitely they are supposed to get 16.66%, 33.33%, 50% cpu time
> respectively. This
> -test case tests that each group should get the cpu time in the same(above)
> ratio irrespective
> -of the shares values provided the ratio is not changed i.e. the cpu time per
> group should not
> -change if we change the shares from 1, 2, 3 to 100, 200, 300 or to 10K, 20K,
> 30K etc.
> -
> -How to view the results:
> ------------------------
> +General Note:-
> +============
> +How to calculate expected cpu time:
>
> Say if there are n groups then
>
> @@ -28,21 +14,108 @@ total share weight = grp1 share + grp2 s
> then expected time of any grp (say grp1)
> =100 * (grp1 share) /(total share weight) %
>
> -Note:-
> -(*If there were say 2 task in this group then both task might devide the
> time of this group
> - *and it would create no effect on tasks in other group. So a task will
> always get time from
> - *it's group time only and will never affect other groups share of time )
> +(* If there were say 2 task in a group then both task might devide the time
divide
> + * of this group and it would create no effect on tasks in other group. So a
another (perhaps?)
> + * task will always get time from it's group time only and will never affect
> + * other groups share of time.
groups'
> + * Even if task is run with different nice value it will affect the tasks in
a/the task
> + * that group only.
> +)
> +
> +!!! User need not worry about this calculation at all as the test dynamicaly
dynamically
> +calculates expected cpu time of a task and prints for each task in results
prints it in the results
file.
> +file. This calculation is given for refernce to interested users.
reference
> +For any other information please refer to cgroup.txt in kernel documentation.
>
> +TESTCASE DESCRIPTION:
> +====================
>
> -The cpu time for each group(task ) is calculated in %. There are two
> outcomes of the test:
> +Test 01: FAIRNESS TEST
> +---------------------
>
> -1. A group should get cpu time in the same ratio as it's shares.
> +This test consists of two test cases.
> +The test plan for this testcase is as below:
>
> +First of all mount the cpu controller on /dev/cpuctl and create n groups.
> +The number of groups should be > the number of cpus for checking scheduling
> +fairness(as we will run 1 task per group). then we create say n groups. By
> +default each group is assigned 1024 shares. The cpu controller schedules the
> +tasks in different groups on the basis of the shares assigned to that group.
> +So the cpu usage of a task depends on the amount of shares it's group has out
> +of the total number of shares(no upper limiit but a lower limit of 2). Let us
limit
> +say that we have 3 groups(1 task each) A,B,C each having 2, 4, 6 shares
> +respectively. Hence if the tasks are running infinitely they are supposed to
> +get 16.66%, 33.33%, 50% cpu time respectively. This test case tests that each
> +group should get the cpu time in the same(above) ratio irrespective of the
> +shares values provided the ratio is not changed i.e. the cpu time per group
> +should not change if we change the shares from 2, 4, 6 to 200, 400, 600 or
> +to 20K, 40K, 60K etc.
> +
Provided the conditions are the same :)
> +
> +How to view the results:
> +-----------------------
> +
> +The cpu time for each group(task ) is calculated in %. There are two
> outcomes of the test:
> +1. A group should get cpu time in the same ratio as it's shares.
> 2. This time should not change with the changes in share values while the
> ratio in those
> values is same.
>
> NOTE: In case 1 a variation of 1-2 % is acceptable.
> +(here we have 1 task per group)
>
> -(remember here we have 1 task per group)
> -
> -For any other information please refer to cgroup.txt in kernel documentation.
> +Test 03: GRANULARITY TEST
(Either i missed something, or there is not test 02)
> +-------------------------
> +Granularity test with respect to shares values
> +In this test the shares value of some of the groups is increased and for some
> +groups is decreased. Accordingly the expected cpu time of a task is
> calculated.
> +The outcome of the test is that calc cpu time must change in accordance with
> +change in shares value.
> +(i.e. calculated cpu time and expected cpu time should be same)
> +
> +Test 04: NICE VALUE TEST
> +-------------------------
> +Renice all tasks of a group to -20 and let tasks in all other groups run with
> +normal priority. The aim is to test that nice effect is within the groups and
> +not shares are dominant over nice values.
> +
> +Test 05: TASK MIGRATION TEST
> +----------------------------
> +In this test for the first run the test is run with say n tasks in m groups.
> +After first run a task is migrated from one group to another group. This task
> +now consumes the cpu time from the new group.
> +
> +Test 06-08 : NUM GROUPS vs NUMBER of TASKS TEST
> +----------------------------------------------
> +In the following three testcases a total tasks are same and each task is
> +expected to get same cpu time. Theses testcases test the effect of creating
> +more groups on fairness.(however latency check will be done in future)
> +
> +Test 06: N X M (N groups with M tasks each)
> +-------
> +
> +Test 07: N*M X 1 (N*M groups with 1 task each)
> +-------
> +
> +Test 08: 1 X N*M (1 group with N*M tasks)
> +-------
> +
> +Test 09-10: STRESS TEST
> +----------
> +The next two testcases put stress on system and create a large number of
> groups
> +each running a large number of tasks in it.
> +
> +Test 09: Heavy stress test with nice value change
> +-------
> +Creates 4 windows of different NICE values. Each window runs some n groups.
> +
> +Test 10: Heavy stress test (effect of heavy group on light group)
> +-------
> +In this test one group has very less tasks while others have large number of
> +tasks. This tests if fairness still remains.
> +
> +
> +(
> + In all tests calc cpu time and exp cpu time should be same
> + Time:: calc:- calculated cpu time obtained for a running task
> + exp:- expected time as per the shares of the group and num of tasks in
> group.
> +)
>
> Thanks,
> Sudhir Kumar
> LTC, ISTL
--
regards,
Dhaval
-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
_______________________________________________
Ltp-list mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/ltp-list
