[openstack-dev] [Rally] Load testing of dependent resources

2014-09-18 Thread Ilya Shakhat
Ralliers,

I'd like to share with you an idea on how to test dependent resources.

Let's consider the following example: there is a network and ports
belonging to it and we would like to test port creation process. Currently
I see 2 options of writing scenario:
 a) The scenario creates network and then creates a specified number of
ports. Runner executes the scenario specified number of times. -- In this
case ports belonging to different networks are created in parallel, but all
ports belonging to the same network are created serially.
 b) The network already exists and the scenario just creates ports, -- This
case allows to test concurrent creation of ports belonging to the same
network but not between multiple networks.
It would be really cool to get benefits from both cases, i.e. to test
parallel port creation but without limiting to parent network resource.

One of possible approaches may be to construct chain of scenarios where
preceding scenario prepares context for the further one. From the above
example the execution would be:
 1) The task contains sequence of 2 scenarios:
* network creation - it creates network and returns it
* port creation - it picks the network from incoming params and creates
port on it
 2) The runner has execution pipeline (which currently is represented as
generator producing identical scenario runners). The pipeline starts with
sequence of network creation scenario runners. Once one of runner finishes,
it then puts the further scenario and the result into pipeline. The
pipeline ends once no further scenarios left.

From the above example execution flow would be like:
 * pipeline is filled with N network scenarios (not actually filled, but
contains a generator that is able to do this)
 * R runners pick scenarios from pipeline
 * pipeline contains N-R network scenarios
 * one of runners finishes the scenario and updates pipeline with P port
scenarios
 * pipeline contains N-R-1 network scenarios followed by P port scenarios
 * work-work-work until pipeline is empty

This execution covers case from a) and b) but there still will be sequences
of child resources belonging to the same parent. The improvement may be to
pick scenario from pipeline randomly.

How does it sound?

Thanks,
Ilya
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [Rally] Load testing of dependent resources

2014-09-18 Thread Boris Pavlovic
Ilya,

It sounds good, moreover few other devs asked about similar feature=)
So I am working on proposal how to implement this=)


Best regards,



On Thu, Sep 18, 2014 at 4:35 PM, Ilya Shakhat ishak...@mirantis.com wrote:

 Ralliers,

 I'd like to share with you an idea on how to test dependent resources.

 Let's consider the following example: there is a network and ports
 belonging to it and we would like to test port creation process. Currently
 I see 2 options of writing scenario:
  a) The scenario creates network and then creates a specified number of
 ports. Runner executes the scenario specified number of times. -- In this
 case ports belonging to different networks are created in parallel, but all
 ports belonging to the same network are created serially.
  b) The network already exists and the scenario just creates ports, --
 This case allows to test concurrent creation of ports belonging to the same
 network but not between multiple networks.
 It would be really cool to get benefits from both cases, i.e. to test
 parallel port creation but without limiting to parent network resource.

 One of possible approaches may be to construct chain of scenarios where
 preceding scenario prepares context for the further one. From the above
 example the execution would be:
  1) The task contains sequence of 2 scenarios:
 * network creation - it creates network and returns it
 * port creation - it picks the network from incoming params and
 creates port on it
  2) The runner has execution pipeline (which currently is represented as
 generator producing identical scenario runners). The pipeline starts with
 sequence of network creation scenario runners. Once one of runner finishes,
 it then puts the further scenario and the result into pipeline. The
 pipeline ends once no further scenarios left.

 From the above example execution flow would be like:
  * pipeline is filled with N network scenarios (not actually filled, but
 contains a generator that is able to do this)
  * R runners pick scenarios from pipeline
  * pipeline contains N-R network scenarios
  * one of runners finishes the scenario and updates pipeline with P port
 scenarios
  * pipeline contains N-R-1 network scenarios followed by P port scenarios
  * work-work-work until pipeline is empty

 This execution covers case from a) and b) but there still will be
 sequences of child resources belonging to the same parent. The improvement
 may be to pick scenario from pipeline randomly.

 How does it sound?

 Thanks,
 Ilya

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev