Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-29 Thread Alexander Kostrikov 
Boris, can You take a look on HA scenarios [0] to give a quick feedback on
how it should be implemented with Rally in right way? Plugins/templates/how
yaml should look like.
To make correct comparison of ostf and rally-based version.

Thanks!
[0] http://paste.openstack.org/show/523983/

On Tue, Jun 28, 2016 at 3:34 AM, Boris Pavlovic 
wrote:

> Igor,
>
>
> I wonder what are the benefits of using Rally then? We can run whatever
>> we want by means of MCollective or Ansible. Is there something that could
>> help us? I don't know, maybe some dashboard integration with per test
>> results or running by tests by tag?
>
>
> Benefits are in the Rally framework, engine and integrated tooling, that
> are doing very hard things to provide simple interfaces for writing simple
> plugins that are emulating complicated test cases.
>
> The major benefits are next:
>
> *1) Generalization*
> 1.1) One tool with one reporting system and one output format for all
> kinds of testing strategies (functional, load, perf, scale, ...)
> 1.2) One set of plugins (code) that can be used to generate all kinds of
> testing strategies
> 1.3.) One API for all kinds of testing strategies
>
> *2) Simplicity *
> 2.1) Plugins are really simple to write, usually requires one method to be
> implemented
> 2.2) Auto discovery: adding plugins == add code in special (or specified)
> directory
>
> *3) Reusability & Data Driven approach: *
> 3.1) Split code (plugins) & tests cases (yaml files)
> 3.2) Test cases is mixture of different plugins
> 3.3) Plugins accept arguments
>
> *4) Integrated tooling*
> 4.1) All results are persisted in Rally DB and you can access it in any
> moment
> 4.2) Results can be exported in different formats (you can write even own
> plugins for simplifying integration)
> 4.3) Detailed HMTL reports with task results overview and trends
>
>
>
>
> Before switching Fuel from ostf to rally, I would like to see feature
>> parity comparison. It's very necessary to understand how much work we need
>> to spend to rewrite all our tests in rally way.
>
>
> Totally agree, let's do it.
>
>
> Best regards,
> Boris Pavlovic
>
>
>
> On Mon, Jun 27, 2016 at 8:58 AM, Vladimir Kuklin 
> wrote:
>
>> +1 to initial suggestion, but I guess we need to have a full feature
>> equality (e.g. HA tests for mysql and rabbitmq replication) before
>> switching to Rally.
>>
>> On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk <
>> sgolovat...@mirantis.com> wrote:
>>
>>> Hi,
>>>
>>> Before switching Fuel from ostf to rally, I would like to see feature
>>> parity comparison. It's very necessary to understand how much work we need
>>> to spend to rewrite all our tests in rally way.
>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Sergii Golovatiuk,
>>> Skype #golserge
>>> IRC #holser
>>>
>>> On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <
>>> akostri...@mirantis.com> wrote:
>>>
 Hello, everybody!
 Hello, Alex!
 >I thought Rally was more for benchmarking.  Wouldn't Tempest make
 more sense?
 Rally is a good tool with nice api/usage/extensibility.
 I really liked "up and running tests in 5 minutes" in Rally with clear
 picture of what I am doing.
 So, I 100% for a Rally as a QA.

 Another note:
 We will need to implement some HA tests, probably not in Rally.

 On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin 
 wrote:

>
>
> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky <
> ikalnit...@mirantis.com> wrote:
>
>>
>> > On Jun 27, 2016, at 16:23, Alex Schultz 
>> wrote:
>> >
>> > I thought Rally was more for benchmarking.  Wouldn't Tempest make
>> more sense?
>>
>> According to Rally wiki page [1], it seems they have a verification
>> layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite
>> our scenarios for Tempest?
>>
>>
> Rally consists of two main components: Rally Task and Rally
> Verification. They are totally separated.
> Task component is fully pluggable and you can run there whatever you
> want in whatever you want way.
> Verification component is hardcoded now. It was designed for
> managing(install, configure) and launching(execute and store results)
> Tempest. But we have a spec to make this component pluggable too.
>
>
>> - igor
>>
>>
>> [1] https://wiki.openstack.org/wiki/Rally
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Boris Pavlovic 
Igor,


I wonder what are the benefits of using Rally then? We can run whatever we
> want by means of MCollective or Ansible. Is there something that could help
> us? I don't know, maybe some dashboard integration with per test results or
> running by tests by tag?


Benefits are in the Rally framework, engine and integrated tooling, that
are doing very hard things to provide simple interfaces for writing simple
plugins that are emulating complicated test cases.

The major benefits are next:

*1) Generalization*
1.1) One tool with one reporting system and one output format for all kinds
of testing strategies (functional, load, perf, scale, ...)
1.2) One set of plugins (code) that can be used to generate all kinds of
testing strategies
1.3.) One API for all kinds of testing strategies

*2) Simplicity *
2.1) Plugins are really simple to write, usually requires one method to be
implemented
2.2) Auto discovery: adding plugins == add code in special (or specified)
directory

*3) Reusability & Data Driven approach: *
3.1) Split code (plugins) & tests cases (yaml files)
3.2) Test cases is mixture of different plugins
3.3) Plugins accept arguments

*4) Integrated tooling*
4.1) All results are persisted in Rally DB and you can access it in any
moment
4.2) Results can be exported in different formats (you can write even own
plugins for simplifying integration)
4.3) Detailed HMTL reports with task results overview and trends




Before switching Fuel from ostf to rally, I would like to see feature
> parity comparison. It's very necessary to understand how much work we need
> to spend to rewrite all our tests in rally way.


Totally agree, let's do it.


Best regards,
Boris Pavlovic



On Mon, Jun 27, 2016 at 8:58 AM, Vladimir Kuklin 
wrote:

> +1 to initial suggestion, but I guess we need to have a full feature
> equality (e.g. HA tests for mysql and rabbitmq replication) before
> switching to Rally.
>
> On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk <
> sgolovat...@mirantis.com> wrote:
>
>> Hi,
>>
>> Before switching Fuel from ostf to rally, I would like to see feature
>> parity comparison. It's very necessary to understand how much work we need
>> to spend to rewrite all our tests in rally way.
>>
>>
>>
>> --
>> Best regards,
>> Sergii Golovatiuk,
>> Skype #golserge
>> IRC #holser
>>
>> On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <
>> akostri...@mirantis.com> wrote:
>>
>>> Hello, everybody!
>>> Hello, Alex!
>>> >I thought Rally was more for benchmarking.  Wouldn't Tempest make more
>>> sense?
>>> Rally is a good tool with nice api/usage/extensibility.
>>> I really liked "up and running tests in 5 minutes" in Rally with clear
>>> picture of what I am doing.
>>> So, I 100% for a Rally as a QA.
>>>
>>> Another note:
>>> We will need to implement some HA tests, probably not in Rally.
>>>
>>> On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin 
>>> wrote:
>>>


 On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky <
 ikalnit...@mirantis.com> wrote:

>
> > On Jun 27, 2016, at 16:23, Alex Schultz 
> wrote:
> >
> > I thought Rally was more for benchmarking.  Wouldn't Tempest make
> more sense?
>
> According to Rally wiki page [1], it seems they have a verification
> layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite
> our scenarios for Tempest?
>
>
 Rally consists of two main components: Rally Task and Rally
 Verification. They are totally separated.
 Task component is fully pluggable and you can run there whatever you
 want in whatever you want way.
 Verification component is hardcoded now. It was designed for
 managing(install, configure) and launching(execute and store results)
 Tempest. But we have a spec to make this component pluggable too.


> - igor
>
>
> [1] https://wiki.openstack.org/wiki/Rally
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



 --
 Best regards,
 Andrey Kurilin.


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> --
>>>
>>> Kind Regards,
>>>
>>> Alexandr Kostrikov,
>>>
>>> Mirantis, Inc.
>>>
>>> 35b/3, Vorontsovskaya St., 109147, Moscow, Russia
>>>
>>>
>>> Tel.: +7 (495) 640-49-04
>>> Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79>
>>>
>>> Skype: akostrikov_mirantis
>>>
>>> E-mail: akostri...@mirantis.com 
>>>
>>> *www.mirantis.com *
>>> *www.mirantis.r

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Vladimir Kuklin 
+1 to initial suggestion, but I guess we need to have a full feature
equality (e.g. HA tests for mysql and rabbitmq replication) before
switching to Rally.

On Mon, Jun 27, 2016 at 6:17 PM, Sergii Golovatiuk  wrote:

> Hi,
>
> Before switching Fuel from ostf to rally, I would like to see feature
> parity comparison. It's very necessary to understand how much work we need
> to spend to rewrite all our tests in rally way.
>
>
>
> --
> Best regards,
> Sergii Golovatiuk,
> Skype #golserge
> IRC #holser
>
> On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <
> akostri...@mirantis.com> wrote:
>
>> Hello, everybody!
>> Hello, Alex!
>> >I thought Rally was more for benchmarking.  Wouldn't Tempest make more
>> sense?
>> Rally is a good tool with nice api/usage/extensibility.
>> I really liked "up and running tests in 5 minutes" in Rally with clear
>> picture of what I am doing.
>> So, I 100% for a Rally as a QA.
>>
>> Another note:
>> We will need to implement some HA tests, probably not in Rally.
>>
>> On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin 
>> wrote:
>>
>>>
>>>
>>> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky >> > wrote:
>>>

 > On Jun 27, 2016, at 16:23, Alex Schultz 
 wrote:
 >
 > I thought Rally was more for benchmarking.  Wouldn't Tempest make
 more sense?

 According to Rally wiki page [1], it seems they have a verification
 layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite
 our scenarios for Tempest?


>>> Rally consists of two main components: Rally Task and Rally
>>> Verification. They are totally separated.
>>> Task component is fully pluggable and you can run there whatever you
>>> want in whatever you want way.
>>> Verification component is hardcoded now. It was designed for
>>> managing(install, configure) and launching(execute and store results)
>>> Tempest. But we have a spec to make this component pluggable too.
>>>
>>>
 - igor


 [1] https://wiki.openstack.org/wiki/Rally

 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>>
>>>
>>>
>>> --
>>> Best regards,
>>> Andrey Kurilin.
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>>
>> Kind Regards,
>>
>> Alexandr Kostrikov,
>>
>> Mirantis, Inc.
>>
>> 35b/3, Vorontsovskaya St., 109147, Moscow, Russia
>>
>>
>> Tel.: +7 (495) 640-49-04
>> Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79>
>>
>> Skype: akostrikov_mirantis
>>
>> E-mail: akostri...@mirantis.com 
>>
>> *www.mirantis.com *
>> *www.mirantis.ru *
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Yours Faithfully,
Vladimir Kuklin,
Fuel Library Tech Lead,
Mirantis, Inc.
+7 (495) 640-49-04
+7 (926) 702-39-68
Skype kuklinvv
35bk3, Vorontsovskaya Str.
Moscow, Russia,
www.mirantis.com 
www.mirantis.ru
vkuk...@mirantis.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Sergii Golovatiuk 
Hi,

Before switching Fuel from ostf to rally, I would like to see feature
parity comparison. It's very necessary to understand how much work we need
to spend to rewrite all our tests in rally way.



--
Best regards,
Sergii Golovatiuk,
Skype #golserge
IRC #holser

On Mon, Jun 27, 2016 at 4:32 PM, Alexander Kostrikov <
akostri...@mirantis.com> wrote:

> Hello, everybody!
> Hello, Alex!
> >I thought Rally was more for benchmarking.  Wouldn't Tempest make more
> sense?
> Rally is a good tool with nice api/usage/extensibility.
> I really liked "up and running tests in 5 minutes" in Rally with clear
> picture of what I am doing.
> So, I 100% for a Rally as a QA.
>
> Another note:
> We will need to implement some HA tests, probably not in Rally.
>
> On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin 
> wrote:
>
>>
>>
>> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky 
>> wrote:
>>
>>>
>>> > On Jun 27, 2016, at 16:23, Alex Schultz  wrote:
>>> >
>>> > I thought Rally was more for benchmarking.  Wouldn't Tempest make more
>>> sense?
>>>
>>> According to Rally wiki page [1], it seems they have a verification
>>> layer (Tempest so far). Hm, I wonder does it mean we will need to rewrite
>>> our scenarios for Tempest?
>>>
>>>
>> Rally consists of two main components: Rally Task and Rally Verification.
>> They are totally separated.
>> Task component is fully pluggable and you can run there whatever you want
>> in whatever you want way.
>> Verification component is hardcoded now. It was designed for
>> managing(install, configure) and launching(execute and store results)
>> Tempest. But we have a spec to make this component pluggable too.
>>
>>
>>> - igor
>>>
>>>
>>> [1] https://wiki.openstack.org/wiki/Rally
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>>
>>
>> --
>> Best regards,
>> Andrey Kurilin.
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
>
> Kind Regards,
>
> Alexandr Kostrikov,
>
> Mirantis, Inc.
>
> 35b/3, Vorontsovskaya St., 109147, Moscow, Russia
>
>
> Tel.: +7 (495) 640-49-04
> Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79>
>
> Skype: akostrikov_mirantis
>
> E-mail: akostri...@mirantis.com 
>
> *www.mirantis.com *
> *www.mirantis.ru *
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Alexander Kostrikov 
Hello, everybody!
Hello, Alex!
>I thought Rally was more for benchmarking.  Wouldn't Tempest make more
sense?
Rally is a good tool with nice api/usage/extensibility.
I really liked "up and running tests in 5 minutes" in Rally with clear
picture of what I am doing.
So, I 100% for a Rally as a QA.

Another note:
We will need to implement some HA tests, probably not in Rally.

On Mon, Jun 27, 2016 at 4:57 PM, Andrey Kurilin 
wrote:

>
>
> On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky 
> wrote:
>
>>
>> > On Jun 27, 2016, at 16:23, Alex Schultz  wrote:
>> >
>> > I thought Rally was more for benchmarking.  Wouldn't Tempest make more
>> sense?
>>
>> According to Rally wiki page [1], it seems they have a verification layer
>> (Tempest so far). Hm, I wonder does it mean we will need to rewrite our
>> scenarios for Tempest?
>>
>>
> Rally consists of two main components: Rally Task and Rally Verification.
> They are totally separated.
> Task component is fully pluggable and you can run there whatever you want
> in whatever you want way.
> Verification component is hardcoded now. It was designed for
> managing(install, configure) and launching(execute and store results)
> Tempest. But we have a spec to make this component pluggable too.
>
>
>> - igor
>>
>>
>> [1] https://wiki.openstack.org/wiki/Rally
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> --
> Best regards,
> Andrey Kurilin.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 

Kind Regards,

Alexandr Kostrikov,

Mirantis, Inc.

35b/3, Vorontsovskaya St., 109147, Moscow, Russia


Tel.: +7 (495) 640-49-04
Tel.: +7 (925) 716-64-52 <%2B7%20%28906%29%20740-64-79>

Skype: akostrikov_mirantis

E-mail: akostri...@mirantis.com 

*www.mirantis.com *
*www.mirantis.ru *
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Igor Kalnitsky 
> On Jun 27, 2016, at 16:57, Andrey Kurilin  wrote:
> 
> Rally consists of two main components: Rally Task and Rally Verification. 
> They are totally separated.
> Task component is fully pluggable and you can run there whatever you want in 
> whatever you want way.

Andrey, thanks for explanation!

I wonder what are the benefits of using Rally then? We can run whatever we want 
by means of MCollective or Ansible. Is there something that could help us? I 
don't know, maybe some dashboard integration with per test results or running 
by tests by tag?

Besides, we don't run some tests if deployed configuration doesn't assume it. 
For instance, no need to run ceilometer test (and even show one) if we deploy 
env without it. Is it possible to achieve it with Rally?

Thanks,
Igor
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Andrey Kurilin 
On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky 
wrote:

>
> > On Jun 27, 2016, at 16:23, Alex Schultz  wrote:
> >
> > I thought Rally was more for benchmarking.  Wouldn't Tempest make more
> sense?
>
> According to Rally wiki page [1], it seems they have a verification layer
> (Tempest so far). Hm, I wonder does it mean we will need to rewrite our
> scenarios for Tempest?
>
>
Rally consists of two main components: Rally Task and Rally Verification.
They are totally separated.
Task component is fully pluggable and you can run there whatever you want
in whatever you want way.
Verification component is hardcoded now. It was designed for
managing(install, configure) and launching(execute and store results)
Tempest. But we have a spec to make this component pluggable too.


> - igor
>
>
> [1] https://wiki.openstack.org/wiki/Rally
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Best regards,
Andrey Kurilin.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Vladimir Kozhukalov 
As far as I understand, Rally is pluggable and one can implement any
scenario we need in pure Python (totally out of Tempest). See for example
[1].


[1] http://rally.readthedocs.io/en/latest/plugins/scenario_plugin.html

Vladimir Kozhukalov

On Mon, Jun 27, 2016 at 4:46 PM, Igor Kalnitsky 
wrote:

>
> > On Jun 27, 2016, at 16:23, Alex Schultz  wrote:
> >
> > I thought Rally was more for benchmarking.  Wouldn't Tempest make more
> sense?
>
> According to Rally wiki page [1], it seems they have a verification layer
> (Tempest so far). Hm, I wonder does it mean we will need to rewrite our
> scenarios for Tempest?
>
> - igor
>
>
> [1] https://wiki.openstack.org/wiki/Rally
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Igor Kalnitsky 

> On Jun 27, 2016, at 16:23, Alex Schultz  wrote:
> 
> I thought Rally was more for benchmarking.  Wouldn't Tempest make more sense? 
>  

According to Rally wiki page [1], it seems they have a verification layer 
(Tempest so far). Hm, I wonder does it mean we will need to rewrite our 
scenarios for Tempest? 

- igor


[1] https://wiki.openstack.org/wiki/Rally
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Andrey Kurilin 
Hi!

On Mon, Jun 27, 2016 at 4:23 PM, Alex Schultz  wrote:

>
>
> On Mon, Jun 27, 2016 at 7:10 AM, Vladimir Kozhukalov <
> vkozhuka...@mirantis.com> wrote:
>
>> Dear colleagues,
>>
>> I'd like to suggest to replace Fuel-ostf with Rally. Rally is quite
>> popular project and as far as I know it has all necessary features
>> (including dashboard). We only need to implement testing scenarios.
>>
>> In particular the suggestion is as follows
>>
>> 1) Implement necessary testing scenarios to achieve feature parity with
>> Fuel-ostf (including those which test Fuel HA features).
>> 2) Prepare necessary rpm/deb packages (Rally itself + scenarios)
>> 3) Modify Fuel-qa so it uses Rally instead of Fuel-ostf.
>> 4) Deprecate Fuel-ostf (including removal of ostf tab on UI). I'd prefer
>> Fuel users to rely on Rally user interface (both CLI and dashboard).
>>
>> What do you think? Are there any volunteers to implement this?
>>
>>
> I thought Rally was more for benchmarking.  Wouldn't Tempest make more
> sense?
>
>
Rally is not only about benchmarking, it provides a flexible interface
which allows to do whatever you want:)


> -Alex
>
>
>>
>> Vladimir Kozhukalov
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Best regards,
Andrey Kurilin.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Alex Schultz 
On Mon, Jun 27, 2016 at 7:10 AM, Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> Dear colleagues,
>
> I'd like to suggest to replace Fuel-ostf with Rally. Rally is quite
> popular project and as far as I know it has all necessary features
> (including dashboard). We only need to implement testing scenarios.
>
> In particular the suggestion is as follows
>
> 1) Implement necessary testing scenarios to achieve feature parity with
> Fuel-ostf (including those which test Fuel HA features).
> 2) Prepare necessary rpm/deb packages (Rally itself + scenarios)
> 3) Modify Fuel-qa so it uses Rally instead of Fuel-ostf.
> 4) Deprecate Fuel-ostf (including removal of ostf tab on UI). I'd prefer
> Fuel users to rely on Rally user interface (both CLI and dashboard).
>
> What do you think? Are there any volunteers to implement this?
>
>
I thought Rally was more for benchmarking.  Wouldn't Tempest make more
sense?

-Alex


>
> Vladimir Kozhukalov
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Fuel] Replace OSTF with Rally

2016-06-27 Thread Vladimir Kozhukalov 
Dear colleagues,

I'd like to suggest to replace Fuel-ostf with Rally. Rally is quite popular
project and as far as I know it has all necessary features (including
dashboard). We only need to implement testing scenarios.

In particular the suggestion is as follows

1) Implement necessary testing scenarios to achieve feature parity with
Fuel-ostf (including those which test Fuel HA features).
2) Prepare necessary rpm/deb packages (Rally itself + scenarios)
3) Modify Fuel-qa so it uses Rally instead of Fuel-ostf.
4) Deprecate Fuel-ostf (including removal of ostf tab on UI). I'd prefer
Fuel users to rely on Rally user interface (both CLI and dashboard).

What do you think? Are there any volunteers to implement this?


Vladimir Kozhukalov
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev