Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Sam P]

Hi Boris,

 Thanks for comment.
 Sorry for the misleading context in the etherpad. It sounds like Eris
will do everything, but it is not.
 I think I need to add more details to etherpad (I will do that)
 Short answer is we are not into  "reinventing wheels" and we will use
all possible existing tools.
 # Me and Gautam are working on a document about what are the gaps,
and what tool should use on what purpose and why.
 # I think that would be the long answer and  we definitely need your
feed back on this.
 #  I will share that here..

 For, Rally,
  Actually we are using Rally in Eris. Here is the PoC we are working on (WIP).
 # At this point, PoC is in it's very early stage. Not all the
Concepts are implemented yet..
  code: https://github.com/gautamdivgi/eris
  doc: 
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/22872097/Extreme+Testing+Demo

 You may find lot more doc's about Eris in here,
 
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/13393034/Eris+-+Extreme+Testing+Framework+for+OpenStack

 For an example, we are considering,
 Rally: For control plane load injection (Current PoC we don't use Rally
 Shaker: For data plane load injection
 os-fault: fault injection ( lot of work need to be done here)

 Here is a one example:
 Eris is focus on realistic outage scenarios such as, SDN controller
failure, Storage back-end failure, infra L2/L3 SW failure ,etc,,
 If those are vendor specific appliances, then how should we do
failure injection and metrics gathering?
 Eris will provide plugin mechanism to plug vendor drivers ( provide
by the vendor).
 In testing, Eris will use Rally and Shaker for load generation and
inject deterministic/random HW/SW failures in those
 appliances and gather the metrics. Then,  Eris (or Rally or any other
tool or mix of them) can analyze the result and
 create the final report.

 Sorry for ranting.

--- Regards,
Sampath



On Wed, Nov 1, 2017 at 4:02 PM, Boris Pavlovic <bo...@pavlovic.me> wrote:
> Sam,
>
>>  Etherpad: https://etherpad.openstack.org/p/SYD-extreme-testing
>
>
> I really don't want to sound like a person that say use Rally my best ever
> project blablbab and other BS.
> I think that "reinventing wheels" approach is how humanity evolves and
> that's why I like this effort in any case.
>
> But really, I read carefully etherpad and I really see in description of
> Eris just plain Rally as is:
>
> - Rally allows you to create tests as YAML
> - Rally allows you to inject in various actions during the load (Rally
> Hooks) which makes it easy to do chaos and other kind of testing
> - Rally is pluggable and you can write even your own Runners (scenario
> executors) that will generate load pattern that you need
> - Rally has SLAs plugins (that can deeply analyze result of test cases) and
> say whatever they pass or not
> - We are working on feature that allows you to mix different workloads in
> parallel (and generate more realistic load)
> -.
>
> So it would be really nice if you can share gaps that you faced that are
> blocking you to use directly Rally..
>
> Thanks!
>
> Best regards,
> Boris Pavlovic
>
>
> On Tue, Oct 31, 2017 at 10:50 PM, Sam P <sam47pr...@gmail.com> wrote:
>>
>> Hi All,
>>
>>  Sending out a gentle reminder of Sydney Summit Forum Session
>> regarding this topic.
>>
>>  Extreme/Destructive Testing
>>  Tuesday, November 7, 1:50pm-2:30pm
>>  Sydney Convention and Exhibition Centre - Level 4 - C4.11
>>
>> [https://www.openstack.org/summit/sydney-2017/summit-schedule/events/20470/extremedestructive-testing]
>>  Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing
>>
>>  Your participation in this session would be greatly appreciated.
>> --- Regards,
>> Sampath
>>
>>
>>
>> On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell <tim.b...@cern.ch> wrote:
>> > +1 for Boris’ suggestion. Many of us use Rally to probe our clouds and
>> > have
>> > significant tooling behind it to integrate with local availability
>> > reporting
>> > and trouble ticketing systems. It would be much easier to deploy new
>> > functionality such as you propose if it was integrated into an existing
>> > project framework (such as Rally).
>> >
>> >
>> >
>> > Tim
>> >
>> >
>> >
>> > From: Boris Pavlovic <bo...@pavlovic.me>
>> > Reply-To: "OpenStack Development Mailing List (not for usage questions)"
>> > <openstack-dev@lists.openstack.org>
>> > Date: Monday, 14 August 2017 at 12:57
>> > To: "OpenStack Development Mailing List (not for usage questions)"
>> > <openstack-dev@lists.openst

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Boris Pavlovic]

Sam,

 Etherpad: https://etherpad.openstack.org/p/SYD-extreme-testing


I really don't want to sound like a person that say use Rally my best ever
project blablbab and other BS.
I think that "reinventing wheels" approach is how humanity evolves and
that's why I like this effort in any case.

But really, I read carefully etherpad and I really see in description of
Eris just plain Rally as is:

- Rally allows you to create tests as YAML
- Rally allows you to inject in various actions during the load (Rally
Hooks) which makes it easy to do chaos and other kind of testing
- Rally is pluggable and you can write even your own Runners (scenario
executors) that will generate load pattern that you need
- Rally has SLAs plugins (that can deeply analyze result of test cases) and
say whatever they pass or not
- We are working on feature that allows you to mix different workloads in
parallel (and generate more realistic load)
-.

So it would be really nice if you can share gaps that you faced that are
blocking you to use directly Rally..

Thanks!

Best regards,
Boris Pavlovic


On Tue, Oct 31, 2017 at 10:50 PM, Sam P <sam47pr...@gmail.com> wrote:

> Hi All,
>
>  Sending out a gentle reminder of Sydney Summit Forum Session
> regarding this topic.
>
>  Extreme/Destructive Testing
>  Tuesday, November 7, 1:50pm-2:30pm
>  Sydney Convention and Exhibition Centre - Level 4 - C4.11
>  [https://www.openstack.org/summit/sydney-2017/summit-
> schedule/events/20470/extremedestructive-testing]
>  Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing
>
>  Your participation in this session would be greatly appreciated.
> --- Regards,
> Sampath
>
>
>
> On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell <tim.b...@cern.ch> wrote:
> > +1 for Boris’ suggestion. Many of us use Rally to probe our clouds and
> have
> > significant tooling behind it to integrate with local availability
> reporting
> > and trouble ticketing systems. It would be much easier to deploy new
> > functionality such as you propose if it was integrated into an existing
> > project framework (such as Rally).
> >
> >
> >
> > Tim
> >
> >
> >
> > From: Boris Pavlovic <bo...@pavlovic.me>
> > Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Date: Monday, 14 August 2017 at 12:57
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Cc: openstack-operators <openstack-operat...@lists.openstack.org>
> > Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme
> > Testing
> >
> >
> >
> > Sam,
> >
> >
> >
> > Seems like a good plan and huge topic ;)
> >
> >
> >
> > I would as well suggest to take a look at the similar efforts in
> OpenStack:
> >
> > - Failure injection: https://github.com/openstack/os-faults
> >
> > - Rally Hooks Mechanism (to inject in rally scenarios failures):
> > https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_
> trigger_plugins.html
> >
> >
> >
> >
> >
> > Best regards,
> > Boris Pavlovic
> >
> >
> >
> >
> >
> > On Mon, Aug 14, 2017 at 2:35 AM, Sam P <sam47pr...@gmail.com> wrote:
> >
> > Hi All,
> >
> > This is a follow up for OpenStack Extreme Testing session[1]
> > we did in MEX-ops-meetup.
> >
> > Quick intro for those who were not there:
> > In this work, we proposed to add new testing framework for openstack.
> > This framework will provides tool for create tests with destructive
> > scenarios which will check for High Availability, failover and
> > recovery of OpenStack cloud.
> > Please refer the link on top of the [1] for further details.
> >
> > Follow up:
> > We are planning periodic irc meeting and have an irc
> > channel for discussion. I will get back to you with those details soon.
> >
> > At that session, we did not have time to discuss last 3 items,
> > Reference architectures
> >  We are discussing about the reference architecture in [2].
> >
> > What sort of failures do you see today in your environment?
> >  Currently we are considering, service failures, backend services (mq,
> > DB, etc.) failures,
> >  Network sw failures..etc. To begin with the implementation, we are
> > considering to start with
> >  service failures. Please let us know what failures are more frequent
> > in your environment.
> >
> > Emulation/Simulation mechanisms, etc.
> 

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Sam P]

Hi All,

 Sending out a gentle reminder of Sydney Summit Forum Session
regarding this topic.

 Extreme/Destructive Testing
 Tuesday, November 7, 1:50pm-2:30pm
 Sydney Convention and Exhibition Centre - Level 4 - C4.11
 
[https://www.openstack.org/summit/sydney-2017/summit-schedule/events/20470/extremedestructive-testing]
 Eatherpad: https://etherpad.openstack.org/p/SYD-extreme-testing

 Your participation in this session would be greatly appreciated.
--- Regards,
Sampath



On Mon, Aug 14, 2017 at 11:43 PM, Tim Bell <tim.b...@cern.ch> wrote:
> +1 for Boris’ suggestion. Many of us use Rally to probe our clouds and have
> significant tooling behind it to integrate with local availability reporting
> and trouble ticketing systems. It would be much easier to deploy new
> functionality such as you propose if it was integrated into an existing
> project framework (such as Rally).
>
>
>
> Tim
>
>
>
> From: Boris Pavlovic <bo...@pavlovic.me>
> Reply-To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Date: Monday, 14 August 2017 at 12:57
> To: "OpenStack Development Mailing List (not for usage questions)"
> <openstack-dev@lists.openstack.org>
> Cc: openstack-operators <openstack-operat...@lists.openstack.org>
> Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme
> Testing
>
>
>
> Sam,
>
>
>
> Seems like a good plan and huge topic ;)
>
>
>
> I would as well suggest to take a look at the similar efforts in OpenStack:
>
> - Failure injection: https://github.com/openstack/os-faults
>
> - Rally Hooks Mechanism (to inject in rally scenarios failures):
> https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html
>
>
>
>
>
> Best regards,
> Boris Pavlovic
>
>
>
>
>
> On Mon, Aug 14, 2017 at 2:35 AM, Sam P <sam47pr...@gmail.com> wrote:
>
> Hi All,
>
> This is a follow up for OpenStack Extreme Testing session[1]
> we did in MEX-ops-meetup.
>
> Quick intro for those who were not there:
> In this work, we proposed to add new testing framework for openstack.
> This framework will provides tool for create tests with destructive
> scenarios which will check for High Availability, failover and
> recovery of OpenStack cloud.
> Please refer the link on top of the [1] for further details.
>
> Follow up:
> We are planning periodic irc meeting and have an irc
> channel for discussion. I will get back to you with those details soon.
>
> At that session, we did not have time to discuss last 3 items,
> Reference architectures
>  We are discussing about the reference architecture in [2].
>
> What sort of failures do you see today in your environment?
>  Currently we are considering, service failures, backend services (mq,
> DB, etc.) failures,
>  Network sw failures..etc. To begin with the implementation, we are
> considering to start with
>  service failures. Please let us know what failures are more frequent
> in your environment.
>
> Emulation/Simulation mechanisms, etc.
>  Rather than doing actual scale, load, or performance tests, we are
> thinking to build a emulation/simulation mechanism
> to get the predictions or result of how will openstack behave on such
> situations.
> This interesting idea was proposed by the Gautam and need more
> discussion on this.
>
> Please let us know you questions or comments.
>
> Request to Mike Perez:
>  We discussed about synergies with openstack assertion tags and other
> efforts to do similar testing in openstack.
>  Could you please give some info or pointer of previous discussions.
>
> [1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
> [2]
> https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch
>
> --- Regards,
> Sampath
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Tim Bell]

+1 for Boris’ suggestion. Many of us use Rally to probe our clouds and have 
significant tooling behind it to integrate with local availability reporting 
and trouble ticketing systems. It would be much easier to deploy new 
functionality such as you propose if it was integrated into an existing project 
framework (such as Rally).

Tim

From: Boris Pavlovic <bo...@pavlovic.me>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Date: Monday, 14 August 2017 at 12:57
To: "OpenStack Development Mailing List (not for usage questions)" 
<openstack-dev@lists.openstack.org>
Cc: openstack-operators <openstack-operat...@lists.openstack.org>
Subject: Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme 
Testing

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:
- Failure injection: https://github.com/openstack/os-faults
- Rally Hooks Mechanism (to inject in rally scenarios failures): 
https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html


Best regards,
Boris Pavlovic


On Mon, Aug 14, 2017 at 2:35 AM, Sam P 
<sam47pr...@gmail.com<mailto:sam47pr...@gmail.com>> wrote:
Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
 We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
 Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
 Network sw failures..etc. To begin with the implementation, we are
considering to start with
 service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
 Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
 We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
 Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2] 
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Ilya Shakhat]

2017-08-14 13:41 GMT+02:00 Ghanshyam Mann :

> On Mon, Aug 14, 2017 at 7:56 PM, Boris Pavlovic  wrote:
> > Sam,
> >
> > Seems like a good plan and huge topic ;)
> >
> > I would as well suggest to take a look at the similar efforts in
> OpenStack:
> > - Failure injection: https://github.com/openstack/os-faults
>
> Yea, we are considering this lib, more details in this spec -
> https://review.openstack.org/#/c/443504/
>
> - Rally Hooks Mechanism (to inject in rally scenarios failures):
> > https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_
> trigger_plugins.html
> >
>

As a little demonstration of Rally and os-faults working together:
OpenStack reliability tests from performance documentation:
 * test plan:
https://docs.openstack.org/performance-docs/latest/test_plans/reliability/version_2/plan.html#reliability-testing-version-2
 * example of report:
https://docs.openstack.org/performance-docs/latest/test_results/reliability/version_2/reports/neutron/create_and_list_networks_with_kill_mysql_service_on_one_node/index.html

Best regards,
Ilya Shakhat
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Ghanshyam Mann]

On Mon, Aug 14, 2017 at 7:56 PM, Boris Pavlovic  wrote:
> Sam,
>
> Seems like a good plan and huge topic ;)
>
> I would as well suggest to take a look at the similar efforts in OpenStack:
> - Failure injection: https://github.com/openstack/os-faults

Yea, we are considering this lib, more details in this spec -
https://review.openstack.org/#/c/443504/

> - Rally Hooks Mechanism (to inject in rally scenarios failures):
> https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html
>
>
> Best regards,
> Boris Pavlovic
>
>
> On Mon, Aug 14, 2017 at 2:35 AM, Sam P  wrote:
>>
>> Hi All,
>>
>> This is a follow up for OpenStack Extreme Testing session[1]
>> we did in MEX-ops-meetup.
>>
>> Quick intro for those who were not there:
>> In this work, we proposed to add new testing framework for openstack.
>> This framework will provides tool for create tests with destructive
>> scenarios which will check for High Availability, failover and
>> recovery of OpenStack cloud.
>> Please refer the link on top of the [1] for further details.
>>
>> Follow up:
>> We are planning periodic irc meeting and have an irc
>> channel for discussion. I will get back to you with those details soon.

Thanks SamP for updates.
Or another option is use qa channel itself as it is going to be under
qa projects. Similar to patrole etc.

Anyways I have added this in QA PTG agenda [1] to discuss the
architecture and how we can help on this to speedup this.

>>
>> At that session, we did not have time to discuss last 3 items,
>> Reference architectures
>>  We are discussing about the reference architecture in [2].
>>
>> What sort of failures do you see today in your environment?
>>  Currently we are considering, service failures, backend services (mq,
>> DB, etc.) failures,
>>  Network sw failures..etc. To begin with the implementation, we are
>> considering to start with
>>  service failures. Please let us know what failures are more frequent
>> in your environment.
>>
>> Emulation/Simulation mechanisms, etc.
>>  Rather than doing actual scale, load, or performance tests, we are
>> thinking to build a emulation/simulation mechanism
>> to get the predictions or result of how will openstack behave on such
>> situations.
>> This interesting idea was proposed by the Gautam and need more
>> discussion on this.
>>
>> Please let us know you questions or comments.
>>
>> Request to Mike Perez:
>>  We discussed about synergies with openstack assertion tags and other
>> efforts to do similar testing in openstack.
>>  Could you please give some info or pointer of previous discussions.
>>
>> [1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
>> [2]
>> https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch
>>
>> --- Regards,
>> Sampath
>>

..1 https://etherpad.openstack.org/p/qa-queens-ptg

-gmann


>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Boris Pavlovic]

Sam,

Seems like a good plan and huge topic ;)

I would as well suggest to take a look at the similar efforts in OpenStack:
- Failure injection: https://github.com/openstack/os-faults
- Rally Hooks Mechanism (to inject in rally scenarios failures):
https://rally.readthedocs.io/en/latest/plugins/implementation/hook_and_trigger_plugins.html


Best regards,
Boris Pavlovic


On Mon, Aug 14, 2017 at 2:35 AM, Sam P  wrote:

> Hi All,
>
> This is a follow up for OpenStack Extreme Testing session[1]
> we did in MEX-ops-meetup.
>
> Quick intro for those who were not there:
> In this work, we proposed to add new testing framework for openstack.
> This framework will provides tool for create tests with destructive
> scenarios which will check for High Availability, failover and
> recovery of OpenStack cloud.
> Please refer the link on top of the [1] for further details.
>
> Follow up:
> We are planning periodic irc meeting and have an irc
> channel for discussion. I will get back to you with those details soon.
>
> At that session, we did not have time to discuss last 3 items,
> Reference architectures
>  We are discussing about the reference architecture in [2].
>
> What sort of failures do you see today in your environment?
>  Currently we are considering, service failures, backend services (mq,
> DB, etc.) failures,
>  Network sw failures..etc. To begin with the implementation, we are
> considering to start with
>  service failures. Please let us know what failures are more frequent
> in your environment.
>
> Emulation/Simulation mechanisms, etc.
>  Rather than doing actual scale, load, or performance tests, we are
> thinking to build a emulation/simulation mechanism
> to get the predictions or result of how will openstack behave on such
> situations.
> This interesting idea was proposed by the Gautam and need more
> discussion on this.
>
> Please let us know you questions or comments.
>
> Request to Mike Perez:
>  We discussed about synergies with openstack assertion tags and other
> efforts to do similar testing in openstack.
>  Could you please give some info or pointer of previous discussions.
>
> [1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
> [2] https://openstack-lcoo.atlassian.net/wiki/spaces/
> LCOO/pages/15477787/Extreme+Testing-Vision+Arch
>
> --- Regards,
> Sampath
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA][LCOO] MEX-ops-meetup: OpenStack Extreme Testing

[Sam P]

Hi All,

This is a follow up for OpenStack Extreme Testing session[1]
we did in MEX-ops-meetup.

Quick intro for those who were not there:
In this work, we proposed to add new testing framework for openstack.
This framework will provides tool for create tests with destructive
scenarios which will check for High Availability, failover and
recovery of OpenStack cloud.
Please refer the link on top of the [1] for further details.

Follow up:
We are planning periodic irc meeting and have an irc
channel for discussion. I will get back to you with those details soon.

At that session, we did not have time to discuss last 3 items,
Reference architectures
 We are discussing about the reference architecture in [2].

What sort of failures do you see today in your environment?
 Currently we are considering, service failures, backend services (mq,
DB, etc.) failures,
 Network sw failures..etc. To begin with the implementation, we are
considering to start with
 service failures. Please let us know what failures are more frequent
in your environment.

Emulation/Simulation mechanisms, etc.
 Rather than doing actual scale, load, or performance tests, we are
thinking to build a emulation/simulation mechanism
to get the predictions or result of how will openstack behave on such
situations.
This interesting idea was proposed by the Gautam and need more
discussion on this.

Please let us know you questions or comments.

Request to Mike Perez:
 We discussed about synergies with openstack assertion tags and other
efforts to do similar testing in openstack.
 Could you please give some info or pointer of previous discussions.

[1] https://etherpad.openstack.org/p/MEX-ops-extreme-testing
[2] 
https://openstack-lcoo.atlassian.net/wiki/spaces/LCOO/pages/15477787/Extreme+Testing-Vision+Arch

--- Regards,
Sampath

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev