Re: [openstack-dev] [tc][all] A culture change (nitpicking)
I think this is not only for code but also for doc and reno. We should fix it basically, especially about doc/reno. But I don't think it should be in the same patch if the mistake isn't critical which means *nitpicks*. I think we can fix them with following patches if we need to fix it. Otherwise, writing reno/doc might be a really difficult and hard work for ESL people like me. -- Masayuki On 05/30, Ghanshyam Mann wrote: > Thanks for making it formal process which really helps. I think most > of the people usually does that but yes it is always helpful to be > added as principles. > > I have gotten mix feedback on fixing other patches in past and when i > got anger by author i try to leave comment for a day or two then fix > if it is really needed otherwise just go with follow-up. > > One question - This only applies to code nitpick only right? not > documentation or releasenotes. For doc and reno, we should fix spell > or grammar mistake etc in same patch. > > -gmann > > > On Tue, May 29, 2018 at 10:55 PM, Julia Kreger > wrote: > > During the Forum, the topic of review culture came up in session after > > session. During these discussions, the subject of our use of nitpicks > > were often raised as a point of contention and frustration, especially > > by community members that have left the community and that were > > attempting to re-engage the community. Contributors raised the point > > of review feedback requiring for extremely precise English, or > > compliance to a particular core reviewer's style preferences, which > > may not be the same as another core reviewer. > > > > These things are not just frustrating, but also very inhibiting for > > part time contributors such as students who may also be time limited. > > Or an operator who noticed something that was clearly a bug and that > > put forth a very minor fix and doesn't have the time to revise it over > > and over. > > > > While nitpicks do help guide and teach, the consensus seemed to be > > that we do need to shift the culture a little bit. As such, I've > > proposed a change to our principles[1] in governance that attempts to > > capture the essence and spirit of the nitpicking topic as a first > > step. > > > > -Julia > > - > > [1]: https://review.openstack.org/570940 > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Happy Hacking!! Masayuki Igawa GPG fingerprint = C27C 2F00 3A2A 999A 903A 753D 290F 53ED C899 BF90 signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] Proposing Felipe Monteiro for Tempest core
+1 -- Masayuki Igawa Key fingerprint = C27C 2F00 3A2A 999A 903A 753D 290F 53ED C899 BF89 On Sat, Apr 28, 2018, at 19:27, Ghanshyam Mann wrote: > Hi Tempest Team, > > I would like to propose Felipe Monteiro (irc: felipemonteiro) to Tempest > core. > > Felipe has been an active contributor to the Tempest since the Pike > cycle. He has been doing lot of review and commits since then. Filling > the gaps on service clients side and their testing and lot other > areas. He has demonstrated the good quality and feedback while his > review. > > He has good understanding of Tempest source code and project missions > & goal. IMO his efforts are highly valuable and it will be great to > have him in team. > > > As per usual practice, please vote +1 or -1 to the nomination. I will > keep this nomination open for a week or until everyone voted. > > Felipe Reviews and Commit - > https://review.openstack.org/#/q/reviewer:felipe.monte...@att.com+project:openstack/tempest > https://review.openstack.org/#/q/owner:felipe.monte...@att.com+project:openstack/tempest > > -gmann > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] QA Office Hours 9:00 UTC is cancelled
Hi All, Today, QA Office Hours @9:00 UTC is canceled due to unavailability of members. Happy Hacking!! -- Masayuki Igawa signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][infra] PTG Infra Helproom Info and Signup
On 02/14, Andrea Frittoli wrote: > On Wed, Feb 14, 2018 at 10:42 AM Thierry Carrez> wrote: > > > Clark Boylan wrote: > > > Last PTG the infra helproom seemed to work out for projects that knew > > about it. The biggest problem seemed to be that other projects either just > > weren't aware that there is/was an Infra helproom or didn't know when an > > appropriate time to show up would be. We are going to try a couple things > > this time around to try and address those issues. > > > > > > First of all the Infra team is hosting a helproom at the Dublin PTG. Now > > you should all know :) The idea is that if projects or individuals have > > questions for the infra team or problems that we can help you with there is > > time set aside specifically for this. I'm not sure what room we will be in, > > you will have to look at the map, but we have the entirety of Monday and > > Tuesday set aside for this. > > > > Also worth noting that it is a "project infrastructure" helproom, in the > > largest sense. It goes beyond the "Infra" team: you can bring any > > question around project support from horizontal support teams like QA, > > > > Indeed, thanks for pointing that out. > > A lot of us from the QA team will be in Dublin, available during the help > ours for questions or topics you may want to discuss. > There's usually enough time to sit down and hack a few things on the > spot... and there are enough infra/qa cores around to get things reviewed > and merged during the week. > > On the QA side we don't have an ethercalc (yet?) but if there are topics > you would like to discuss / develop please add something to the etherpad. > > Andrea Frittoli (andreaf) > > [1] https://etherpad.openstack.org/p/qa-rocky-ptg Yeah, actually, I was thinking to have a dedicated session for stestr Q if someone wants it. Does anybody want to join the stestr Q session? Or, we can talk about it during the PTG anytime :) -- Masayuki > > > > release management, requirements, stable team... > > > > -- > > Thierry Carrez (ttx) > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] [tc] Need champion for "cold upgrades capabilities" goal
Hi, Thank you so much for your warm comments/links/etc,etc! I'll check them and start to write a proposal for the goal for Rocky early next week. -- Masayuki On 01/11, Emilien Macchi wrote: > Sorry I forgot an important action: propose the goal for Rocky :-) > > An example with https://review.openstack.org/#/c/532361/ (from Sean). > > Thanks, > > On Thu, Jan 11, 2018 at 9:29 PM, Emilien Macchi <emil...@redhat.com> wrote: > > On Thu, Jan 11, 2018 at 7:13 PM, Masayuki Igawa > > <masayuki.ig...@gmail.com> wrote: > >> Hi Emilien, > >> > >> I'd love to take this role! > > > > Wow, this is AWESOME. > > > >> I have some tempest experience but not QA work which you mentioned > >> (devstack, grenade, zuul layout) that much. However, I think it will > >> be a very good opportunity to step-up. > >> > >> So, for me, the next step might be to know grenade / upgrade jobs / > >> plugin mechanizm deeply. And I suppose we need some documentation > >> about "how to support upgrade" based on the requirements for the other > >> projects (and me :). > >> > >> Any suggestion and/or comments are welcome! > > > > OK so the steps are documented here: > > https://governance.openstack.org/tc/reference/tags/assert_supports-upgrade.html#requirements > > > > So first of all, it only affects projects that are "services" (e.g. Tacker). > > The main requirement is to have grenade scripts in-tree for the > > projects which miss the tag now. > > Look at Ironic which already has the tag: > > https://github.com/openstack/ironic/tree/master/devstack/upgrade > > > > At the same time, we need to change the zuul layout for the projects > > which don't have the tag yet. > > Still example with Ironic: > > https://github.com/openstack/ironic/blob/master/zuul.d/project.yaml#L6-L7 > > > > Once a project has a grenade job (voting) that runs Grenade with the > > specs described here: > > https://governance.openstack.org/tc/reference/tags/assert_supports-upgrade.html#requirements > > - the project can apply to the tag and the goal is complete once the > > tag is approved. > > > > I think the first step is to make sure each project which doesn't have > > the tag has to understand what the tag means, in term of requirements. > > > > If they say: > > > > "Yeah, our project can already do that" (because they did some manual > > testing etc): then the work is purely grenade/zuul-layout. > > > > "No, we don't support upgrades at all": then the goal might take one > > or two cycles, depending on the amount of work. > > > > This is the list of service projects that don't have the tag yet: > > aodh > > blazar > > cloudkitty > > congress > > dragonflow > > ec2api > > freezer > > karbor > > kuryr > > masakari > > mistral > > monasca > > murano > > octavia > > panko > > searchlight > > senlin > > tacker > > trove > > vitrage > > watcher > > zaqar > > zun > > > > (I might have miss some, please fix it). > > > > That's it for now, I hope it helped, please let us know if more questions. > > Thanks again for stepping up and you have all our support at any time. > > > > [...] > > -- > > Emilien Macchi > > > > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] [tc] Need champion for "cold upgrades capabilities" goal
Hi Emilien, I'd love to take this role! I have some tempest experience but not QA work which you mentioned (devstack, grenade, zuul layout) that much. However, I think it will be a very good opportunity to step-up. So, for me, the next step might be to know grenade / upgrade jobs / plugin mechanizm deeply. And I suppose we need some documentation about "how to support upgrade" based on the requirements for the other projects (and me :). Any suggestion and/or comments are welcome! -- Masayuki On 01/11, Emilien Macchi wrote: > Some projects are still not testing cold upgrades and therefore don't > have the "supports-upgrade" tag. > > https://governance.openstack.org/tc/reference/tags/assert_supports-upgrade.html > > This goal would mostly benefit the operators community as we would > continue to ensure OpenStack can be upgraded and it's something that > we actually test in the gate. > In term of actions, we would need to run grenade / upgrade jobs for > the projects which don't have this tag yet, so it's mostly QA work > (devstack, grenade, zuul layout). > > We're now looking for someone willing to lead this effort. Someone > with a little bit of experience on QA and upgrades would work. > However, our community is strong and we always help each others so no > big deal if someone volunteers without all knowledge. > A Champion is someone who coordinates the work to make a goal happen, > and not supposed to do all the work. The Champion gets support from > the whole community at any time. > > Please step-up if you're willing to take this role! > > Thanks, > -- > Emilien Macchi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Dublin PTG format
On 11/28, Ghanshyam Mann wrote: > Mon-Tue/Wed-Fri works as most suitable format. There are always > conflict for many of us but that can be adjusted by working with team > planning. > > Another thing can help is to give flexibility to team to have team > topic even on Mon-Tue, which mean team need dedicated room on Mon-Tue > also. For example, if QA want to discuss few of the topic on Mon-Tue > and have Code sprint/Help room etc on rest of week. This can help me > or other QA members to join other team discussion on Wed-Fri. But that > is based on special request and team requirement of number of topics > to discuss. > > morning/afternoon format will be little hectic and less productive due > to changing rooms and topic etc, at least for me :) Yeah, I totally agree with that. Regarding to the QA team members, most of members are not dedicated to the upstream work. So, if we can concentrate on it during the rest of 3 days, the days could be really productive. And I felt it in the past PTG. -- Masayuki > > -gmann > > > On Mon, Nov 27, 2017 at 7:58 PM, Thierry Carrezwrote: > > Hi everyone, > > > > We are in the final step in the process of signing the contract with the > > PTG venue. We should be able to announce the location this week ! > > > > So it's time to start preparing. We'll have 5 days, like in Denver. One > > thing we'd like to change for this round is to give a bit more > > flexibility in the topics being discussed, especially in the first two days. > > > > In Denver, we selected a number of general "themes" and gave them all a > > room for 2 days on Monday-Tuesday. Then all the "teams" that wanted a > > project team meeting could get a room for 2 or 3 days on > > Wednesday-Friday. That resulted in a bit of flux during the first two > > days, with a lot of empty rooms as most of the themes did not really > > need 2 days, and a lot of conflicts were present. > > > > For Dublin, the idea would be to still predetermine topics (themes and > > teams) and assign them rooms in advance. But we would be able to assign > > smaller amounts of time (per half-day) based on the expressed needs. > > Beyond those pre-assigned themes/teams we'd add flexibility for other > > groups to book the remaining available rooms in 90-min slots on-demand. > > A bit like how we did reservable rooms in the past, but more integrated > > with the rest of the event. It would all be driven by the PTGbot, which > > would show which topic is being discussed in which room, in addition to > > the current discussion subject within each topic. > > > > We have two options in how we do the split for predetermined topics. We > > used to split the week between Mon-Tue (themes) and Wed-Fri (teams). The > > general idea there was to allow some people only interested in a team > > meeting to only attend the second part of the week. However most people > > attend all 5 days, and during event feedback some people suggested that > > "themes" should be in the mornings and "teams" in the afternoons (and > > all Friday). > > > > What would be your preference ? The Mon-Tue/Wed-Fri split means less > > room changes, which make it easier on the events team. So all else being > > equal we'd rather keep it the way it is, but I'm open to changing it if > > attendees think it's a good idea. > > > > If you have any other suggestion (that we could implement in the 3 > > months we have between now and the event) please let me know :) > > > > -- > > Thierry Carrez (ttx) > > > > __ > > OpenStack Development Mailing List (not for usage questions) > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa][tc][all] Tempest to reject trademark tests
gt; > release to have significant changes in behavior in the exact same > > tests for the next release. > > I actually find this to be kinda misleading. Tempest has always had > running on any cloud as part of it's mission. I think you're referring > to the monster defcore thread from last summer about proprietary nova > extensions > adding on to API responses. This is honestly a completely separate > problem > which is not something I want to dive into again, because that was a much > more > nuanced problem that involved much more than just code review. > > > > > At the early stage, when the DefCore team was still figuring out > > these issues, it made sense to put all of the tests in one place > > with a review team that was actively participating in establishing > > the process. If we better understand the "rules" for these tests > > now, we can document them and distribute the work of maintaining the > > test suites. > > I think you're overestimating how much work is actually being done > bidirectionally here. The interaction with defcore is more straight > consumption > then you might think. They tend to just pick and choose from what tempest > has > and don't actually identify gaps or contribute back into tempest much. > > This actually is the crux of my entire concern, that assuming we do > widely > expand the number of trademark programs there is an assumption that a > bunch of > people are going to show up, write tests, and maintain them. However, all > past > evidence shows that this just doesn't ever happen. I linked to that graph > from > one of my talks to try and help illustrate this. That was back in the > days when > tempest supported all official openstack projects and was a > **requirement** for > projects to graduate. Even then the contribution in this space was > minimal or > non-existent for newer projects. > > But that being said it's just a concern, we have yet to actually reach > this > point because of defcore requirements, and we might never actually get > there. > > > > > And yes, I agree with the argument that we should be fair and treat > > all projects the same way. If we're going to move tests out of the > > tempest repository, we should move all of them. The QA team can > > still help maintain the test suites for whatever projects they want, > > even if those tests are in plugins. > > Again, where has this been proposed? I've yet to see anything where > tests being removed from tempest has being proposed anywhere. Also, moving > tests from tempest to some other place doesn't magically solve any of the > issues. > The fundamental problem I'm concerned with is a large expansion in the number > of trademark programs overloading a small team. Just saying the QA team can > still help maintain them doesn't change the scaling problem. It just shifts > that from the tempest repo to another repo. (or multiples) Yeah, we should define and clear the problems before discussing the solution. I think Thierry already mentioned about that like below. --- > > > I fear that we are discussing solutions before defining the problem. We > > > want: > > > > > > 1- Decentralize test maintenance, through more tempest plugins, to > > > account for limited QA resources > > > 2- Additional codereview constraints and approval rules for tests that > > > happen to be used in trademark programs > > > 3- Discoverability/ease-of-install of the set of tests that happen to be > > > used in trademark programs > > > 4- A git repo layout that can be simply explained, for new teams to > > > understand --- > > But as I've said so many times, where is there a formal proposal for the > program expansion? Where has a proposed test for defcore been rejected from > tempest? This all is still very in loose terms and seems to be a completely > hypothetical discussion. I feel like we're searching for a problem to solve > before there is actually one. I agree here, too. Of course, we can have casual conversation. But I feel this topic it too big for a *casual*. And if we should change something, I think it's better to have concrete patches, specs and/or *formal* proposal like Matthew said. Otherwise, discussion may not be productive. -- Masayuki Igawa > > -Matt Treinish > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > Email had 1 attachment: > + signature.asc > 1k (application/pgp-signature) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Proposed changes to the team meeting time
Thanks Andrea, +1 But my only one concern is this change is not good for you, right? Of course, you don't need to attend both meetings, though :) -- Masayuki Igawa On Fri, May 26, 2017, at 10:41 AM, zhu.fang...@zte.com.cn wrote: > > +1, thanks! > > zhufl > > > > Original Mail > *Sender: * <andrea.fritt...@gmail.com>; > *To: * <openstack-dev@lists.openstack.org>; > *Date: *2017/05/25 21:19 > *Subject: **[openstack-dev] [QA] Proposed changes to the team > meeting time*> > Hello team, > > our current QA team meeting schedule alternates between 9:00 UTC and > 17:00 UTC.> The 9:00 meetings is a bit towards the end of the day for out > contributors in APAC, so I'm proposing to move the meeting to > 8:00 UTC.> > Please respond with +1 / -1 and/or comments, I will leave the poll > open for about 10 days to make sure everyone interested gets a chance > to comment.> > Thank you > > andrea > > > - > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] Proposing Fanglei Zhu for Tempest core
+1! -- Masayuki Igawa masay...@igawa.me On Tue, May 16, 2017, at 05:22 PM, Andrea Frittoli wrote: > Hello team, > > I'm very pleased to propose Fanglei Zhu (zhufl) for Tempest core. > > Over the past two cycle Fanglei has been steadily contributing to > Tempest and its community.> She's done a great deal of work in making Tempest > code cleaner, easier > to read, maintain and> debug, fixing bugs and removing cruft. Both her code > as well as her > reviews demonstrate a> very good understanding of Tempest internals and of > the project future > direction.> I believe Fanglei will make an excellent addition to the team. > > As per the usual, if the current Tempest core team members would > please vote +1> or -1(veto) to the nomination when you get a chance. We'll > keep the > polls open> for 5 days or until everyone has voted. > > References: > https://review.openstack.org/#/q/owner:zhu.fanglei%2540zte.com.cn > https://review.openstack.org/#/q/reviewer:zhufl > > Thank you, > > Andrea (andreaf) > - > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] Proposing Fanglei Zhu for Tempest core
+1! -- Masayuki Igawa masay...@igawa.me On Tue, May 16, 2017, at 05:22 PM, Andrea Frittoli wrote: > Hello team, > > I'm very pleased to propose Fanglei Zhu (zhufl) for Tempest core. > > Over the past two cycle Fanglei has been steadily contributing to > Tempest and its community.> She's done a great deal of work in making Tempest > code cleaner, easier > to read, maintain and> debug, fixing bugs and removing cruft. Both her code > as well as her > reviews demonstrate a> very good understanding of Tempest internals and of > the project future > direction.> I believe Fanglei will make an excellent addition to the team. > > As per the usual, if the current Tempest core team members would > please vote +1> or -1(veto) to the nomination when you get a chance. We'll > keep the > polls open> for 5 days or until everyone has voted. > > References: > https://review.openstack.org/#/q/owner:zhu.fanglei%2540zte.com.cn > https://review.openstack.org/#/q/reviewer:zhufl > > Thank you, > > Andrea (andreaf) > - > > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: OpenStack-dev- > requ...@lists.openstack.org?subject:unsubscribe> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA]Refactoring Scenarios manager.py
Hi, Thank you for bringing this up. Yeah, I understand your frustration. We already have the document about our stable interface[1]. It says -- Stability Any code that lives in tempest/lib will be treated as a stable interface. This means that any public interface under the tempest/lib directory is expected to be a stable interface suitable for public consumption. However, for any interfaces outside of tempest/lib in the tempest tree (unless otherwise noted) or any private interfaces the same stability guarantees don't apply. -- So we can change private interfaces basically. However, I also assume that this document is not well known(or people just ignore it, though, maybe). So I'd like to advertise this policy here, and discuss it (if the discussion is needed.) [1] https://docs.openstack.org/developer/tempest/library.html#stability On Sat, Feb 25, 2017 at 22:39 Jordan Pittier <jordan.pitt...@scality.com> wrote: > Hi guys, > So I have a problem with these 2 patches here [1] and here [2]. You > basically are blocking any attempt of refactoring manager.py. Refactoring > that file has been our number one priority for 2 cycles, and so far hardly > no one stepped up really to do the work, except me with these 2 patches. > Let me remind you that that file is a gigantic mess, an so are our network > scenarios. > > The manager.py file in the scenarios directory has no stable interface, > and it was never "advertised" so. That some plugins decided to use some > private methods (such as this _get_network_by_name) is unfortunate but that > should not block us from moving. > > So just to be clear, if we really want to refactor our scenarios (and we > must in my opinion), things will break for projects that are importing > Tempest and using it outside of its stable interface. I am not interested > in being the good Samaritan for the whole OpenStack galaxy, I have enough > with the 6 core projects and the Tempest stable interface. So guys, if you > are and don't want to go forward with [1] and [2], be sure I'll never touch > those scenarios again. I am not upset, but we have to make clear decisions, > sometimes difficult. > > [1] : https://review.openstack.org/#/c/436555/ > [2] : https://review.openstack.org/#/c/438097/ > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] PTL non-candidacy
Thank you for your great effort! I'm very proud of you as a colleague :) -- Masayuki Igawa On Fri, Jan 20, 2017 at 6:16 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> wrote: > Hi, > > I will step down as PTL after this Ocata cycle. > I was happy to see new ideas and folks who try making ideas true in > this 2 cycles. > Now QA project has a lot of components with many people's effort and > we help each other as a community. > This experience is very exciting for me, I am proud to being a member > in this community. > > Today, I'd like to concentrate on coding and reviewing again as a developer. > I think we have good candidates for a next PTL, and I will keep active > under the next PTL's leadership. > > Thanks for choosing me anyways, let's make OpenStack quality better together > :-) > > Thanks > Ken Ohmichi > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] [openstack-health] Avoid showing non-official projects failure ratios
Hi, On Fri, Dec 2, 2016 at 6:29 PM, Andreas Jaeger <a...@suse.com> wrote: > On 12/02/2016 10:03 AM, Thierry Carrez wrote: >> Ken'ichi Ohmichi wrote: >>> Hi QA-team, >>> >>> In the big-tent policy, we continue creating new projects. >>> On the other hand, some projects became non-active. >>> That seems natural thing. >>> >>> Now openstack-health[1] shows non-active project as 100% failure ratio >>> on "Project Status". >>> The project has became non-official since >>> https://review.openstack.org/#/c/324412/ >>> So I feel it is nice to have black-list or something to make it >>> disappear from the dashboard for concentrating on active projects' >>> failures. >>> >>> Any thoughts? >> >> Yes, I totally agree we should only list active official projects in >> there, otherwise long-dead things like Cue will make the view look bad. >> Looks like the system adds new ones but does not remove anything ? It >> should probably take its list from [1]. >> >> [1] >> http://git.openstack.org/cgit/openstack/governance/tree/reference/projects.yaml > > Is cue completely dead? Should we then retire it completely following > http://docs.openstack.org/infra/manual/drivers.html#retiring-a-project ? > > It still has jobs setup and I see people submitting typo fixes etc. I'm not sure the cue is dead or not. But I think we should fix the failure of the job or remove the periodic jobs. Otherwise, the job just waste the resource of the OpenStack infra.. But we should have the filter feature like a 'Project Type' of stackalitics, probably. I think it's useful for openstack-health users. Best Regards, -- Masayuki Igawa > > > Andreas > -- > Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi > SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany >GF: Felix Imendörffer, Jane Smithard, Graham Norton, >HRB 21284 (AG Nürnberg) > GPG fingerprint = 93A3 365E CE47 B889 DF7F FED1 389A 563C C272 A126 > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Cancel this week QA meeting
Hi! We cancel this week QA meeting because of too few attendees this timing. See you next time :) Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] The end-user test suite for OpenStack clusters
Hi, My first impression is "It's interesting" :) I actually don't read whole of this threads, but, I re-read the QA mission statement[1]. I feel os-faults is a little bit far from the mission statement. Because os-faults seems like focusing on operator testing. But I think this can be under the QA project. Because tempest scope has also operator testing. [1] https://wiki.openstack.org/wiki/QA#Project_Team_Definition On Sat, Oct 8, 2016 at 4:43 AM, Ken'ichi Ohmichiwrote: > Hi Timur, > > 2016-10-06 4:08 GMT-07:00 Timur Nurlygayanov : >> Ken, it is a good idea! >> >> The plan is to develop os-faults as a library which will be able >> to manage cluster nodes and OpenStack services on these nodes. >> It is a good idea to add some Tempest tests which will use >> os-faults library as well for some API tests. > > Sorry for my misleading, that is not I wanted to say. > I am thinking Tempest doesn't use os-faults as library. > I just wanted to say some destructive tests which will use REST APIs > can be implemented in Tempest instead of os-faults. > > For example, the compute service client contains disable_service which > disables nova service but Tempest doesn't contain the corresponding > test cases because Tempest tests need to run in parallel. > https://github.com/openstack/tempest/blob/master/tempest/lib/services/compute/services_client.py#L54 > > If some tests use this method, the other tests which run in parallel > can be failed as unexpected on the gate. > Such tests also are destructive and I am hoping these tests also will > be able to run the gate by finding better way. > >> The Stepler framework [1] will use os-faults to perform all destructive >> actions in the clouds (the reboot of nodes, restart of OpenStack services, >> enable/disable network interfaces or some firewall rules and etc). >> >> We need to get +1 from you in [1] to create the repository with >> advanced end-user scenario tests. > > OK, I'd like to summarize my thinking here for adding os-faults under > QA project: > > Under QA project, the first target of most programs(tempest, devstack, > etc) is for the gate testing. > Of course, some of them are available on production clouds also as the > design, but the first is for the gate. > That means devstack clouds as the first target/purpose. > At this time, this os-faults doesn't seem the gate as the first > purpose based on current implementation. > So I feel os-faults seems different from the existing programs. > > os-faults is very young and we will be able to extend it for devstack > clouds also later, I hope. > In addition, I heard this kind of tests(destructive, HA, etc) are > requested from the other users. > So this os-faults seems very valuable for users. > > Just in my opinion, I am find to add this os-faults under QA project. > But before that, I'd like to get opinions from the other people of QA team. > > Thanks > Ken Ohmichi > > > > > > > > > > > > >> On Tue, Oct 4, 2016 at 8:53 PM, Yaroslav Lobankov >> wrote: >>> >>> Hi Ken, >>> >>> OS-Faults doesn't have any scenarios in the tree yet (the project is two >>> months old), but you can find some examples of the use in the >>> os-faults/examples directory. >>> >>> Regards, >>> Yaroslav Lobankov. >>> >>> On Tue, Oct 4, 2016 at 8:02 PM, Ken'ichi Ohmichi >>> wrote: Hi Timur, Thanks for your explanation. 2016-09-29 6:22 GMT-07:00 Timur Nurlygayanov : > >> I am guessing the above "restart nodes" is for verifying each >> OpenStack service restarts successfully, right? > > Yes, this is right. And we also will check that HA logic for these > services works correctly (for example, rescheduling of L3 Neutron > agents for networks). > >> But these service scripts are provided by distributors, and Devstack >> itself doesn't contain service scripts IIUC. >> So I'd like to know how to verify it on Devstack clouds. > > Yes, DevStack doesn't support many scenarios which are actual > and should be supported on the production clouds. > It will be not possible to run all advanced test scenarios for DevStack > clouds, > just because DevStack can't deploy OpenStack cloud with 3 controllers > now (so, probably it will be possible in the future). > > Of course, some advanced scenarios will support DevStack clouds, > for example, some test scenarios which are based on customer-found > issues from the real production clouds, like upload of the large images > (100+ Gb) > to Glance with Swift backend. Such cases are important for verification > of > pre-production environments, but not very important for CI gate jobs. > > It is also important to note that in these advanced cases we are > targeting > to check not only the logic of Python
Re: [openstack-dev] [QA] Presence at PTG Atlanta in February
Thank you for clarifying Thierry! I got it! I'm looking forward to see the registration:) Best Regards, -- Masayuki Igawa On Wed, 12 Oct 2016 at 18:46 Thierry Carrez <thie...@openstack.org> wrote: > Masayuki Igawa wrote: > > Hi Ken'ichi, > > > > Thanks for bring this up. > > > > On Tue, Oct 11, 2016 at 6:50 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> > wrote: > >> Hi QA team, > >> > >> As you know, the first PTG(Project Teams Gathering) happens at Atlanta > >> 20th-24th February the next year. > >> After Barcelona, OpenStack Summit will be separated into two parts > >> (conferences and design sessions) and PTG is a new format for design > >> sessions for developers(Please see the detail on [1]). > >> QA is always important factor of developments, and I am thinking the > >> presence of QA project is good for the community. > > > > Yeah, I agree with you. (I'm not sure I'll be there, though.) > > Do we(QA team) need to register or something for the PTG? > > Current plan is to have registration open shortly after Barcelona. > > -- > Thierry Carrez (ttx) > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Presence at PTG Atlanta in February
Hi Ken'ichi, Thanks for bring this up. On Tue, Oct 11, 2016 at 6:50 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> wrote: > Hi QA team, > > As you know, the first PTG(Project Teams Gathering) happens at Atlanta > 20th-24th February the next year. > After Barcelona, OpenStack Summit will be separated into two parts > (conferences and design sessions) and PTG is a new format for design > sessions for developers(Please see the detail on [1]). > QA is always important factor of developments, and I am thinking the > presence of QA project is good for the community. Yeah, I agree with you. (I'm not sure I'll be there, though.) Do we(QA team) need to register or something for the PTG? Best Regards, -- Masayuki Igawa > > Thanks > Ken Omichi > > --- > [1]: http://www.openstack.org/ptg > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Meeting Thursday Oct 6th at 9:00 UTC
Hello everyone, Please reminder that the weekly OpenStack QA team IRC meeting will be Thursday, Oct 6th at 9:00 UTC in the #openstack-meeting channel. The agenda for the meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_October_6th_2016_.280900_UTC.29 Anyone is welcome to add an item to the agenda. To help people figure out what time 9:00 UTC is in other timezones the next meeting will be at: 04:00 EST 18:00 JST 18:30 ACST 11:00 CEST 04:00 CDT 02:00 PDT Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] tempest tests in Horizon GUI
Hi Ofer, As Andrea said, Tempest shouldn't leave any test resource after a Tempest run. And Horizon is not mandatory in a Tempest run. But if you want to verify it, you can do that through your openstack-client or Horizon dashboard just in case. And you can find your credentials in your tempest.conf and accounts.yaml file. If you want to know more details, some documents[1] might be helpful. [1] http://docs.openstack.org/developer/tempest/configuration.html#tempest-configuration . Best Regards, -- Masayuki Igawa On Thu, Sep 22, 2016 at 7:27 AM, Andrea Frittoli <andrea.fritt...@gmail.com> wrote: > Hi Ofer, > > Tempest tests try to cleanup after themselves, so in theory after a Tempest > run you shouldn't see any test resource left around. > > If you run your tests using dynamic credentials, test accounts are created > on the fly, and deleted after they're used. If you use preprovisioned > credentials, you need to setup a network for them before the test run > starts. > > Andrea > > > On Thu, 22 Sep 2016, 7:02 a.m. Ken'ichi Ohmichi, <ken1ohmi...@gmail.com> > wrote: >> >> Hi Ofer, >> >> The scenario test of Horizon has been removed from Tempest since >> https://review.openstack.org/#/c/313713/ >> And now the test[1] exists in openstack/tempest-horizon. >> The test is very simple like >> * checks that the login page is available >> * logs in as a regular user >> * checks that the user home page loads without error >> >> > When I run a tempest test/scenario-test, should I see the components >> > (network, subnet, router etc.) in the horizon GUI ? >> >> The test doesn't create any resources(network, subnet, etc), so any >> changes would not happen during the test. >> >> Thanks >> Ken Ohmichi >> >> --- >> [1]: >> https://github.com/openstack/tempest-horizon/blob/master/tempest_horizon/tests/scenario/test_dashboard_basic_ops.py >> >> 2016-09-21 15:01 GMT+02:00 Barber, Ofer <ofer.bar...@hpe.com>: >> > I have a basic question about tempest. >> > >> > >> > >> > When I run a tempest test/scenario-test, should I see the components >> > (network, subnet, router etc.) in the horizon GUI ? >> > >> > >> > >> > If yes, for what username or what project those are created ? >> > >> > >> > >> > Thank you, >> > >> > Ofer >> > >> > >> > >> > >> > >> > __ >> > OpenStack Development Mailing List (not for usage questions) >> > Unsubscribe: >> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Announcing my candidacy for PTL of the Ocata
Hi everyone, I'd like to announce my candidacy for Ocata cycle as the PTL for the Quality Assurance program/team/project/etc. First off, I would like to thank you all the contributors, core reviewers, PTLs and anyone who involves and makes the OpenStack better. Let me introduce myself, briefly. I have joined the OpenStack community since 2012 as a developer. Now, I'm a core member of some QA/Infra projects such as Tempest, Tempest-lib(deprecated), OpenStack-health, stackviz, subunit2sql[1]. And I played the mentors/instructors role at the upstream training in Japan several times. It was a great experience to know the difficulty of telling people how OpenStack community is going. This Newton cycle has also been a very exciting one for the QA program. New QA projects like stackviz and OpenStack-health are emerging and growing. For distributed and stable project testing, we have been working on making the tempest service clients consistent and migrating it to the lib. It is also an excellent job in progress. And we improved Tempest-cli and its user workflow. Thanks to the improvements, we can use Tempest in very simple steps now. And I especially focused on improving UI side of tempest, OpenStack-health, etc. I think this is also a great step for better UX. In Ocata cycle, I believe cleaning up Tempest/DevStack will be one of the top priority works such as making consistency, cleaning up pluggable modules. We will continue to do it this cycle, too. And I also think improving UI and UX is one of the most important things for QA. For example, OpenStack-health should show more variety of data and be improved its performance. And regarding the tempest cli, the first implementation phase was almost done, and we need to get user feedbacks and improve them. I think our OpenStack QA work is unique and excellent compare to the other open source projects. And as you know, it is very exciting not boring these days. I hope more and more contributors will participate in the OpenStack development, especially the QA. So I will continue to advertise our great the OpenStack QA works and review and make patches for better QA, of course. And any contribution like reporting bugs, suggesting a lack of documentations, are appreciated, too. Join us and happy hacking! [1] http://stackalytics.com/?user_id=igawa Thanks for your consideration! Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Infra] Announcing Gertty 1.4.0
Hi! On Wed, Jul 27, 2016 at 11:50 PM, James E. Blair <cor...@inaugust.com> wrote: > Michał Dulko <michal.du...@intel.com> writes: > >> Just wondering - were there tries to implement syntax highlighting in >> diff view? I think that's the only thing that keeps me from switching to >> Gertty. > > I don't know of anyone working on that, but I suspect it could be done > using the pygments library. Oh, it's an interesting feature to me :) I'll try to investigate and implement in next couple of days :) Thanks, -- Masayuki Igawa > > -Jim > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest] Discussion on to enable "Citrix XenServer CI" to vote openstack/tempest
Hi Jianghua and QA team, Thank you for bringing this up. IMO, it's good to make it voting in openstack/tempest. Because it's already a voting job in Nova and seems having enough stability. And we should keep test stability for projects. My concern is that the resource of "Citrix XenServer CI". Do you have enough resource for it? And it seems like "Citrix XenServer CI" job execution time is the longest one in Tempest jobs. Best regards, -- Masayuki On Tue, Jun 14, 2016 at 2:36 PM, Jianghua Wangwrote: > Added project prefix in the subject and loop in Masayuki and Ghanshyam who > know the background as well. Thanks. > > > > Jianghua > > > > From: Jianghua Wang > Sent: Tuesday, June 14, 2016 12:46 PM > To: OpenStack Development Mailing List (not for usage questions) > Cc: Jianghua Wang > Subject: Discussion on to enable "Citrix XenServer CI" to vote > openstack/tempest > > > > Hi all, > >Recently the "Citrix XenServer CI" was broken due to a bad commit[1] to > openstack/tempest. As the commit was merged on Friday which is vacation at > here, it had been in failure for more than three days before we noticed and > fixed[2] this problem. As this CI votes for openstack/nova, it had been > keeping to vote -1 until disabled voting. > >So I suggest we also enable this XenServer CI voting on tempest change to > avoid similar cases in the future. We see in this case, the tempest commit > didn’t consider the different case for type-1 hypervisors, so it broke > XenServer test. Actually “Citrix XenServer CI” verified that patch set with > failure result but which got ignored due to no voting. So let’s enable the > voting to make life easierJ > > Currently we have this CI voting for openstack/nova. Per the history > experience, it has been a stable CI(more stable than the Jenkins check) > normally if there is no bad commit breaking it. > > Thanks for any comments. > > > > [1] https://review.openstack.org/#/c/316672 > > [2] https://review.openstack.org/#/c/328836/ > > > > Regards, > > Jianghua > > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] StackViz is now enabled for all devstack-gate jobs
Congrats! I'm looking forward to seeing an integration/collaboration with openstack-heath :) On Tue, Jun 7, 2016 at 8:32 AM, Buckley, Tim Jasonwrote: > Hello all, > > I'd like to announce that StackViz will now be running at the end all > tempest-dsvm jobs and saving visualization output to the log server. > > StackViz is a visualization utility for generating interactive visualizations > of > jobs in the OpenStack QA pipeline and aims to ease debugging and performance > analysis tasks. Currently it renders an interactive timeline for subunit > results and dstat data, but we are actively working to visualize more log > types > in the future. > > StackViz instances are saved as a 'stackviz' directory under 'logs' for each > job > run on http://logs.openstack.org/. For an example, see: > > http://logs.openstack.org/07/212207/8/check/gate-tempest-dsvm-full/2d30217/logs/stackviz/ > > For more information StackViz, see the project page at: > https://github.com/openstack/stackviz > > Bugs can also be reported at: > https://bugs.launchpad.net/stackviz > > Feedback is greatly appreciated! > > Thanks, > Tim Buckley > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Meeting Thursday June 2nd at 9:00 UTC
Hello everyone, Please reminder that the weekly OpenStack QA team IRC meeting will be Thursday, June 2nd at 9:00 UTC in the #openstack-meeting channel. The agenda for the meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_June_2nd_2016_.280900_UTC.29 Anyone is welcome to add an item to the agenda. To help people figure out what time 9:00 UTC is in other timezones the next meeting will be at: 04:00 EST 18:00 JST 18:30 ACST 11:00 CEST 04:00 CDT 02:00 PDT Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] [Tempest] Abondoned old code reviews
On Wed, Jun 1, 2016 at 3:05 AM, Andrea Frittoliwrote: > On Mon, 30 May 2016, 6:25 p.m. Ken'ichi Ohmichi, > wrote: >> >> Hi, >> >> There are many patches which are not updated in Tempest review queue >> even if having gotten negative feedback from reviewers or jenkins. >> Nova team is abandoning such patches like [1]. >> I feel it would be nice to abandone such patches which are not updated >> since the end of 2015. >> Any thoughts? +1 I think 5 months is enough to abandon :) Best Regards, -- Masayuki > > > I don't mind either way, if you prefer abandoning them it's ok with me. > I rely on gerrit dashboards and IRC communication to decide which patches I > should review; but I understand it would be nice to remove some clutter. > > Andrea > >> >> [1]: >> http://lists.openstack.org/pipermail/openstack-dev/2016-May/096112.html >> >> Thanks >> Ken Ohmichi >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA][RFC] Skip this week meeting
Hi QA guys, We have very few topic this week to discuss. So I think we should skip this week meeting. Any thoughts? Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [openstack-health][QA] We need your feedbacks
Hi, Now, we are making OpenStack-Health[1] which is a dashboard for visualizing test results of OpenStack CI jobs. This is heavily under development mode but works, you can use it now. We'd like to get your feedback to make it better UI/UX. So, if you have any comments or feedbacks, please feel free to file it as a bug[2] and/or submit patches[3]. This is a kind of advertisements, however, I think openstack-health could be useful for all projects through knowing the development status of the OpenStack projects. [1] http://status.openstack.org/openstack-health/ [2] https://bugs.launchpad.net/openstack-health [3] http://git.openstack.org/cgit/openstack/openstack-health Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][stackalytics] Gaming the Stackalytics stats
2016-04-11 10:31 GMT+09:00 Sheel Rana Insaan <ranasheel2...@gmail.com>: >> So should we add what we don't want > to see people -1 for? > >>[1] http://docs.openstack.org/infra/manual/developers.html#peer-review > > This seems right way.. but concern is do everyone follow all docs? Nice point. Yeah, I suppose that not everyone read all docs. But some reviewers can know our custom/culture through the docs. And we can indicate the pointer if reviewers don't know it. Best Regards, -- Masayuki Igawa > > But atleast we should document it somewhere. > > Regards, > Sheel Rana > > On Apr 11, 2016 6:52 AM, "Masayuki Igawa" <masayuki.ig...@gmail.com> wrote: > > 2016-04-11 9:46 GMT+09:00 Matt Riedemann <mrie...@linux.vnet.ibm.com>: >> >> >> On 4/10/2016 6:37 PM, Clint Byrum wrote: >>> >>> Excerpts from Matt Riedemann's message of 2016-04-09 06:42:54 -0700: >>>> >>>> There is also disincentive in +1ing a change that you don't understand >>>> and is wrong and then a core comes along and -1s it (you get dinged for >>>> the disagreement). And there is disincentive in -1ing a change for the >>>> wrong reasons (silly nits or asking questions for understanding). I ask >>>> a lot of questions in a lot of changes and I don't vote on those because >>>> it would be inappropriate. >>>> >>> >>> Why is disagreement a negative thing? IMO, reviewers who agree too much >>> are just part of the echo chamber. >>> >>> >>> __ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >> >> I'm not saying disagreement is a negative thing, I was saying there are >> times when I've seen people -1 for crazy nits, e.g. there should be a >> blank >> line between the bug ref and change-id in the commit message, or for >> asking >> questions for understanding (which, btw, I'm fine with -1 for 'add a >> comment >> because this is complicated and I didn't get it at first'). And I'm also >> not >> crazy about piling on or agreeing with everything either. My point is I >> think it's appropriate in a lot of cases to just not vote but still >> comment. > > I think we have some/many implicit rules for our review. There's a > document[1] for review > but it doesn't mention crazy nits. So should we add what we don't want > to see people -1 for? > > [1] http://docs.openstack.org/infra/manual/developers.html#peer-review > >> >> -- >> >> Thanks, >> >> Matt Riedemann >> >> >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][stackalytics] Gaming the Stackalytics stats
2016-04-11 9:46 GMT+09:00 Matt Riedemann: > > > On 4/10/2016 6:37 PM, Clint Byrum wrote: >> >> Excerpts from Matt Riedemann's message of 2016-04-09 06:42:54 -0700: >>> >>> There is also disincentive in +1ing a change that you don't understand >>> and is wrong and then a core comes along and -1s it (you get dinged for >>> the disagreement). And there is disincentive in -1ing a change for the >>> wrong reasons (silly nits or asking questions for understanding). I ask >>> a lot of questions in a lot of changes and I don't vote on those because >>> it would be inappropriate. >>> >> >> Why is disagreement a negative thing? IMO, reviewers who agree too much >> are just part of the echo chamber. >> >> __ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > I'm not saying disagreement is a negative thing, I was saying there are > times when I've seen people -1 for crazy nits, e.g. there should be a blank > line between the bug ref and change-id in the commit message, or for asking > questions for understanding (which, btw, I'm fine with -1 for 'add a comment > because this is complicated and I didn't get it at first'). And I'm also not > crazy about piling on or agreeing with everything either. My point is I > think it's appropriate in a lot of cases to just not vote but still comment. I think we have some/many implicit rules for our review. There's a document[1] for review but it doesn't mention crazy nits. So should we add what we don't want to see people -1 for? [1] http://docs.openstack.org/infra/manual/developers.html#peer-review > > -- > > Thanks, > > Matt Riedemann > > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][all] Propose to remove negative tests from Tempest
gt;> to feedback, but I'm happy to get feedback to see our direction :-) >> >> The guideline is a nice idea. >> If necessary to add more negative tests into Tempest, how about >> requiring to write the reason which explains new tests are not corner >> cases in the commit message? >> We can know the merit of new negative ones when reviewing. >> >>> However Tempest is also interoperability, so we should keep at least a few >>> negative API checks in tempest (for the core six services) to enforce that >>> return codes do not change inadvertently in negative cases, which could >>> break existing clients and applications. >> >> This also is a nice point. >> How to change error return codes is unclear to me at this time. >> In Nova, there are some exceptions for changing error return code >> without microversion bumping as [1]. This kind of guideline will be >> discussed later. > > This makes Tempest scope little bit unclear again. If we want to > verify all error codes in Tempest then it leads to have all surface > level negative testing also in Tempest. There are lot of scenario > where error codes can be verified and will be difficult to cover all > in Tempest. > > Current negative tests does not cover all error codes for all APIs. If > we try to implement all then it will be huge tests number. > I think its project which should be verifying those. > > In Summary - > > 1. If we choose to have only valid negative tests (other than surface > level negative testing) which can verify stability/integration of API > and used by Def-core too then, > - We should remove all existing tests which only touch surface of > APIs (wrong input validation). > 2. If we want to verify error codes also in Tempest then, > - first point becomes invalid and we need to implement all > possible error code testing. > > Having mix of those always leads to have issue in reviews/development etc. I think it doesn't depend on surface or deeply but it depends on the interface is stable or not. Because Tempest is a blackbox testing suite, Tempest shouldn't care about the individual project implementation, basically. So the issue is we don't know that which interface(especially negative case) is stable or not, actually. I think individual projects (and/or Def-Core) can define that, though. In other words, when we have a tempest negative/positive test case, it defines as a stable interface for its project, IMO. Best Regards, -- Masayuki Igawa (IRC: masayukig) > >> >> Now I am fine to keep the existing negative tests in Tempest, anyway. >> >> Thanks >> Ken Ohmichi >> >> --- >> [1]: >> https://github.com/openstack/nova/blob/master/doc/source/api_microversion_dev.rst#f2 >> __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Not running for PTL
From: Matthew Treinish <mtrein...@kortar.org> Subject: [openstack-dev] [QA] Not running for PTL Date: Fri, 11 Mar 2016 14:34:10 -0500 > Hi everyone, > > I'm writing this to announce that I am not running for QA PTL this cycle. I've > been the QA PTL for the past 4 cycles and I think it's time for another person > to take over the role. I think during the past 4 cycles the QA community has > grown greatly and become a much larger and stronger community compared to when > I first took on the position in the Juno cycle. > > I strongly believe in the diversity of leadership and ideas, and I don't want > the community to grow stagnant because it becomes synonymous with just my > voice. > Being a PTL is not the same as being an autocrat and I think it's time for > another person to step up and take over the QA PTL spot. > > That being said, I'm not planning on going anywhere or leaving the project. I > fully intend to continue working and being heavily involved with the QA > program, > as I have for been the past 2 years. (although maybe with a bit more free time > now) Thank you very very very much for your really great work! And I hope you still continue to deeply involved with the QA program for the better OpenStack quality, too :) Best Regards, -- Masayuki Igawa(IRC: masayukig) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Use tempest-config for tempest-cli-improvements
Hi, I agree with Ken'ichi's opinion, basically. Tempest users should know "what do we test for?" and we shouldn't discover values that we test for automatically. If users seems that "My current cloud is good. This is what I expect.", discovering function could work. But I suppose many of users would use tempest-config-generator for a new their cloud. So I feel the tool users could be misunderstanding easily. But I also think that tempest users don't need to know all of the configurations. So, how about something like introducing "a configuration wizard" for tempest configuration? This is a just idea, though.. Anyway, if you have the motivation to introduce tempest-config, how about writing a spec for the feature for a concrete discussion? (I think we should have agreements about the target issues, users, solutions, etc.) Best Regards, -- Masayuki Igawa 2015年11月29日(日) 22:07 Yair Fried <yfr...@redhat.com>: > Hi, > I agree with Jordan. > We don't have to use the tool as part of the gate. It's target audience is > people and not CI systems. More specifically - new users. > However, we could add a gate (or a few) for the tool that makes sure a > proper conf file is generated. It doesn't have to run the tests, just > compare the output of the script to the conf generated by devstack. > > Re Rally - I believe the best place for tempest configuration script is > within tempest. That said, if the Tempest community doesn't want this tool, > we'll have to settle for the Rally solution. > > Regards > Yair > > On Fri, Nov 27, 2015 at 11:31 AM, Jordan Pittier < > jordan.pitt...@scality.com> wrote: > >> Hi, >> I think this script is valuable to some users: Rally and Red Hat >> expressed their needs, they seem clear. >> >> This tool is far from bullet proof and if used blindly or in case of >> bugs, Tempest could be misconfigured. So, we could have this tool inside >> the Tempest repository (in the tools/) but not use it at all for the Gate. >> >> I am not sure I fully understand the resistance for this, if we don"t use >> this config generator for the gate, what's the risk ? >> >> Jordan >> >> On Fri, Nov 27, 2015 at 8:05 AM, Ken'ichi Ohmichi <ken1ohmi...@gmail.com> >> wrote: >> >>> 2015-11-27 15:40 GMT+09:00 Daniel Mellado <daniel.mellado...@ieee.org>: >>> > I still do think that even if there are some issues addressed to the >>> > feature, such as skipping tests in the gate, the feature itself it's >>> still >>> > good -we just won't use it for the gates- >>> > Instead it'd be used as a wrapper for a user who would be interested on >>> > trying it against a real/reals clouds. >>> > >>> > Ken, do you really think a tempest user should know all tempest >>> options? >>> > As you pointed out there are quite a few of them and even if they >>> should at >>> > least know their environment, this script would set a minimum >>> acceptable >>> > default. Do you think PTL and Pre-PTL concerns that we spoke of would >>> still >>> > apply to that scenario? >>> >>> If Tempest users run part of tests of Tempest, they need to know the >>> options which are used with these tests only. >>> For example, current Tempest contains ironic API tests and the >>> corresponding options. >>> If users don't want to run these tests because the cloud don't support >>> ironic API, they don't need to know/setup these options. >>> I feel users need to know necessary options which are used on tests >>> they want, because they need to investigate the reason if facing a >>> problem during Tempest tests. >>> >>> Now Tempest options contain their default values, but you need a >>> script for changing them from the default. >>> Don't these default values work for your cloud at all? >>> If so, these values should be changed to better. >>> >>> Thanks >>> Ken Ohmichi >>> >>> --- >>> >>> > Andrey, Yaroslav. Would you like to revisit the blueprint to adapt it >>> to >>> > tempest-cli improvements? What do you think about this, Masayuki? >>> > >>> > Thanks for all your feedback! ;) >>> > >>> > El 27/11/15 a las 00:15, Andrey Kurilin escribió: >>> > >>> > Sorry for wrong numbers. The bug-fix for issue with counters is merged. >>> > Correct numbers(latest result from rally's gate[1]): >>> > - total number of executed tests: 1689 >>> > -
[openstack-dev] [QA][Tempest][Keystone] Re: [openstack-qa] [tempest][keystone]Inherit API was not found in API tempest.
Hi Koshiya-san, Actually, we don't use openstack-qa ML for discussion now. So I added openstack-dev ML to CC. 2015年11月18日(水) 16:31 koshiya maho <koshiya.m...@po.ntts.co.jp>: > > Hi all, > > I'm creating a tempest that conforms to operational environment > of OpenStack that we have built. > > So far as I've seen the API tempest in the community, > Inherit API of Keystone was not found. > > Does this exist somewhere? Other task is moving in relation to this? > > If not, I want to challenge to post a patch of this. Do you mean this API? http://developer.openstack.org/api-ref-identity-v3-ext.html#identity_v3_OS-INHERIT-ext I've found a client for OS-TRUST extension api in tempest/services/identity/v3/json/identity_client.py but not about OS-INHERIT. So I think you can add OS-INHERIT api client and test then post it :) Best regards, -- Masayuki Igawa > -- > Maho Koshiya > NTT Software Corporation > E-Mail : koshiya.m...@po.ntts.co.jp __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Deprecate or drop about Tempest CLI
2015年11月18日(水) 5:37 Matthew Treinish <mtrein...@kortar.org>: > On Tue, Nov 17, 2015 at 01:39:34AM +, Masayuki Igawa wrote: > > Hi tempest CLI users and developers, > > > > Now, we are improving Tempest CLI as we discussed at the summit[1] and > > migrating legacy commands to tempest cli[2]. > > > > In this situation, my concern is 'CLI compatibility'. > > If we'll drop old CLIs support(like my patch), it might break existing > > workflows. > > > > So I think there are two options. > > 1. Deprecate old tempest CLIs in this Mitaka cycle and we'll drop them > at > > the beginning of the N cycle. > > 2. Drop old tempest CLIs in this Mitaka cycle when new CLIs will be > > implemented. > > # Actually, I'd like to just drop old CLIs. :) > > As would I, but I agree we need to worry about existing users out there and > being nice to them. > > I think we should do option #1, but in [2] we also maintain the old entry > point. > We also have the function called by the old entry points emit a single > warning > that says to use the new CLI. This way we have the new entry points all > setup > and we indicate that everyone should move over to them. > > If you started with the tempest-verify-config script you actually would > would > fail because devstack currently uses it. So doing the switchover > gracefully I > think would be best, because this is the same issue users will potentially > hit. > Thank you for clarifying. Sure, it sounds reasonable to me. I'll try to maintain the old entry point, too. Best Regards, -- Masayuki Igawa > > > > > If you have question and/or opinion, please let me know. > > > > [1] https://etherpad.openstack.org/p/tempest-cli-improvements > > [2] https://review.openstack.org/#/c/240399/ > > > > -Matt Treinish > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA][Tempest] Deprecate or drop about Tempest CLI
Hi tempest CLI users and developers, Now, we are improving Tempest CLI as we discussed at the summit[1] and migrating legacy commands to tempest cli[2]. In this situation, my concern is 'CLI compatibility'. If we'll drop old CLIs support(like my patch), it might break existing workflows. So I think there are two options. 1. Deprecate old tempest CLIs in this Mitaka cycle and we'll drop them at the beginning of the N cycle. 2. Drop old tempest CLIs in this Mitaka cycle when new CLIs will be implemented. # Actually, I'd like to just drop old CLIs. :) If you have question and/or opinion, please let me know. [1] https://etherpad.openstack.org/p/tempest-cli-improvements [2] https://review.openstack.org/#/c/240399/ Best Regards, -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] OpenStack Meiji (明治) - Our next release name has been selected
Hi, # I'm a native Japanese speaker :-) 2015年7月7日(火) 20:33 Amrith Kumar amr...@tesora.com: Maybe this (the google answer)? www.youtube.com/watch?v=Qvw0A12aOGQ Yeah, this is correct pronunciation. But someone who is a native Japanese speaker should confirm. https://www.youtube.com/watch?v=D9vwge557hY I think this is a casual version and Japanese people are familiar with this. Thanks, -- Masayuki Igawa -amrith P.S. the yahoo answer suggests that you pronounce it as Meh - gee with the emphasis on the meh ;) -Original Message- From: Matthias Runge [mailto:mru...@redhat.com] Sent: Tuesday, July 07, 2015 6:08 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] OpenStack Meiji (明治) - Our next release name has been selected On Mon, Jul 06, 2015 at 11:48:13PM -0400, Monty Taylor wrote: significant identified problems... but the third in the list, Meiji (明 治) is clear. So please join me in welcoming the latest name to our family ... and if you, like me, are not a native Japanese speaker, in learning two new characters. Could someone provide a hint please, how to pronounce this properly? -- Matthias Runge mru...@redhat.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] OpenStack Meiji (明治) - Our next release name has been selected
2015年7月7日(火) 21:14 Matthias Runge mru...@redhat.com: On 07/07/15 13:55, Masayuki Igawa wrote: https://www.youtube.com/watch?v=D9vwge557hY I think this is a casual version and Japanese people are familiar with this. Thank you! I have an idea now. About the youtube snippet: Is that a commercial? Yes, it's a chocolate commercial:-) https://www.meiji.co.jp/ It's a very famous company which make snack foods and medicines in Japan. Thanks, -- Masayuki Igawa Matthias __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core
+1 :) On 2015年6月23日(火) at 5:30 Matthew Treinish mtrein...@kortar.org wrote: Hi Everyone, I'd like to propose we add Jordan Pittier (jordanP) to the tempest core team. Jordan has been a steady contributor and reviewer on tempest over the past few cycles and he's been actively engaged in the Tempest community. Jordan has had one of the higher review counts on Tempest for the past cycle, and he has consistently been providing reviews that show insight into both the project internals and it's future direction. I feel that Jordan will make an excellent addition to the core team. As per the usual, if the current Tempest core team members would please vote +1 or -1(veto) to the nomination when you get a chance. We'll keep the polls open for 5 days or until everyone has voted. Thanks, Matt Treinish References: https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z http://stackalytics.com/?metric=marksuser_id=jordan-pittier __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- -- Masayuki on a tiny device __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Meeting Thursday June 11th at 22:00 UTC
Hi everyone, Just a quick reminder that the weekly OpenStack QA team IRC meeting will be tomorrow Thursday, June 11th at 22:00 UTC in the #openstack-meeting channel. The agenda for tomorrow's meeting can be found here: https://wiki.openstack.org/wiki/Meetings/QATeamMeeting Anyone is welcome to add an item to the agenda. To help people figure out what time 22:00 UTC is in other timezones tomorrow's meeting will be at: 18:00 EDT 07:00 JST 07:30 ACST 00:00 CEST 17:00 CDT 15:00 PDT -- Masayuki Igawa __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Proposing Ghanshyam Mann for Tempest Core
+1! On 2014年11月22日(土) 3:29 Matthew Treinish mtrein...@kortar.org wrote: Hi Everyone, I'd like to propose we add Ghanshyam Mann (gmann) to the tempest core team. Over the past couple of cycles Ghanshyam has been actively engaged in the Tempest community. Ghanshyam has had one of the highest review counts on Tempest for the past cycle, and he has consistently been providing reviews that have been of consistently high quality that show insight into both the project internals and it's future direction. I feel that Ghanshyam will make an excellent addition to the core team. As per the usual, if the current Tempest core team members would please vote +1 or -1(veto) to the nomination when you get a chance. We'll keep the polls open for 5 days or until everyone has voted. Thanks, Matt Treinish References: https://review.openstack.org/#/q/reviewer:%22Ghanshyam+Mann+ %253Cghanshyam.mann%2540nectechnologies.in%253E%22,n,z http://stackalytics.com/?user_id=ghanshyammannmetric=marks ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA][Tempest] Gap analysis for using Tempest in production environments
Hi Timur, Thank you for your reply and adding interesting topics! I'll look at it. By the way, could you please add the link to etherpad [2] to the event description [1], it will help to find the correct link from mobile devices. Actually, I don't have the permission to add/change the description on the sched.org. I think Matthew(mtreinish) can do it. Thanks. On Sat, Nov 1, 2014 at 6:21 PM, Timur Nurlygayanov tnurlygaya...@mirantis.com wrote: Hello, it is the really important design session, because we need to break the ice and implement several good ideas, which we have. The first one is https://review.openstack.org/#/c/94473, we have the blueprint for this spec and one commit https://review.openstack.org/#/c/75425 in Abandoned state. Can we continue to work on this commit or we need to redesign it and make new commits? The other specs which we also can discuss on this design session (added to etherpad): https://review.openstack.org/#/c/86967/ https://review.openstack.org/#/c/89322/ https://review.openstack.org/#/c/92804/ and again, we need to update them, review and merge (if we want to see them in Tempest). We are using Tempest tests for verification of different OpenStack clouds and we really have issues with many custom configuration files and additional scripts, which allow to configure OpenStack before the tests. I like the idea, which Boris and Andrey described in https://review.openstack.org/#/c/94473 and I want to participate in implementation of this feature and use it to automate OpenStack clouds verification on many projects. I hope we will discuss this spec on the design session and will update and merge it soon. By the way, could you please add the link to etherpad [2] to the event description [1], it will help to find the correct link from mobile devices. Thank you! On Fri, Oct 31, 2014 at 4:23 AM, Masayuki Igawa masayuki.ig...@gmail.com wrote: Hi everyone, I'd like to advertise the session[1] and gather ideas before we will start the session to make it more productive. So please put your ideas for this topic on the pad[2] if you have ideas. I think the goal of this session is to make decisions what we should do or not in Kilo development cycle for filling the gap. And that will be an input to make blurprints. [1] http://kilodesignsummit.sched.org/event/6d91b3205804c930c2a067b23e7aab76#.VErHcH96c-U [2] https://etherpad.openstack.org/p/kilo-summit-gap-analysis-tempest-in-production Thanks, -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Timur, Senior QA Engineer OpenStack Projects Mirantis Inc ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA][Tempest] Gap analysis for using Tempest in production environments
Hi everyone, I'd like to advertise the session[1] and gather ideas before we will start the session to make it more productive. So please put your ideas for this topic on the pad[2] if you have ideas. I think the goal of this session is to make decisions what we should do or not in Kilo development cycle for filling the gap. And that will be an input to make blurprints. [1] http://kilodesignsummit.sched.org/event/6d91b3205804c930c2a067b23e7aab76#.VErHcH96c-U [2] https://etherpad.openstack.org/p/kilo-summit-gap-analysis-tempest-in-production Thanks, -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Proposal: A launchpad bug description template
Hi Markus, Thank you for bringing up this. On Thu, Oct 16, 2014 at 9:13 PM, Markus Zoeller mzoel...@de.ibm.com wrote: TL;DR: A proposal for a template for launchpad bug entries which asks for the minimal needed data to work on a bug. Hi, As a new guy in OpenStack I find myself often in the situation that I'm not able to work on a bug because I don't understand the problem itself. If I don't understand the problem I'm not able to locate the source of the issue and to provide an appropriate patch. As the new guy I don't want to flood the bug entry with questions what it should do which might be obvious to other members. And so, wrt Igawas comment in the mail thread [openstack-dev] [qa] Tempest Bug triage (http://openstack.markmail.org/thread/jcwgdcjxylqw2vyk) I'd like to propose a template for bug entries in launchpad (see below). Even if I'm not able to solve a bug due to the lack of knowledge, reading the bug description written according to this template could help me build up knowledge in this area. Maybe this or a similar template can be preloaded into the description panel of launchpad when creating a bug entry. What are your thoughts about this? Regards, Markus Zoeller IRC: markus_z -- template start - Problem description: == # Summarize the issue in as few words as possible and as much words as # necessary. A reader should have the chance to decide if this is in his # expertise without reading the following sections. Steps to reproduce: == # Explain where you start and under which conditions. # List every input you gave and every action you made. # E.g.: # Prereqs: Installed devstack on x86 machine with icehouse release # 1) In Horizon, launch an instance with name test an flavor m1.tiny #from image cirros-0.3.1-x86_64-uec # 2) Launch 2 other instances with different names but with the same #flavor and image. Expected behavior: == # Describe what you thought what should happen. If applicable provide # sources (e.g. wiki pages, manuals or similar) why to expected this. # E.g.: # Each instance should be launched successfully. Observed behavior: == # Describe the observed behavior and how it differs from the expected # one. List the error messages you get. Link to debug data. # E.g.: # The third instance was not launched. It went to an error state. The # error message was host not found. Reproducibility: == # How often will the steps to reproduce result in the described # observed behavior? # This could give a hint if the bug is related to race conditions # Try to give a good estimate. # E.g. 4/5 (read four times out of five tries) # 5/5 Environment: == # Which version/branch did you use? # Details of the machine? Additional data: == # Links to http://paste.openstack.org/ with content of log files. # Links to possible related bugs. # Things which might be helpful to debug this but doesn't fit into the # other sections. -- template end - +1! I think many of bug descriptions don't have enough information for debugging now. So we should have enough information like this as possible. However, one thing I'm concerned about this template is a little heavy process for submitting an easy bug. We might have some templates for depending on the situation. Thanks, -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Proposed Changes to Tempest Core
+1 ! On Jul 22, 2014 7:36 AM, Matthew Treinish mtrein...@kortar.org wrote: Hi Everyone, I would like to propose 2 changes to the Tempest core team: First, I'd like to nominate Andrea Fritolli to the Tempest core team. Over the past cycle Andrea has been steadily become more actively engaged in the Tempest community. Besides his code contributions around refactoring Tempest's authentication and credentials code, he has been providing reviews that have been of consistently high quality that show insight into both the project internals and it's future direction. In addition he has been active in the qa-specs repo both providing reviews and spec proposals, which has been very helpful as we've been adjusting to using the new process. Keeping in mind that becoming a member of the core team is about earning the trust from the members of the current core team through communication and quality reviews, not simply a matter of review numbers, I feel that Andrea will make an excellent addition to the team. As per the usual, if the current Tempest core team members would please vote +1 or -1(veto) to the nomination when you get a chance. We'll keep the polls open for 5 days or until everyone has voted. References: https://review.openstack.org/#/q/reviewer:%22Andrea+Frittoli+%22,n,z http://stackalytics.com/?user_id=andrea-frittolimetric=marksmodule=qa-group The second change that I'm proposing today is to remove Giulio Fidente from the core team. He asked to be removed from the core team a few weeks back because he is no longer able to dedicate the required time to Tempest reviews. So if there are no objections to this I will remove him from the core team in a few days. Sorry to see you leave the team Giulio... Thanks, Matt Treinish ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [tempest][scenario] independece test between Stack and backing services?
Hi, On 04/02, Tomoya Goto wrote: Thanks for quick replies Igawa-san and Mr.Sean! :) and sorry foy my slow reply :( np :) The task I wantetd to conduct is not only for upgrading but also for rather small maintenace, say stopping openstack-cinder* for changing configuration. Now, Grenade is for upgrade purpose but not for such small maintenace, right? So I think tempest is more suitable than Grenade for such task. what do you say? This kind (fault injection?) of tests that you said are interesting and we should have them in future. But Tempest should not operate OpenStack components directly. e.g. stop/start Cinder/Glance/Nova services. This is one of design principles[1]. So I think we need a new project for these types of tests or need to change the principles. [1] http://docs.openstack.org/developer/tempest/overview.html#design-principles Thanks, -- Masayuki Igawa - Tomoya Goto You are correct. The testing we do for this is in Grenade, which we run in the gate. Grenade tests an upgrade from last stable release to current master. It creates a few resources before the upgrade, and fails if those are interupted after the upgrade. Grenade is still pretty light on the number of resources it creates before the upgrade, and is definitely a place where enhancement is welcome. -Sean On 04/01/2014 04:18 AM, Masayuki Igawa wrote: Hi Goto-san, I think this is an interesting test case. But AFAIK, Tempest and its scenario tests don't have such test cases now, and we can't stop the OpenStack processes through Tempest. Do you know Grenade[1]? I think Grenade is the only one upgrading test in the OpenStack community now. So I guess Grenade can test these kind of tests but not yet though. [1] https://wiki.openstack.org/wiki/Grenade On 04/01, 後藤 僚哉 wrote: Hello everyone. I'm looking for an independence test between an OpenStack environment and virtual environments. In case of updating an openstack environment, you need to stop each OpenStack process, but you don't want the instances to be affected by OpenStack outages. So before maintenane, you want to make sure OpenStack and backing services(KVM, OVS, storage,.) are separate. For example. 1) Creating a virtual environment on a OpenStack environment. this includes Nova instances, Neutron L2/3 networks, Cinder volumes and etc. I'd like to clarify more. Do you mean OpenStack on OpenStack environment? Or just mean VMs on OpenStack env? I meant just VMs on OpenStack env. When you stop some processes for update, say openstack-cinder-*, you want to make sure it won't disconnect volume/VM. Thanks, I got it. 2) Stopping one or more OpenStack's processes. Currently, Tempest can't stop the OpenStack processes. Because Tempest can operate OpenStack components through OpenStack APIs only for now. oh yes. Just like I feard. Thanks, -- Masayuki Igawa 3) Running this test, and checking that each resource doesn't stop. 4) Updating an OpenStack, editing configurations or etc. I assume such test is coverd by tempest. Dose Tempest have those test methods? or if not, do you think it's going to be handy if I make such test? Best regards, Tomoya Goto ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa][Nova] Update: Nova v2/v3 API List for Missing Tempest Tests
Hi, Thanks many guys for updating these spreadsheets! I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of these spreadsheets: - Nova V2 APIs --- different count from Tested or not # of APIs ratio the last time --- Tested 152 60.1% +10 Not Tested[1]42 16.6% -2 Not Need to Test[2] 59 23.3% -2 --- Total: 253 100.0% +6 [1] included 5 Doings [2] Because they are deprecated APIs such as nova-network and volume. - Nova V3 APIs --- different count from Tested or not # of APIs ratio the last time --- Tested 111 78.2% +23 Not Tested[1]28 19.7% -25 Not Need to Test[2] 3 2.1% +3 --- Total: 142 100.0% +1 [1] included 6 Doings [2] Nova APIs are not implemented yet. Additional information: I made this API list with these nova's patch https://review.openstack.org/#/c/25882/ https://review.openstack.org/#/c/72277/ https://review.openstack.org/#/c/65615/ (Actually, I need to extract and summarize the data from its screen-n-api.txt.gz manually because of some reasons.) This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. And if you find any mistakes on this list, feel free to correct/update it please. Best Regards, -- Masayuki Igawa smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Run Tempest with new test case
Hi, Oscar First, openstack-qa ML is deprecated now as I said before. So please do not send an e-mail to openstack-qa ML. On 03/10, Oscar Correia wrote: I have new test case and I would like add inside Tempest directory and after execute, example: test case Swift for object and container. How is the process? Have you read How_To_Contribute wiki page[1]? Before contributing your code, you need to agree with the contributors license agreement. And, you also need to know Gerrit Workflow[2]. This is the process. [1] https://wiki.openstack.org/wiki/How_To_Contribute [2] https://wiki.openstack.org/wiki/GerritWorkflow Best Regards, -- Masayuki Igawa Oscar CorreiaCTwww.lettersvitae.comSkype: oscar.correiaFacebook: Letters VitaeE-Mail: oscar_correiaj@hotmail.comoscarmo...@lettersvitae.com Regis Regum servitium -- ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] API weekly meeting
I'm interested in this. I can join except 13-21 UTC. Thanks, -- Masayuki IGAWA 2014/03/07 9:53 Christopher Yeoh cbky...@gmail.com: Hi, I'd like to start a weekly IRC meeting for those interested in discussing Nova API issues. I think it would be a useful forum for: - People to keep up with what work is going on the API and where its headed. - Cloud providers, SDK maintainers and users of the REST API to provide feedback about the API and what they want out of it. - Help coordinate the development work on the API (both v2 and v3) If you're interested in attending please respond and include what time zone you're in so we can work out the best time to meet. Chris ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Does scenario.test_minimum_basic need to upload ami images?
Hi, I've created a patch for qcow2 on scenario tests part.[1] But I'd like to know someone already do like this or not because I'd like to avoid conflict works. [1] https://review.openstack.org/#/c/75312/ On Fri, Feb 21, 2014 at 7:01 AM, David Kranz dkr...@redhat.com wrote: On 02/20/2014 04:53 PM, Sean Dague wrote: On 02/20/2014 04:31 PM, David Kranz wrote: Running this test in tempest requires an ami image triple to be on the disk where tempest is running in order for the test to upload it. It would be a lot easier if this test could use a simple image file instead. That image file could even be obtained from the cloud being tested while configuring tempest. Is there a reason to keep the three-part image? I have no issue changing this to a single part image, as long as we could find a way that we can make it work with cirros in the gate (mostly because it can run in really low mem footprint). Is there a cirros single part image somewhere? Honestly it would be much simpler even in the devstack environment. -Sean http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [openstack-qa] Unit Tests for Cinder and Swift
Hi Oscar, I'm sorry for my delayed reply. I think this is a development topic. I added openstack-dev ML to CC this time. Actually, I'm not familiar with Swift unit tests. So I just CCed and I'm expecting answers from Swift people. -- Masayuki Igawa Thank you Masayuki for everything. I woud like help the community Openstack QA writing the Unit tests for Swift module (rings, object, container, accounts and replication), what is the next step? *Oscar Correia* CT www.lettersvitae.com Skype: oscar.correia Facebook: Letters Vitae E-Mail: oscar_corre...@hotmail.com oscarmo...@lettersvitae.com Regis Regum servitium Date: Tue, 4 Feb 2014 09:28:08 +0900 Subject: Re: [openstack-qa] Unit Tests for Cinder and Swift From: masayuki.ig...@gmail.com To: openst...@lists.openstack.org; oscar_corre...@hotmail.com CC: openstack...@lists.openstack.org Hi Oscar, As I already replied to you in another mail[1], unfortunately, the openstack...@lists.openstack.org has been deprecated. And for asking about usages, you send things to openst...@lists.openstack.org or access to https://ask.openstack.org/ . [1] http://lists.openstack.org/pipermail/openstack/2014-February/005070.html On Sun, Feb 2, 2014 at 11:45 PM, Oscar Jr oscar_corre...@hotmail.com wrote: Hi Jenkins, where I can get full Unit tests for Cinder and Swift module? You can get them from their source code: http://git.openstack.org/cgit/openstack/cinder/tree/cinder/tests http://git.openstack.org/cgit/openstack/swift/tree/test/ Thanks. Oscar Correia CT www.lettersvitae.com Skype: oscar.correia Facebook: Letters Vitae E-Mail: oscar_corre...@hotmail.com oscarmo...@lettersvitae.com Regis Regum servitium -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] [openstack-qa] Unit Tests for Cinder and Swift
Hi Oscar, I'm sorry for my delayed reply. I think this is a development topic. I added openstack-dev ML to CC this time. Actually, I'm not familiar with Swift unit tests. So I just CCed and I'm expecting answers from Swift people. -- Masayuki Igawa Thank you Masayuki for everything. I woud like help the community Openstack QA writing the Unit tests for Swift module (rings, object, container, accounts and replication), what is the next step? *Oscar Correia* CT www.lettersvitae.com Skype: oscar.correia Facebook: Letters Vitae E-Mail: oscar_corre...@hotmail.com oscarmo...@lettersvitae.com Regis Regum servitium Date: Tue, 4 Feb 2014 09:28:08 +0900 Subject: Re: [openstack-qa] Unit Tests for Cinder and Swift From: masayuki.ig...@gmail.com To: openstack@lists.openstack.org; oscar_corre...@hotmail.com CC: openstack...@lists.openstack.org Hi Oscar, As I already replied to you in another mail[1], unfortunately, the openstack...@lists.openstack.org has been deprecated. And for asking about usages, you send things to openstack@lists.openstack.org or access to https://ask.openstack.org/ . [1] http://lists.openstack.org/pipermail/openstack/2014-February/005070.html On Sun, Feb 2, 2014 at 11:45 PM, Oscar Jr oscar_corre...@hotmail.com wrote: Hi Jenkins, where I can get full Unit tests for Cinder and Swift module? You can get them from their source code: http://git.openstack.org/cgit/openstack/cinder/tree/cinder/tests http://git.openstack.org/cgit/openstack/swift/tree/test/ Thanks. Oscar Correia CT www.lettersvitae.com Skype: oscar.correia Facebook: Letters Vitae E-Mail: oscar_corre...@hotmail.com oscarmo...@lettersvitae.com Regis Regum servitium -- Masayuki Igawa ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
[openstack-dev] [qa][Nova] Update: Nova *V3* API List for Missing Tempest Tests
Hi, I have updated: Nova *V3* API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc#gid=6 The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 88 62.4% 0 Not Tested API[1]53 37.6% 0 --- Total[2]: 141 100.0% 0 [1] including some Doings [2] only v3 APIs Additional information: I made this API list with this nova's patch https://review.openstack.org/#/c/25882/ https://review.openstack.org/#/c/72277/ https://review.openstack.org/#/c/65615/ (Actually, I need to extract and summarize the data from its screen-n-api.txt.gz manually because of some reasons.) This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. And if you notice any mistakes on this list, feel free to correct/update it please. Best Regards, -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] [openstack-qa] openstack-qa Digest, Vol 15, Issue 28
Hi Oscar, Unfortunately, the openstack...@lists.openstack.org has been deprecated. For asking about usages, you send things to openstack@lists.openstack.org or access to https://ask.openstack.org/ . The only reason we keep it around is for the periodic job logs. Hi Jenkins, how I configure the Tempest to run against real deployments? My recommendation is reading the Quickstart of Tempest's README, first. http://git.openstack.org/cgit/openstack/tempest/tree/README.rst#n37 BTW, 'Jenkins' is not a real man. Just a bot :) Thanks. On Fri, Jan 31, 2014 at 12:35 AM, Oscar Jr oscar_corre...@hotmail.com wrote: Hi Jenkins, how I configure the Tempest to run against real deployments? Oscar Correia CT www.lettersvitae.com Skype: oscar.correia Facebook: Letters Vitae E-Mail: oscar_corre...@hotmail.com oscarmo...@lettersvitae.com Regis Regum servitium From: openstack-qa-requ...@lists.openstack.org Subject: openstack-qa Digest, Vol 15, Issue 28 To: openstack...@lists.openstack.org Date: Thu, 30 Jan 2014 12:00:01 + Send openstack-qa mailing list submissions to openstack...@lists.openstack.org To subscribe or unsubscribe via the World Wide Web, visit http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa or, via email, send a message with subject or body 'help' to openstack-qa-requ...@lists.openstack.org You can reach the person managing the list at openstack-qa-ow...@lists.openstack.org When replying, please edit your Subject line so it is more specific than Re: Contents of openstack-qa digest... Today's Topics: 1. Periodic jobs for openstack/tempest failed (jenk...@openstack.org) -- Message: 1 Date: Thu, 30 Jan 2014 06:35:02 + From: jenk...@openstack.org To: openstack...@lists.openstack.org Subject: [openstack-qa] Periodic jobs for openstack/tempest failed Message-ID: e1w8ldk-0005ee...@zuul.openstack.org Content-Type: text/plain; charset=us-ascii Build failed. - periodic-tempest-dsvm-all-havana http://logs.openstack.org/periodic-qa/periodic-tempest-dsvm-all-havana/84cc12c : FAILURE in 3m 38s - periodic-tempest-dsvm-stress-havana http://logs.openstack.org/periodic-qa/periodic-tempest-dsvm-stress-havana/5428519 : FAILURE in 3m 24s - periodic-tempest-dsvm-neutron-pg-havana http://logs.openstack.org/periodic-qa/periodic-tempest-dsvm-neutron-pg-havana/56f9e92 : SUCCESS in 31m 41s - periodic-tempest-dsvm-large-ops-havana http://logs.openstack.org/periodic-qa/periodic-tempest-dsvm-large-ops-havana/8093002 : SUCCESS in 11m 49s - periodic-tempest-dsvm-neutron-large-ops-havana http://logs.openstack.org/periodic-qa/periodic-tempest-dsvm-neutron-large-ops-havana/3dd9940 : SUCCESS in 13m 47s -- ___ openstack-qa mailing list openstack...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa End of openstack-qa Digest, Vol 15, Issue 28 ___ openstack-qa mailing list openstack...@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-qa -- -- Masayuki Igawa ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
[openstack-dev] [qa][Nova] Update: Nova API List for Missing Tempest Tests
Hi, Thanks many guys for updating this list! I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 142 57.5% +16 Not Tested API[1]44 17.8% -20 Not Need to Test[2] 61 24.7% +1 --- Total[3]: 247 100.0% 0 [1] included 3 Doings [2] Because they are deprecated APIs such as nova-network and volume. [3] not included v3 APIs Additional information: I made this API list with this nova's patch https://review.openstack.org/#/c/25882/ and https://review.openstack.org/#/c/65615/ (Actually, I need to extract and summarize the data from its screen-n-api.txt.gz manually because of some reasons.) This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. And if you notice any mistakes on this list, feel free to correct/update it please. Best Regards, -- Masayuki Igawa smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] Organizing a Gate Blocking Bug Fix Day
I'm in! Is the day Jan 27th? On Fri, Jan 10, 2014 at 6:33 AM, Anita Kuno ante...@anteaya.info wrote: On 01/10/2014 02:53 AM, Jay Pipes wrote: On Thu, 2014-01-09 at 07:46 -0500, Sean Dague wrote: I think we are all agreed that the current state of Gate Resets isn't good. Unfortunately some basic functionality is really not working reliably, like being able to boot a guest to a point where you can ssh into it. These are common bugs, but they aren't easy ones. We've had a few folks digging deep on these, but we, as a community, are not keeping up with them. So I'd like to propose Gate Blocking Bug Fix day, to be Monday Jan 20th. On that day I'd ask all core reviewers (and anyone else) on all projects to set aside that day to *only* work on gate blocking bugs. We'd like to quiet the queues to not include any other changes that day so that only fixes related to gate blocking bugs would be in the system. This will have multiple goals: #1 - fix some of the top issues #2 - ensure we classify (ER fingerprint) and register everything we're seeing in the gate fails #3 - ensure all gate bugs are triaged appropriately I'm hopefully that if we can get everyone looking at this one a single day, we can start to dislodge the log jam that exists. Specifically I'd like to get commitments from as many PTLs as possible that they'll both directly participate in the day, as well as encourage the rest of their project to do the same. I'm in. Due to what ttx mentioned about I-2, I think the 13th Jan or 27th Jan Mondays would be better. Personally, I think sooner is better. The severity of the disruption is quite high, and action is needed ASAP. Best, -jay I'm in. Jan. 13th is a transportation day for me as I wend my way to the Neutron Tempest code sprint in Montreal. I am operating on the belief that since other Neutron Tempest folks might also be having a transportation day, Sean has steered away from this date as an option. Thanks, Anita. ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Tempest][qa] Adding tags to commit messages
Hi, On Tue, Dec 24, 2013 at 3:47 PM, Yair Fried yfr...@redhat.com wrote: Hi, Suggestion: Please consider tagging your Tempest commit messages the same way you do your mails in the mailing list Explanation: Since tempest is a single project testing multiple Openstack project we have a very diverse collection of patches as well as reviewers. Tagging our commit messages will allow us to classify patches and thus: 1. Allow reviewer to focus on patches related to their area of expertise 2. Track trends in patches - I think we all know that we lack in Neutron testing for example, but can we assess how many network related patches are for awaiting review 3. Future automation of flagging interesting patches You can usually tell all of this from reviewing the patch, but by then - you've spent time on a patch you might not even be qualified to review. I suggest we tag our patches with, to start with, the components we are looking to test, and the type of test (sceanrio, api, ...) and that reviewers should -1 untagged patches. I think the tagging should be the 2nd line in the message: == Example commit message [Neutron][Nova][Network][Scenario] Explanation of how this scenario tests both Neutron and Nova Network performance Chang-id XXX === I would like this to start immediately but what do you guys think? +1 And, how about do we the tagging about the services in the subject(1st line)? For example: Neutron:Example commit subject Because the dashboard of the gerrit shows the subject only now. I think reviewers can find interesting patches easily if the dashboard shows the tags. This is not so strong opinion because some scenario tests may have several services tags. -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
Hi, I read the qa-meeting log[1]. And I registered a blueprint[2] for tracking and avoiding duplication. I think if we put Partially Implements: blueprint add-scenario-tests-in-icehouse in the commit message, we can avoid the duplication and tracking the scenarios. Because the commit subject and the link will be wrote automatically in the whiteboard. However, I'm not sure whether we can associate with multiple blueprints such as BP:lbaas-scenario-tests and add-scenario-tests-in-icehouse though. Is this make sense? [1] http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-11-14-17.02.log.html [2] https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse Any comments and suggestions are welcome. Best Regards, -- Masayuki Igawa On Mon, Nov 18, 2013 at 10:07 PM, Salvatore Orlando sorla...@nicira.com wrote: I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Tracking development of scenario tests
Hi, David Thanks for your reply. On Tue, Nov 19, 2013 at 12:37 AM, David Kranz dkr...@redhat.com wrote: On 11/18/2013 09:34 AM, Masayuki Igawa wrote: Hi, I read the qa-meeting log[1]. And I registered a blueprint[2] for tracking and avoiding duplication. I think if we put Partially Implements: blueprint add-scenario-tests-in-icehouse in the commit message, we can avoid the duplication and tracking the scenarios. Because the commit subject and the link will be wrote automatically in the whiteboard. However, I'm not sure whether we can associate with multiple blueprints such as BP:lbaas-scenario-tests and add-scenario-tests-in-icehouse though. Is this make sense? [1] http://eavesdrop.openstack.org/meetings/qa/2013/qa.2013-11-14-17.02.log.html [2] https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse Any comments and suggestions are welcome. Best Regards, -- Masayuki Igawa I think there could also be links to other blueprints either in the whiteboard or main section of the blueprint. At the meeting we just said there should be some way to get from the master blueprint to information about each new scenario being created. -David I've added three links on the whiteboard of the blueprint. https://blueprints.launchpad.net/tempest/+spec/add-scenario-tests-in-icehouse --- * https://blueprints.launchpad.net/tempest/+spec/neutron-advanced-scenarios Advanced scenario tests for Neutron * https://blueprints.launchpad.net/tempest/+spec/lbaas-scenario-tests Add advanced scenario tests for Neutron LBaaS sevice * https://review.openstack.org/#/c/56923/ zeroth version of l3 topology scenario --- Every developers can modify the whiteboard. So developers can add their scenario to this white board by themselves or automatically. I hope this BP could be useful for tracking scenarios and avoiding duplicate development. Best Regards, -- Masayuki Igawa On Mon, Nov 18, 2013 at 10:07 PM, Salvatore Orlando sorla...@nicira.com wrote: I've pushed https://review.openstack.org/#/c/56923/ trying to follow this protocol. Salvatore On 14 November 2013 16:38, Zhi Kun Liu liuzhi...@gmail.com wrote: +1, This is a great idea. We could consider it as a general process for all tests. 2013/11/14 Koderer, Marc m.kode...@telekom.de Hi all, I think we have quite the same issue with the neutron testing. I already put it on the agenda for the QA meeting for today. Let's make it to a general topic. Regards Marc From: Giulio Fidente [gfide...@redhat.com] Sent: Thursday, November 14, 2013 6:23 AM To: openstack-dev@lists.openstack.org Subject: Re: [openstack-dev] [qa] Tracking development of scenario tests On 11/14/2013 12:24 AM, David Kranz wrote: 1. Developer checks in the zeroth version of a scenario test as work in progress. It contains a description of the test, and possibly work items. This will claim the area of the proposed scenario to avoid duplication and allow others to comment through gerrit. 2. The developer pushes new versions, removing work in progress if the code is in working state and a review is desired and/or others will be contributing to the scenario. 3. When finished, any process-oriented content such as progress tracking is removed and the test is ready for final review. +1 , the description will eventually contribute to documenting the scenarios yet the submitter (step 1) remains in charge of adding to the draft the reviewers how about we map at least one volunteer to each service (via the HACKING file) and ask submitters to add such a person as reviewer of its drafts when the tests touch the service? this should help avoid tests duplication. I very much like the idea of using gerrit for this -- Giulio Fidente GPG KEY: 08D733BA | IRC: giulivo ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Masayuki Igawa ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests
Hi, Ivan Thank you for your information. And I'm sorry for delay reply. Hi, I also collect the tests status for nova v3 api manually. You can find the status of v3 tests in this link: https://etherpad.openstack.org/p/nova-v3-tests Because there are some extensions that just extend the attribute of specific api reponse, or convert some input before specific api called. There is no url for these extension, but I think we also need check them. I collect the tempest tests according to the nova api code. check tests to the action one by one, list the status file by file. I have a question. Is there any way to list the nova v3 apis? If so, I think the checking process of the test implementation can be automatically. Best Regards, -- Masayuki Igawa Due to these patch https://review.openstack.org/#/c/39609/ and https://review.openstack.org/#/c/39621/ are still under reviewing. so we can't add tempest test for nova v3 api. (almost existing tests have been ported into v3, but these patches depend on the 39609 and 39621). The status of porting existing tests is also listed in this link. we will add the missing tests in v2 firstly, when it can be ported into v3, will port it. Best Regards Ivan Zhu On 2013年10月15日 14:25, Masayuki Igawa wrote: Hi, First, thank you to an anonymous for updating this list! - GET /{project_id}/servers/:server_id/diagnostics And, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 124 49.6% +2 Not Tested API 66 26.4% -2 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs The tempest version is: commit f55f4e54ceab7c6a4d330f92c8059e46233e3560 Merge: 86ab238 062e30a Author: Jenkins jenk...@review.openstack.org mailto:jenk...@review.openstack.org Date: Mon Oct 14 15:55:59 2013 + By the way, I saw a design summit proposal related to this topic(*3). I think this information should be generated automatically. So I'd like to talk about this topic at the summit session. (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171 This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. Best Regards, -- Masayuki Igawa Hi, # I'm sorry for this resending because my last mail has unnecessary messages. I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not# of APIs ratio the last time --- Tested API 122 48.8% +5 Not Tested API 68 27.2% -5 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs I hope this information would be helpful for creating Tempest tests. Any comments and questions are welcome. Best Regards, -- Masayuki Igawa Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs
[openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests
Hi, Thanks many guys for updating this list! And, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 126 50.4% +2 Not Tested API(*1) 64 25.6% -2 Not Need to Test(*2) 60 24.0% 0 --- Total(*3): 250 100.0% 0 (*1) included 9 Doings (*2) Because they are deprecated APIs such as nova-network and volume. (*3) not included v3 APIs The tempest version is: - commit 7ca13ed9daa710cbe1ac348cb903ffada4f8f6d2 Merge: 66ff406 72d7d5b Author: Jenkins jenk...@review.openstack.org Date: Mon Oct 21 04:29:49 2013 + Merge add test for updating server's disk_config test - This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. And if you notice any mistakes on this list, feel free to correct/update it please. Best Regards, -- Masayuki Igawa Hi, First, thank you to an anonymous for updating this list! - GET /{project_id}/servers/:server_id/diagnostics And, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API124 49.6% +2 Not Tested API 66 26.4% -2 Not Need to Test(*1) 60 24.0% 0 --- Total(*2):250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs The tempest version is: commit f55f4e54ceab7c6a4d330f92c8059e46233e3560 Merge: 86ab238 062e30a Author: Jenkins jenk...@review.openstack.org Date: Mon Oct 14 15:55:59 2013 + By the way, I saw a design summit proposal related to this topic(*3). I think this information should be generated automatically. So I'd like to talk about this topic at the summit session. (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171 This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. Best Regards, -- Masayuki Igawa Hi, # I'm sorry for this resending because my last mail has unnecessary messages. I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not# of APIs ratio the last time --- Tested API 122 48.8% +5 Not Tested API 68 27.2% -5 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs I hope this information would be helpful for creating Tempest tests. Any comments and questions are welcome. Best Regards, -- Masayuki Igawa Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs are not executed. But they maybe not need to test. - Because they are deprecated APIs such as nova-network and volume. So I think we need more tempest test cases. If this idea is acceptable, can you put your name to 'assignee' at your favorites, and implement tempest tests. Any comments are welcome. Additional information: I made this API list with modification of nova's code that based on https://review.openstack.org/#/c/25882/ (Abandoned). Best Regards, -- Masayuki Igawa smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests
Hi, First, thank you to an anonymous for updating this list! - GET /{project_id}/servers/:server_id/diagnostics And, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 124 49.6% +2 Not Tested API 66 26.4% -2 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs The tempest version is: commit f55f4e54ceab7c6a4d330f92c8059e46233e3560 Merge: 86ab238 062e30a Author: Jenkins jenk...@review.openstack.org Date: Mon Oct 14 15:55:59 2013 + By the way, I saw a design summit proposal related to this topic(*3). I think this information should be generated automatically. So I'd like to talk about this topic at the summit session. (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171 This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. Best Regards, -- Masayuki Igawa Hi, # I'm sorry for this resending because my last mail has unnecessary messages. I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not# of APIsratio the last time --- Tested API122 48.8% +5 Not Tested API 68 27.2% -5 Not Need to Test(*1) 60 24.0% 0 --- Total(*2):250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs I hope this information would be helpful for creating Tempest tests. Any comments and questions are welcome. Best Regards, -- Masayuki Igawa Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs are not executed. But they maybe not need to test. - Because they are deprecated APIs such as nova-network and volume. So I think we need more tempest test cases. If this idea is acceptable, can you put your name to 'assignee' at your favorites, and implement tempest tests. Any comments are welcome. Additional information: I made this API list with modification of nova's code that based on https://review.openstack.org/#/c/25882/ (Abandoned). Best Regards, -- Masayuki Igawa smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests
Hi, Matthew Thank you very much for your reply. I'm glad that others have an interest in this topic. I've started an etherpad for that discussion here: https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling Right now it's a very rough outline, without much on it. I'm planning to add more later. But, feel free to add any discussion points or information that you think needs to be a part of the session. Thanks. I'll add discussion points and information. By the way, I think another way to increase test coverage. That is Test Driven Development by using Tempest. https://etherpad.openstack.org/p/icehouse-summit-qa-tdd-by-tempest I'd like to propose this topic to Icehouse design summit.(not yet) Because we implement a Tempest test code after implementing an execution code such as Nova, Cinder.. and so on now. But I think this flow is one of the missing Tempest test reasons. I think we can increase the test coverage of Tempest if we write a Tempest code first. Best Regards, -- Masayuki Igawa On 2013/10/15 23:24:56 +0900, Matthew Treinish wrote: On Tue, Oct 15, 2013 at 06:25:28AM +, Masayuki Igawa wrote: Hi, First, thank you to an anonymous for updating this list! - GET /{project_id}/servers/:server_id/diagnostics And, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not # of APIs ratio the last time --- Tested API 124 49.6% +2 Not Tested API 66 26.4% -2 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs The tempest version is: commit f55f4e54ceab7c6a4d330f92c8059e46233e3560 Merge: 86ab238 062e30a Author: Jenkins jenk...@review.openstack.org Date: Mon Oct 14 15:55:59 2013 + By the way, I saw a design summit proposal related to this topic(*3). I think this information should be generated automatically. So I'd like to talk about this topic at the summit session. (*3) Coverage analysis tooling: http://summit.openstack.org/cfp/details/171 I'm glad that others have an interest in this topic. I've started an etherpad for that discussion here: https://etherpad.openstack.org/p/icehouse-summit-qa-coverage-tooling Right now it's a very rough outline, without much on it. I'm planning to add more later. But, feel free to add any discussion points or information that you think needs to be a part of the session. -Matt Treinish This information would be useful for creating Tempest tests. Any comments/questions/suggestions are welcome. Best Regards, -- Masayuki Igawa Hi, # I'm sorry for this resending because my last mail has unnecessary messages. I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not# of APIsratio the last time --- Tested API122 48.8% +5 Not Tested API 68 27.2% -5 Not Need to Test(*1) 60 24.0% 0 --- Total(*2):250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs I hope this information would be helpful for creating Tempest tests. Any comments and questions are welcome. Best Regards, -- Masayuki Igawa Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs are not executed. But they maybe not need to test. - Because they are deprecated APIs such as nova-network and volume. So I think we need more tempest test cases. If this idea is acceptable, can you put your name to 'assignee' at your favorites, and implement tempest tests. Any comments are welcome. Additional information: I made this API list with modification of nova's code that based on https://review.openstack.org/#/c/25882/ (Abandoned
[openstack-dev] [qa] Update: Nova API List for Missing Tempest Tests
Hi, I have updated: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc The summary of this list: different count from Tested or not# of APIs ratio the last time --- Tested API 122 48.8% +5 Not Tested API 68 27.2% -5 Not Need to Test(*1) 60 24.0% 0 --- Total(*2): 250 100.0% 0 (*1) Because they are deprecated APIs such as nova-network and volume. (*2) not included v3 APIs I hope this information would be helpful for creating Tempest tests. Any comments and questions are welcome. Best Regards, -- Masayuki Igawa Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs are not executed. But they maybe not need to test. - Because they are deprecated APIs such as nova-network and volume. So I think we need more tempest test cases. If this idea is acceptable, can you put your name to 'assignee' at your favorites, and implement tempest tests. Any comments are welcome. Additional information: I made this API list with modification of nova's code that based on https://review.openstack.org/#/c/25882/ (Abandoned). Best Regards, -- Masayuki Igawa - 《本メールの取り扱い》・区分:秘密 ・開示:配布先限り ・制限条件:重要扱い ・持出:禁止 ・期限:無期限 ・用済後:廃棄 - -- ITシステム事業部 クラウド基盤サービス開発G 井川 征幸 - 《本メールの取り扱い》・区分:秘密 ・開示:配布先限り ・制限条件:重要扱い ・持出:禁止 ・期限:無期限 ・用済後:廃棄 - -- ITシステム事業部 クラウド基盤サービス開発G 井川 征幸 smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [QA] Nova API List for Missing Tempest Tests
Hi, Tempest developers I have made: Nova API List for Missing Tempest Tests. https://docs.google.com/spreadsheet/ccc?key=0AmYuZ6T4IJETdEVNTWlYVUVOWURmOERSZ0VGc1BBQWc This list shows what we should test. That is: * Nova has 250 APIs(not include v3 APIs). * 117 APIs are executed(maybe tested). * 73 APIs are not executed. * 60 APIs are not executed. But they maybe not need to test. - Because they are deprecated APIs such as nova-network and volume. So I think we need more tempest test cases. If this idea is acceptable, can you put your name to 'assignee' at your favorites, and implement tempest tests. Any comments are welcome. Additional information: I made this API list with modification of nova's code that based on https://review.openstack.org/#/c/25882/ (Abandoned). Best Regards, -- Masayuki Igawa smime.p7s Description: S/MIME cryptographic signature ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev