Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
Mike, This effort has taken quite some time and was going to require hard decisions to be made at some point. You have been more than patient in this process and I commend you for that as well as all the communication. Thank you for continuing to drive this! Jay On Mar 24, 2015 10:55 PM, Monty Taylor mord...@inaugust.com wrote: On 03/24/2015 06:05 PM, Kyle Mestery wrote: On Tue, Mar 24, 2015 at 9:56 AM, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Mark McClain's message of 2015-03-24 10:25:31 -0400: Echoing both Thierry and John. I support Mike’s decision to enforce the requirement. Maintaining drivers in the tree comes with responsibilities to the community and 3rd party CI is one of the them. Mike enforcing this requirement was the right action even if a hard one to take. mark Indeed. The deadlines were communicated clearly, and frequently. Making phone calls to reach out to contributors for issues like this is exceptional. +1000. Enabling contributors is great, but CI systems are an important part of that enablement. I appreciate what Mike has done to drive Cinder towards a quality level for all contributors here. It's a hard stance to take, but saying No can sometimes be the right decision. I'm just going to pile on. People keep asking us to focus on quality over landing features. It was the #1 request from EVERY operator at the recent Ops summit. This is one of that facets of doing that. It's hard, and it doesn't always feel good to all the parties involved - but it's important. Thank you, Mike, for sticking to your deadline. OpenStack will be better for it. If it's not tested, it's broken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Tue, Mar 24, 2015 at 4:52 AM, Thierry Carrez thie...@openstack.org wrote: Walter A. Boring IV wrote: Thanks Mike for all of your efforts on this, +1 I think Mike checked all the possible boxes to give fair warning to driver owners. It's hard to say no in the name of quality. It's so much easier to just say yes and avoid all the hatemail and the pressure. Mike did the right thing, he did it the right way, and he needs all the support the rest of our community can give him. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Just adding my support to the very hard thing that Mike is doing here. As mentioned discussions and warnings have gone on ad nauseum over the last year. I may not agree with some of the wording or depictions, and I certainly am empathetic here; but the fact is this has been a year long process that was communicated, discussed and help provided. NOTE this started at the summit in Atlanta!!! CI can be hard, the work of a lot of people in Cinder, the Infra team and others have made it a lot easier. People have also spent countless hours writing code for this, setting up their own systems and helping others out via IRC and even a dedicated weekly meeting as well as a time slot every week in Cinders meeting. If the reasons were different than my data center went offline or I can't host a public web server I might have a different opinion. But I have a really hard time with this sort of thing coming from companies the size and scale of Microsoft, NetApp and Oracle. Anyway, I do feel bad but not as bad as I'd feel for everybody that worked their butts off on this whole topic for the last year if we turn around and punt it again. John __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Mar 24, 2015, at 9:30 AM, John Griffith john.griffi...@gmail.com wrote: On Tue, Mar 24, 2015 at 4:52 AM, Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org wrote: Walter A. Boring IV wrote: Thanks Mike for all of your efforts on this, +1 I think Mike checked all the possible boxes to give fair warning to driver owners. It's hard to say no in the name of quality. It's so much easier to just say yes and avoid all the hatemail and the pressure. Mike did the right thing, he did it the right way, and he needs all the support the rest of our community can give him. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://openstack-dev-requ...@lists.openstack.org/?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev Just adding my support to the very hard thing that Mike is doing here. As mentioned discussions and warnings have gone on ad nauseum over the last year. I may not agree with some of the wording or depictions, and I certainly am empathetic here; but the fact is this has been a year long process that was communicated, discussed and help provided. NOTE this started at the summit in Atlanta!!! CI can be hard, the work of a lot of people in Cinder, the Infra team and others have made it a lot easier. People have also spent countless hours writing code for this, setting up their own systems and helping others out via IRC and even a dedicated weekly meeting as well as a time slot every week in Cinders meeting. If the reasons were different than my data center went offline or I can't host a public web server I might have a different opinion. But I have a really hard time with this sort of thing coming from companies the size and scale of Microsoft, NetApp and Oracle. Anyway, I do feel bad but not as bad as I'd feel for everybody that worked their butts off on this whole topic for the last year if we turn around and punt it again. John Echoing both Thierry and John. I support Mike’s decision to enforce the requirement. Maintaining drivers in the tree comes with responsibilities to the community and 3rd party CI is one of the them. Mike enforcing this requirement was the right action even if a hard one to take. mark__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
Walter A. Boring IV wrote: Thanks Mike for all of your efforts on this, +1 I think Mike checked all the possible boxes to give fair warning to driver owners. It's hard to say no in the name of quality. It's so much easier to just say yes and avoid all the hatemail and the pressure. Mike did the right thing, he did it the right way, and he needs all the support the rest of our community can give him. -- Thierry Carrez (ttx) __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Tue, Mar 24, 2015 at 2:51 AM, Walter A. Boring IV walter.bor...@hp.com wrote: On 03/23/2015 01:50 PM, Mike Perez wrote: On 12:59 Mon 23 Mar , Stefano Maffulli wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? Just to be clear, here's all the communication attempts made to vendors: 1) Talks during the design summit and the meetup on Friday at the summit. 2) Discussions at the Cinder midcycle meetups in Fort Collins and Austin. 4) Individual emails to driver maintainers. This includes anyone else who has worked on the driver file according to the git logs. 5) Reminders on the mailing list. 6) Reminders on IRC and Cinder IRC meetings every week. 7) If you submitted a new driver in Kilo, you had the annoying reminder from reviewers that your driver needs to have a CI by Kilo. And lastly I have made phone calls to companies that have shown zero responses to my emails or giving me updates. This is very difficult with larger companies because you're redirected from one person to another of who their OpenStack person is. I've left reminders on given voice mail extensions. I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. I completely agree here Mike. The Cinder cores, PTL, and the rest of the community have been talking about getting CI as a requirement for quite some time now. It's really not the fault of the Cinder PTL, or core members, that your driver got pulled from the Kilo release, because you had issues getting your CI up and stable in the required time frame. Mike made every possible attempt to let folks know, up front, that the deadline was going to happen. Getting CI in place is critical for the stability of Cinder in general. We have already benefited from having 3rd Party CI in place. It wasn't but a few weeks ago that a change that was submitted actually broke the HP drivers. The CI we had in place discovered it, and brought it to the surface. Without having that CI in place for our drivers, we would be in a bad spot now. +1, we (GlusterFS) too discovered issues with live snapshot (being one of the very few that uses it in cinder) tests failing as part of CI and we fixed it [1] [1]: https://review.openstack.org/#/c/156940/ thanx, deepak __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
I also absolutely agree that Mike did a great job on the communication with the driver maintainers and a lot more, especially in the hectic days around the K-3 deadline. Removing any driver lacking CI testing was just the right thing to do, even if this affected our SMB3 driver. Hopefully this is just temporary as the related CI is currently under test as well (whose delays are totally unrelated), so I dont see it as a particularly dramatic decision. It also aligns well with the policies applied it other major OpenStack projects like Nova or Neutron. I'd be even in favor of a driver decomposition approach as Neutron did, but that's another topic. So, thanks again for all your great work and help! Alessandro On 24 Mar 2015, at 16:41, Joshua Harlow harlo...@outlook.com wrote: +10 to mike; I have no doubt this is an uneasy and tough task. Thanks mike for pushing this through; given all the challenges and hard work (and likely not fun work) that had to be done. I salute u! :) -Josh Duncan Thomas wrote: On 23 March 2015 at 22:50, Mike Perez thin...@gmail.com mailto:thin...@gmail.com wrote: I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. As somebody who has previously attempted to drive 3rd party CI in Cinder forward and completely burnt out on the process, I have to applaud Mike's efforts. We needed a line in the sand to force the issue, or we were never going to get to 100% coverage of drivers, which was what we desperately needed. For those who've have issues getting CI to be stable, this is a genuine reflection of the stability of Openstack in general, but it is not something we're going to be able to make progress on without CI systems exposing problems. That's the entire point of the 3rd party CI program - we knew there were bugs, stability issues and drivers that just plain didn't work, but we couldn't do much about it without being able to test changes. Thanks to Mike for the finally push on this, and to all of the various people in both cinder and infra who've been very active in helping people get their CI running, sometimes in very trying circumstances. -- Duncan Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Tue, Mar 24, 2015 at 9:56 AM, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Mark McClain's message of 2015-03-24 10:25:31 -0400: Echoing both Thierry and John. I support Mike’s decision to enforce the requirement. Maintaining drivers in the tree comes with responsibilities to the community and 3rd party CI is one of the them. Mike enforcing this requirement was the right action even if a hard one to take. mark Indeed. The deadlines were communicated clearly, and frequently. Making phone calls to reach out to contributors for issues like this is exceptional. +1000. Enabling contributors is great, but CI systems are an important part of that enablement. I appreciate what Mike has done to drive Cinder towards a quality level for all contributors here. It's a hard stance to take, but saying No can sometimes be the right decision. Thanks, Kyle Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
+10 to mike; I have no doubt this is an uneasy and tough task. Thanks mike for pushing this through; given all the challenges and hard work (and likely not fun work) that had to be done. I salute u! :) -Josh Duncan Thomas wrote: On 23 March 2015 at 22:50, Mike Perez thin...@gmail.com mailto:thin...@gmail.com wrote: I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. As somebody who has previously attempted to drive 3rd party CI in Cinder forward and completely burnt out on the process, I have to applaud Mike's efforts. We needed a line in the sand to force the issue, or we were never going to get to 100% coverage of drivers, which was what we desperately needed. For those who've have issues getting CI to be stable, this is a genuine reflection of the stability of Openstack in general, but it is not something we're going to be able to make progress on without CI systems exposing problems. That's the entire point of the 3rd party CI program - we knew there were bugs, stability issues and drivers that just plain didn't work, but we couldn't do much about it without being able to test changes. Thanks to Mike for the finally push on this, and to all of the various people in both cinder and infra who've been very active in helping people get their CI running, sometimes in very trying circumstances. -- Duncan Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 03/24/2015 06:05 PM, Kyle Mestery wrote: On Tue, Mar 24, 2015 at 9:56 AM, Doug Hellmann d...@doughellmann.com wrote: Excerpts from Mark McClain's message of 2015-03-24 10:25:31 -0400: Echoing both Thierry and John. I support Mike’s decision to enforce the requirement. Maintaining drivers in the tree comes with responsibilities to the community and 3rd party CI is one of the them. Mike enforcing this requirement was the right action even if a hard one to take. mark Indeed. The deadlines were communicated clearly, and frequently. Making phone calls to reach out to contributors for issues like this is exceptional. +1000. Enabling contributors is great, but CI systems are an important part of that enablement. I appreciate what Mike has done to drive Cinder towards a quality level for all contributors here. It's a hard stance to take, but saying No can sometimes be the right decision. I'm just going to pile on. People keep asking us to focus on quality over landing features. It was the #1 request from EVERY operator at the recent Ops summit. This is one of that facets of doing that. It's hard, and it doesn't always feel good to all the parties involved - but it's important. Thank you, Mike, for sticking to your deadline. OpenStack will be better for it. If it's not tested, it's broken __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
One other item I’d like to bring up. While Nova and Neutron are well distributed around the globe and have Core Reviewers on IRC in the Asia daytime, some other projects are not so well distributed as yet. A problem I noticed a number of times is that an Asia based developer will post to the mailing list to get some attention for his/her patch. This is frowned upon in the community, but when there are few to no Core Reviewers in IRC, getting that first core review can be difficult. Emailing the PTL is something I’m sure the PTLs would like to limit as they are already swamped. So, how do we get timely first core review of patches in areas of the world where Core presence in IRC is slim to none? I can think of a few options but they don’t seem great: · A filter for dashboards that flags reviews with multiple +1s and no core along with a commitment of the Core team to perform a review within x number of days · A separate mailing list for project review requests · Somehow queueing requests in the IRC channel so that offline developers can easily find review requests when looking at channel logs · ??? Solving this issue could help not just Third Party developers, but all of OpenStack and make the community more inviting to Asian and Australian (and maybe European and African) developers. --Rocky From: Rochelle Grober [mailto:rochelle.gro...@huawei.com] Sent: Monday, March 23, 2015 14:51 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI)) I’d like to suggest that the myriad wiki pages and spreadsheets for Third Party CI also be consolidated to a more manageable count. Just looking for maintainers contact, you can find information (often conflicting) in Stackalytics, on the ThirdPartyDrivers page, on the Cinder PTL’s google doc and who knows where else for the Neutron maintainers. Even finding which tests to run takes linking through a number of Cinder wiki pages. The teams have done a great job documenting a process that started out as lore, but I think the beginning of L would be a great time to revisit and reorganize the documentation for clarity, conciseness and single locations (version controlled) of critical information. --Rocky From: Patrick East [mailto:patrick.e...@purestorage.com] Sent: Monday, March 23, 2015 14:38 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI)) On Mon, Mar 23, 2015 at 12:59 PM, Stefano Maffulli stef...@openstack.orgmailto:stef...@openstack.org wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. I think the only failure is on the side of any driver maintainers who did not make the deadlines. From my perspective (as one of the driver maintainers who did setup a CI system and a developer working on Cinder) this whole process has been a success. The test coverage has sky rocketed for Cinder, driver maintainers are forced to be a bit more active in the community, and the code base (in theory) no longer has volume drivers in tree that we don't know if they actually work or not. This is, in my opinion, a huge win for the project. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? For anyone who struggled with getting a CI system operational there are numerous resources at your disposal (all of which have been advertised in Cinder meetings and the #openstack-cinder IRC channel). There are three meetings every week where you can get help setting them up [1]. There are a few different Cinder developers who have set up their own CI systems and shared code/instructions [2][3]. I have seen those same devs supporting them via IRC and have enabled several other companies to successfully use their tools. Between these resources I don't think anyone who has actually showed up at the meetings, asked for help, and make a good faith effort to keep everyone in the loop and show progress
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
Excerpts from Mark McClain's message of 2015-03-24 10:25:31 -0400: Echoing both Thierry and John. I support Mike’s decision to enforce the requirement. Maintaining drivers in the tree comes with responsibilities to the community and 3rd party CI is one of the them. Mike enforcing this requirement was the right action even if a hard one to take. mark Indeed. The deadlines were communicated clearly, and frequently. Making phone calls to reach out to contributors for issues like this is exceptional. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Mon, Mar 23, 2015 at 9:23 PM, Anita Kuno ante...@anteaya.info wrote: I'm really disappointed that there hasn't been more support for Mike in this process. I can see how everyone thought I was the problem last year when I had to endure this kind of treatment in Neutron, but I would think after seeing the exact same kind of behaviour a second time folks might be starting to see the pattern. This CI is in my opinion one of the most important undertakings in Cinder to date. Cinder is basically an abstraction over a set of drivers, so testing those drivers is very important. This is especially true for those that deploy OpenStack at various customer sites, each with different hardware, and up to now just had to pray that things worked. The discussion about the CI has been going on forever. Mike brought it up so many times in every forum possible and did a great job with this difficult task. While I understand that setting this up is not a simple task, I think there was enough time. We have been discussing this CI forever, and if action is not taken now, it will never happen. This is not the end of the world for drivers that are removed. Some drivers are already hosted in their own github repos as well as in Cinder's, so vendors can go with that route. Or maybe there will be an exception made to allow backports for removed drivers (I'm not sure this is a good idea). Anyway, I'm very happy to finally have a release where I know that all drivers are more-or-less working. Kudos to Mike, and to all of the Cinder folks that pioneered the effort and provided support to those that followed. -- *Avishay Traeger* *Storage RD* Mobile: +972 54 447 1475 E-mail: avis...@stratoscale.com Web http://www.stratoscale.com/ | Blog http://www.stratoscale.com/blog/ | Twitter https://twitter.com/Stratoscale | Google+ https://plus.google.com/u/1/b/108421603458396133912/108421603458396133912/posts | Linkedin https://www.linkedin.com/company/stratoscale __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 23 March 2015 at 22:50, Mike Perez thin...@gmail.com wrote: I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. As somebody who has previously attempted to drive 3rd party CI in Cinder forward and completely burnt out on the process, I have to applaud Mike's efforts. We needed a line in the sand to force the issue, or we were never going to get to 100% coverage of drivers, which was what we desperately needed. For those who've have issues getting CI to be stable, this is a genuine reflection of the stability of Openstack in general, but it is not something we're going to be able to make progress on without CI systems exposing problems. That's the entire point of the 3rd party CI program - we knew there were bugs, stability issues and drivers that just plain didn't work, but we couldn't do much about it without being able to test changes. Thanks to Mike for the finally push on this, and to all of the various people in both cinder and infra who've been very active in helping people get their CI running, sometimes in very trying circumstances. -- Duncan Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
Dell - Internal Use - Confidential +2 on Mike’s job From: Duncan Thomas [mailto:duncan.tho...@gmail.com] Sent: Tuesday, March 24, 2015 3:07 AM To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI)) On 23 March 2015 at 22:50, Mike Perez thin...@gmail.commailto:thin...@gmail.com wrote: I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. As somebody who has previously attempted to drive 3rd party CI in Cinder forward and completely burnt out on the process, I have to applaud Mike's efforts. We needed a line in the sand to force the issue, or we were never going to get to 100% coverage of drivers, which was what we desperately needed. For those who've have issues getting CI to be stable, this is a genuine reflection of the stability of Openstack in general, but it is not something we're going to be able to make progress on without CI systems exposing problems. That's the entire point of the 3rd party CI program - we knew there were bugs, stability issues and drivers that just plain didn't work, but we couldn't do much about it without being able to test changes. Thanks to Mike for the finally push on this, and to all of the various people in both cinder and infra who've been very active in helping people get their CI running, sometimes in very trying circumstances. -- Duncan Thomas __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 03/23/2015 03:59 PM, Stefano Maffulli wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. The thing that quite visibly that happened is that some folks don't recognize that their actions affect others and take responsibility for those actions. I have learned much to my dismay that there is a certain percentage of a group that will not follow the intent of the group despite any amount of support given to them. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? Asking this question dismisses all the hard work that both Mike and myself and many others including some very committed operators have put into this space. Some folks just will not respect other people's time. To pretend otherwise is a huge dis-service to folks trying their hardest to support those worthy of the support. I'm really disappointed that there hasn't been more support for Mike in this process. I can see how everyone thought I was the problem last year when I had to endure this kind of treatment in Neutron, but I would think after seeing the exact same kind of behaviour a second time folks might be starting to see the pattern. Anita. How should the processes and tool be different so you'll be successful with your OpenStack-based products? Cheers, stef __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? Cheers, stef __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 03/23/2015 01:50 PM, Mike Perez wrote: On 12:59 Mon 23 Mar , Stefano Maffulli wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? Just to be clear, here's all the communication attempts made to vendors: 1) Talks during the design summit and the meetup on Friday at the summit. 2) Discussions at the Cinder midcycle meetups in Fort Collins and Austin. 4) Individual emails to driver maintainers. This includes anyone else who has worked on the driver file according to the git logs. 5) Reminders on the mailing list. 6) Reminders on IRC and Cinder IRC meetings every week. 7) If you submitted a new driver in Kilo, you had the annoying reminder from reviewers that your driver needs to have a CI by Kilo. And lastly I have made phone calls to companies that have shown zero responses to my emails or giving me updates. This is very difficult with larger companies because you're redirected from one person to another of who their OpenStack person is. I've left reminders on given voice mail extensions. I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. I completely agree here Mike. The Cinder cores, PTL, and the rest of the community have been talking about getting CI as a requirement for quite some time now. It's really not the fault of the Cinder PTL, or core members, that your driver got pulled from the Kilo release, because you had issues getting your CI up and stable in the required time frame. Mike made every possible attempt to let folks know, up front, that the deadline was going to happen. Getting CI in place is critical for the stability of Cinder in general. We have already benefited from having 3rd Party CI in place. It wasn't but a few weeks ago that a change that was submitted actually broke the HP drivers. The CI we had in place discovered it, and brought it to the surface. Without having that CI in place for our drivers, we would be in a bad spot now. In other words, it should be a top priority for vendors to get CI in place, if for the selfish reason of protecting their code!!! That being said, I look forward to seeing folks submit their drivers back in the early L time frame. If my driver got pulled for K, It would be my top priority to get CI working NOW, and the day L opens up, I have my driver patch up, with CI reporting. Thanks Mike for all of your efforts on this, Walt __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
I’d like to suggest that the myriad wiki pages and spreadsheets for Third Party CI also be consolidated to a more manageable count. Just looking for maintainers contact, you can find information (often conflicting) in Stackalytics, on the ThirdPartyDrivers page, on the Cinder PTL’s google doc and who knows where else for the Neutron maintainers. Even finding which tests to run takes linking through a number of Cinder wiki pages. The teams have done a great job documenting a process that started out as lore, but I think the beginning of L would be a great time to revisit and reorganize the documentation for clarity, conciseness and single locations (version controlled) of critical information. --Rocky From: Patrick East [mailto:patrick.e...@purestorage.com] Sent: Monday, March 23, 2015 14:38 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI)) On Mon, Mar 23, 2015 at 12:59 PM, Stefano Maffulli stef...@openstack.orgmailto:stef...@openstack.org wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. I think the only failure is on the side of any driver maintainers who did not make the deadlines. From my perspective (as one of the driver maintainers who did setup a CI system and a developer working on Cinder) this whole process has been a success. The test coverage has sky rocketed for Cinder, driver maintainers are forced to be a bit more active in the community, and the code base (in theory) no longer has volume drivers in tree that we don't know if they actually work or not. This is, in my opinion, a huge win for the project. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? For anyone who struggled with getting a CI system operational there are numerous resources at your disposal (all of which have been advertised in Cinder meetings and the #openstack-cinder IRC channel). There are three meetings every week where you can get help setting them up [1]. There are a few different Cinder developers who have set up their own CI systems and shared code/instructions [2][3]. I have seen those same devs supporting them via IRC and have enabled several other companies to successfully use their tools. Between these resources I don't think anyone who has actually showed up at the meetings, asked for help, and make a good faith effort to keep everyone in the loop and show progress failed to get their system online and keep their driver in Cinder its not a coincidence. There are also efforts to provide an easier to use CI system that is shared with the OpenStack infra team [4]. I would recommend anyone who wants to help ease this process for new drivers/maintainers to help contribute to this effort. I think this is going to be the best route forward to ensure people have the tools they need to setup and operate a stable third party ci system. 1 - https://wiki.openstack.org/wiki/Meetings/ThirdParty#Weekly_Third_Party_meetings 2 - https://github.com/rasselin/os-ext-testing 3 - https://github.com/j-griffith/sos-ci 4 - https://review.openstack.org/#/c/139745/ -Patrick __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 03/23/2015 05:51 PM, Rochelle Grober wrote: I’d like to suggest that the myriad wiki pages and spreadsheets for Third Party CI also be consolidated to a more manageable count. Just looking for maintainers contact, you can find information (often conflicting) in Stackalytics, on the ThirdPartyDrivers page, on the Cinder PTL’s google doc and who knows where else for the Neutron maintainers. Even finding which tests to run takes linking through a number of Cinder wiki pages. The teams have done a great job documenting a process that started out as lore, but I think the beginning of L would be a great time to revisit and reorganize the documentation for clarity, conciseness and single locations (version controlled) of critical information. --Rocky Sure. Since that is the first goal of the third-party meetings, everyone is welcome to attend and assist with this ongoing effort: https://wiki.openstack.org/wiki/Meetings/ThirdParty#Goals_for_Third_Party_meetings Thanks Rocky, Anita. From: Patrick East [mailto:patrick.e...@purestorage.com] Sent: Monday, March 23, 2015 14:38 To: OpenStack Development Mailing List (not for usage questions) Subject: Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI)) On Mon, Mar 23, 2015 at 12:59 PM, Stefano Maffulli stef...@openstack.orgmailto:stef...@openstack.org wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. I think the only failure is on the side of any driver maintainers who did not make the deadlines. From my perspective (as one of the driver maintainers who did setup a CI system and a developer working on Cinder) this whole process has been a success. The test coverage has sky rocketed for Cinder, driver maintainers are forced to be a bit more active in the community, and the code base (in theory) no longer has volume drivers in tree that we don't know if they actually work or not. This is, in my opinion, a huge win for the project. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? For anyone who struggled with getting a CI system operational there are numerous resources at your disposal (all of which have been advertised in Cinder meetings and the #openstack-cinder IRC channel). There are three meetings every week where you can get help setting them up [1]. There are a few different Cinder developers who have set up their own CI systems and shared code/instructions [2][3]. I have seen those same devs supporting them via IRC and have enabled several other companies to successfully use their tools. Between these resources I don't think anyone who has actually showed up at the meetings, asked for help, and make a good faith effort to keep everyone in the loop and show progress failed to get their system online and keep their driver in Cinder its not a coincidence. There are also efforts to provide an easier to use CI system that is shared with the OpenStack infra team [4]. I would recommend anyone who wants to help ease this process for new drivers/maintainers to help contribute to this effort. I think this is going to be the best route forward to ensure people have the tools they need to setup and operate a stable third party ci system. 1 - https://wiki.openstack.org/wiki/Meetings/ThirdParty#Weekly_Third_Party_meetings 2 - https://github.com/rasselin/os-ext-testing 3 - https://github.com/j-griffith/sos-ci 4 - https://review.openstack.org/#/c/139745/ -Patrick __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 12:59 Mon 23 Mar , Stefano Maffulli wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? Just to be clear, here's all the communication attempts made to vendors: 1) Talks during the design summit and the meetup on Friday at the summit. 2) Discussions at the Cinder midcycle meetups in Fort Collins and Austin. 4) Individual emails to driver maintainers. This includes anyone else who has worked on the driver file according to the git logs. 5) Reminders on the mailing list. 6) Reminders on IRC and Cinder IRC meetings every week. 7) If you submitted a new driver in Kilo, you had the annoying reminder from reviewers that your driver needs to have a CI by Kilo. And lastly I have made phone calls to companies that have shown zero responses to my emails or giving me updates. This is very difficult with larger companies because you're redirected from one person to another of who their OpenStack person is. I've left reminders on given voice mail extensions. I've talked to folks at the OpenStack foundation to get feedback on my communication, and was told this was good, and even better than previous communication to controversial changes. I expected nevertheless people to be angry with me and blame me regardless of my attempts to help people be successful and move the community forward. -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On Mon, Mar 23, 2015 at 12:59 PM, Stefano Maffulli stef...@openstack.org wrote: On Mon, 2015-03-23 at 11:43 -0700, Mike Perez wrote: We've been talking about CI's for a year. We started talking about CI deadlines in August. If you post a driver for Kilo, it was communicated that you're required to have a CI by the end of Kilo [1][2][3][4][5][6][7][8]. This should've been known by your engineers regardless of when you submitted your driver. Let's work to fix the CI bits for Liberty and beyond. I have the feeling that despite your best effort to communicate deadlines, some quite visible failure has happened. I think the only failure is on the side of any driver maintainers who did not make the deadlines. From my perspective (as one of the driver maintainers who did setup a CI system and a developer working on Cinder) this whole process has been a success. The test coverage has sky rocketed for Cinder, driver maintainers are forced to be a bit more active in the community, and the code base (in theory) no longer has volume drivers in tree that we don't know if they actually work or not. This is, in my opinion, a huge win for the project. You've been clear about Cinder's deadlines, I've been trying to add them also to the weekly newsletter, too. To the people whose drivers don't have their CI completed in time: what do you suggest should change so that you won't miss the deadlines in the future? How should the processes and tool be different so you'll be successful with your OpenStack-based products? For anyone who struggled with getting a CI system operational there are numerous resources at your disposal (all of which have been advertised in Cinder meetings and the #openstack-cinder IRC channel). There are three meetings every week where you can get help setting them up [1]. There are a few different Cinder developers who have set up their own CI systems and shared code/instructions [2][3]. I have seen those same devs supporting them via IRC and have enabled several other companies to successfully use their tools. Between these resources I don't think anyone who has actually showed up at the meetings, asked for help, and make a good faith effort to keep everyone in the loop and show progress failed to get their system online and keep their driver in Cinder its not a coincidence. There are also efforts to provide an easier to use CI system that is shared with the OpenStack infra team [4]. I would recommend anyone who wants to help ease this process for new drivers/maintainers to help contribute to this effort. I think this is going to be the best route forward to ensure people have the tools they need to setup and operate a stable third party ci system. 1 - https://wiki.openstack.org/wiki/Meetings/ThirdParty#Weekly_Third_Party_meetings 2 - https://github.com/rasselin/os-ext-testing 3 - https://github.com/j-griffith/sos-ci 4 - https://review.openstack.org/#/c/139745/ -Patrick __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
On 21:51 Mon 23 Mar , Rochelle Grober wrote: I’d like to suggest that the myriad wiki pages and spreadsheets for Third Party CI also be consolidated to a more manageable count. Just looking for maintainers contact, you can find information (often conflicting) in Stackalytics, on the ThirdPartyDrivers page, on the Cinder PTL’s google doc and who knows where else for the Neutron maintainers. Even finding which tests to run takes linking through a number of Cinder wiki pages. The teams have done a great job documenting a process that started out as lore, but I think the beginning of L would be a great time to revisit and reorganize the documentation for clarity, conciseness and single locations (version controlled) of critical information. More than happy to update my doc. My doc was purely for me to expose a non-editable doc of what I was seeing since I made myself the point of contact for Cinder CI's. My spreadsheet came before the thirdparty CI maintainer wiki page had any useful information on it. I got the contacts from the git logs of the people who actually worked on the drivers. I also was dealing with cases where a vendor hired an outside company to do their driver, which made things difficult for contact. The one wiki page people should pay attention to is the Cinder Third Party wiki page which now has a link to the status page: https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Cinder Third-Party CI: what next? (was Re: [cinder] Request exemption for removal of NetApp FC drivers (no voting CI))
As an Engineer running the NetApp CI, I thought it would be a good time to chime in here. While I have many opinions around this whole process, I will try my best to avoid any judgement and minimize ratholes. Over the past year, we have implemented a scalable CI system that is now running tests against 5 NetApp drivers for every patch (including stable branches). We have already prevented numerous bugs, some that would completely break OpenStack/Netapp integrations and other times we caught issues with the gate before it had time to break. We promptly worked with the code contributors in each of those cases. We have run over 50k test runs in total. Now I realize *those* CI tests have nothing to do with the FibreChannel drivers specifically, however, it took significant resources to get where we are now and CI has been our top priority. In the case of FC, we do test it regularly but atm we only have one FC capable server and 2 FC capable storage controllers. We have been working diligently with IT to acquire more for CI use but as most of you know, FC gear is not cheap and IT can take time. I would agree that we haven’t been panicking and calling IT every day at 8am under the belief that the community was aware of our situation and was ok with these drivers taking a bit longer. I might add that the FC drivers are just a wrapper around our iscsi drivers (the only difference being the zoning decorator). Now enough about NetApp, I wish folks would consider their perspective on the situation. It’s a huge ask to implement a CI system that tests every patch and not everyone has unlimited resources. Third party CI systems should be a huge deal for the third party, they should care way more than Cinder core. Although I understand Cinder not wanting deployers to attempt to use broken code in their project, I am certain a vendor does not want broker integration. Given the nights and weekends I have spent on third party CI, I would appreciate some empathy on the matter. I’m sure there are plenty of other folks that would agree. Please ask me any questions out right and I will give you an answer. I may have been confused but I was under the impression that NetApp FC had an exception to the deadline given the many conversations that had occurred. -Alex On Mon, Mar 23, 2015 at 6:30 PM, Mike Perez thin...@gmail.com wrote: On 21:51 Mon 23 Mar , Rochelle Grober wrote: I’d like to suggest that the myriad wiki pages and spreadsheets for Third Party CI also be consolidated to a more manageable count. Just looking for maintainers contact, you can find information (often conflicting) in Stackalytics, on the ThirdPartyDrivers page, on the Cinder PTL’s google doc and who knows where else for the Neutron maintainers. Even finding which tests to run takes linking through a number of Cinder wiki pages. The teams have done a great job documenting a process that started out as lore, but I think the beginning of L would be a great time to revisit and reorganize the documentation for clarity, conciseness and single locations (version controlled) of critical information. More than happy to update my doc. My doc was purely for me to expose a non-editable doc of what I was seeing since I made myself the point of contact for Cinder CI's. My spreadsheet came before the thirdparty CI maintainer wiki page had any useful information on it. I got the contacts from the git logs of the people who actually worked on the drivers. I also was dealing with cases where a vendor hired an outside company to do their driver, which made things difficult for contact. The one wiki page people should pay attention to is the Cinder Third Party wiki page which now has a link to the status page: https://wiki.openstack.org/wiki/Cinder/tested-3rdParty-drivers -- Mike Perez __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
