On Wed, 2014-07-16 at 16:15 +0200, Sean Dague wrote: .. > Based on these experiences, libvirt version differences seem to be as > substantial as major hypervisor differences. There is a proposal here - > https://review.openstack.org/#/c/103923/ to hold newer versions of > libvirt to the same standard we hold xen, vmware, hyperv, docker, > ironic, etc.
That's a bit of a mis-characterization - in terms of functional test coverage, the libvirt driver is the bar that all the other drivers struggle to meet. And I doubt any of us pay too close attention to the feature coverage that the 3rd party CI test jobs have. > I'm somewhat concerned that the -2 pile on in this review is a double > standard of libvirt features, and features exploiting really new > upstream features. I feel like a lot of the language being used here > about the burden of doing this testing is exactly the same as was > presented by the docker team before their driver was removed, which was > ignored by the Nova team at the time. Personally, I wasn't very comfortable with the docker driver move. It certainly gave an outward impression that we're an unfriendly community. The mitigating factor was that a lot of friendly, collaborative, coaching work went on in the background for months. Expectations were communicated well in advance. Kicking the docker driver out of the tree has resulted in an uptick in the amount of work happening on it, but I suspect most people involved have a bad taste in their mouths. I guess there's incentives at play which mean they'll continue plugging away at it, but those incentives aren't always at play. > It was the concern by the freebsd > team, which was also ignored and they were told to go land libvirt > patches instead. > > I'm ok with us as a project changing our mind and deciding that the test > bar needs to be taken down a notch or two because it's too burdensome to > contributors and vendors, but if we are doing that, we need to do it for > everyone. A lot of other organizations have put a ton of time and energy > into this, and are carrying a maintenance cost of running these systems > to get results back in a timely basis. I don't agree that we need to apply the same rules equally to everyone. At least part of the reasoning behind the emphasis on 3rd party CI testing was that projects (Neutron in particular) were being overwhelmed by contributions to drivers from developers who never contributed in any way to the core. The corollary of that is the contributors who do contribute to the core should be given a bit more leeway in return. There's a natural building of trust and element of human relationships here. As a reviewer, you learn to trust contributors with a good track record and perhaps prioritize contributions from them. > As we seem deadlocked in the review, I think the mailing list is > probably a better place for this. > > If we want to reduce the standards for libvirt we should reconsider > what's being asked of 3rd party CI teams, and things like the docker > driver, as well as the A, B, C driver classification. Because clearly > libvirt 1.2.5+ isn't actually class A supported. No, there are features or code paths of the libvirt 1.2.5+ driver that aren't as well tested as the "class A" designation implies. And we have a proposal to make sure these aren't used by default: https://review.openstack.org/107119 i.e. to stray off the "class A" path, an operator has to opt into it by changing a configuration option that explains they will be enabling code paths which aren't yet tested upstream. These features have value to some people now, they don't risk regressing the "class A" driver and there's a clear path to them being elevated to "class A" in time. We should value these contributions and nurture these contributors. Appending some of my comments from the review below. The tl;dr is that I think we're losing sight of the importance of welcoming and nurturing contributors, and valuing whatever contributions they can make. That terrifies me. Mark. --- Compared to other open source projects, we have done an awesome job in OpenStack of having good functional test coverage. Arguably, given the complexity of the system, we couldn't have got this far without it. I can take zero credit for any of it. However, not everything is tested now, nor is the tests we have foolproof. When you consider the number of configuration options we have, the supported distros, the ranges of library versions we claim to support, etc., etc. I don't think we can ever get to an "everything is tested" point. In the absence of that, I think we should aim to be more clear what *is* tested. The config option I suggest does that, which is a big part of its merit IMHO. We've had some success with the "be nasty enough to driver contributors and they'll do what we want" approach so far, but IMHO that was an exceptional approach for an exceptional situation - drivers that were completely broken, and driver developers who didn't contribute to the core project or, worse, didn't maintain their drivers adequately. Not every contributor can be bullied like this. And even where it succeeds, I don't much love the perception it gives the project. In this case, the effect I could imagine us having would be that rather than us successfully bullying someone into setting up "latest libvirt CI", we drive development of features requiring newer libvirt into a separate tree. There would even be a perfectly defensible rationale for doing this - "we're building features that can't go upstream until a newer libvirt shows up there". If these are NFV features, there'd likely be a bunch of downstreams that ship it. And then when the newer libvirt shows up, all the features requiring it get proposed together as fully-baked patch sets? No thank you. The desire to have the best possible test coverage is great. The desire to document our policies around this stuff is great. But let's not forget that we're talking about contributors that need to be nurtured and contributions that should be valued. _______________________________________________ OpenStack-dev mailing list OpenStackemail@example.com http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev