It has been 5 weeks since Emilien has asked Fuel developers to contribute more actively to Puppet OpenStack project [0]. We had a lively discussion on openstack-dev, myself and other Fuel developers proposed some concrete steps that could be done to reconcile the two projects, the whole thread was reported and discussed on Linux Weekly News [1], and the things were looking up.
[0] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066544.html [1] https://lwn.net/Articles/648331/ And now, 5 weeks later, Emilien has reported to the Technical Committee that there has been no progress on reconciliation between Fuel and Puppet OpenStack, and used his authority as a PTL to request that the Fuel's proposal to join OpenStack [2] is rejected. [2] https://review.openstack.org/199232 In further comments on the same review, Emilien has claimed that there's "clearly less" contribution to Puppet OpenStack from Fuel developers than before, and even brought up an example of a review in puppet-horizon that was proposed and then abandoned by Fuel team [3]. Jay went as far as calling that example "an obvious failure of working with the upstream Puppet-OpenStack community". [3] https://review.openstack.org/198119 Needless to say, I found all these claims deeply disturbing, and had to look closely into what's going on. The case of the puppet-horizon commit has turned out to be surprisingly obvious. Even before looking into the review comments, I could see a technical reason for abandoning the commit: if there is a bug in a component, fixing that bug in the package is preferrable to fixing it in puppet, because it allows anybody to benefit from the fix, not just the people deploying that package with puppet. And if you do look at the review in question, you'll find that immediately (14 minutes, and that at 6pm on Friday night!) after Jay has asked in the comments to the review why it was abandoned, the commit author from the Fuel team has explained that this patch was a workaround for a packaging problem, and that this was pointed out in the review by a Horizon core reviewer more than a week ago, and later corroborated by a Puppet OpenStack core reviewer. Further confirming that fixing this in the package instead of in puppet-horizon was an example of Fuel developers agreeing with other Puppet OpenStack contributors and doing the right thing. Emilien has clearly found this case important enough to report to the TC, and yet didn't find it important enough to simply ask Fuel developers why they chose to abandon the commit. I guess you can call that an obvious failure to work together. Looking further into Fuel team's reviews for puppet-horizon, I found another equally disturbing example [4]. [4] https://review.openstack.org/190548 Here's what I see in this review: a) Fuel team has spent more than a month (since June 11) on trying to land this patch. b) 2 out of 5 negative reviews are from Fuel team, proving that Fuel developers are not "rubberstamping" each other's commits as was implied by Emilien's comments on the TC review above. c) There was one patch set that was pushed 3 business days after a negative review, all other patch sets (11 total) were pushed no later than next day after a negative review. All in all, I see great commitment and effort on behalf of Fuel team, eventually awarded by a +2 from Michael Chapman. On the same day (June 30), Emilien votes -2 for a correction in a comment, and walks away from the review for good. 18 days and 4 patch sets and 2 outstanding +1's later, the review remains blocked by that -2. Reaching out on #puppet-openstack [5] didn't help, either. [5] http://irclog.perlgeek.de/puppet-openstack/2015-07-08#i_10867124 At the same time, Emilien has commented on the TC review that the only metric he considers reflective of Fuel's contribution to Puppet OpenStack is merged commits. Isn't that a Catch-22 situation, requesting more merged commits and refusing to merge them? When I compare the trends in the number of patch sets pushed by Fuel developers [6] against the number of merged commits [7], I see the same picture. It's really obvious when comparing both graphs side by side. [6] http://stackalytics.com/?module=puppetopenstack-group&metric=patches&company=mirantis [7] http://stackalytics.com/?module=puppetopenstack-group&metric=commits&company=mirantis Between May and July, the peak number of patchsets in Puppet OpenStack from all contributors increased by a factor of 1.78, while patchsets associated with Mirantis jumped up by a factor of 6.75. At the same time, merged commits from all contributors increased by 2.44 (meaning 30% less patch sets per commit than in May), for Mirantis the factor is only 2.0 (meaning 3 times as many patch sets per commit as in May). Just look at the attached picture to see how obviously massive is the increase in the effort that Fuel team contributes to Puppet OpenStack. One can argue for different ways to explain why this effort does not convert into merged commits, but based on two examples above I find it very hard to believe that it can be explained by a lack of trying. In fact, this is exactly the situation that I have predicted in the original thread when I said that commmitment to collaborate has to be mutual [8]. [8] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066731.html Fuel team can commit to reporting bugs and posting patches, Fuel team can spend reasonable amount of time and effort to respond to review comments, but all that is worth nothing if there's no commitment from Puppet OpenStack team to review and merge these patches in a timely manner. Looking through the collaboration thread, the only commitment I see is Emilien's promise that "if you submit a good patch now, it will land in maximum one week or so", and his comment on the TC review: "The end of the thread was kind of consensus to me and I was pretty happy to see Dmitry was really open to the discussion". Clearly this kind of consensus is not working out, and I think at least one reason is that my proposal for Puppet OpenStack developers to take over stalled commits was rejected. One way or another, cases like "16 days since the first patch set that had no negative reviews" that obviously violate the "land within a week" commitment have to be identified and prioritized. Being on the hook to take them over if they don't land on time is one way to introduce a motivation to make that happen. There could be other ways to address it, for example having a review dashboard that highlights stuck reviews (as I proposed in the same email) would help add some visibility into the problem. Please don't get me wrong. In the same reviews I've highlighted above I see excellent example of collaboration from Puppet OpenStack team, insightful and timely comments, and I'm not denying the effort the team spends on onboarding new contributors flooding in from Fuel. But I don't see how this situation can be called "no progress" and "reduced contribution from Fuel" when making further progress is limited by Puppet OpenStack team's ability to review incoming patches much more than by Fuel team's commitment to proposing and updating them. -- Dmitry Borodaenko
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev