On 07/07/2014 12:33 PM, Sean Dague wrote: > On 07/04/2014 10:54 AM, Mark McLoughlin wrote: >> On Fri, 2014-07-04 at 15:31 +0200, Ihar Hrachyshka wrote: >>> Hi all, >>> at the moment we have several bot jobs that sync contents to affected >>> projects: >>> >>> - translations are copied from transifex; >>> - requirements are copied from global requirements repo. >>> >>> We have another source of common code - oslo-incubator, though we >>> still rely on people manually copying the new code from there to >>> affected projects. This results in old, buggy, and sometimes >>> completely different versions of the same code in all projects. >>> >>> I wonder why don't we set another bot to sync code from incubator? In >>> that way, we would: >>> - reduce work to do for developers [I hope everyone knows how boring >>> it is to fill in commit message with all commits synchronized and >>> create sync requests for > 10 projects at once]; >>> - make sure all projects use (almost) the same code; >>> - ensure projects are notified in advance in case API changed in one >>> of the modules that resulted in failures in gate; >>> - our LOC statistics will be a bit more fair ;) (currently, the one >>> who syncs a large piece of code from incubator to a project, gets all >>> the LOC credit at e.g. stackalytics.com). >>> >>> The changes will still be gated, so any failures and incompatibilities >>> will be caught. I even don't expect most of sync requests to fail at >>> all, meaning it will be just a matter of two +2's from cores. >>> >>> I know that Oslo team works hard to graduate lots of modules from >>> incubator to separate libraries with stable API. Still, I guess we'll >>> live with incubator at least another cycle or two. >>> >>> What are your thoughts on that? >> >> Just repeating what I said on IRC ... >> >> The point of oslo-incubator is that it's a place where APIs can be >> cleaned up so that they are ready for graduation. Code living in >> oslo-incubator for a long time with unchanging APIs is not the idea. An >> automated sync job would IMHO discourage API cleanup work. I'd expect >> people would start adding lots of ugly backwards API compat hacks with >> their API cleanups just to stop people complaining about failing >> auto-syncs. That would be the opposite of what we're trying to achieve. > > The problem is in recent times we've actually seen the opposite happen. > Code goes into oslo-incubator working. It gets "cleaned up". It syncs > back to the projects and break things. olso.db was a good instance of that. > > Because during the get "cleaned up" phase it's not being tested in the > whole system. It's only being unit tested. > > Basically code goes from working in place, drops 95% of it's testing, > then gets refactored. Which is exactly what you don't want to be doing > to refactor code. > > So I think the set of trade offs for oslo looked a lot different when > only a couple projects were using it, and the amount of code is small, > vs. where we stand now. > > What it's produced is I think the opposite of what we're trying to > achieve (as people are pretty gunshy now on oslo syncs), because the > openstack/ tree across projects is never the same. So you'll have 12 > different versions of log.py in a production system.
This is indeed an unfortunate situation that needs to be fixed but I think it's related to the current testing/reviewing strategies rather than just syncing code back to projects. > What I really want is forward testing of oslo interfaces. Because most > of the breaks in oslo weren't because there was a very strong view that > an certain interface or behavior needed to change. It was because after > all the testing was removed from the code, and the people working on it > in oslo didn't have the context on how the code was used in a project, > behavior changed. Not intentionally, just as a side effect. > > I think the goal of oslo is really common code for OpenStack. I would > much rather have all the projects running the same oslo code, even if it > meant a few compat interfaces in there, than having the wild divergence > of olso code in the current model. This would be the ideal scenario but I don't believe automatic syncs are the right solution for the problems you mentioned nor for the keep-projects-updated issue. Nevertheless, assuming we wanted to do so, this is the list of things I believe we would need to do to get there, OTOH: #. Improve update.py #. Complete the first round of libraries graduation #. Sync all projects and port them to the latest version of whatever there is in oslo-inc that they're using. #. Add jobs to auto-sync projects in a weekly(?) basis #. Hope that these reviews won't be ignored when there's a failure. Flavio -- @flaper87 Flavio Percoco _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
