One last question: do we know how other organizations solve these types of central library problems? Though perhaps the conditions are different enough that their experience is not relevant.
--- Tangent: I'd like to point out that *things will only get better from here!* 😃 The components team has been in the process of *reimplementing* *the core functionality of our apps *while the product teams are creating complex feature releases (Focus w/ GV and Fire TV redesign). It's incredible we haven't had more bugs, actually. :P Future releases may not be so risky and when the core functionality is completed, upgrade timelines will be less demanding and bugs will be less critical to fix quickly. Also, perhaps more importantly,* our teams are showing a commitment to learning and iterating to improve our processes *(thank you again, Stefan, for starting this thread!). Thanks for all your hard work everyone and I look forward to seeing where we go next! 🚀 - Mike On Tue, Oct 2, 2018 at 3:07 PM Michael Comella <mcome...@mozilla.com> wrote: > Thanks for starting this discussion, Stefan! I think it's helpful to > better understand everyone's problems so we can come to a shared process > that benefits us all! > > To take a step back, on Fire TV some of the issues we've faced with > components integration are: > > 1) The way the product and components release schedules happen to align *makes > us find and fix bugs/design issues later than we'd like.* The product > sprints/releases are every two weeks, we code freeze on the second > Wednesday, and finish our build end-of-day Friday. afaik, components > releases sometime between Wednesday and Friday which means we're sometimes > updating the potentially unstable components [1] after code freeze, > creating a lot of stress on both teams to fix bugs or design issues by the > end of the week. > > Some solutions, with varying trade-offs: > - Upgrade components once a sprint (during week one): I think this is what > Susheel proposes by pinning and backporting dependencies > - *Desynchronize the two release schedules,* e.g. release components > earlier in the week > - Release components more often so we can test earlier before an official > components release: this is what Stefan proposes > > It's important not to fall too far behind components due to the additional > engineering overhead [2] so I'd rather upgrade the components frequently, > if possible, removing option #1. *#2 seems like a great way to keep our > products more stable* but it may not be logistically feasible (e.g. does > sprint planning align?). I think #3 is also a good option to improve > stability but we may still find the final upgrade from final RC to release > creates bugs. > > 2) *Components updates unexpectedly break existing integrations* (e.g. new > Settings <https://github.com/mozilla-mobile/firefox-tv/issues/1204>), > adding unplanned work to our sprints. > > Some solutions: > - Explicit notifications about potentially breaking changes early: this > can let us plan for the fixes in sprint planning > - Only upgrade components early in the sprint to find and fix such issues > early > > As before, I'd prefer to upgrade components as often as we can while > maintaining stability so I'd go with #1. I want to see more false > positives! :) A solution to the first problem will also help. > > Chris' later email points out semantic versioning but I think some our > breakage often comes from 1) incorrect integrations, 2) design issues, 3) > missing functionality in components that replaces WebView-based > functionality, and 4) new components code breaking existing work-arounds: > semantic versioning wouldn't address this. > > 3) *Patching previous releases seems to have a lot of overhead.* > Sometimes a product is forced to pin their versions for stability or lack > of engineering time to integrate breaking changes (see above) but it's > seemed difficult to quickly backport bug fixes. From this thread, it sounds > like the components team is addressing this though. 👍 Also, finding bugs > sooner (the point of this thread!) would make *quick* backports less > necessary. > > --- > > Finally, wrt snapshot vs. RC, I think it's better to have explicitly > pinned versions so we're never in a situation where behavior is not > reproducible. We can also release with an RC when necessary. > > Hope this helps! > - Mike > > [1]: They're stable against the components test suite but that doesn't > mean they integrate well with our app design or that they're bug free when > used in our product (e.g. Fire OS keyboard bugs). > > [2]: Additional engineering overhead comes from the time spent backporting > and, when upgrading to the latest version, finding and fixing bugs/design > issues that were fixed some time ago in components so it's harder to track > down. > > On Tue, Oct 2, 2018 at 11:31 AM Stefan Arentz <sare...@mozilla.com> wrote: > >> >> >> On Tue, Oct 2, 2018 at 1:39 PM Susheel Daswani <sdasw...@mozilla.com> >> wrote: >> >>> Yes Stefan, I don't think the snapshots and rc releases are super useful >>> *yet*. We do use them with GV but that's really just because GV >>> development is so active and the engine is so core that updating it more >>> frequently overrides the risk. But being on the bleeding edge does have >>> it's issues (e.g., yesterday we had to downgrade GV because it was too >>> bleeding edge). >>> >> >> I think what we will recommend to all teams is actually this: >> >> - Always develop against components -SNAPSHOT so that you stay up to >> date. Snapshot is the place where you will get the changes, bug fixes and >> features that you have requested. And you will get them real-time as those >> changes land. Using snapshots is the best way to work together with us >> because not only will you get the things you need soon, you will also find >> issues sooner which means we get to a stable release sooner, which means >> less chance of point releases. >> >> - When your project reaches code freeze or feature complete, that is the >> moment you stop using snapshots and pin to a stable version. In an ideal >> world the timing is such that our calendars are in sync enough that you can >> just pin the upcoming release. But if they are not, you can tell us the >> point in time when you would like to pin components and we can make a >> release happen on that date. (Note that this does not scale well beyond one >> or two products, it is simpler to take our release dates into account and >> align schedules.) >> >> The point releases are to handle unexpected (aren't they always) critical >> bugs, they do not exist to accommodate a development schedule. However, >> they can happen after you pinned to a specific version or even after you >> shipped. But I think that with a good schedule and good testing, these >> releases can mostly be avoided. >> >> S. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Android Product Team (APT)" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to android-product-team+unsubscr...@mozilla.com. >> To post to this group, send email to android-product-t...@mozilla.com. >> To view this discussion on the web visit >> https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CAKyLfD4OXSfV%2Bf-4AmWALEJtKT%2BhwZoOp1F%3D%2BFh%3DY%3D2LhMR9CQ%40mail.gmail.com >> <https://groups.google.com/a/mozilla.com/d/msgid/android-product-team/CAKyLfD4OXSfV%2Bf-4AmWALEJtKT%2BhwZoOp1F%3D%2BFh%3DY%3D2LhMR9CQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >
_______________________________________________ Sync-dev mailing list Sync-dev@mozilla.org https://mail.mozilla.org/listinfo/sync-dev
