Since there seems to be a general agreement on these I think we can start a vote.
Possible post-vote tasks include following. * Generate human readable reports on status of Beam dependencies. * Automatically create JIRAs for significantly outdated dependencies based on above reports. * Copy component level dependency version declarations to top level. * Try to identify owners for dependencies and specify owners in comments close to dependency declarations. * Shade any dependencies that can cause issues if leaked to other components (for example, gRPC). Thanks, Cham On Tue, Jun 5, 2018 at 4:38 PM Chamikara Jayalath <chamik...@google.com> wrote: > Thanks everybody for all the comments in the doc. > > Based on the reaction so far, seems like community generally agrees to > introducing policies around managing dependencies. I updated the suggested > policies based on comments and rewrote them in a more concise form and > added room for more human intervention when needed. Updated policies are > given below (and mentioned in bold in the document). Please see the > document for details. > > (1) Human readable reports on status of Beam dependencies are generated > weekly and shared with the Beam community through the dev list. > > (2) Beam components should always have dependencies and their versions > defined at the top level. > > (3) A significantly outdated dependency (identified manually or through > tooling) should result in a JIRA that is a blocker for the next release. > Release manager may choose to push the blocker to the subsequent release or > downgrade from a blocker. > > (4) Dependency declarations may identify owners that are responsible for > upgrading the respective dependencies. > > (5) Dependencies of Java SDK components that may cause issues to other > components if leaked should be shaded. > > Thanks, > Cham > > On Fri, Jun 1, 2018 at 2:44 PM Chamikara Jayalath <chamik...@google.com> > wrote: > >> Thanks. I left these ideas bit detailed for clarification. I'll rewrite >> them in a more concise form when we have enough feedback. >> >> - Cham >> >> On Fri, Jun 1, 2018 at 1:58 PM Kenneth Knowles <k...@google.com> wrote: >> >>> This does seem really useful. I appreciate the detailed explanations. If >>> we formalize it into policy, I'd love to make it a bit more concise, and >>> with appropriate room for human contestation of the guidelines. >>> >>> On Fri, Jun 1, 2018 at 1:47 PM Scott Wegner <sweg...@google.com> wrote: >>> >>>> Thanks Cham. Overall this seems like a useful hygiene improvement for >>>> the project. I've left some comments in the doc. >>>> >>>> On Fri, Jun 1, 2018 at 10:48 AM Chamikara Jayalath < >>>> chamik...@google.com> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I've copied ideas proposed in the doc below for more visibility. Any >>>>> comments are welcome. >>>>> >>>>> >>>>> >>>>> * - Human readable per-SDK reports on status of Beam dependencies are >>>>> generated weekly and shared with the Beam community through the dev list. >>>>> These reports should be concise and should highlight the cases where the >>>>> community has to act on. See [4] for more details on this.- Beam >>>>> Components >>>>> (IO connectors, runners, etc) should always try to use versions of >>>>> dependencies that are defined at the top level. Per-component dependency >>>>> version overrides should only be performed in rare cases and should come >>>>> with clear warnings for users.- Upgrading a dependency with an outdated >>>>> major version becomes a blocker for next major version release of Beam and >>>>> for any minor version releases after next immediate minor version release. >>>>> For example, if a dependency is identified to be outdated while the latest >>>>> release is x.y.z, upgrading this dependency becomes a blocker for releases >>>>> (x+1).0.0 and x.(y+2).0 of Beam. Additionally, upgrading to a major >>>>> version >>>>> of a dependency will only be enforced if the new major version of the >>>>> dependency can be adapted without a significant rewrite to any Beam >>>>> component. Note that this policy intentionally allows one of the minor >>>>> version releases to proceed without upgrading the dependency which I >>>>> believe will give Beam community enough breathing room to upgrade >>>>> dependencies without significantly affecting the release frequency.- >>>>> Upgrading a dependency with a significantly outdated minor version (based >>>>> on methodology defined in [4]) becomes a blocker for next major version >>>>> release of Beam and for any minor version releases of Beam after next >>>>> immediate minor version release. Note that this policy does not force Beam >>>>> to adapt every minor version release of a dependency.- When performing a >>>>> release, release manager should make sure that blockers identified through >>>>> above process are resolved before the release candidate is cut.- >>>>> Optionally, dependency declarations may have comments that identify owners >>>>> that should be responsible for upgrading the respective dependencies. >>>>> Release manager may choose to assign a blocking JIRA for a dependency to >>>>> its owner.* >>>>> Thanks, >>>>> Cham >>>>> >>>>> On Thu, May 31, 2018 at 7:11 PM Chamikara Jayalath < >>>>> chamik...@google.com> wrote: >>>>> >>>>>> Hi All, >>>>>> >>>>>> We recently ran into many issues due to Beam dependencies being >>>>>> significantly out of date. For example see [1], [2], and [3]. >>>>>> >>>>>> Yifan Zou recently introduced a proposal [4] that would allow us to >>>>>> identify outdated dependencies. But to really make sure that this helps >>>>>> the >>>>>> Beam project and community I believe we should adapt several small policy >>>>>> changes to our development and release process. >>>>>> >>>>>> To this end, I have created following short document that identifies >>>>>> the dependency issue and proposes several policy changes. I greatly >>>>>> appreciate if you can take a look and comment. >>>>>> >>>>>> >>>>>> https://docs.google.com/document/d/15m1MziZ5TNd9rh_XN0YYBJfYkt0Oj-Ou9g0KFDPL2aA/edit?usp=sharing >>>>>> >>>>>> Thanks, >>>>>> Cham >>>>>> >>>>>> [1] https://issues.apache.org/jira/browse/BEAM-3098 >>>>>> [2] https://issues.apache.org/jira/browse/BEAM-3991 >>>>>> [3] https://issues.apache.org/jira/browse/BEAM-4229 >>>>>> [4] >>>>>> https://lists.apache.org/thread.html/758625106a6cfe9ba23d7b39625da20e050c6279b138b18b3f0013e7@%3Cdev.beam.apache.org%3E >>>>>> >>>>>
