Hi Chamikara, Ahmet is asking for your input in https://github.com/apache/beam/pull/11156#issuecomment-602716275 . Would you check this?
On Mon, Mar 23, 2020 at 3:05 PM Tomo Suzuki <suzt...@google.com> wrote: > (Applied Ahmet's feedback in https://github.com/apache/beam/pull/11156) > > Hi Luke, > > > the pom files that are produced > > Build.gradle that uses the BOM will produce pom files that have > corresponding dependencies without versions, and have a > "dependencyManagement" section through which they import the GCP Libraries > BOM. This fills the versions of the versionless dependencies when build > systems evaluates the pom files. > > > how the GCP BOM impacts the release process > > I'm afraid I don't have knowledge on how dependencies affect the Beam > release process. Would you be willing to break down this concern into some > questions I can answer? > > > > > On Fri, Mar 20, 2020 at 6:13 PM Luke Cwik <lc...@google.com> wrote: > >> My concern would be to validate how the GCP BOM impacts the release >> process and the pom files that are produced otherwise the next person >> running a release may run into trouble. >> >> On Fri, Mar 20, 2020 at 3:00 PM Pablo Estrada <pabl...@google.com> wrote: >> >>> +1 >>> >>> On Fri, Mar 20, 2020 at 2:41 PM Ismaël Mejía <ieme...@gmail.com> wrote: >>> >>>> Well why we just don't merge it? I am unfamiliar with GCP deps to be >>>> confident to LGTM it. but given that 22 checks pass and given that >>>> Tomo addressed most comments and he has already a consistent track of >>>> good work on dependency improvements I think it is worth to merge it >>>> as it is. We still have some time to fix stuff if we find any >>>> regression. WDYT? >>>> >>>> On Thu, Mar 19, 2020 at 9:36 PM Luke Cwik <lc...@google.com> wrote: >>>> > >>>> > As much as I would like to spend time on these reviews, I believe >>>> I'll be delayed from reviewing them thoroughly till I finish other work >>>> that I'm targeting for the 2.21 release related to portability. It would be >>>> greatly appreciated if there are others that could review this in the >>>> meantime. >>>> > >>>> > On Thu, Mar 19, 2020 at 7:09 AM Tomo Suzuki <suzt...@google.com> >>>> wrote: >>>> >> >>>> >> PR is ready (22 successful check) >>>> >> https://github.com/apache/beam/pull/11156 >>>> >> (Luke assigned himself as a reviewer) >>>> >> >>>> >> Regards, >>>> >> Tomo >>>> >> >>>> >> On Wed, Mar 11, 2020 at 3:50 PM Tomo Suzuki <suzt...@google.com> >>>> wrote: >>>> >>> >>>> >>> Thank you for favorable responses. I'll start implementation. >>>> >>> >>>> >>> On Thu, Mar 5, 2020 at 2:22 PM Tomo Suzuki <suzt...@google.com> >>>> wrote: >>>> >>>> >>>> >>>> > Do Spark or Flink have BOMs? >>>> >>>> >>>> >>>> Not that I know of. I couldn't find "bom" in their artifacts [1, >>>> 2]. >>>> >>>> >>>> >>>> [1]: https://search.maven.org/search?q=g:org.apache.flink >>>> >>>> [2]: https://search.maven.org/search?q=g:org.apache.spark >>>> >>>> >>>> >>>> >>>> >>>> On Thu, Mar 5, 2020 at 1:46 PM Kenneth Knowles <k...@apache.org> >>>> wrote: >>>> >>>>> >>>> >>>>> +1 and you have phrased the benefits and limitations well. We >>>> have plenty of not-Google-related dependencies that use Guava and protobuf >>>> (I know of Calcite, Cassandra, Kinesis, and Spark) so there's still work in >>>> managing deps, but the BOM should make it a lot easier to upgrade all these >>>> tightly coupled libraries that Google ships from their head. >>>> >>>>> >>>> >>>>> Do Spark or Flink have BOMs? I wonder if there's an opportunity >>>> to catch incompatible deps at a larger scale by comparing and merging a >>>> half dozen BOMs (although in the limit it approximately expands to one per >>>> runner and one per IO and projects mature and become independent) >>>> >>>>> >>>> >>>>> Kenn >>>> >>>>> >>>> >>>>> On Thu, Mar 5, 2020 at 10:05 AM Luke Cwik <lc...@google.com> >>>> wrote: >>>> >>>>>> >>>> >>>>>> How would the Apache Beam BOM and GCP BOM work together? >>>> >>>>>> >>>> >>>>>> On Thu, Mar 5, 2020 at 7:25 AM Filipe Regadas < >>>> filiperega...@gmail.com> wrote: >>>> >>>>>>> >>>> >>>>>>> Big +1, this is a step in the right direction and checking with >>>> other Beam's direct and transitive deps is crucial since the referred bom >>>> only convers a small part of it. Apache Commons, Jackson, `com.google.{api, >>>> apis, cloud}`, slf4j comes to mind. >>>> >>>>>>> >>>> >>>>>>> Filipe Regadas >>>> >>>>>>> >>>> >>>>>>> >>>> >>>>>>> On Thu, Mar 5, 2020 at 3:33 AM Ismaël Mejía <ieme...@gmail.com> >>>> wrote: >>>> >>>>>>>> >>>> >>>>>>>> +1 Sounds like a good improvement for users and maintainers ! >>>> >>>>>>>> >>>> >>>>>>>> On Thu, Mar 5, 2020 at 6:59 AM Alex Van Boxel < >>>> a...@vanboxel.be> wrote: >>>> >>>>>>>> > >>>> >>>>>>>> > +1, I can remember the countless hours that we fought with >>>> Google dependencies. >>>> >>>>>>>> > >>>> >>>>>>>> > On Thu, Mar 5, 2020, 04:07 Chamikara Jayalath < >>>> chamik...@google.com> wrote: >>>> >>>>>>>> >> >>>> >>>>>>>> >> +1 for this. >>>> >>>>>>>> >> >>>> >>>>>>>> >> This will make life easy for many of our users and will >>>> help us keep GCP related dependencies compatible (which has not been easy >>>> in the past). >>>> >>>>>>>> >> >>>> >>>>>>>> >> On Wed, Mar 4, 2020 at 2:16 PM Tomo Suzuki < >>>> suzt...@google.com> wrote: >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> Hi Beam developers, >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> Shall we use GCP Libraries BOM [1] to specify the >>>> Google-related library versions in Beam? >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> I've been working on Beam's dependency upgrades in the >>>> past few months. It's time to consider a long-term solution to keep the >>>> libraries up-to-date with small maintenance effort. To achieve that, I >>>> propose Beam to use GCP Libraries BOM to set the Google-related library >>>> versions, rather than the current way of making changes in each of ~30 >>>> Google libraries with individual PRs [2]. >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> After the proposal is implemented, Beam project upgrades >>>> the BOM version to upgrade these Google-related libraries. This still needs >>>> to ensure the libraries in GCP Library BOM are compatible with Beam's other >>>> dependencies. (Linkage Checker will help with this job.) I believe >>>> onboarding GCP Libraries BOM will solve lots of incompatibilities which we >>>> have seen in gax, gRPC, google-cloud-core, and so on with minimal effort in >>>> Beam's developers. >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> Created an issue to track this: BEAM-9444 [3]. I >>>> appreciate if you can share questions or feedback (thumbs-up / concerns). >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> [1]: >>>> https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/The-Google-Cloud-Platform-Libraries-BOM >>>> >>>>>>>> >>> [2]: >>>> https://github.com/apache/beam/pulls?page=1&q=is%3Apr+author%3Asuztomo >>>> >>>>>>>> >>> [3]: https://issues.apache.org/jira/browse/BEAM-9444 >>>> >>>>>>>> >>> >>>> >>>>>>>> >>> -- >>>> >>>>>>>> >>> Regards, >>>> >>>>>>>> >>> Tomo >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> -- >>>> >>>> Regards, >>>> >>>> Tomo >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> Regards, >>>> >>> Tomo >>>> >> >>>> >> >>>> >> >>>> >> -- >>>> >> Regards, >>>> >> Tomo >>>> >>> > > -- > Regards, > Tomo > -- Regards, Tomo
