I can try triggering a one-off maven snapshot release. On Thu, Apr 26, 2018 at 1:59 PM Eric Beach <[email protected]> wrote:
> Is it possible to trigger a manual task to update the snapshot artifacts > here in the interim until a decision has been made? It is blocking dev > toward a public release. > > On Thu, Apr 26, 2018 at 2:51 AM Romain Manni-Bucau <[email protected]> > wrote: > >> >> >> 2018-04-26 7:54 GMT+02:00 Reuven Lax <[email protected]>: >> >>> This seems more than just Gradle, as they are using internal details in >>> the parent pom (all the Google Cloud dependency version). Saying that >>> nothing should break means that we can never rename or refactor anything in >>> our poms because some user might be depending on it; that would be true on >>> Maven just as much as on Gradle. >>> >>> Maybe this particular use is important enough that it needs to be >>> maintained, I'm not sure. However we also need to decide what our stable >>> pom interface is for users. >>> >> >> Browsing pom hierarchy to grab artifacts (including properties or not) >> and plugin configurations (it is common to extract compiler config and >> shade configs) is widely spread cause maven is the standard exchange format >> so I hope beam will respect that at least. >> The other usage is to check the dependencies between modules and >> determine the impact of a change ("what is the common part" ~ parent+bom, >> "what does it impact if we change that dep"). It can be key for beam to >> keep that to ensure upstream projects can determine some of their impacts >> statically through mavenrepo tools. >> >> >>> >>> Reuven >>> >>> On Wed, Apr 25, 2018 at 9:34 PM Romain Manni-Bucau < >>> [email protected]> wrote: >>> >>>> Hmm, gradle move not supposed to break any user and there are other >>>> parents usages, as mentionned previously, so not sure beam can really >>>> bypass it. Maven discussed consumer vs build pom since years for that >>>> reason ;). >>>> >>>> Le 26 avr. 2018 01:59, "Eric Beach" <[email protected]> a écrit : >>>> >>>>> Thanks Scott and Ben. >>>>> >>>>> This makes sense, and of course the same error happens when I tried >>>>> beam-parent instead of beam-examples-parent. >>>>> >>>>> There is a big benefit for supporting explicit dependence on parent >>>>> (either beam-examples-parent or beam-parent). In the project I'm involved >>>>> in, I have Dataflow API (to query metrics for completed jobs), Google >>>>> Cloud >>>>> Storage, and Cloud Spanner (to execute queries outside of Beam). >>>>> >>>>> >>>>> On Wed, Apr 25, 2018 at 7:55 PM Scott Wegner <[email protected]> >>>>> wrote: >>>>> >>>>>> Interesting. So two things: >>>>>> >>>>>> 1) As you correctly deduced, we switched to producing snapshots from >>>>>> the Gradle build, which doesn't include parent pom's. As a result, the >>>>>> parent pom snapshot you're referencing is stale. We should probably >>>>>> remove >>>>>> it from the repository to prevent accidental usage until we get this >>>>>> sorted >>>>>> out. >>>>>> >>>>>> 2) Depending on the parent pom directly isn't an intentionally >>>>>> supported scenario, but your use-case seems valid: successful execution >>>>>> on >>>>>> Dataflow requires specific version sets for Google Cloud dependencies. >>>>>> And >>>>>> those dependency versions are conveniently defined as properties in the >>>>>> beam-parent >>>>>> pom >>>>>> <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/beam-parent-2.5.0-20180408.070218-37.pom> >>>>>> . >>>>>> >>>>>> Note that simply generating parent pom's from Gradle wouldn't fix >>>>>> this; we need to explicitly generate these version properties in the pom >>>>>> file. >>>>>> >>>>>> We could publish a special GCP parent pom just for this purpose. >>>>>> Although forcing a parent relationship seems unnecessary; is there a >>>>>> better >>>>>> convention within Maven for importing a set property values? >>>>>> >>>>>> On Wed, Apr 25, 2018 at 4:37 PM Eric Beach <[email protected]> wrote: >>>>>> >>>>>>> If I run $ mvn clean compile exec:java ... on my project, I get the >>>>>>> following error stack trace: >>>>>>> >>>>>>> java.lang.reflect.InvocationTargetException >>>>>>> >>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >>>>>>> Method) >>>>>>> >>>>>>> at >>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>>>>>> >>>>>>> at >>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>>>>> >>>>>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>>>>> >>>>>>> at >>>>>>> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:294) >>>>>>> >>>>>>> at java.lang.Thread.run(Thread.java:748) >>>>>>> >>>>>>> Caused by: java.lang.NoClassDefFoundError: >>>>>>> com/google/api/services/clouddebugger/v2/CloudDebugger >>>>>>> >>>>>>> at java.lang.Class.getDeclaredMethods0(Native Method) >>>>>>> >>>>>>> at >>>>>>> java.lang.Class.privateGetDeclaredMethods(Class.java:2703) >>>>>>> >>>>>>> at java.lang.Class.getDeclaredMethod(Class.java:2130) >>>>>>> >>>>>>> at >>>>>>> org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:206) >>>>>>> >>>>>>> at >>>>>>> org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:162) >>>>>>> >>>>>>> at >>>>>>> org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55) >>>>>>> >>>>>>> at org.apache.beam.sdk.Pipeline.create(Pipeline.java:150) >>>>>>> >>>>>>> at >>>>>>> com.google.cloud.pontem.CloudSpannerDatabaseBackup.main(CloudSpannerDatabaseBackup.java:359) >>>>>>> >>>>>>> ... 6 more >>>>>>> >>>>>>> Caused by: java.lang.ClassNotFoundException: >>>>>>> com.google.api.services.clouddebugger.v2.CloudDebugger >>>>>>> >>>>>>> at >>>>>>> java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>>>>> >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>>>>> >>>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>>>>> >>>>>>> >>>>>>> When I dig deeper, the root problem seems to be that my project is >>>>>>> pulling in v2-rev8-1.22.0 instead of v2-rev233-1.23.0 (changing >>>>>>> commit here >>>>>>> <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> >>>>>>> ). >>>>>>> >>>>>>> In fact, all the version changes in this commit >>>>>>> <https://github.com/apache/beam/commit/66fcf4bff19cda5831465ccedbbaa6fbaa648234#diff-600376dffeb79835ede4a0b285078036> >>>>>>> are missing. >>>>>>> >>>>>>> My project's pom.xml contains the following: >>>>>>> >>>>>>> <parent> >>>>>>> >>>>>>> <artifactId>beam-examples-parent</artifactId> >>>>>>> >>>>>>> <groupId>org.apache.beam</groupId> >>>>>>> >>>>>>> <version>2.5.0-SNAPSHOT</version> >>>>>>> >>>>>>> </parent> >>>>>>> >>>>>>> <modelVersion>4.0.0</modelVersion> >>>>>>> >>>>>>> >>>>>>> <groupId>com.google.cloud</groupId> >>>>>>> >>>>>>> <artifactId>xyz</artifactId> >>>>>>> >>>>>>> >>>>>>> <repositories> >>>>>>> >>>>>>> <repository> >>>>>>> >>>>>>> <id>apache.snapshots</id> >>>>>>> >>>>>>> <name>Apache Development Snapshot Repository</name> >>>>>>> >>>>>>> <url> >>>>>>> https://repository.apache.org/content/repositories/snapshots/</url> >>>>>>> >>>>>>> <releases> >>>>>>> >>>>>>> <enabled>false</enabled> >>>>>>> >>>>>>> </releases> >>>>>>> >>>>>>> <snapshots> >>>>>>> >>>>>>> <enabled>true</enabled> >>>>>>> >>>>>>> </snapshots> >>>>>>> >>>>>>> </repository> >>>>>>> >>>>>>> </repositories> >>>>>>> >>>>>>> My first round of attempts to debug included removing ~/.m2/ and a >>>>>>> number of other techniques. >>>>>>> >>>>>>> When I look at >>>>>>> https://repository.apache.org/content/repositories/snapshots, I notice >>>>>>> that beam-parent with 2.5.0-SNAPSHOT >>>>>>> <https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/> >>>>>>> is >>>>>>> very outdated and so all the version numbers inherited from the POM are >>>>>>> wrong (see >>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/). >>>>>>> If you look at >>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/ >>>>>>> you will see that the beam-parent is significantly old. >>>>>>> >>>>>>> So, it seems imperative that the snapshot be rebuilt before the >>>>>>> finalized 2.5.0 version is set or any projects depending on the Beam >>>>>>> parent >>>>>>> will break. (I depend upon the Beam parent because trying to keep the >>>>>>> versions of all the different Google projects in sync is a total >>>>>>> nightmare). >>>>>>> >>>>>>> >>>>>>> On Wed, Apr 25, 2018 at 6:58 PM Chamikara Jayalath < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Ccing Eric for providing more context. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Cham >>>>>>>> >>>>>>>> On Wed, Apr 25, 2018 at 3:38 PM Scott Wegner <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Do you have any more context on how they were using the parent >>>>>>>>> pom? In the Gradle build, instead of using a parent pom hierarchy we >>>>>>>>> are >>>>>>>>> embedding the complete set of dependencies and metadata in each >>>>>>>>> generated >>>>>>>>> pom. They should be functionally equivalent. >>>>>>>>> >>>>>>>>> On Wed, Apr 25, 2018 at 3:09 PM Chamikara Jayalath < >>>>>>>>> [email protected]> wrote: >>>>>>>>> >>>>>>>>>> At least one user was depending on 'beam-parent' and ran into >>>>>>>>>> issues due to dependency update >>>>>>>>>> https://github.com/apache/beam/pull/5046 not being reflected >>>>>>>>>> there. I'm not sure if this is a usage pattern that we should >>>>>>>>>> discourage. >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Cham >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Apr 25, 2018 at 2:36 PM Alan Myrvold <[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> The gradle build is currently only generating pom files when >>>>>>>>>>> there is a jar file. >>>>>>>>>>> What are these parent poms used for? Do all the parent poms need >>>>>>>>>>> to be generated or just the top level beam-parent? >>>>>>>>>>> >>>>>>>>>>> On Wed, Apr 25, 2018 at 2:07 PM Chamikara Jayalath < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Thanks. Should this be a 2.5.0 blocker ? >>>>>>>>>>>> >>>>>>>>>>>> On Wed, Apr 25, 2018 at 2:05 PM Alan Myrvold < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I think this corresponds with the move from maven to gradle >>>>>>>>>>>>> snapshot releases. >>>>>>>>>>>>> Issue logged as >>>>>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-4170 >>>>>>>>>>>>> >>>>>>>>>>>>> On Wed, Apr 25, 2018 at 1:57 PM Chamikara Jayalath < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Seems like we are not building some of the SNAPSHOT artifacts >>>>>>>>>>>>>> since 4/9/2018. >>>>>>>>>>>>>> For example: >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-parent/2.5.0-SNAPSHOT/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> https://repository.apache.org/content/repositories/snapshots/org/apache/beam/beam-examples-parent/2.5.0-SNAPSHOT/ >>>>>>>>>>>>>> >>>>>>>>>>>>>> Users who depend on these artifacts are getting outdated >>>>>>>>>>>>>> dependencies. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Any idea why ? >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Cham >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>> >>>>>>>>> >>>>>>>>> Got feedback? http://go/swegner-feedback >>>>>>>>> >>>>>>>> -- >>>>>> >>>>>> >>>>>> Got feedback? http://go/swegner-feedback >>>>>> >>>>> >>
