Tim and Boyuan were previously discussing similar issues in the Slack channel [1], and the root cause was related to JAR corruption by the signing plugin when using parallel builds. There was also some investigation in BEAM-4328 [2].
I believe fixes for all known-issues are now merged. The Gradle-based release guide is in review [3] but already includes instructions for --no-parallel. If there are still additional issues we should create JIRAs for them. [1] https://the-asf.slack.com/archives/C9H0YNP3P/p1526496272000381 [2] https://issues.apache.org/jira/browse/BEAM-4328 [3] https://github.com/apache/beam-site/pull/424 On Wed, Jun 6, 2018 at 9:13 AM Robert Bradshaw <[email protected]> wrote: > Are there JIRAs filed for these? I have yet to have a corrupt cache, but > it would be nice to know how to avoid and fix it. Did --no-parallel make > the ErrorProne error go away? > > On Tue, Jun 5, 2018 at 11:39 PM Romain Manni-Bucau <[email protected]> > wrote: > >> Also maybe deactivate the daemon (--no-daemon) since its cache can get >> corrupted ~easily. >> >> Romain Manni-Bucau >> @rmannibucau <https://twitter.com/rmannibucau> | Blog >> <https://rmannibucau.metawerx.net/> | Old Blog >> <http://rmannibucau.wordpress.com> | Github >> <https://github.com/rmannibucau> | LinkedIn >> <https://www.linkedin.com/in/rmannibucau> | Book >> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >> >> >> Le mer. 6 juin 2018 à 08:35, Jean-Baptiste Onofré <[email protected]> a >> écrit : >> >>> It looks better with --no-parallel >>> >>> Regards >>> JB >>> >>> On 06/06/2018 07:49, Jean-Baptiste Onofré wrote: >>> > New issue during: >>> > >>> > ./gradlew publish -PisRelease >>> > >>> >> Task :beam-runners-apex:compileTestJava >>> > >>> /home/jbonofre/Workspace/beam/runners/apex/src/test/java/org/apache/beam/runners/apex/translation/utils/ApexStateInternalsTest.java:55: >>> > error: An unhandled exception was thrown by the Error Prone static >>> > analysis plugin. >>> > public static class StandardStateInternalsTests extends >>> > StateInternalsTest { >>> > ^ >>> > Please report this at >>> > https://github.com/google/error-prone/issues/new and include the >>> following: >>> > >>> > error-prone version: 2.3.1 >>> > BugPattern: HidingField >>> > Stack Trace: >>> > com.sun.tools.javac.code.ClassFinder$BadClassFile: bad class file: >>> > >>> /home/jbonofre/Workspace/beam/runners/core-java/build/libs/beam-runners-core-java-2.5.0-tests.jar(/org/apache/beam/runners/core/StateInternalsTest$MapEntry.class) >>> > unable to access file: java.io.EOFException: Unexpected end of ZLIB >>> > input stream >>> > Please remove or make sure it appears in the correct subdirectory >>> of >>> > the classpath. >>> > at >>> > com.sun.tools.javac.jvm.ClassReader.badClassFile(ClassReader.java:298) >>> > at >>> > >>> com.sun.tools.javac.jvm.ClassReader.readClassFile(ClassReader.java:2830) >>> > >>> > I'm trying with --no-parallel. >>> > >>> > Regards >>> > JB >>> > >>> > On 06/06/2018 05:18, Lukasz Cwik wrote: >>> >> JB, I believe many people are waiting on the release to happen and the >>> >> release branch is yet to be cut. It has been almost a week since you >>> >> said you would cut the release branch. It seems like your very busy, >>> can >>> >> you explain clearly what is slowing you down so people can help or >>> would >>> >> you like to defer doing the release at this point in time? >>> >> >>> >> On Tue, Jun 5, 2018 at 10:16 AM Jean-Baptiste Onofré <[email protected] >>> >> <mailto:[email protected]>> wrote: >>> >> >>> >> Thanks, yeah I saw that but I'm planning to add some additional >>> notes. >>> >> >>> >> Regards >>> >> JB >>> >> Le 5 juin 2018, à 19:02, Boyuan Zhang <[email protected] >>> >> <mailto:[email protected]>> a écrit: >>> >> >>> >> Hey JB, >>> >> >>> >> We have some updates in : >>> >> >>> https://github.com/pabloem/beam-site/blob/372c93ecbafbf3a1440492df1e12050dfe939e91/src/contribute/release-guide.md >>> >> < >>> https://github.com/pabloem/beam-site/blob/372c93ecbafbf3a1440492df1e12050dfe939e91/src/contribute/release-guide.md >>> >and >>> >> https://github.com/apache/beam-site/pull/424/files >>> >> < >>> https://www.google.com/url?q=https://github.com/apache/beam-site/pull/424/files&sa=D&usg=AFQjCNHm8O1DwRKZ1828EQxzmx881O5aWA >>> >. >>> >> They may be helpful. >>> >> >>> >> Boyuan >>> >> >>> >> On Tue, Jun 5, 2018 at 9:50 AM Jean-Baptiste Onofré < >>> >> [email protected] <mailto:[email protected]>> wrote: >>> >> >>> >> Was checking if there was something like the release >>> branch >>> >> of the maven release plugin, but there's not with the >>> gradle >>> >> one. >>> >> >>> >> I'm creating the branch by hand and I'm updating the >>> release >>> >> guide in the mean time. >>> >> >>> >> Regards >>> >> JB >>> >> Le 5 juin 2018, à 18:44, "Jean-Baptiste Onofré" < >>> >> [email protected] <mailto:[email protected]>> a écrit: >>> >> >>> >> On the way >>> >> Le 5 juin 2018, à 18:23, Kenneth Knowles < >>> >> [email protected] <mailto:[email protected]>> a écrit: >>> >> >>> >> Have you cut the release branch? It is much easier >>> >> to stabilize a cut branch that is separated from >>> >> continued development on master. I think we have >>> to >>> >> cut it before continuing. >>> >> >>> >> Kenn >>> >> >>> >> On Tue, Jun 5, 2018 at 1:14 AM Jean-Baptiste >>> Onofré >>> >> < [email protected] <mailto:[email protected]>> >>> wrote: >>> >> >>> >> Sorry for the noise: this build error was due >>> to >>> >> a corrupted file in my >>> >> .m2/repository. >>> >> >>> >> The HDFS extension build is now OK. I'm >>> >> launching a full build. >>> >> >>> >> Regards >>> >> JB >>> >> >>> >> On 05/06/2018 07:43, Jean-Baptiste Onofré >>> wrote: >>> >> > New failure on the build: >>> >> > >>> >> > FAILURE: Build failed with an exception. >>> >> > >>> >> > * What went wrong: >>> >> > Could not resolve all files for >>> configuration >>> >> > >>> >> >>> ':beam-sdks-java-io-hadoop-file-system:testCompileClasspath'. >>> >> >>> >> >> Could not find zookeeper-tests.jar >>> >> (org.apache.zookeeper:zookeeper:3.4.6). >>> >> > Searched in the following locations: >>> >> > >>> >> > >>> >> >>> >>> file:/home/jbonofre/.m2/repository/org/apache/zookeeper/zookeeper/3.4.6/zookeeper-3.4.6-tests.jar >>> >> >>> >> > >>> >> > I'm fixing the HDFS extension. >>> >> > >>> >> > Regards >>> >> > JB >>> >> > >>> >> > On 05/06/2018 07:18, Jean-Baptiste Onofré >>> wrote: >>> >> >> Hi, >>> >> >> >>> >> >> yes, it's release blocker: the build is not >>> >> fully stable. I'm trying to >>> >> >> build the release for one week and it fails >>> >> with different errors. >>> >> >> >>> >> >> I have a new build in progress. I hope it >>> >> will be good. I keep you posted. >>> >> >> >>> >> >> Regards >>> >> >> JB >>> >> >> >>> >> >> On 05/06/2018 01:38, Scott Wegner wrote: >>> >> >>> Hey JB, you mentioned some build issues on >>> >> Slack [1]. Is this blocking >>> >> >>> the release? Let me know if there's >>> anything >>> >> I can help with. >>> >> >>> >>> >> >>> [1] >>> >> >>> https://the-asf.slack.com/archives/C9H0YNP3P/p1528133545000136 >>> >> >>> >> >>> >>> >> >>> On Sun, Jun 3, 2018 at 10:58 PM >>> >> Jean-Baptiste Onofré < [email protected] >>> >> <mailto:[email protected]> >>> >> >>> <mailto: [email protected] >>> >> <mailto:[email protected]>>> wrote: >>> >> >>> >>> >> >>> Hi guys, >>> >> >>> >>> >> >>> just to let you know that the build is >>> >> now OK. I'm completing the Jira >>> >> >>> triage this morning (my time) and cut >>> >> the release branch (starting the >>> >> >>> release process). I will validate the >>> >> release guide in the mean time. >>> >> >>> >>> >> >>> Thanks, >>> >> >>> Regards >>> >> >>> JB >>> >> >>> >>> >> >>> On 06/04/2018 10:48, Jean-Baptiste >>> >> Onofré wrote: >>> >> >>> > Hi guys, >>> >> >>> > >>> >> >>> > Apache Beam 2.4.0 has been released >>> on >>> >> March 20th. >>> >> >>> > >>> >> >>> > According to our cycle of release >>> >> (roughly 6 weeks), we should >>> >> >>> think about 2.5.0. >>> >> >>> > >>> >> >>> > I'm volunteer to tackle this >>> release. >>> >> >>> > >>> >> >>> > I'm proposing the following items: >>> >> >>> > >>> >> >>> > 1. We start the Jira triage now, up >>> to >>> >> Tuesday >>> >> >>> > 2. I would like to cut the release >>> on >>> >> Tuesday night (Europe time) >>> >> >>> > 2bis. I think it's wiser to still >>> use >>> >> Maven for this release. Do >>> >> >>> you think we >>> >> >>> > will be ready to try a release with >>> >> Gradle ? >>> >> >>> > >>> >> >>> > After this release, I would like a >>> >> discussion about: >>> >> >>> > 1. Gradle release (if we release >>> 2.5.0 >>> >> with Maven) >>> >> >>> > 2. Isolate release cycle per Beam >>> >> part. I think it would be >>> >> >>> interesting to have >>> >> >>> > different release cycle: SDKs, DSLs, >>> >> Runners, IOs. That's another >>> >> >>> discussion, I >>> >> >>> > will start a thread about that. >>> >> >>> > >>> >> >>> > Thoughts ? >>> >> >>> > >>> >> >>> > Regards >>> >> >>> > JB >>> >> >>> > >>> >> >>> >>> >> >>> -- >>> >> >>> Jean-Baptiste Onofré >>> >> >>> [email protected] >>> >> <mailto:[email protected]> <mailto: >>> >> [email protected] <mailto: >>> [email protected]>> >>> >> >>> http://blog.nanthrax.net >>> >> >>> Talend - http://www.talend.com >>> >> >>> >>> >> >> >>> >> > >>> >> >>> >> -- >>> >> Jean-Baptiste Onofré >>> >> [email protected] <mailto: >>> [email protected]> >>> >> http://blog.nanthrax.net >>> >> Talend - http://www.talend.com >>> >> >>> > >>> >>> -- >>> Jean-Baptiste Onofré >>> [email protected] >>> http://blog.nanthrax.net >>> Talend - http://www.talend.com >>> >>
