Allright. The travis tests are green and I tested it again with Tachyon on a cluster.
My pull request also fixes some of the issues mentioned earlier in this thread by Stephan (the warnings from shading regarding duplicate classes). I can not fix all of them because some projects like hadoop just have the "conflicts" in their dependencies. I'll merge the pull request to master. Marton can merge it to "release-0.8". On Thu, Jan 8, 2015 at 2:42 PM, Stephan Ewen <se...@apache.org> wrote: > Robert, can you let us know once the PR is merged? > > Then we can start with the next steps... > > > > On Thu, Jan 8, 2015 at 2:07 PM, Robert Metzger <rmetz...@apache.org> > wrote: > > > Please wait with merging 268. There is still a minor open issue. > > > > On Thu, Jan 8, 2015 at 1:28 PM, Márton Balassi <balassi.mar...@gmail.com > > > > wrote: > > > > > +1 for merging PR 268, volunteering for the merge. It should also solve > > the > > > issue mentioned by Stephan with the yarn uberjar. > > > > > > Updating the release scripts and doing a double-check on the cosmetics. > > > > > > On Thu, Jan 8, 2015 at 12:55 PM, Stephan Ewen <se...@apache.org> > wrote: > > > > > > > We just found and fixed another highly critical bug. > > > > > > > > I vote to also cherry pick commit > > > > 19066b520435528e104a69ccf372f56811123ee3 ([FLINK-1371] > > > > [runtime] Fix KryoSerializer to not swallow EOFExceptions) > > > > > > > > > > > > > > > > On Thu, Jan 8, 2015 at 11:55 AM, Robert Metzger <rmetz...@apache.org > > > > > > wrote: > > > > > > > > > +1 for merging pull request 268. It makes Flink even more > compatible > > > with > > > > > hadoop. > > > > > > > > > > > > > > > regarding the documentation: I already updated the build > > instructions. > > > > But > > > > > we should double check everything again. > > > > > I probably won't have time to look into this today however. > > > > > > > > > > On Thu, Jan 8, 2015 at 11:50 AM, Stephan Ewen <se...@apache.org> > > > wrote: > > > > > > > > > > > After the dependencies are cleaned up (Robert's pull request > > > > > > https://github.com/apache/flink/pull/268 solves that), we need > to > > > also > > > > > do > > > > > > the following: > > > > > > > > > > > > - Make a pass over dependencies to ensure that NOTICE and > LICENSE > > > > files > > > > > > are up to date (both source and bin distribution) > > > > > > - Adjust the variables in the docs to reflect the fact that > > hadoop2 > > > is > > > > > now > > > > > > the default profile > > > > > > > > > > > > I would volunteer for the first task. > > > > > > > > > > > > Stephan > > > > > > > > > > > > > > > > > > On Wed, Jan 7, 2015 at 10:01 PM, Stephan Ewen <se...@apache.org> > > > > wrote: > > > > > > > > > > > > > I also see a warning when building the yarn fat jar. > > > > > > > > > > > > > > This is a consequence of the duplicate dependencies. > > > > > > > > > > > > > > [WARNING] servlet-api-3.0.20100224.jar, > > > javax.servlet-api-3.0.1.jar, > > > > > > > javax.servlet-3.1.jar, servlet-api-2.5.jar define 42 > overlappping > > > > > > classes: > > > > > > > [WARNING] - javax.servlet.http.Cookie > > > > > > > [WARNING] - javax.servlet.http.HttpSessionBindingEvent > > > > > > > [WARNING] - javax.servlet.http.NoBodyResponse > > > > > > > [WARNING] - javax.servlet.ServletContext > > > > > > > [WARNING] - javax.servlet.ServletOutputStream > > > > > > > [WARNING] - javax.servlet.http.HttpSessionListener > > > > > > > [WARNING] - javax.servlet.http.HttpSessionContext > > > > > > > [WARNING] - javax.servlet.FilterChain > > > > > > > [WARNING] - javax.servlet.GenericServlet > > > > > > > [WARNING] - javax.servlet.http.HttpServletRequestWrapper > > > > > > > [WARNING] - 32 more... > > > > > > > [WARNING] hadoop-yarn-api-2.2.0.jar, > hadoop-yarn-common-2.2.0.jar > > > > > define > > > > > > 3 > > > > > > > overlappping classes: > > > > > > > [WARNING] - org.apache.hadoop.yarn.util.package-info > > > > > > > [WARNING] - org.apache.hadoop.yarn.factories.package-info > > > > > > > [WARNING] - > > org.apache.hadoop.yarn.factory.providers.package-info > > > > > > > [WARNING] asm-4.0.jar, asm-3.1.jar define 21 overlappping > > classes: > > > > > > > [WARNING] - org.objectweb.asm.Type > > > > > > > [WARNING] - org.objectweb.asm.AnnotationVisitor > > > > > > > [WARNING] - org.objectweb.asm.MethodVisitor > > > > > > > [WARNING] - org.objectweb.asm.Attribute > > > > > > > [WARNING] - org.objectweb.asm.FieldWriter > > > > > > > [WARNING] - org.objectweb.asm.signature.SignatureWriter > > > > > > > [WARNING] - org.objectweb.asm.MethodWriter > > > > > > > [WARNING] - org.objectweb.asm.Edge > > > > > > > [WARNING] - org.objectweb.asm.Handler > > > > > > > [WARNING] - org.objectweb.asm.ByteVector > > > > > > > [WARNING] - 11 more... > > > > > > > [WARNING] javax.servlet-api-3.0.1.jar, javax.servlet-3.1.jar, > > > > > > > servlet-api-3.0.20100224.jar define 32 overlappping classes: > > > > > > > [WARNING] - javax.servlet.annotation.HttpConstraint > > > > > > > [WARNING] - javax.servlet.DispatcherType > > > > > > > [WARNING] - > javax.servlet.descriptor.JspPropertyGroupDescriptor > > > > > > > [WARNING] - javax.servlet.Registration > > > > > > > [WARNING] - javax.servlet.SessionTrackingMode > > > > > > > [WARNING] - > > > > > javax.servlet.annotation.ServletSecurity$EmptyRoleSemantic > > > > > > > [WARNING] - javax.servlet.annotation.HandlesTypes > > > > > > > [WARNING] - javax.servlet.ServletRegistration > > > > > > > [WARNING] - javax.servlet.annotation.ServletSecurity > > > > > > > [WARNING] - javax.servlet.ServletContainerInitializer > > > > > > > [WARNING] - 22 more... > > > > > > > [WARNING] jasper-compiler-5.5.23.jar, jasper-runtime-5.5.23.jar > > > > define > > > > > 1 > > > > > > > overlappping classes: > > > > > > > [WARNING] - org.apache.jasper.compiler.Localizer > > > > > > > [WARNING] commons-collections-3.2.1.jar, > > > commons-beanutils-1.7.0.jar, > > > > > > > commons-beanutils-core-1.8.0.jar define 10 overlappping > classes: > > > > > > > [WARNING] - > org.apache.commons.collections.FastHashMap$EntrySet > > > > > > > [WARNING] - org.apache.commons.collections.FastHashMap$KeySet > > > > > > > [WARNING] - org.apache.commons.collections.ArrayStack > > > > > > > [WARNING] - > > > > > > > > > > > > > > > > > > > > > > > > > > > > org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator > > > > > > > [WARNING] - org.apache.commons.collections.FastHashMap$Values > > > > > > > [WARNING] - > > > > org.apache.commons.collections.FastHashMap$CollectionView > > > > > > > [WARNING] - org.apache.commons.collections.FastHashMap$1 > > > > > > > [WARNING] - org.apache.commons.collections.Buffer > > > > > > > [WARNING] - org.apache.commons.collections.FastHashMap > > > > > > > [WARNING] - > > > org.apache.commons.collections.BufferUnderflowException > > > > > > > [WARNING] commons-beanutils-1.7.0.jar, > > > > commons-beanutils-core-1.8.0.jar > > > > > > > define 82 overlappping classes: > > > > > > > [WARNING] - org.apache.commons.beanutils.ConvertUtilsBean > > > > > > > [WARNING] - > > > > org.apache.commons.beanutils.converters.SqlTimeConverter > > > > > > > [WARNING] - org.apache.commons.beanutils.Converter > > > > > > > [WARNING] - > > > > > org.apache.commons.beanutils.converters.FloatArrayConverter > > > > > > > [WARNING] - org.apache.commons.beanutils.NestedNullException > > > > > > > [WARNING] - > org.apache.commons.beanutils.ConvertingWrapDynaBean > > > > > > > [WARNING] - > > > > > org.apache.commons.beanutils.converters.LongArrayConverter > > > > > > > [WARNING] - > > > > org.apache.commons.beanutils.converters.SqlDateConverter > > > > > > > [WARNING] - > > > > > > org.apache.commons.beanutils.converters.BooleanArrayConverter > > > > > > > [WARNING] - > > > org.apache.commons.beanutils.converters.StringConverter > > > > > > > [WARNING] - 72 more... > > > > > > > > > > > > > > > > > > > > > On Wed, Jan 7, 2015 at 9:08 PM, Robert Metzger < > > > rmetz...@apache.org> > > > > > > > wrote: > > > > > > > > > > > > > >> ... I did some more checks: > > > > > > >> > > > > > > >> The hadoop200alpha version has some issues with the hadoop > > > > dependency > > > > > > >> exclusions. I fixed the issues in this pull request: > > > > > > >> https://github.com/apache/flink/pull/268. The dependency > > > exclusions > > > > > for > > > > > > >> hadoop are basically disabled because they are overwritten by > > the > > > > > > >> hadoop200alpha profile. This causes issues like two asm > versions > > > > (4.0 > > > > > > and > > > > > > >> 3.2) in the lib/ folder. > > > > > > >> > > > > > > >> Very cosmetic: > > > > > > >> - We can update the urls in the README.txt / README.md of the > > > binary > > > > > > >> releases. They contain > > > > > > >> http://flink.incubator.apache.org > > > > > > >> https://github.com/apache/incubator-flink > > > > > > >> u...@flink.incubator.apache.org > > > > > > >> dev@flink.incubator.apache.org > > > > > > >> Which all don't need the "incubator" anymore (mailinglists > > should > > > > work > > > > > > as > > > > > > >> well, but we need to verify) > > > > > > >> > > > > > > >> > > > > > > >> Positive checks: > > > > > > >> - No incubator notices > > > > > > >> - Example plan preview is working for all examples > > > > > > >> - All examples are running with build-in data, submitted from > > the > > > > web > > > > > > >> interface > > > > > > >> - scala examples (some) > > > > > > >> - streaming examples (some) > > > > > > >> - No exceptions or WARN log messages occurred after a "start > jm > > / > > > > web > > > > > > --> > > > > > > >> all java batch examples --> stop everything" cycle. > > > > > > >> - ./bin/flink also works for the wordcount example ;) > > > > > > >> - no binaries in the src archive > > > > > > >> - the source archive builds (mvn clean install) (including rat > > > > license > > > > > > >> header check, checkstyle and all tests) > > > > > > >> - gpg signatures are valid: gpg --verify > > flink-0.8.0-src.tgz.asc > > > > > > >> flink-0.8.0-src.tgz > > > > > > >> > > > > > > >> > > > > > > >> On Wed, Jan 7, 2015 at 7:55 PM, Robert Metzger < > > > rmetz...@apache.org > > > > > > > > > > > >> wrote: > > > > > > >> > > > > > > >> > -1 > > > > > > >> > > > > > > > >> > There is a major issue with the hadoop1 and hadoop2 > versions: > > We > > > > > made > > > > > > >> > hadoop2 the default profile, so the 0.8.0 version will pull > > the > > > > > > hadoop2 > > > > > > >> > dependencies. > > > > > > >> > We basically need an explicit 0.8.0-hadoop1 version now. > > > > > > >> > The release candidate contains a 0.8.0 version which has > > hadoop2 > > > > > > >> activated > > > > > > >> > by default, the 0.8.0-hadoop2 version also has hadoop2 > > activated > > > > by > > > > > > >> > default. In this release candidate, there is no way for > users > > to > > > > > > depend > > > > > > >> on > > > > > > >> > a hadoop1 flink dependency. > > > > > > >> > > > > > > > >> > In general, we should warn users that the default hadoop > > > > > dependencies > > > > > > >> have > > > > > > >> > changed with the release! > > > > > > >> > > > > > > > >> > > > > > > > >> > I suggest to wait a bit with the next RC to collect more > > > feedback > > > > on > > > > > > the > > > > > > >> > release candidate. > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > On Wed, Jan 7, 2015 at 7:37 PM, Márton Balassi < > > > > mbala...@apache.org > > > > > > > > > > > > >> > wrote: > > > > > > >> > > > > > > > >> >> Please vote on releasing the following candidate as Apache > > > Flink > > > > > > >> version > > > > > > >> >> 0.8.0 > > > > > > >> >> > > > > > > >> >> This release will be the first major release for Flink as a > > top > > > > > level > > > > > > >> >> project. > > > > > > >> >> > > > > > > >> >> > ------------------------------------------------------------- > > > > > > >> >> The commit to be voted on is in the branch > > "release-0.8.0-rc1" > > > > > > >> >> (commit 8c30f60): > > > > > > >> >> > https://git-wip-us.apache.org/repos/asf/flink/commit/8c30f60 > > > > > > >> >> > > > > > > >> >> The release artifacts to be voted on can be found at: > > > > > > >> >> http://people.apache.org/~mbalassi/flink-0.8.0-rc1/ > > > > > > >> >> > > > > > > >> >> Release artifacts are signed with the following key: > > > > > > >> >> https://people.apache.org/keys/committer/mbalassi.asc > > > > > > >> >> > > > > > > >> >> The staging repository for this release can be found at: > > > > > > >> >> > > > > > > > > > https://repository.apache.org/content/repositories/orgapacheflink-1021 > > > > > > >> >> > ------------------------------------------------------------- > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> > > > > > > >> >> Please vote on releasing this package as Apache Flink > 0.8.0. > > > > > > >> >> > > > > > > >> >> The vote is open for the next 72 hours and passes if a > > majority > > > > of > > > > > at > > > > > > >> >> least > > > > > > >> >> three +1 PMC votes are cast. > > > > > > >> >> > > > > > > >> >> [ ] +1 Release this package as Apache Flink 0.8.0 > > > > > > >> >> [ ] -1 Do not release this package because ... > > > > > > >> >> > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
