I vote to include it as well. It is sort of vital for advanced use of the Scala API.
It is also an isolated change that does not affect other components, so it should be testable very well. On Fri, Jan 9, 2015 at 7:05 PM, Robert Metzger <rmetz...@apache.org> wrote: > I'd say yes because its affecting a user. > > On Fri, Jan 9, 2015 at 7:00 PM, Aljoscha Krettek <aljos...@apache.org> > wrote: > > > I have a fix for this issue: > > > https://issues.apache.org/jira/browse/FLINK-1378?jql=project%20%3D%20FLINK > > and I think this should also make it into the 0.8.0 release. > > > > What do you think? > > > > On Fri, Jan 9, 2015 at 3:28 PM, Márton Balassi <balassi.mar...@gmail.com > > > > wrote: > > > Sure, thanks Ufuk. > > > > > > On Fri, Jan 9, 2015 at 3:15 PM, Ufuk Celebi <u...@apache.org> wrote: > > > > > >> Marton, could you also cherry-pick 7f659f6 and 7e08fa1 for the next > RC? > > >> It's a minor update to the README describing the IDE setup. > > >> > > >> I will closed the respective issue FLINK-1109. > > >> > > >> On 08 Jan 2015, at 23:50, Henry Saputra <henry.sapu...@gmail.com> > > wrote: > > >> > > >> > Marton, could you close this VOTE thread by replying to the original > > >> > email and append [CANCEL] in the subject line. > > >> > > > >> > - Henry > > >> > > > >> > On Thu, Jan 8, 2015 at 9:35 AM, Márton Balassi < > > balassi.mar...@gmail.com> > > >> wrote: > > >> >> Cherry-picked and tested: found no duplicate dependencies in lib, > > yarn > > >> >> uberjar build goes without the mentioned warns. > > >> >> Travis tests are passing, pushing soon. > > >> >> > > >> >> On Thu, Jan 8, 2015 at 4:57 PM, Stephan Ewen <se...@apache.org> > > wrote: > > >> >> > > >> >>> Nice. > > >> >>> > > >> >>> @Marton: As soon as as you are done, I make a pass over the > > licenses... > > >> >>> > > >> >>> Stephan > > >> >>> > > >> >>> > > >> >>> On Thu, Jan 8, 2015 at 4:42 PM, Robert Metzger < > rmetz...@apache.org > > > > > >> >>> wrote: > > >> >>> > > >> >>>> Allright. The travis tests are green and I tested it again with > > >> Tachyon > > >> >>> on > > >> >>>> a cluster. > > >> >>>> > > >> >>>> My pull request also fixes some of the issues mentioned earlier > in > > >> this > > >> >>>> thread by Stephan (the warnings from shading regarding duplicate > > >> >>> classes). > > >> >>>> I can not fix all of them because some projects like hadoop just > > have > > >> the > > >> >>>> "conflicts" in their dependencies. > > >> >>>> > > >> >>>> I'll merge the pull request to master. > > >> >>>> Marton can merge it to "release-0.8". > > >> >>>> > > >> >>>> On Thu, Jan 8, 2015 at 2:42 PM, Stephan Ewen <se...@apache.org> > > >> wrote: > > >> >>>> > > >> >>>>> Robert, can you let us know once the PR is merged? > > >> >>>>> > > >> >>>>> Then we can start with the next steps... > > >> >>>>> > > >> >>>>> > > >> >>>>> > > >> >>>>> On Thu, Jan 8, 2015 at 2:07 PM, Robert Metzger < > > rmetz...@apache.org> > > >> >>>>> wrote: > > >> >>>>> > > >> >>>>>> Please wait with merging 268. There is still a minor open > issue. > > >> >>>>>> > > >> >>>>>> On Thu, Jan 8, 2015 at 1:28 PM, Márton Balassi < > > >> >>>> balassi.mar...@gmail.com > > >> >>>>>> > > >> >>>>>> wrote: > > >> >>>>>> > > >> >>>>>>> +1 for merging PR 268, volunteering for the merge. It should > > also > > >> >>>> solve > > >> >>>>>> the > > >> >>>>>>> issue mentioned by Stephan with the yarn uberjar. > > >> >>>>>>> > > >> >>>>>>> Updating the release scripts and doing a double-check on the > > >> >>>> cosmetics. > > >> >>>>>>> > > >> >>>>>>> On Thu, Jan 8, 2015 at 12:55 PM, Stephan Ewen < > se...@apache.org > > > > > >> >>>>> wrote: > > >> >>>>>>> > > >> >>>>>>>> We just found and fixed another highly critical bug. > > >> >>>>>>>> > > >> >>>>>>>> I vote to also cherry pick commit > > >> >>>>>>>> 19066b520435528e104a69ccf372f56811123ee3 ([FLINK-1371] > > >> >>>>>>>> [runtime] Fix KryoSerializer to not swallow EOFExceptions) > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> > > >> >>>>>>>> On Thu, Jan 8, 2015 at 11:55 AM, Robert Metzger < > > >> >>>> rmetz...@apache.org > > >> >>>>>> > > >> >>>>>>>> wrote: > > >> >>>>>>>> > > >> >>>>>>>>> +1 for merging pull request 268. It makes Flink even more > > >> >>>>> compatible > > >> >>>>>>> with > > >> >>>>>>>>> hadoop. > > >> >>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>>> regarding the documentation: I already updated the build > > >> >>>>>> instructions. > > >> >>>>>>>> But > > >> >>>>>>>>> we should double check everything again. > > >> >>>>>>>>> I probably won't have time to look into this today however. > > >> >>>>>>>>> > > >> >>>>>>>>> On Thu, Jan 8, 2015 at 11:50 AM, Stephan Ewen < > > >> >>> se...@apache.org> > > >> >>>>>>> wrote: > > >> >>>>>>>>> > > >> >>>>>>>>>> After the dependencies are cleaned up (Robert's pull > request > > >> >>>>>>>>>> https://github.com/apache/flink/pull/268 solves that), we > > >> >>> need > > >> >>>>> to > > >> >>>>>>> also > > >> >>>>>>>>> do > > >> >>>>>>>>>> the following: > > >> >>>>>>>>>> > > >> >>>>>>>>>> - Make a pass over dependencies to ensure that NOTICE and > > >> >>>>> LICENSE > > >> >>>>>>>> files > > >> >>>>>>>>>> are up to date (both source and bin distribution) > > >> >>>>>>>>>> - Adjust the variables in the docs to reflect the fact that > > >> >>>>>> hadoop2 > > >> >>>>>>> is > > >> >>>>>>>>> now > > >> >>>>>>>>>> the default profile > > >> >>>>>>>>>> > > >> >>>>>>>>>> I would volunteer for the first task. > > >> >>>>>>>>>> > > >> >>>>>>>>>> Stephan > > >> >>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>>> On Wed, Jan 7, 2015 at 10:01 PM, Stephan Ewen < > > >> >>>> se...@apache.org> > > >> >>>>>>>> wrote: > > >> >>>>>>>>>> > > >> >>>>>>>>>>> I also see a warning when building the yarn fat jar. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> This is a consequence of the duplicate dependencies. > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> [WARNING] servlet-api-3.0.20100224.jar, > > >> >>>>>>> javax.servlet-api-3.0.1.jar, > > >> >>>>>>>>>>> javax.servlet-3.1.jar, servlet-api-2.5.jar define 42 > > >> >>>>> overlappping > > >> >>>>>>>>>> classes: > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.Cookie > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.HttpSessionBindingEvent > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.NoBodyResponse > > >> >>>>>>>>>>> [WARNING] - javax.servlet.ServletContext > > >> >>>>>>>>>>> [WARNING] - javax.servlet.ServletOutputStream > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.HttpSessionListener > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.HttpSessionContext > > >> >>>>>>>>>>> [WARNING] - javax.servlet.FilterChain > > >> >>>>>>>>>>> [WARNING] - javax.servlet.GenericServlet > > >> >>>>>>>>>>> [WARNING] - javax.servlet.http.HttpServletRequestWrapper > > >> >>>>>>>>>>> [WARNING] - 32 more... > > >> >>>>>>>>>>> [WARNING] hadoop-yarn-api-2.2.0.jar, > > >> >>>>> hadoop-yarn-common-2.2.0.jar > > >> >>>>>>>>> define > > >> >>>>>>>>>> 3 > > >> >>>>>>>>>>> overlappping classes: > > >> >>>>>>>>>>> [WARNING] - org.apache.hadoop.yarn.util.package-info > > >> >>>>>>>>>>> [WARNING] - > org.apache.hadoop.yarn.factories.package-info > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>> org.apache.hadoop.yarn.factory.providers.package-info > > >> >>>>>>>>>>> [WARNING] asm-4.0.jar, asm-3.1.jar define 21 overlappping > > >> >>>>>> classes: > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.Type > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.AnnotationVisitor > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.MethodVisitor > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.Attribute > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.FieldWriter > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.signature.SignatureWriter > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.MethodWriter > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.Edge > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.Handler > > >> >>>>>>>>>>> [WARNING] - org.objectweb.asm.ByteVector > > >> >>>>>>>>>>> [WARNING] - 11 more... > > >> >>>>>>>>>>> [WARNING] javax.servlet-api-3.0.1.jar, > > >> >>> javax.servlet-3.1.jar, > > >> >>>>>>>>>>> servlet-api-3.0.20100224.jar define 32 overlappping > > >> >>> classes: > > >> >>>>>>>>>>> [WARNING] - javax.servlet.annotation.HttpConstraint > > >> >>>>>>>>>>> [WARNING] - javax.servlet.DispatcherType > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>> javax.servlet.descriptor.JspPropertyGroupDescriptor > > >> >>>>>>>>>>> [WARNING] - javax.servlet.Registration > > >> >>>>>>>>>>> [WARNING] - javax.servlet.SessionTrackingMode > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>>> javax.servlet.annotation.ServletSecurity$EmptyRoleSemantic > > >> >>>>>>>>>>> [WARNING] - javax.servlet.annotation.HandlesTypes > > >> >>>>>>>>>>> [WARNING] - javax.servlet.ServletRegistration > > >> >>>>>>>>>>> [WARNING] - javax.servlet.annotation.ServletSecurity > > >> >>>>>>>>>>> [WARNING] - javax.servlet.ServletContainerInitializer > > >> >>>>>>>>>>> [WARNING] - 22 more... > > >> >>>>>>>>>>> [WARNING] jasper-compiler-5.5.23.jar, > > >> >>>> jasper-runtime-5.5.23.jar > > >> >>>>>>>> define > > >> >>>>>>>>> 1 > > >> >>>>>>>>>>> overlappping classes: > > >> >>>>>>>>>>> [WARNING] - org.apache.jasper.compiler.Localizer > > >> >>>>>>>>>>> [WARNING] commons-collections-3.2.1.jar, > > >> >>>>>>> commons-beanutils-1.7.0.jar, > > >> >>>>>>>>>>> commons-beanutils-core-1.8.0.jar define 10 overlappping > > >> >>>>> classes: > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>> org.apache.commons.collections.FastHashMap$EntrySet > > >> >>>>>>>>>>> [WARNING] - > > >> >>>> org.apache.commons.collections.FastHashMap$KeySet > > >> >>>>>>>>>>> [WARNING] - org.apache.commons.collections.ArrayStack > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> > > > org.apache.commons.collections.FastHashMap$CollectionView$CollectionViewIterator > > >> >>>>>>>>>>> [WARNING] - > > >> >>>> org.apache.commons.collections.FastHashMap$Values > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>> org.apache.commons.collections.FastHashMap$CollectionView > > >> >>>>>>>>>>> [WARNING] - org.apache.commons.collections.FastHashMap$1 > > >> >>>>>>>>>>> [WARNING] - org.apache.commons.collections.Buffer > > >> >>>>>>>>>>> [WARNING] - org.apache.commons.collections.FastHashMap > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>> org.apache.commons.collections.BufferUnderflowException > > >> >>>>>>>>>>> [WARNING] commons-beanutils-1.7.0.jar, > > >> >>>>>>>> commons-beanutils-core-1.8.0.jar > > >> >>>>>>>>>>> define 82 overlappping classes: > > >> >>>>>>>>>>> [WARNING] - > org.apache.commons.beanutils.ConvertUtilsBean > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>> org.apache.commons.beanutils.converters.SqlTimeConverter > > >> >>>>>>>>>>> [WARNING] - org.apache.commons.beanutils.Converter > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>>> org.apache.commons.beanutils.converters.FloatArrayConverter > > >> >>>>>>>>>>> [WARNING] - > > >> >>>> org.apache.commons.beanutils.NestedNullException > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>> org.apache.commons.beanutils.ConvertingWrapDynaBean > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>>> org.apache.commons.beanutils.converters.LongArrayConverter > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>> org.apache.commons.beanutils.converters.SqlDateConverter > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>>>>> > org.apache.commons.beanutils.converters.BooleanArrayConverter > > >> >>>>>>>>>>> [WARNING] - > > >> >>>>>>> org.apache.commons.beanutils.converters.StringConverter > > >> >>>>>>>>>>> [WARNING] - 72 more... > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> On Wed, Jan 7, 2015 at 9:08 PM, Robert Metzger < > > >> >>>>>>> rmetz...@apache.org> > > >> >>>>>>>>>>> wrote: > > >> >>>>>>>>>>> > > >> >>>>>>>>>>>> ... I did some more checks: > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> The hadoop200alpha version has some issues with the > hadoop > > >> >>>>>>>> dependency > > >> >>>>>>>>>>>> exclusions. I fixed the issues in this pull request: > > >> >>>>>>>>>>>> https://github.com/apache/flink/pull/268. The dependency > > >> >>>>>>> exclusions > > >> >>>>>>>>> for > > >> >>>>>>>>>>>> hadoop are basically disabled because they are > overwritten > > >> >>>> by > > >> >>>>>> the > > >> >>>>>>>>>>>> hadoop200alpha profile. This causes issues like two asm > > >> >>>>> versions > > >> >>>>>>>> (4.0 > > >> >>>>>>>>>> and > > >> >>>>>>>>>>>> 3.2) in the lib/ folder. > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> Very cosmetic: > > >> >>>>>>>>>>>> - We can update the urls in the README.txt / README.md of > > >> >>>> the > > >> >>>>>>> binary > > >> >>>>>>>>>>>> releases. They contain > > >> >>>>>>>>>>>> http://flink.incubator.apache.org > > >> >>>>>>>>>>>> https://github.com/apache/incubator-flink > > >> >>>>>>>>>>>> u...@flink.incubator.apache.org > > >> >>>>>>>>>>>> d...@flink.incubator.apache.org > > >> >>>>>>>>>>>> Which all don't need the "incubator" anymore > (mailinglists > > >> >>>>>> should > > >> >>>>>>>> work > > >> >>>>>>>>>> as > > >> >>>>>>>>>>>> well, but we need to verify) > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> Positive checks: > > >> >>>>>>>>>>>> - No incubator notices > > >> >>>>>>>>>>>> - Example plan preview is working for all examples > > >> >>>>>>>>>>>> - All examples are running with build-in data, submitted > > >> >>>> from > > >> >>>>>> the > > >> >>>>>>>> web > > >> >>>>>>>>>>>> interface > > >> >>>>>>>>>>>> - scala examples (some) > > >> >>>>>>>>>>>> - streaming examples (some) > > >> >>>>>>>>>>>> - No exceptions or WARN log messages occurred after a > > >> >>> "start > > >> >>>>> jm > > >> >>>>>> / > > >> >>>>>>>> web > > >> >>>>>>>>>> --> > > >> >>>>>>>>>>>> all java batch examples --> stop everything" cycle. > > >> >>>>>>>>>>>> - ./bin/flink also works for the wordcount example ;) > > >> >>>>>>>>>>>> - no binaries in the src archive > > >> >>>>>>>>>>>> - the source archive builds (mvn clean install) > (including > > >> >>>> rat > > >> >>>>>>>> license > > >> >>>>>>>>>>>> header check, checkstyle and all tests) > > >> >>>>>>>>>>>> - gpg signatures are valid: gpg --verify > > >> >>>>>> flink-0.8.0-src.tgz.asc > > >> >>>>>>>>>>>> flink-0.8.0-src.tgz > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>> On Wed, Jan 7, 2015 at 7:55 PM, Robert Metzger < > > >> >>>>>>> rmetz...@apache.org > > >> >>>>>>>>> > > >> >>>>>>>>>>>> wrote: > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>>>> -1 > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> There is a major issue with the hadoop1 and hadoop2 > > >> >>>>> versions: > > >> >>>>>> We > > >> >>>>>>>>> made > > >> >>>>>>>>>>>>> hadoop2 the default profile, so the 0.8.0 version will > > >> >>>> pull > > >> >>>>>> the > > >> >>>>>>>>>> hadoop2 > > >> >>>>>>>>>>>>> dependencies. > > >> >>>>>>>>>>>>> We basically need an explicit 0.8.0-hadoop1 version now. > > >> >>>>>>>>>>>>> The release candidate contains a 0.8.0 version which has > > >> >>>>>> hadoop2 > > >> >>>>>>>>>>>> activated > > >> >>>>>>>>>>>>> by default, the 0.8.0-hadoop2 version also has hadoop2 > > >> >>>>>> activated > > >> >>>>>>>> by > > >> >>>>>>>>>>>>> default. In this release candidate, there is no way for > > >> >>>>> users > > >> >>>>>> to > > >> >>>>>>>>>> depend > > >> >>>>>>>>>>>> on > > >> >>>>>>>>>>>>> a hadoop1 flink dependency. > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> In general, we should warn users that the default hadoop > > >> >>>>>>>>> dependencies > > >> >>>>>>>>>>>> have > > >> >>>>>>>>>>>>> changed with the release! > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> I suggest to wait a bit with the next RC to collect more > > >> >>>>>>> feedback > > >> >>>>>>>> on > > >> >>>>>>>>>> the > > >> >>>>>>>>>>>>> release candidate. > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> On Wed, Jan 7, 2015 at 7:37 PM, Márton Balassi < > > >> >>>>>>>> mbala...@apache.org > > >> >>>>>>>>>> > > >> >>>>>>>>>>>>> wrote: > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> Please vote on releasing the following candidate as > > >> >>>> Apache > > >> >>>>>>> Flink > > >> >>>>>>>>>>>> version > > >> >>>>>>>>>>>>>> 0.8.0 > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> This release will be the first major release for Flink > > >> >>>> as a > > >> >>>>>> top > > >> >>>>>>>>> level > > >> >>>>>>>>>>>>>> project. > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> > > >> >>>>> ------------------------------------------------------------- > > >> >>>>>>>>>>>>>> The commit to be voted on is in the branch > > >> >>>>>> "release-0.8.0-rc1" > > >> >>>>>>>>>>>>>> (commit 8c30f60): > > >> >>>>>>>>>>>>>> > > >> >>>>> https://git-wip-us.apache.org/repos/asf/flink/commit/8c30f60 > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> The release artifacts to be voted on can be found at: > > >> >>>>>>>>>>>>>> http://people.apache.org/~mbalassi/flink-0.8.0-rc1/ > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> Release artifacts are signed with the following key: > > >> >>>>>>>>>>>>>> https://people.apache.org/keys/committer/mbalassi.asc > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> The staging repository for this release can be found > > >> >>> at: > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>> > > >> >>>> > > >> > https://repository.apache.org/content/repositories/orgapacheflink-1021 > > >> >>>>>>>>>>>>>> > > >> >>>>> ------------------------------------------------------------- > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> Please vote on releasing this package as Apache Flink > > >> >>>>> 0.8.0. > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> The vote is open for the next 72 hours and passes if a > > >> >>>>>> majority > > >> >>>>>>>> of > > >> >>>>>>>>> at > > >> >>>>>>>>>>>>>> least > > >> >>>>>>>>>>>>>> three +1 PMC votes are cast. > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>>> [ ] +1 Release this package as Apache Flink 0.8.0 > > >> >>>>>>>>>>>>>> [ ] -1 Do not release this package because ... > > >> >>>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>>> > > >> >>>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>>> > > >> >>>>>>>>>> > > >> >>>>>>>>> > > >> >>>>>>>> > > >> >>>>>>> > > >> >>>>>> > > >> >>>>> > > >> >>>> > > >> >>> > > >> > > >> > > >
