I agree mostly with Robert. However, one could also argue that by not including the libraries in the dist package, the user code jar will also be blown up by the dependencies added by the library. This will slow down job submission, because it has to be distributed on the cluster. Furthermore, I wouldn't expect all our users to use the quickstarts archetypes or to set up maven such that it builds a fat jar.
I think the best is to explicitly document how to use the libraries and what to do in order to run it on the cluster. On Jun 12, 2015 9:15 PM, "Robert Metzger" <rmetz...@apache.org> wrote: > Regarding the discussion with including ML, Gelly, streaming connectors > into "flink-dist". > I'm strongly against adding those into our jar because they blow up the > dependencies we are shipping by default. > > Also, the maven archetype sets up everything so that the dependencies are > packaged into the usercode jar. > I'd say most of the time users are using custom dependencies anyways > (Guava), so they need to set this up properly. > > I would not start recommending our users putting their dependencies into > the lib/ folder. Its much more convenient to let maven do the fat-jar > packaging. > > > On Fri, Jun 12, 2015 at 9:44 AM, Till Rohrmann <trohrm...@apache.org> > wrote: > > > I've finished the legal check of the source and binary distribution. The > PR > > with the LICENSE and NOTICE file updates can be found here [1]. > > > > What I haven't done yet is addressing the issue with the shaded > > dependencies. I think that we have to add to all jars which contain > > dependencies as binary data a LICENSE/NOTICE file referencing the > included > > dependencies if they are not licensed under Apache-2.0 or contain a > special > > NOTICE portion. > > > > Cheers, > > Till > > > > [1] https://github.com/apache/flink/pull/830 > > > > On Fri, Jun 12, 2015 at 5:44 PM Maximilian Michels <m...@apache.org> > wrote: > > > > > I almost finished creating the new release candidate. Then the maven > > deploy > > > command failed on me for the hadoop1 profile: > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD FAILURE > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 19:15.388s > > > [INFO] Finished at: Fri Jun 12 15:25:50 UTC 2015 > > > [INFO] Final Memory: 126M/752M > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [ERROR] Failed to execute goal > > > org.apache.maven.plugins:maven-checkstyle-plugin:2.12.1:check > (validate) > > on > > > project flink-language-binding-generic: Failed during checkstyle > executi > > > on: Unable to find suppressions file at location: > > > /tools/maven/suppressions.xml: Could not find resource > > > '/tools/maven/suppressions.xml'. -> [Help 1] > > > [ERROR] > > > [ERROR] To see the full stack trace of the errors, re-run Maven with > the > > -e > > > switch. > > > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > > > [ERROR] > > > [ERROR] For more information about the errors and possible solutions, > > > please read the following articles: > > > [ERROR] [Help 1] > > > > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > > > [ERROR] > > > [ERROR] After correcting the problems, you can resume the build with > the > > > command > > > [ERROR] mvn <goals> -rf :flink-language-binding-generic > > > > > > I need to look into this later. Unfortunately, I'm traveling this > > weekend. > > > > > > On Fri, Jun 12, 2015 at 3:34 PM, Fabian Hueske <fhue...@gmail.com> > > wrote: > > > > > > > OK, guys. I merged and pushed the last outstanding commits to the > > > > release-0.9 branch. > > > > Good to go for a new candidate. > > > > > > > > 2015-06-12 14:30 GMT+02:00 Maximilian Michels <m...@apache.org>: > > > > > > > > > +1 Let's constitute the changes in a new release candidate. > > > > > > > > > > On Fri, Jun 12, 2015 at 2:06 PM, Fabian Hueske <fhue...@gmail.com> > > > > wrote: > > > > > > > > > > > +1 for b) > > > > > > > > > > > > I'm organizing + merging the commits that need to go the new > > > candidate > > > > > > right now. Will let you know, when I am done. > > > > > > > > > > > > 2015-06-12 14:03 GMT+02:00 Till Rohrmann < > till.rohrm...@gmail.com > > >: > > > > > > > > > > > > > I'm in favour of option b) as well. > > > > > > > > > > > > > > On Fri, Jun 12, 2015 at 12:05 PM Ufuk Celebi <u...@apache.org> > > > wrote: > > > > > > > > > > > > > > > Yes, the LICENSE files are definitely a release blocker. > > > > > > > > > > > > > > > > a) Either we wait with the RC until we have fixed the > LICENSES, > > > or > > > > > > > > > > > > > > > > b) Put out next RC to continue with testing and then update > it > > > with > > > > > the > > > > > > > > LICENSE [either we find something before the LICENSE update > or > > we > > > > > only > > > > > > > have > > > > > > > > to review the LICENSE change] > > > > > > > > > > > > > > > > Since this is not a vote yet, it doesn't really matter, but > I'm > > > > > leaning > > > > > > > > towards b). > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Jun 12, 2015 at 11:43 AM, Till Rohrmann < > > > > > > till.rohrm...@gmail.com > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > What about the shaded jars? > > > > > > > > > > > > > > > > > > On Fri, Jun 12, 2015 at 11:32 AM Ufuk Celebi < > u...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > @Max: for the new RC. Can you make sure to set the > > variables > > > > > > > correctly > > > > > > > > > > with regard to stable/snapshot versions in the docs? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
