Hi, thank you Ron and Jörg for the responses!
Using the dependency management in the common parent pom resolved my problem! Thank you again, Robert On Fri, Nov 14, 2014 at 7:55 PM, Jörg Schaible <[email protected]> wrote: > Hi Robert, > > Robert Metzger wrote: > > > Hi, > > > > I'm a developer at the Apache Flink (incubating) project. I'm trying to > > pass the version of a dependency using a property from the command line > to > > the binary build. But it seems that the version I've set is not respected > > everywhere. > > > > > > We have the following setup: > > - flink-parent > > - flink-core > > - ... more modules ... > > - flink-yarn > > - flink-dist > > > > The "flink-yarn" module (and others) have dependencies to Apache Hadoop. > > We have a property (called hadoop.version) to have one place for setting > > the Hadoop version. The default Hadoop version 2.2.0. > > > > The "flink-dist" module is creating the binary release for Flink. It is > > using the maven-assembly-plugin for that. All jars we depend on are > placed > > in the "lib" directory. > > The "flink-dist" module has most of our modules (also the flink-yarn) > > module as a dependency so that we can use the modules in the assembly. > > > > > > > > What I'm trying to do is building Flink for a different Hadoop version, > > say "2.2.0-cdh5.0.0-beta-2". Therefore, I'm passing this version as > > "-Dhadoop.version=2.2.0-cdh5.0.0-beta-2". > > > > However, some files from Hadoop "2.2.0" end up in the "lib" directory. > > > > mvn dependency:tree reveals for "flink-dist": > > > > [INFO] +- org.apache.flink:flink-yarn:jar:0.8-incubating-SNAPSHOT:compile > > [INFO] | +- org.apache.hadoop:hadoop-yarn-client:jar:2.2.0:compile > > [INFO] | | +- org.apache.hadoop:hadoop-yarn-api:jar:2.2.0:compile > > [INFO] | | +- org.apache.hadoop:hadoop-yarn-common:jar:2.2.0:compile > > > > however, for "flink-yarn" the versions are correct: > > > > [INFO] +- > > org.apache.hadoop:hadoop-yarn-client:jar:2.2.0-cdh5.0.0-beta-2:compile > > [INFO] | +- commons-logging:commons-logging:jar:1.1.3:compile > > [INFO] | +- commons-lang:commons-lang:jar:2.6:compile > > > > Here is the full dependency:tree output: > > https://gist.github.com/rmetzger/70c5f35d4cfd06a91169 > > > > > > How can I create builds for different Hadoop versions using Maven? > > General rule: > - don't use a version tag for direct dependencies > - use a common parent for all modules with > - a dependency management section for any dependency you want to control > - use properties for these versions > > These properties can be overwritten from command line or setting the > property again explicitly in a module. > > Cheers, > Jörg > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
