Hello all, I am running into a general problem with trying to use bigtop/maven
to resolve dependencies. The common pattern seems to be, define your
dependencies as properties, for example if your dependency is hadoop define a
variable named “hadoop.version” and put a default version string into that
property. In the Bigtop world, we override this value with a
–Dhadoop.version=${HADOOP_VERSION} in the do-component-build script.
The problem is the scenario sort of listed here:
http://stackoverflow.com/questions/12212440/overriding-maven-properties-on-command-line-maven-2-2-1.
Namely, say I bring in a new version of Hadoop (let’s call it 2.6.0-rob) and
my Bigtop has a version of Mahout which depends on a previous custom version of
hadoop, let’s call it 2.4.0-rob. Also this bigtop has a project named
“leidle-project" that depends on Mahout. I can build Mahout just fine with the
–Dhadoop.version=2.6.0-rob, however, when I build my “leidle-project” which
depends on Mahout it will try to bring in this 2.4.0-rob version of Hadoop as a
transitive dependency of Mahout and the build will fail if maven cannot find
it. In addition, I would like to produce a complete Maven repository output as
a result of my Bigtop build and I would like all the projects in this
repository to depend on the same Hadoop version.
An analogous situation in Bigtop is Hive which brings in HBase and Hadoop,
which I believe has undergone several changes:
https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hive/do-component-build
Any thoughts on this scenario? I could fix it by doing a search and replace in
the pom file prior to the build but this seems clunky and problematic.
