I'd like to add a core module to hadoop, but I'm running into some issues getting started. What I want is to be able to add a native library and codec to some stable build of hadoop, build, debug, experiment, and benchmark. Currently, I'm trying to rebuild the Cloudera rpms so I can get a complete stable set of source to start from. (When I tried working from the SVN trunk, it seemed there was so much active development going on, it was hard to get something stable to compile.) So, I'm working from the Cloudera instructions - https://ccp.cloudera.com/display/CDHDOC/Building+RPMs+from+CDH+Source+RPMs. I downloaded hadoop-0.20-0.20.2+923.97-1.src.rpm, installed jdk, ant, maven, and set various environment variables: export PATH=$PATH:/usr/local/apache-maven-3.0.3/bin export JAVA_HOME=/usr/java/jdk1.7.0 export HADOOP_HOME=/usr/lib/hadoop-0.20 export HADOOP_VERSION=`hadoop version | head -n 1 | cut -f 2 -d " "` I wasn't sure what to do about ANT_HOME, FORREST_HOME, or JAVA5_HOME as forrest wasn't requested to be installed, ant doesn't appear to have a special directory anywhere, and just generally not sure what's up with JAVA5_HOME, but this appears to be forrest related? None of these is generating complaints when I build. When I build, I get a whole bunch of output including the following: rpmbuild --rebuild $SRPM .........<snip>............ compile: [echo] contrib: gridmix [javac] Compiling 31 source files to /home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/build/contrib/gridmix/classes [javac] /home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java:396: error: type argument ? extends T is not within bounds of type-variable E [javac] private <T> String getEnumValues(Enum<? extends T>[] e) { [javac] ^ [javac] where T,E are type-variables: [javac] T extends Object declared in method <T>getEnumValues(Enum<? extends T>[]) [javac] E extends Enum<E> declared in class Enum [javac] /home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java:399: error: type argument ? extends T is not within bounds of type-variable E [javac] for (Enum<? extends T> v : e) { [javac] ^ [javac] where T,E are type-variables: [javac] T extends Object declared in method <T>getEnumValues(Enum<? extends T>[]) [javac] E extends Enum<E> declared in class Enum [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 2 errors
Questions: 1 - Is there a more appropriate environment to work from than the Cloudera distribution for developing codecs? 2 - If not, is this an appropriate place to ask about how to build Cloudera? 3 - Any suggestions for getting this rpm to rebuild? 4 - Any suggestions for editing the rpm so I can just wipe out gridmix altogether? TIA - Tim.
