Thanks Hyunsik and Avery for your comments!

I've added a JIRA here: to present a patch that adds two new profiles:


hadoop_0.20.203 is the default (i.e. same as if you do not give a -P argument).

I tried "mvn clean verify && mvn -Phadoop_0.20.203 clean verify && mvn -Phadoop_trunk clean verify" and this worked with the GIRAPH-168 patch applied.

As I said on GIRAPH-168, I could not get hadoop_non_secure or hadoop_facebook to compile.

I agree that munge is hacky - in my patch I tried to modify our use of munge a bit to make it simpler: I removed the usages of HADOOP (e.g. "/*if_not[HADOOP]"). I felt like this usage is misleading, because it might make you think that Giraph can be compiled without Hadoop support at all, which is not what is meant by "/*if_not[HADOOP]": in fact, what it actually means is that the Hadoop to be used is the default version (Hadoop 0.205.0).


On 3/21/12 3:46 AM, Hyunsik Choi wrote:
This approach looks good.

On Wed, Mar 21, 2012 at 3:53 PM, Avery Ching<>  wrote:

I agree with this approach, although munge is kinda hacky.  It is easy
though. =)


On 3/20/12 5:52 PM, Eugene Koontz wrote:

Hi Giraphers,
    I think it might be good to look at how we can add support for new
hadoop versions. Currently we have hadoop_facebook (**jira/browse/GIRAPH-14<>).
I am considering adding new ones such as hadoop_0.24. Looking at the code,
it seems that the main hadoop variation between the stock hadoop used
(0.203.0) versus facebook has to do with the new security-related APIs in
the latter that is, fortunately, also available in hadoop 0.23 and 0.24.
So, hopefully we can make use of the existing work that Avery has done for
hadoop_facebook and apply it to other hadoop versions. Therefore I would
propose that:

1. a new munge flag HADOOP_SECURE to be used in and
a few other places, where we are currently checking for HADOOP_FACEBOOK and
2. we make a new profile called hadoop_secure, which, as with
hadoop_facebook, will use the above munge flag.
3. we make a new profile "hadoop_0.20.203" for the existing default
hadoop and make it the default profile (activeByDefault=true). This will
makes it easier to handle the differences in the hadoop library dependency
set that have happened between 0.20.203 and hadoop trunk.

Please see**giraph/tree/security-profile<>for
 my branch that implements the above.



Reply via email to