Ed Kohlwey commented on GIRAPH-111:

Maybe I'm getting ahead of myself. I discovered the need for such a thing while 
working on my patch for GIRAPH-108, but others might have better ideas on how 
to address the problem.

The issue comes from the desire to create a giraph-specific context that can 
use delegation to either hook into the existing hadoop context/reporting system 
or report back to a context system that is specific to the resource allocator 
being used on the cluster (Mesos, YARN). Furthermore, this system should be 
backwards-compatible with (meaning, they should inherit from) Hadoop's system 
in order to instantiate InputFormats, etc. This is where the problem occurs.

Since Hadoop's io libraries use class-based inheritance rather than interfaces, 
you have to use the hack I reference in GIRAPH-108 (creating a wrapper class 
where every method is overrided) that uses a sort of nasty constructor with 
null arguments to trick the underlying superclass's code into being 
instantiated, when in reality you're really ignoring the superclass 
implementation entirely. You can check the code out in the patch - 
GraphMapper.HadoopContext and GraphProcess.Context.
> Refactor I/O to be independent of Map/Reduce
> --------------------------------------------
>                 Key: GIRAPH-111
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-111
>             Project: Giraph
>          Issue Type: Improvement
>          Components: graph
>            Reporter: Ed Kohlwey
> The I/O mechanisms should probably be abstracted entirely from Map/Reduce in 
> order to support making Giraph an independent framework.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
For more information on JIRA, see: http://www.atlassian.com/software/jira


Reply via email to