[
https://issues.apache.org/jira/browse/GIRAPH-564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603558#comment-13603558
]
Eli Reisman commented on GIRAPH-564:
------------------------------------
@Avery: yes this will be a big help. On the other hand, I have gotten the YARN
impl to work without it, and in a stable way, so we're free to let this take a
while if we want. Replacing the Mapper#Context connection that has been exposed
now in GraphMapper/GraphTaskManager will get rid of the umbilical cord to
Hadoop MRv1. On the other hand, this also means a big refactor to the IO
formats since they depend on various Task-related objects handed off to us by
the Mapper#Context, our Configuration is the "easy" one to deal with.
@Alessandro: I like this idea, and I like the simplification. I'm thinking
there were some places outside IO where the Immutable version is non-negotiable
to get the generics plumbing to work on that reference down the road. So just
placing the wrapper in the IO might not work. I think there were at least 2
places just internal to the YARN setup code and ConfigurationUtils where I had
to wrap the class to keep the generics working. Other times it doesn't seem to
matter.
A cleaner solution here is inevitable soon. The Mapper#Context is the key to
the whole thing. I was actually going to put up this JIRA this week myself ;)
In the GiraphYarnTask in GIRAPH-13 you can see what stuff the Mapper#Context
replacement will need to carry in for the engine to turn over on the Giraph
side. So it could be simpler than Mapper#context in a bunch of ways also.
> Input formats should provide GiraphContext
> ------------------------------------------
>
> Key: GIRAPH-564
> URL: https://issues.apache.org/jira/browse/GIRAPH-564
> Project: Giraph
> Issue Type: Improvement
> Affects Versions: 0.2.0
> Reporter: Avery Ching
>
> Context is a MapReduce Context that input classes have to explicitly create a
> ImmutableGiraphClassesConfiguration from (which is not intuitive). It would
> be better to provide a GiraphContext that would provide a
> ImmutableGiraphClassesConfiguration directly for the user, while still
> providing the user access to the MapReduce Context if really necessary. This
> might also help with the YARN port? Not sure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
