[
https://issues.apache.org/jira/browse/HAMA-431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098250#comment-13098250
]
Thomas Jungblut commented on HAMA-431:
--------------------------------------
Sorry for my late feedback and thank you for your help and information.
Currently our codebase has a majority of the "old" architecture of Hadoop. We
changed parts of the computation model, but task execution and job livecycle
stays the same as it was in the "old" Hadoop architecture. We put a
synchronization service on top of it which is working (most of the time it is
not working:P) with zookeeper. In addition we have RPC connections between the
servers to message each other.
I suggest to implement our BSPMaster as an "Application Master". He must take
care of allocating new Containers which then will be a "Groom" in our
namespace. Each groom needs a ZNode and some kind of identifier.
But there is a question of security in my mind. Do you mind when we don't care
about security in the first version? I'm not an expert in these authentication
systems like Kerberos.
So everything is actually implemented some way, but we need to port this code
to YARN. I have alot of time tomorrow so I just start. I also think we are
going to split this task up to several smaller pieces, so our other developers
can contribute to it, too.
But I have a more general question:
Should we make this task part of our framework? Like another maven module which
can be plugged-in into Hadoop?
> MapReduce NG integration
> ------------------------
>
> Key: HAMA-431
> URL: https://issues.apache.org/jira/browse/HAMA-431
> Project: Hama
> Issue Type: New Feature
> Reporter: Thomas Jungblut
> Assignee: Thomas Jungblut
>
> We should take a look at how to integrate Hama's BSP Engine to Hadoop's
> nextGen application platform.
> Can be currently found in the 0.23 branch.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
