[
https://issues.apache.org/jira/browse/HAMA-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865430#action_12865430
]
Hyunsik Choi commented on HAMA-251:
-----------------------------------
Before I complete the UML diagram and the design of the job manager, I would
like to discuss my plan with you. Overall, the job management system of BSP is
resemble to that of MR. The below summary includes not only a job management
but also an overview of bsp cluster. It is not completed yet, and I'll update
this summary later.
----
Summary:
One BSP cluster consists of one BSPMaster, multiple GroomServer and one or more
zookeeper servers in a network environment. Like JobTracker of MR, the bsp
master plays the following roles:
* Maintaining groom server status.
* Controlling supersteps in a cluster.
* Maintaining job progress information.
* Assining tasks to groom servers.
* Job Scheduing
* Controlling fault.
* Providing users with the cluster control interface.
Like TaskTracker of MR, a Groom Server (shortly referred to as groom) is a
process that performs bsp tasks assigned by BSPMaster. Each groom contacts the
BSPMaster, and it takes assigned tasks and reports its status by means of
periodical piggybacks with BSPMaster. Each groom is designed to run with HDFS
or other distributed storages. Basically, a groom server and a data node should
be run on one physical node.
A BSP Master and multiple grooms are started by the script. Then, the bsp
master starts up with a RPC server for groom servers. Groom servers starts up
with a BSPPeer instance - later, BSPPeer needs to be integrated with
GroomServer - and a RPC proxy to contact the bsp master. After started, each
groom periodically sends a heartbeat message that encloses its groom server
status, including maximum task capacity, unused memory, and so on.
Each time the bsp master receives a heartbeat message, it brings up-to-date
groom server status - the bsp master makes use of groom servers' status in
order to effectively assign tasks to idle groom servers - and returns a
heartbeat response that contains assigned tasks and others actions that a groom
server has to do. For now, I'll develop a FIFO job scheduler and very simple
task assignment algorithms.
> Add job manager to BSPMaster
> ----------------------------
>
> Key: HAMA-251
> URL: https://issues.apache.org/jira/browse/HAMA-251
> Project: Hama
> Issue Type: Sub-task
> Affects Versions: 0.2.0
> Reporter: Hyunsik Choi
> Assignee: Hyunsik Choi
> Fix For: 0.2.0
>
>
> In regard to BSPMaster, I plan to design as follows:
> * BSPMaster is responsible to manage jobs submitted by clients.
> * BSPMaster is responsible to disseminate submitted jobs to BSPPeers.
> * BSPMaster is responsible to control job progress and BSPPeers actions.
> It will be similar to JobTracker. In this issue, let's discuss the above
> features of BSPMaster.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
