[
https://issues.apache.org/jira/browse/HAMA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13284432#comment-13284432
]
praveen sripati commented on HAMA-569:
--------------------------------------
Having a single master sending data to multiple bsp nodes will not scale. The
single master will be the bottle neck.
Two approaches I can think of
- Have multiple masters. The client gets the list of masters and sends the
messages in a round-robin fashion.
- As in the hdfs, the client can get the list of bsp nodes and send the
messages directly to the bsp nodes avoiding the master.
In either case, the client should be notified of any changes.
Any thoughts.
> Make Hama scalable as more processing is done
> ---------------------------------------------
>
> Key: HAMA-569
> URL: https://issues.apache.org/jira/browse/HAMA-569
> Project: Hama
> Issue Type: Improvement
> Components: bsp core
> Affects Versions: 0.4.0, 0.5.0
> Reporter: praveen sripati
> Fix For: 0.6.0
>
>
> Currently Hama doesn't scale. Once the job has been submitted, the # of the
> bsp tasks is fixed. So, there are fixed costs associated with the job. The
> JIRA is to evaluate if Hama can be made scalable automatically once the job
> has been submitted and provide a solution for the same. This applies to both
> batch and real time processing.
> For ex., in the case of real time processing the # of bsp tasks once the job
> has been submitted remain the same for 1 or a million inputs per second.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
