[
https://issues.apache.org/jira/browse/HAMA-569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13279500#comment-13279500
]
praveen sripati commented on HAMA-569:
--------------------------------------
In case, where the processing of a certain event is independent of the earlier
event then it should be theoretically possible (keeping aside the Hama
limitations if any) to scale up/down the # of bsp nodes for processing.
Whenever there is any dependency, there is a need to keep track of # of bsp
nodes at any instant of time to know which bsp node is processing a certain
event.
Couldn't find much literature on using BSP for real time or on scaling BSP
up/down. The closest I could get is Adaptive Parallelism in the
Bulk-Synchronous Parallel Model (quickly glanced through it)
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.53.9686
Also, looks like cloudscale
(http://www.cloudscale.com/index.php/technology/cloudscale-bsp) uses BSP which
is scalable.
> Make Hama scalable as more processing is done
> ---------------------------------------------
>
> Key: HAMA-569
> URL: https://issues.apache.org/jira/browse/HAMA-569
> Project: Hama
> Issue Type: Improvement
> Components: bsp core
> Affects Versions: 0.4.0, 0.5.0
> Reporter: praveen sripati
> Fix For: 0.6.0
>
>
> Currently Hama doesn't scale. Once the job has been submitted, the # of the
> bsp tasks is fixed. So, there are fixed costs associated with the job. The
> JIRA is to evaluate if Hama can be made scalable automatically once the job
> has been submitted and provide a solution for the same. This applies to both
> batch and real time processing.
> For ex., in the case of real time processing the # of bsp tasks once the job
> has been submitted remain the same for 1 or a million inputs per second.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
