[
https://issues.apache.org/jira/browse/HAMA-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13456657#comment-13456657
]
Thomas Jungblut edited comment on HAMA-642 at 9/17/12 8:31 AM:
---------------------------------------------------------------
First try of implementing. Adding the updated lib into our source path.
No Apache headers yet.
Should we fork this into a submodule? I think this can be majorly improved over
time and would be a good start in making Hama more scalable.
was (Author: thomas.jungblut):
First try of implementing. Adding the updated lib into our source path.
No Apache headers yet.
> Make GraphRunner disk based
> ---------------------------
>
> Key: HAMA-642
> URL: https://issues.apache.org/jira/browse/HAMA-642
> Project: Hama
> Issue Type: Improvement
> Components: graph
> Reporter: Thomas Jungblut
> Attachments: HAMA-scale_1.patch
>
>
> To improve scalability we can improve the graph runner to be disk based.
> Which basically means:
> - We have just a single Vertex instance that get's refilled.
> - We directly write vertices to disk after partitioning
> - In every superstep we iterate over the vertices on disk, fill the vertex
> instance and call the users compute functions
> Problems:
> - State other than vertex value can't be stored easy
> - How do we deal with random access after messages have arrived?
> So I think we should make the graph runner more hybrid, like using the queues
> we have implemented in the messaging. So the graphrunner can be configured to
> run completely on disk, in cached mode or in in-memory mode.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
