[
https://issues.apache.org/jira/browse/HAMA-629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13925652#comment-13925652
]
Edward J. Yoon commented on HAMA-629:
-------------------------------------
Thanks for your links.
Currently every vertex sends aggregate value to master task. And, the machine
consists of multiple tasks, and each task contains multiple vertices. So, if we
aggregate first at machine level (local) and merge finally at master task, I
think we can easily implement it and improve the performance.
> Improve RPC Scalability Part 2
> ------------------------------
>
> Key: HAMA-629
> URL: https://issues.apache.org/jira/browse/HAMA-629
> Project: Hama
> Issue Type: Sub-task
> Components: graph
> Affects Versions: 0.5.0
> Reporter: Thomas Jungblut
> Fix For: 0.7.0
>
>
> There is a problem when all 1k peers would attempt to send to a single peer
> (let's say a master task in a graph algorithm that aggregates). In this case
> the peer will start 1k-threads which is using enourmous amount of memory.
> I think we can coordinate the message sending either with Zookeeper or by
> using the task id and do a smarter sending chain.
> By the last, I mean, that each task can start at a different offset in the
> peer array to start sending messages to the other peers. But this won't solve
> the problem DDoS'ing a single master task.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
