Thomas Jungblut created HAMA-629:
------------------------------------
Summary: Improve RPC Scalability Part 2
Key: HAMA-629
URL: https://issues.apache.org/jira/browse/HAMA-629
Project: Hama
Issue Type: Sub-task
Components: bsp core, messaging
Affects Versions: 0.5.0
Reporter: Thomas Jungblut
There is a problem when all 1k peers would attempt to send to a single peer
(let's say a master task in a graph algorithm that aggregates). In this case
the peer will start 1k-threads which is using enourmous amount of memory.
I think we can coordinate the message sending either with Zookeeper or by using
the task id and do a smarter sending chain.
By the last, I mean, that each task can start at a different offset in the peer
array to start sending messages to the other peers. But this won't solve the
problem DDoS'ing a single master task.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
