[
https://issues.apache.org/jira/browse/HAMA-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13455888#comment-13455888
]
Suraj Menon commented on HAMA-643:
----------------------------------
Cool. This is something similar to what I wanted to propose for broadcast API.
Except that instead of having a round-robin implementation, we should have a
tree based communication. Making it round robin would increase the worst case
latency by O(N) where N is the number of tasks in the barrier synchronization.
For tree based, we can hopefully reduce the latency to O(log(N)). Both
implementation would need asynchronous communication with some state maintained
in the process. This increases the priority of Asynchronous communication
implementation. We should target it for 0.7. It should also improve
performance. Pregel paper mentions a bit about their implementation. I shall
detail the broadcastAndSync API in JIRA. Also we need to think about handling
link faults in this.
> Introduce BSP_OBLSYNC
> ---------------------
>
> Key: HAMA-643
> URL: https://issues.apache.org/jira/browse/HAMA-643
> Project: Hama
> Issue Type: New Feature
> Components: bsp core, messaging
> Reporter: Thomas Jungblut
>
> The PUB Library [1] a german BSP lib contains a very interesting feature
> called "oblivious synchronization" or in short "BSP_OBLSYNC".
> The signature of the method looks like this:
> bq.bsp oblsync (bsp, numMsgs)
> {{{
> The oblivious synchronization should be used if the programmer knows the
> number of messages
> each processor will receive in a superstep. E.g., if each processor sends a
> message to its right
> neighbour then every processor knows that it will receive exactly one
> message. Thus, in the oblivious
> synchronization each processor waits until nmsgs are received. This type of
> synchronization
> is much faster than the other one since no additional communication is
> needed. The standard
> synchronization should be used if the number of messages to receive is
> unknown. Supersteps with
> standard synchronization can alternate with oblivious synchronizations, but
> within one superstep
> each processor has to use the same type of synchronization
> }}}
> With such an feature, we can greatly speedup kmeans clustering by avoiding
> barrier synchronization, because the message exchanged are constant
> throughout all supersteps.
> [1] http://www2.cs.uni-paderborn.de/~pub/
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
