[
https://issues.apache.org/jira/browse/HAMA-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13221896#comment-13221896
]
Apurv Verma commented on HAMA-515:
----------------------------------
Hii,
Here is a solution I have come up with.
Basically we maintain two counter values. One for the value of the counter in
the current superstep and the other for the value of the counter at the end of
the last superstep.
{noformat}
@Override
public final void sync() throws IOException, SyncException,
InterruptedException {
enterBarrier();
Iterator<Entry<InetSocketAddress, LinkedList<M>>> it = messenger
.getMessageIterator();
while (it.hasNext()) {
Entry<InetSocketAddress, LinkedList<M>> entry = it.next();
final InetSocketAddress addr = entry.getKey();
final Iterable<M> messages = entry.getValue();
final BSPMessageBundle<M> bundle = combineMessages(messages);
if (conf.getBoolean("bsp.checkpoint.enabled", false)) {
checkpoint(checkpointedPath(), bundle);
}
// remove this message during runtime to save a bit of memory
it.remove();
messenger.transfer(addr, bundle);
}
lock();
data = read(); //read the data from znode.
write(currentCounterValue - lastCounterValue) + data;
unlock();
leaveBarrier();
lastCounterVal = currentCounterVal;
currentCounterVal = read();
incrCounter(PeerCounter.SUPERSTEP_SUM, 1L);
currentTaskStatus.setCounters(counters);
umbilical.statusUpdate(taskId, currentTaskStatus);
// Clear outgoing queues.
messenger.clearOutgoingQueues();
}
{noformat}
Thomas told me that sync() already has a lock so I don't have to worry about
the lock part of it. As far as my understanding about barriers goes it allows
multiple threads (in our case bsp tasks) to enter and only when all have
arrived it allows them to leave.
I still don't see where they acquire the lock. I still have to see that up.
http://pastebin.com/XHHm0q1X
> Counter should be synchronized by sync()
> ----------------------------------------
>
> Key: HAMA-515
> URL: https://issues.apache.org/jira/browse/HAMA-515
> Project: Hama
> Issue Type: New Feature
> Affects Versions: 0.4.0
> Reporter: Thomas Jungblut
> Priority: Minor
> Fix For: 0.5.0
>
>
> We should synchronize the counters in all tasks after a sync() call.
> Then someone can use it for flow control. E.G. graph related algorithms, to
> get the number of globally updated vertices.
> Two options to solve this:
> - Sync the whole stuff over RPC from the BSPMaster
> - Use Zookeeper for counter handling
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
