[
https://issues.apache.org/jira/browse/HAMA-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021130#comment-13021130
]
Thomas Jungblut commented on HAMA-380:
--------------------------------------
I like that +1 and reviewed the patch.
Do you have a comparision?
I believe that using reflection here without caching can be a bottleneck too.
Using the void constructor (searching it once) and cache it is a lot better.
{noformat}
try {
clazz = (Class<? extends BSPMessage>) Class.forName(className);
} catch (ClassNotFoundException ex) {
throw new IOException(ex);
}
// Deserialize messages.
messages = new ArrayList<BSPMessage>(numMessages);
for (int i = 0; i < numMessages; ++i) {
try {
// Instantiate new message and deserialize it.
BSPMessage newMessage = clazz.newInstance();
{noformat}
Have a look at org.apache.hadoop.util.ReflectionUtils and check out:
newInstance(Class<T> theClass, Configuration conf).
Maybe we should sort the messages by their class, this will reduce traffic with
sending the name of the class for each message.
Otherwise nice work ;)
> Send messages in batches to reduce RPC overhead.
> ------------------------------------------------
>
> Key: HAMA-380
> URL: https://issues.apache.org/jira/browse/HAMA-380
> Project: Hama
> Issue Type: Improvement
> Components: bsp
> Affects Versions: 0.3.0
> Reporter: Miklos Erdelyi
> Fix For: 0.3.0
>
> Attachments: bspmsgbundle.patch, bspmsgbundle.v2.patch
>
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> When profiling graph computations running on HAMA it turned out that huge
> amount of time is spent in RPC calls.
> To reduce this overhead I propose extending the GroomServer's API with a
> put(BSPMessageBundle) method so that a group of messages can be transferred
> from the caller to the callee at once. Also, the sync() operation should be
> improved by sending messages in batches rather than one-by-one.
> Using BSPMessageBundle makes it easier to add support for compression of
> messages later.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
