[
https://issues.apache.org/jira/browse/HAMA-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688219#comment-13688219
]
MaoYuan Xian commented on HAMA-759:
-----------------------------------
Add my modifications for current phase anyway. Send message bundle by bundle.
> When peer sends message, not matter what kinds of queue used, all messages
> read into memory.
> --------------------------------------------------------------------------------------------
>
> Key: HAMA-759
> URL: https://issues.apache.org/jira/browse/HAMA-759
> Project: Hama
> Issue Type: Improvement
> Components: bsp core
> Affects Versions: 0.6.1
> Reporter: MaoYuan Xian
> Attachments: HAMA-759.patch
>
>
> Before bsp peer sends message to network, all messages have to be read into
> memory, No matter what kinds of outgoing queue used. Which means, even when
> user use DiskQueue to sending message, no memory saving exactly. In sync()
> method of BSPPeerImpl.java，
> {code}
> while (it.hasNext()) {
> ...
> BSPMessageBundle<M> bundle = combineMessages(messages);
> // remove this message during runtime to save a bit of memory
> it.remove();
> if (combiner != null) {
> try {
> messenger.transfer(addr, bundle);
> } catch (Exception e) {
> LOG.error("Error while sending messages", e);
> }
> {code}
> In the combineMessages method, all messages are put into memory:
> {code}
> private final BSPMessageBundle<M> combineMessages(Iterable<M> messages) {
> BSPMessageBundle<M> bundle = new BSPMessageBundle<M>();
> if (combiner != null) {
> bundle.addMessage(combiner.combine(messages));
> } else {
> for (M message : messages) {
> bundle.addMessage(message);
> }
> }
> return bundle;
> }
> {code}
> Can we make some modifications here, implements a configurable size sending
> bundle, only collect parts of message instead of all before sends messages to
> network.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
