Claudio Martella updated GIRAPH-45:

    Attachment: GIRAPH-45.diff

This a premature patch not meant for inclusion but as RFC. It passes all local 
unit tests and MR except checkpointing and partitioner tests. 
Apparently I broke something with partitioning.

In case of checkpointing it breaks in 
BasicRPCCommunications#checkForMessageToNonExistentVertex(), with messages sent 
to the wrong worker (see IllegalStateException), while in TestGraphPartitioner 
the output partition files are small than required size.

I'm requesting some comments as apparently I don't get how I broke partitioner 
package by moving some code from prepareSuperstep() to putMsg* methods. There 
must be an assumption I don't get which might be obvious to one of you.

I tried to go incrementally by just refactoring 
BasicRPCCommunications#checkForMessageToNonExistentVertex() and leaving the 
rest AS-IS, so no out-of-core classes, just really trunk with 
BasicRPCCommunications#checkForMessageToNonExistentVertex() logics, and the 
code doesn't break. So... any ideas?
> Improve the way to keep outgoing messages
> -----------------------------------------
>                 Key: GIRAPH-45
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-45
>             Project: Giraph
>          Issue Type: Improvement
>          Components: bsp
>            Reporter: Hyunsik Choi
>         Attachments: GIRAPH-45.diff
> As discussed in GIRAPH-12(http://goo.gl/CE32U), I think that there is a 
> potential problem to cause out of memory when the rate of message generation 
> is higher than the rate of message flush (or network bandwidth).
> To overcome this problem, we need more eager strategy for message flushing or 
> some approach to spill messages into disk.
> The below link is Dmitriy's suggestion.
> https://issues.apache.org/jira/browse/GIRAPH-12?focusedCommentId=13116253&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13116253

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
For more information on JIRA, see: http://www.atlassian.com/software/jira


Reply via email to