One misunderstanding my side. Isn't it true that the messages have to be
buffered as they all have to be collected before they can be processed (by
definition of superstep)? So you cannot really process them as they come?

On Thu, Sep 8, 2011 at 11:33 PM, Avery Ching <ach...@apache.org> wrote:

>  On 9/8/11 2:19 PM, Jake Mannix wrote:
>
>
>
> On Thu, Sep 8, 2011 at 2:01 PM, Avery Ching <ach...@apache.org> wrote:
>
>>  Yes, the messages are delivered to inMessages (map of id -> list of
>> messages).  Then they are transferred to the vertices just before the
>> computation on a worker.
>>
>
>  So just to make sure I understand correctly (I think I do, but reasoning
> about distributed async operations makes my head hurt a bit), I'm going to
> sound dumb and repeat this back to you: they show up in an asynchronous
> manner into BasicRPCCommunications, where they get buffered up into the
> inMessages map (one list for each vertex on the local node), where the
> GraphMapper then loops through and gives them to each Vertex before they do
> their compute() calls?
>
>
> You have got it =).  Although to clarify, all the messages are attached to
> the Vertex in the commService.prepareSuperstep() in GraphMapper.  I'm
> starting to wonder if we should have this conversation on
> giraph-...@incubator.apache.org =).  Never tried that list yet though...
>
>
>
>    On 9/8/11 1:55 PM, Jake Mannix wrote:
>>
>> So in particular, each GraphMapper has a BasicRPCCommunications object
>> (and a CommunicationsInterface proxy object for each of the other nodes),
>> and when other nodes call putMsg(vertexId, message) on their respective
>> proxies, it shows up in the target vertex's BasicRPCCommunions putMsg()
>> call, is that right?
>>
>> On Thu, Sep 8, 2011 at 1:38 PM, Avery Ching <ach...@apache.org> wrote:
>>
>>> If I'm understanding your question correctly, the iterator is created in
>>> GraphMapper, just before compute() is called.  Messages are tranferred to
>>> the vertices prior to the compute portion superstep.
>>>
>>> Avery
>>>
>>>
>>> On 9/8/11 12:59 PM, Jake Mannix wrote:
>>>
>>>> Question about the message passing API:
>>>>
>>>>  BaseVertex#sendMsg(I id, M msg)
>>>>
>>>> sends messages.  And
>>>>
>>>>  BaseVertex#compute(Iterator<M> msgIterator)
>>>>
>>>> deals with them in a big iterator provided by the framework.  My
>>>> question is where the Iterator of messages is created?  If I wanted to say,
>>>> subvert the processing to avoid buffering them up into one big List, and
>>>> instead handle them as they come in, where would I look in the code to see
>>>> where it's buffered up?
>>>>
>>>>  -jake
>>>>
>>>>
>>>>
>>>
>>
>>
>
>


-- 
    Claudio Martella
    claudio.marte...@gmail.com

Reply via email to