Tried it, got the same exact results. Do you think it's a problem with
ArrayListWritable, or with sendMessage()? I don't have a great
understanding of how sendMessage() utilizes the Writable interface
exactly... I'm guessing that at some point it uses readFields() and write()
because those obviously need to be implemented for a class to be considered
"Writable". Looking at those two methods for ArrayListWritable, nothing
jumps out at me as being a bug, though I can see where the minimum number
of values in the ArrayList could be set incorrectly in readFields():
int numValues = in.readInt(); // read number of values
ensureCapacity(numValues);
What do you think?
On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella <
[email protected]> wrote:
> could you try using:
>
> -D giraph.oneToAllMsgSending=true when you run it?
>
>
> On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando
> <[email protected]>wrote:
>
>> Yep, and I've pulled recently as well.
>>
>>
>> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella <
>> [email protected]> wrote:
>>
>>> That's strange. This looks like a bug to me. Are you using trunk?
>>>
>>>
>>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando
>>> <[email protected]>wrote:
>>>
>>>> Also, I realize that I just sent an uncommented version... sorry guys,
>>>> but the algorithm consists of just three supersteps:
>>>>
>>>>
>>>> Superstep 0.) Source sends list of source's neighbors to each neighbor
>>>>
>>>> Superstep 1.) Neighbor receives list of source's neighbors from
>>>> source, counts number of mutual neighbors, sends this value (in a
>>>> 1-element list) back to source
>>>>
>>>> Superstep 2.) Source receives messages from neighbors that contain
>>>> number of mutual neighbors. These are summed up and divided by the
>>>> number of possible connections between neighbors to get the local
>>>> clustering coefficient (between 0 and 1). This is set as the source
>>>> vertex's value.
>>>>
>>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando <[email protected]>
>>>> wrote:
>>>> > Alrighty, here's my code:
>>>> >
>>>> > public void compute(
>>>> > Vertex<IntWritable, DoubleWritable,
>>>> NullWritable> vertex,
>>>> > Iterable<IntArrayListWritable> messages) throws
>>>> IOException {
>>>> >
>>>> > if (isSource(vertex)) {
>>>> > if (getSuperstep() == 0) {
>>>> > System.out.println("\nSUPERSTEP
>>>> 0\n------------------------------");
>>>> > IntArrayListWritable
>>>> sourceNeighbors = new IntArrayListWritable();
>>>> >
>>>> > vertex.setValue(new
>>>> DoubleWritable(0));
>>>> >
>>>> > for (Edge<IntWritable,
>>>> NullWritable> edge : vertex.getEdges()) {
>>>> > IntWritable targetVertex =
>>>> new
>>>> > IntWritable(edge.getTargetVertexId().get());
>>>> >
>>>> sourceNeighbors.add(targetVertex);
>>>> > }
>>>> >
>>>> > for (IntWritable neighbor :
>>>> sourceNeighbors) {
>>>> > sendMessage(neighbor,
>>>> sourceNeighbors);
>>>> > System.out.println("Vertex
>>>> " + vertex.getId() + " sends " +
>>>> > sourceNeighbors + " to " + neighbor);
>>>> > }
>>>> >
>>>> > } else if (getSuperstep() == 2) {
>>>> > int total = vertex.getNumEdges();
>>>> > int possibleCombos = total * (total
>>>> - 1);
>>>> > int mutualNeighbors = 0;
>>>> > double coefficient = 0;
>>>> > int count = 1;
>>>> >
>>>> > System.out.println("\nSUPERSTEP
>>>> 2\n------------------------------");
>>>> >
>>>> > for (IntArrayListWritable message :
>>>> messages) {
>>>> > System.out.println("Message
>>>> " + count + " contains: " + message);
>>>> > mutualNeighbors +=
>>>> message.get(0).get();
>>>> > count++;
>>>> > }
>>>> >
>>>> > if (possibleCombos > 0) {
>>>> > coefficient =
>>>> (double)mutualNeighbors / possibleCombos;
>>>> > vertex.setValue(new
>>>> DoubleWritable(coefficient));
>>>> > }
>>>> >
>>>> > }
>>>> > } else {
>>>> > vertex.setValue(new DoubleWritable(0));
>>>> >
>>>> > if (getSuperstep() == 1) {
>>>> > int mutualNeighbors = 0;
>>>> >
>>>> > System.out.println("\nSUPERSTEP
>>>> 1\n------------------------------");
>>>> > IntArrayListWritable
>>>> sourceNeighbors = messages.iterator().next();
>>>> > System.out.println("Vertex " +
>>>> vertex.getId() + " receives " +
>>>> > sourceNeighbors);
>>>> >
>>>> > for (Edge<IntWritable,
>>>> NullWritable> edge : vertex.getEdges()) {
>>>> > IntWritable targetVertex =
>>>> new
>>>> > IntWritable(edge.getTargetVertexId().get());
>>>> > if
>>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++;
>>>> > }
>>>> >
>>>> > IntArrayListWritable neighborCount
>>>> = new IntArrayListWritable();
>>>> > neighborCount.add(new
>>>> IntWritable(mutualNeighbors));
>>>> >
>>>> > sendMessage(new
>>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount);
>>>> > System.out.println("Vertex " +
>>>> vertex.getId() + " sends " +
>>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf()));
>>>> >
>>>> > }
>>>> > }
>>>> >
>>>> > vertex.voteToHalt();
>>>> > }
>>>> >
>>>> >
>>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella
>>>> > <[email protected]> wrote:
>>>> >> you'd have to show us the code in the compute method. your problem
>>>> might be
>>>> >> caused by object reusal.
>>>> >>
>>>> >>
>>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando <
>>>> [email protected]>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hello,
>>>> >>>
>>>> >>> I am trying to write code to compute the local clustering
>>>> coefficient
>>>> >>> of a vertex/some vertices, and to do this I send a message that
>>>> >>> contains a list of the source's neighbors to each of its neighbors.
>>>> >>> This is in, of course, an IntArrayListWritable. I check the list
>>>> that
>>>> >>> I am sending before invoking sendMessage(), and it appears to be
>>>> >>> correct. However, when I use message.iterator.next() or something
>>>> >>> similar, the IntArrayListWritable objects seem to repeat or
>>>> something.
>>>> >>> I did some logging, and here was the output:
>>>> >>>
>>>> >>> SUPERSTEP 0
>>>> >>> ------------------------------
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 2 receives [2, 3, 4]
>>>> >>> Vertex 2 sends [1] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4]
>>>> >>> Vertex 3 sends [1] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4]
>>>> >>> Vertex 4 sends [0] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 2
>>>> >>> ------------------------------
>>>> >>> Message 1 contains: [1]
>>>> >>> Message 2 contains: [1, 1, 1]
>>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0]
>>>> >>>
>>>> >>> What is happening?
>>>> >>>
>>>> >>> --
>>>> >>> Kyle Orlando
>>>> >>> Computer Engineering Major
>>>> >>> University of Maryland
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >> Claudio Martella
>>>> >> [email protected]
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Kyle Orlando
>>>> > Computer Engineering Major
>>>> > University of Maryland
>>>>
>>>>
>>>>
>>>> --
>>>> Kyle Orlando
>>>> Computer Engineering Major
>>>> University of Maryland
>>>>
>>>
>>>
>>>
>>> --
>>> Claudio Martella
>>> [email protected]
>>>
>>
>>
>>
>> --
>> Kyle Orlando
>> Computer Engineering Major
>> University of Maryland
>>
>
>
>
> --
> Claudio Martella
> [email protected]
>
--
Kyle Orlando
Computer Engineering Major
University of Maryland
