It looks like there's a bug in ArrayListWritable. Try adding a super.clear() at the beginning of readFields(). That should clear the list before it's reused.
On Wed, Aug 14, 2013 at 12:30 AM, Kyle Orlando <[email protected]>wrote: > Tried it, got the same exact results. Do you think it's a problem with > ArrayListWritable, or with sendMessage()? I don't have a great > understanding of how sendMessage() utilizes the Writable interface > exactly... I'm guessing that at some point it uses readFields() and write() > because those obviously need to be implemented for a class to be considered > "Writable". Looking at those two methods for ArrayListWritable, nothing > jumps out at me as being a bug, though I can see where the minimum number > of values in the ArrayList could be set incorrectly in readFields(): > > int numValues = in.readInt(); // read number of values > ensureCapacity(numValues); > > What do you think? > > > On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella < > [email protected]> wrote: > >> could you try using: >> >> -D giraph.oneToAllMsgSending=true when you run it? >> >> >> On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando >> <[email protected]>wrote: >> >>> Yep, and I've pulled recently as well. >>> >>> >>> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella < >>> [email protected]> wrote: >>> >>>> That's strange. This looks like a bug to me. Are you using trunk? >>>> >>>> >>>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando <[email protected] >>>> > wrote: >>>> >>>>> Also, I realize that I just sent an uncommented version... sorry guys, >>>>> but the algorithm consists of just three supersteps: >>>>> >>>>> >>>>> Superstep 0.) Source sends list of source's neighbors to each neighbor >>>>> >>>>> Superstep 1.) Neighbor receives list of source's neighbors from >>>>> source, counts number of mutual neighbors, sends this value (in a >>>>> 1-element list) back to source >>>>> >>>>> Superstep 2.) Source receives messages from neighbors that contain >>>>> number of mutual neighbors. These are summed up and divided by the >>>>> number of possible connections between neighbors to get the local >>>>> clustering coefficient (between 0 and 1). This is set as the source >>>>> vertex's value. >>>>> >>>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando < >>>>> [email protected]> wrote: >>>>> > Alrighty, here's my code: >>>>> > >>>>> > public void compute( >>>>> > Vertex<IntWritable, DoubleWritable, >>>>> NullWritable> vertex, >>>>> > Iterable<IntArrayListWritable> messages) >>>>> throws IOException { >>>>> > >>>>> > if (isSource(vertex)) { >>>>> > if (getSuperstep() == 0) { >>>>> > System.out.println("\nSUPERSTEP >>>>> 0\n------------------------------"); >>>>> > IntArrayListWritable >>>>> sourceNeighbors = new IntArrayListWritable(); >>>>> > >>>>> > vertex.setValue(new >>>>> DoubleWritable(0)); >>>>> > >>>>> > for (Edge<IntWritable, >>>>> NullWritable> edge : vertex.getEdges()) { >>>>> > IntWritable targetVertex = >>>>> new >>>>> > IntWritable(edge.getTargetVertexId().get()); >>>>> > >>>>> sourceNeighbors.add(targetVertex); >>>>> > } >>>>> > >>>>> > for (IntWritable neighbor : >>>>> sourceNeighbors) { >>>>> > sendMessage(neighbor, >>>>> sourceNeighbors); >>>>> > System.out.println("Vertex >>>>> " + vertex.getId() + " sends " + >>>>> > sourceNeighbors + " to " + neighbor); >>>>> > } >>>>> > >>>>> > } else if (getSuperstep() == 2) { >>>>> > int total = vertex.getNumEdges(); >>>>> > int possibleCombos = total * >>>>> (total - 1); >>>>> > int mutualNeighbors = 0; >>>>> > double coefficient = 0; >>>>> > int count = 1; >>>>> > >>>>> > System.out.println("\nSUPERSTEP >>>>> 2\n------------------------------"); >>>>> > >>>>> > for (IntArrayListWritable message >>>>> : messages) { >>>>> > >>>>> System.out.println("Message " + count + " contains: " + message); >>>>> > mutualNeighbors += >>>>> message.get(0).get(); >>>>> > count++; >>>>> > } >>>>> > >>>>> > if (possibleCombos > 0) { >>>>> > coefficient = >>>>> (double)mutualNeighbors / possibleCombos; >>>>> > vertex.setValue(new >>>>> DoubleWritable(coefficient)); >>>>> > } >>>>> > >>>>> > } >>>>> > } else { >>>>> > vertex.setValue(new DoubleWritable(0)); >>>>> > >>>>> > if (getSuperstep() == 1) { >>>>> > int mutualNeighbors = 0; >>>>> > >>>>> > System.out.println("\nSUPERSTEP >>>>> 1\n------------------------------"); >>>>> > IntArrayListWritable >>>>> sourceNeighbors = messages.iterator().next(); >>>>> > System.out.println("Vertex " + >>>>> vertex.getId() + " receives " + >>>>> > sourceNeighbors); >>>>> > >>>>> > for (Edge<IntWritable, >>>>> NullWritable> edge : vertex.getEdges()) { >>>>> > IntWritable targetVertex = >>>>> new >>>>> > IntWritable(edge.getTargetVertexId().get()); >>>>> > if >>>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++; >>>>> > } >>>>> > >>>>> > IntArrayListWritable neighborCount >>>>> = new IntArrayListWritable(); >>>>> > neighborCount.add(new >>>>> IntWritable(mutualNeighbors)); >>>>> > >>>>> > sendMessage(new >>>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount); >>>>> > System.out.println("Vertex " + >>>>> vertex.getId() + " sends " + >>>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf())); >>>>> > >>>>> > } >>>>> > } >>>>> > >>>>> > vertex.voteToHalt(); >>>>> > } >>>>> > >>>>> > >>>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella >>>>> > <[email protected]> wrote: >>>>> >> you'd have to show us the code in the compute method. your problem >>>>> might be >>>>> >> caused by object reusal. >>>>> >> >>>>> >> >>>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando < >>>>> [email protected]> >>>>> >> wrote: >>>>> >>> >>>>> >>> Hello, >>>>> >>> >>>>> >>> I am trying to write code to compute the local clustering >>>>> coefficient >>>>> >>> of a vertex/some vertices, and to do this I send a message that >>>>> >>> contains a list of the source's neighbors to each of its neighbors. >>>>> >>> This is in, of course, an IntArrayListWritable. I check the list >>>>> that >>>>> >>> I am sending before invoking sendMessage(), and it appears to be >>>>> >>> correct. However, when I use message.iterator.next() or something >>>>> >>> similar, the IntArrayListWritable objects seem to repeat or >>>>> something. >>>>> >>> I did some logging, and here was the output: >>>>> >>> >>>>> >>> SUPERSTEP 0 >>>>> >>> ------------------------------ >>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2 >>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3 >>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4 >>>>> >>> >>>>> >>> SUPERSTEP 1 >>>>> >>> ------------------------------ >>>>> >>> Vertex 2 receives [2, 3, 4] >>>>> >>> Vertex 2 sends [1] to Vertex 1 >>>>> >>> >>>>> >>> SUPERSTEP 1 >>>>> >>> ------------------------------ >>>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4] >>>>> >>> Vertex 3 sends [1] to Vertex 1 >>>>> >>> >>>>> >>> SUPERSTEP 1 >>>>> >>> ------------------------------ >>>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4] >>>>> >>> Vertex 4 sends [0] to Vertex 1 >>>>> >>> >>>>> >>> SUPERSTEP 2 >>>>> >>> ------------------------------ >>>>> >>> Message 1 contains: [1] >>>>> >>> Message 2 contains: [1, 1, 1] >>>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0] >>>>> >>> >>>>> >>> What is happening? >>>>> >>> >>>>> >>> -- >>>>> >>> Kyle Orlando >>>>> >>> Computer Engineering Major >>>>> >>> University of Maryland >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Claudio Martella >>>>> >> [email protected] >>>>> > >>>>> > >>>>> > >>>>> > -- >>>>> > Kyle Orlando >>>>> > Computer Engineering Major >>>>> > University of Maryland >>>>> >>>>> >>>>> >>>>> -- >>>>> Kyle Orlando >>>>> Computer Engineering Major >>>>> University of Maryland >>>>> >>>> >>>> >>>> >>>> -- >>>> Claudio Martella >>>> [email protected] >>>> >>> >>> >>> >>> -- >>> Kyle Orlando >>> Computer Engineering Major >>> University of Maryland >>> >> >> >> >> -- >> Claudio Martella >> [email protected] >> > > > > -- > Kyle Orlando > Computer Engineering Major > University of Maryland > -- Claudio Martella [email protected]
