Ah okay, I will try to do this tonight/tomorrow, along with the other issue I should have opened/patched.
On Tue, Aug 13, 2013 at 8:08 PM, Claudio Martella < [email protected]> wrote: > It looks like there's a bug in ArrayListWritable. Try adding a > super.clear() at the beginning of readFields(). That should clear the list > before it's reused. > > > On Wed, Aug 14, 2013 at 12:30 AM, Kyle Orlando > <[email protected]>wrote: > >> Tried it, got the same exact results. Do you think it's a problem with >> ArrayListWritable, or with sendMessage()? I don't have a great >> understanding of how sendMessage() utilizes the Writable interface >> exactly... I'm guessing that at some point it uses readFields() and write() >> because those obviously need to be implemented for a class to be considered >> "Writable". Looking at those two methods for ArrayListWritable, nothing >> jumps out at me as being a bug, though I can see where the minimum number >> of values in the ArrayList could be set incorrectly in readFields(): >> >> int numValues = in.readInt(); // read number of values >> ensureCapacity(numValues); >> >> What do you think? >> >> >> On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella < >> [email protected]> wrote: >> >>> could you try using: >>> >>> -D giraph.oneToAllMsgSending=true when you run it? >>> >>> >>> On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando <[email protected] >>> > wrote: >>> >>>> Yep, and I've pulled recently as well. >>>> >>>> >>>> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella < >>>> [email protected]> wrote: >>>> >>>>> That's strange. This looks like a bug to me. Are you using trunk? >>>>> >>>>> >>>>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando < >>>>> [email protected]> wrote: >>>>> >>>>>> Also, I realize that I just sent an uncommented version... sorry guys, >>>>>> but the algorithm consists of just three supersteps: >>>>>> >>>>>> >>>>>> Superstep 0.) Source sends list of source's neighbors to each neighbor >>>>>> >>>>>> Superstep 1.) Neighbor receives list of source's neighbors from >>>>>> source, counts number of mutual neighbors, sends this value (in a >>>>>> 1-element list) back to source >>>>>> >>>>>> Superstep 2.) Source receives messages from neighbors that contain >>>>>> number of mutual neighbors. These are summed up and divided by the >>>>>> number of possible connections between neighbors to get the local >>>>>> clustering coefficient (between 0 and 1). This is set as the source >>>>>> vertex's value. >>>>>> >>>>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando < >>>>>> [email protected]> wrote: >>>>>> > Alrighty, here's my code: >>>>>> > >>>>>> > public void compute( >>>>>> > Vertex<IntWritable, DoubleWritable, >>>>>> NullWritable> vertex, >>>>>> > Iterable<IntArrayListWritable> messages) >>>>>> throws IOException { >>>>>> > >>>>>> > if (isSource(vertex)) { >>>>>> > if (getSuperstep() == 0) { >>>>>> > System.out.println("\nSUPERSTEP >>>>>> 0\n------------------------------"); >>>>>> > IntArrayListWritable >>>>>> sourceNeighbors = new IntArrayListWritable(); >>>>>> > >>>>>> > vertex.setValue(new >>>>>> DoubleWritable(0)); >>>>>> > >>>>>> > for (Edge<IntWritable, >>>>>> NullWritable> edge : vertex.getEdges()) { >>>>>> > IntWritable targetVertex >>>>>> = new >>>>>> > IntWritable(edge.getTargetVertexId().get()); >>>>>> > >>>>>> sourceNeighbors.add(targetVertex); >>>>>> > } >>>>>> > >>>>>> > for (IntWritable neighbor : >>>>>> sourceNeighbors) { >>>>>> > sendMessage(neighbor, >>>>>> sourceNeighbors); >>>>>> > >>>>>> System.out.println("Vertex " + vertex.getId() + " sends " + >>>>>> > sourceNeighbors + " to " + neighbor); >>>>>> > } >>>>>> > >>>>>> > } else if (getSuperstep() == 2) { >>>>>> > int total = vertex.getNumEdges(); >>>>>> > int possibleCombos = total * >>>>>> (total - 1); >>>>>> > int mutualNeighbors = 0; >>>>>> > double coefficient = 0; >>>>>> > int count = 1; >>>>>> > >>>>>> > System.out.println("\nSUPERSTEP >>>>>> 2\n------------------------------"); >>>>>> > >>>>>> > for (IntArrayListWritable message >>>>>> : messages) { >>>>>> > >>>>>> System.out.println("Message " + count + " contains: " + message); >>>>>> > mutualNeighbors += >>>>>> message.get(0).get(); >>>>>> > count++; >>>>>> > } >>>>>> > >>>>>> > if (possibleCombos > 0) { >>>>>> > coefficient = >>>>>> (double)mutualNeighbors / possibleCombos; >>>>>> > vertex.setValue(new >>>>>> DoubleWritable(coefficient)); >>>>>> > } >>>>>> > >>>>>> > } >>>>>> > } else { >>>>>> > vertex.setValue(new DoubleWritable(0)); >>>>>> > >>>>>> > if (getSuperstep() == 1) { >>>>>> > int mutualNeighbors = 0; >>>>>> > >>>>>> > System.out.println("\nSUPERSTEP >>>>>> 1\n------------------------------"); >>>>>> > IntArrayListWritable >>>>>> sourceNeighbors = messages.iterator().next(); >>>>>> > System.out.println("Vertex " + >>>>>> vertex.getId() + " receives " + >>>>>> > sourceNeighbors); >>>>>> > >>>>>> > for (Edge<IntWritable, >>>>>> NullWritable> edge : vertex.getEdges()) { >>>>>> > IntWritable targetVertex >>>>>> = new >>>>>> > IntWritable(edge.getTargetVertexId().get()); >>>>>> > if >>>>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++; >>>>>> > } >>>>>> > >>>>>> > IntArrayListWritable >>>>>> neighborCount = new IntArrayListWritable(); >>>>>> > neighborCount.add(new >>>>>> IntWritable(mutualNeighbors)); >>>>>> > >>>>>> > sendMessage(new >>>>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount); >>>>>> > System.out.println("Vertex " + >>>>>> vertex.getId() + " sends " + >>>>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf())); >>>>>> > >>>>>> > } >>>>>> > } >>>>>> > >>>>>> > vertex.voteToHalt(); >>>>>> > } >>>>>> > >>>>>> > >>>>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella >>>>>> > <[email protected]> wrote: >>>>>> >> you'd have to show us the code in the compute method. your problem >>>>>> might be >>>>>> >> caused by object reusal. >>>>>> >> >>>>>> >> >>>>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando < >>>>>> [email protected]> >>>>>> >> wrote: >>>>>> >>> >>>>>> >>> Hello, >>>>>> >>> >>>>>> >>> I am trying to write code to compute the local clustering >>>>>> coefficient >>>>>> >>> of a vertex/some vertices, and to do this I send a message that >>>>>> >>> contains a list of the source's neighbors to each of its >>>>>> neighbors. >>>>>> >>> This is in, of course, an IntArrayListWritable. I check the list >>>>>> that >>>>>> >>> I am sending before invoking sendMessage(), and it appears to be >>>>>> >>> correct. However, when I use message.iterator.next() or something >>>>>> >>> similar, the IntArrayListWritable objects seem to repeat or >>>>>> something. >>>>>> >>> I did some logging, and here was the output: >>>>>> >>> >>>>>> >>> SUPERSTEP 0 >>>>>> >>> ------------------------------ >>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2 >>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3 >>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4 >>>>>> >>> >>>>>> >>> SUPERSTEP 1 >>>>>> >>> ------------------------------ >>>>>> >>> Vertex 2 receives [2, 3, 4] >>>>>> >>> Vertex 2 sends [1] to Vertex 1 >>>>>> >>> >>>>>> >>> SUPERSTEP 1 >>>>>> >>> ------------------------------ >>>>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4] >>>>>> >>> Vertex 3 sends [1] to Vertex 1 >>>>>> >>> >>>>>> >>> SUPERSTEP 1 >>>>>> >>> ------------------------------ >>>>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4] >>>>>> >>> Vertex 4 sends [0] to Vertex 1 >>>>>> >>> >>>>>> >>> SUPERSTEP 2 >>>>>> >>> ------------------------------ >>>>>> >>> Message 1 contains: [1] >>>>>> >>> Message 2 contains: [1, 1, 1] >>>>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0] >>>>>> >>> >>>>>> >>> What is happening? >>>>>> >>> >>>>>> >>> -- >>>>>> >>> Kyle Orlando >>>>>> >>> Computer Engineering Major >>>>>> >>> University of Maryland >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> >>>>>> >> -- >>>>>> >> Claudio Martella >>>>>> >> [email protected] >>>>>> > >>>>>> > >>>>>> > >>>>>> > -- >>>>>> > Kyle Orlando >>>>>> > Computer Engineering Major >>>>>> > University of Maryland >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Kyle Orlando >>>>>> Computer Engineering Major >>>>>> University of Maryland >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Claudio Martella >>>>> [email protected] >>>>> >>>> >>>> >>>> >>>> -- >>>> Kyle Orlando >>>> Computer Engineering Major >>>> University of Maryland >>>> >>> >>> >>> >>> -- >>> Claudio Martella >>> [email protected] >>> >> >> >> >> -- >> Kyle Orlando >> Computer Engineering Major >> University of Maryland >> > > > > -- > Claudio Martella > [email protected] > -- Kyle Orlando Computer Engineering Major University of Maryland
