Jake Mannix commented on GIRAPH-34:

I'll definitely open another JIRA for the Vertex subclasses, and dig into that 
a bit.

But on this current topic, I see how users could possibly do something like 
sendMsg(destVertex, getVertexValue()), yes.  But isn't this analogous to in 
regular Hadoop-land, that you simply cannot expect to hang onto your Writable 
instances and use them later.  If you're in 
Mapper.map(SomethingWritableComparable key, SomethingWritable value, Context 
c), you should *never* just buffer up the key and value instances, as this is 
practically guaranteed to break - Hadoop will be re-using the key and value as 
container objects to read new bytes off of disk for the next invocation to 
map(), so that java objects are rarely created, instead you're just constantly 
doing simple bit/byte operations on the disk stream, and setting values inside 
of Writable containers.  

It seems like one of the basic contracts of Writables (at least in Hadoop-land) 
is that they are always to be considered containers: call get() or 
getSomeKindOfThing() on them as soon as you have a handle on one, and use 
whatever *that* is, assuming that the framework can and will reuse your 
original Writable.

> Failure of Vertex reflection for putVertexList from GIRAPH-27 
> --------------------------------------------------------------
>                 Key: GIRAPH-34
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-34
>             Project: Giraph
>          Issue Type: Bug
>            Reporter: Christian Kunz
>            Assignee: Avery Ching
>         Attachments: GIRAPH-34.patch
> Christian actually found this bug.  I am filing the JIRA on his behalf.  
> Here's my error when running TestVertexRangeBalancer.  
> java.lang.RuntimeException: java.io.IOException: Call to 
> returnwhose-lm/ failed on local exception: 
> java.io.EOFException
>       at 
> org.apache.giraph.comm.BasicRPCCommunications.sendVertexListReq(BasicRPCCommunications.java:768)
>       at 
> org.apache.giraph.graph.BspServiceWorker.exchangeVertexRanges(BspServiceWorker.java:1282)
>       at org.apache.giraph.graph.GraphMapper.map(GraphMapper.java:589)
>       at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>       at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:763)
>       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
>       at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
>       at java.security.AccessController.doPrivileged(Native Method)
>       at javax.security.auth.Subject.doAs(Subject.java:396)
>       at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>       at org.apache.hadoop.mapred.Child.main(Child.java:253)
> Caused by: java.io.IOException: Call to returnwhose-lm/ 
> failed on local exception: java.io.EOFException
>       at org.apache.hadoop.ipc.Client.wrapException(Client.java:1065)
>       at org.apache.hadoop.ipc.Client.call(Client.java:1033)
>       at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:224)
>       at $Proxy3.putVertexList(Unknown Source)
>       at 
> org.apache.giraph.comm.BasicRPCCommunications.sendVertexListReq(BasicRPCCommunications.java:766)
>       ... 10 more
> Caused by: java.io.EOFException
>       at java.io.DataInputStream.readInt(DataInputStream.java:375)
>       at 
> org.apache.hadoop.ipc.Client$Connection.receiveResponse(Client.java:767)
>       at org.apache.hadoop.ipc.Client$Connection.run(Client.java:712)
> I identified and fixed the issue by making BasicVertex implement Configurable 
> and making the graph state set in BasicRPCCommunications.  There is one more 
> error though that I'll try and solve before putting up a reviewboard.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


Reply via email to