Eric, There is a subtle difference between how the ipc works in the http and socket cases and it boils down to the part the connection plays in protocol negotiation and the fact that the connection is being reused in the case of the SocketTranciever and not with the HttpTranciever.
Ideally speaking, once the Requestor and the Responder have negotiated a protocol between them (the handshake has been completed and a protocol has been agreed), then so long as the connection is persisted it is redundant to keep sending the handshake data. In the java versions of Requestor and Responder the negotiated protocol is stored along with the connection. The connection is considered 'established' and so long as it persists, neither side expects the handshake to appear in any subsequent call between them. This is all straightforward enough and works perfectly for the Socket case where the connection is long lived and maintained across invocations. In the HTTP case, by the nature of Http, the connection exists only for a single request and response. There may be http keepalive of the underlying socket, but conceptually the http connection is closed after the server responds. Because there is no persistent connection to associate the negotiated protocol with, a handshake ends up being sent with every request, though admittedly only hashes are transferred in the subsequent invocations. >From what I can tell of the Python requestor, its sending the handshake every time and while this will work with the HttpServer for the reason above, it will fail for the SocketServer. In this case, on the second and subequent invocations, the SocketServer thinks that the protocol negotiation is complete and is expecting to read the specific payload data but is instead encountering the retransmitted handshake header. The error it displays is what you're seeing is what happens when the server expects to read one thing (like a handshake or connection metadata) but encounters something else. In order for your python SocketTranciever to work with the Java version, it will similarly need to skip sending the handshake once the connection is established. Hope this helps. Regards, Stephen. On Thu, Nov 18, 2010 at 5:10 PM, Eric Evans <[email protected]> wrote: > > I put together a SocketTransceiver for Python so that I could compare > performance against the HTTP one, but I'm having some trouble getting it > to work. I'm suspect it's something really dumb, that I'm just too > close to spot it, and that someone else might. > > The patch against 1.4 is attached. It causes the Java server to produce > the following exception: > > java.io.EOFException > at > org.apache.avro.ipc.ByteBufferInputStream.getBuffer(ByteBufferInputStream.java:84) > at > org.apache.avro.ipc.ByteBufferInputStream.read(ByteBufferInputStream.java:46) > at org.apache.avro.io.BinaryDecoder > $InputStreamByteSource.readRaw(BinaryDecoder.java:815) > at org.apache.avro.io.BinaryDecoder.doReadBytes(BinaryDecoder.java:340) > at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:265) > at > org.apache.avro.io.ValidatingDecoder.readString(ValidatingDecoder.java:99) > at > org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:318) > at > org.apache.avro.generic.GenericDatumReader.readMap(GenericDatumReader.java:229) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:117) > at > org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:105) > at org.apache.avro.ipc.Responder.respond(Responder.java:112) > at org.apache.avro.ipc.SocketServer > $Connection.run(SocketServer.java:91) > at java.lang.Thread.run(Thread.java:636) > > Anyone? > > -- > Eric Evans > [email protected] >
