Stefan Were you ever able to make any progress on this ?
Just curious :) -- Atin Sood Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Wednesday, May 22, 2013 at 7:36 PM, Stefan Krawczyk wrote: > Yes agreed, that would be a good approach to take, however before I go try to > figure out how to write a NettyTransceiver in python, I'd like to get some > community input on so that: > - I understand why there wasn't one built in the first place, and thus > answer is it actually possible to build that in python > - I'm not duplicating effort if there is somebody already working on it > > So any NettyServer & NettyTransceiver experts in the house? > > :) > > Cheers, > > Stefan > > > > On Wed, May 22, 2013 at 4:22 PM, Atin Sood <[email protected] > (mailto:[email protected])> wrote: > > Sure Stefan > > > > I am a newbie to both python and avro and have limited experience in > > networking. > > > > But if I am not wrong the problem is at python client end as I am guessing > > you must be using s.th (http://s.th) like > > > > client code - attach to the server and send a message > > client = ipc.HTTPTransceiver(server_addr[0], server_addr[1]) > > requestor = ipc.Requestor(PROTOCOL, client) > > > > > > So even though you can go ahead and switch to NettyServer in java you will > > be limited to use httpserver because your python client uses http client > > > > https://github.com/phunt/avro-rpc-quickstart/blob/master/src/main/java/example/Main.java > > > > > > I guess the best way to get around this will be to look at source code of > > avro python client ipc.py file > > > > and add a new implementation besides the one that comes out of the box. > > > > class HTTPTransceiver(object): """ A simple HTTP-based transceiver > > implementation. Useful for clients but not for servers """ > > > > > > That said, again I am new to the whole thing so I might be totally wrong :) > > > > -- > > Atin Sood > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > On Wednesday, May 22, 2013 at 11:55 AM, Stefan Krawczyk wrote: > > > > > Hi Atin, > > > > > > Thanks for the response. Yes I understand I could use HTTPServer on the > > > java side and things would work. However I'm after a solution where I can > > > still have the java side use the NettyServer. > > > > > > Cheers, > > > > > > Stefan > > > > > > > > > On Wed, May 22, 2013 at 4:11 AM, Atin Sood <[email protected] > > > (mailto:[email protected])> wrote: > > > > You can try looking into something that I wrote as an example > > > > > > > > https://github.com/atinsood/HESDataAnalyticsFinalProject/tree/master/javaXPython > > > > > > > > > > > > https://github.com/atinsood/HESDataAnalyticsFinalProject#javaxpython > > > > > > > > -- > > > > Atin Sood > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > > > > > > > > > > On Tuesday, May 21, 2013 at 11:18 PM, Stefan Krawczyk wrote: > > > > > > > > > Hi, > > > > > > > > > > I am trying to use Avro RPC and have a python client talk to a java > > > > > server, using the avro-rpc-quickstart > > > > > (https://github.com/phunt/avro-rpc-quickstart) on github as a base (I > > > > > made sure the avro version being pulled in was 1.7.4). However when I > > > > > get my python client to talk to the java server I see this error: > > > > > > > > > > 2013-05-20 19:38:32,512 (pool-5-thread-2) [WARN - > > > > > org.apache.avro.ipc.NettyServer$NettyServerAvroHandler.exceptionCaught(NettyServer.java:201)] > > > > > Unexpected exception from downstream. > > > > > org.apache.avro.AvroRuntimeException: Excessively large list > > > > > allocation request detected: 539959368 items! Connection closed. > > > > > at > > > > > org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decodePackHeader(NettyTransportCodec.java:167) > > > > > at > > > > > org.apache.avro.ipc.NettyTransportCodec$NettyFrameDecoder.decode(NettyTransportCodec.java:139) > > > > > at > > > > > org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:286) > > > > > at > > > > > org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:208) > > > > > at > > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268) > > > > > at > > > > > org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255) > > > > > at > > > > > org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:94) > > > > > at > > > > > org.jboss.netty.channel.socket.nio.AbstractNioWorker.processSelectedKeys(AbstractNioWorker.java:364) > > > > > at > > > > > org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:238) > > > > > at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:38) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > > > > > at > > > > > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > > > > > at java.lang.Thread.run(Thread.java:722) > > > > > > > > > > From digging around on the web I understand this is a > > > > > NettyTransceiver issue, i.e. the python client isn't using it because > > > > > it uses the HTTPTransceiver. > > > > > > > > > > I was wondering, what are my options for moving forward, other than > > > > > getting the java server to use the HTTPTransceiver? > > > > > > > > > > Apologies if I have overlooked something that points out what I can > > > > > do. > > > > > > > > > > Cheers, > > > > > > > > > > Stefan > > > > > > > > > >
