Thanks Philip, I plan to try the http approach as well for clients that can't use Netty
..sent from my phone On Jan 30, 2013 7:05 PM, "Philip Zeyliger" <[email protected]> wrote: > Matt, > > I misread your question. The multiple-thread approach is with the > Jetty/HTTP mechanism, not the Netty one. Sorry 'bout the noise. > > > On Wed, Jan 30, 2013 at 7:04 PM, Philip Zeyliger <[email protected]>wrote: > >> Hi Matt, >> >> I do this with some frequency. The trick is to use Jetty manually to >> create an appropriate jetty.Server, and then start it. The snippets below >> should help. >> >> -- Philip >> >> import javax.servlet.Servlet; >> >> import org.apache.avro.ipc.ResponderServlet; >> import org.apache.avro.ipc.specific.SpecificResponder; >> import org.mortbay.jetty.Server; >> import org.mortbay.jetty.servlet.Context; >> import org.mortbay.jetty.servlet.ServletHolder; >> import org.mortbay.thread.QueuedThreadPool; >> >> public static Server createAvroServer(int port, String name, int >> maxThreads, >> int maxIdleTimeMs, >> SpecificResponder responder) >> throws EnterpriseServiceException >> { >> try { >> Server httpServer = new Server(port); >> QueuedThreadPool qtp = new QueuedThreadPool(); >> // QueuedThreadPool is jetty's thread pool implementation; >> // this lets us give it a name. >> qtp.setName(name); >> qtp.setDaemon(true); >> qtp.setMaxThreads(maxThreads); >> qtp.setMaxIdleTimeMs(maxIdleTimeMs); >> httpServer.setThreadPool(qtp); >> Servlet servlet = new ResponderServlet(responder); >> new Context(httpServer, "/").addServlet(new ServletHolder(servlet), >> "/*"); >> return httpServer; >> } catch (Exception e) { >> throw new EnterpriseServiceException(e); >> } >> } >> >> // main: >> SpecificResponder responder = new SpecificResponder( >> XXX.class, >> new XXXImpl()); >> server = createAvroServer(listenPort, "somename", >> maxThreads, maxIdleTimeMs, responder); >> server.start(); >> >> >> On Wed, Jan 30, 2013 at 6:38 PM, Matt Corgan <[email protected]> wrote: >> >>> hmm - i haven't heard any suggestions in the past few days... maybe a >>> really short version of the question: Can an Avro-rpc Netty server process >>> requests in parallel? Will it still process in parallel if they're from >>> the same client? >>> >>> Thanks much, >>> Matt >>> >>> >>> On Sun, Jan 27, 2013 at 4:16 PM, Matt Corgan <[email protected]>wrote: >>> >>>> Hi, >>>> I'm trying to familiarize myself with avro-rpc by running and tweaking >>>> the example at https://github.com/jbaldassari/Avro-RPC. By the way, >>>> thanks for creating this example James. >>>> >>>> I'm using the BidderTest.sendBidRequestWithCallback method and am >>>> trying to get the server to process the requests in parallel, which is >>>> something i'll need in my application. I use the DelayInjectingBidder with >>>> delay of 90ms so i can watch what's happening on the server. By debugging >>>> the client, I see that all requests do in fact make it to the server >>>> without blocking, but watching the server log i see that each request is >>>> processed sequentially. So it takes ~9s to process 100 messages. >>>> >>>> Digging into the NettyServer constructor, I see the default >>>> ChannelFactory uses Executors.newCachedThreadPool() for the Boss and Worker >>>> executors. I tried overriding the worker with >>>> Executors.newFixedThreadPool(8) but didn't notice any difference. Looking >>>> at the threads running in Eclipse Debug mode, i see the following threads >>>> which confirms that there is only one worker: >>>> >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner at >>>> localhost:50255 >>>> Thread [main] (Running) >>>> Thread [ReaderThread] (Running) >>>> Thread [New I/O server boss #2 ([id: 0x6a5dd151, >>>> /0:0:0:0:0:0:0:0:45633])] (Running) >>>> Thread [Avro NettyTransceiver Boss 1] (Running) >>>> Thread [New I/O server worker #2-1] (Running) >>>> Thread [New I/O client worker #1-1] (Running) >>>> >>>> I also see that the NettyTransceiver has configurable thread pools, but >>>> I'm not clear on what the Transceiver's role is (client side thing?), and >>>> fiddling with its ChannelFactory is causing exceptions. >>>> >>>> Is is possible that the behavior of the BidderTest client doesn't >>>> trigger the multi-threaded server even though it's configured correctly, or >>>> that the server processes all requests from a single client sequentially? >>>> >>>> Sidenote: i also tried using avro version 1.7.3 with the same results, >>>> although the above thread pools had different names. >>>> >>>> I wonder if someone could shed some light on what the difference is >>>> between the Server and Transceiver thread pools and how to configure them >>>> for multi-threaded server processing. >>>> >>>> Thanks, >>>> Matt >>>> >>>> >>> >> >
