Hi samantp, did you find problem in your setup, it would be great for this thread.
Thanks Peeyush On Tuesday, March 1, 2016 at 2:44:29 AM UTC-8, samantp wrote: > > Certainly Rogan, > > And thanks a ton for spending time to think over it. > > Regards, > --samantp. > > > On Tuesday, 1 March 2016 15:16:40 UTC+5:30, Rogan Dawes wrote: >> >> Ok, it might be interesting to run a few tests, then, scaling the number >> of (simulated) clients from small to large, and see where the performance >> starts to drop off. >> >> Not having done much performance testing myself, I'd nonetheless suggest >> trying to profile your server to see where the bottlenecks appear while >> doing so. >> >> Rogan >> >> >> On Tue, Mar 1, 2016 at 11:33 AM samantp <[email protected]> wrote: >> >>> Hello Rogan, >>> >>> Even tried without the sleep, but similar results. In fact, the sleep >>> was added a bit later in the game, to avoid queing up too many pending >>> requests on the client side channel pool and out of memory errors due to it >>> for very large number of requests. >>> >>> Regards, >>> --Parikshit N. Samant. >>> >>> >>> >>> On Tuesday, 1 March 2016 14:35:09 UTC+5:30, Rogan Dawes wrote: >>> >>>> So, I wonder then, given your description of the client setup, whether >>>> you had periods of "silence" from your client, that would explain why >>>> there >>>> were fewer messages sent/received. >>>> >>>> i.e if the following represents the aggregate message flow over time >>>> for your few clients making continuous requests >>>> >>>> OOOOOOOOOOOOOOOO == 100,000 requests per second >>>> >>>> and the following represents the aggregate message flow over time of >>>> the "many clients making intermittent requests": >>>> >>>> ooOOoo_____ooOOoo___ >>>> >>>> I would hardly be surprised to find that the total message count is >>>> significantly lower. >>>> >>>> I'd be curious to see what happens if you simply remove the sleep from >>>> your clients. >>>> >>>> Rogan >>>> >>>> >>>> On Tue, Mar 1, 2016 at 10:48 AM samantp <[email protected]> wrote: >>>> >>> Hello Rogan, >>>>> >>>>> Please find my responses inlined. >>>>> >>>>> >>>>> On Tuesday, 1 March 2016 12:58:12 UTC+5:30, Rogan Dawes wrote: >>>>>> >>>>>> OK, so now we know what we are looking at :-) >>>>>> >>>>>> A possibly stupid question, how did you simulate your client load? >>>>>> Did you run several instances on a single machine, or several instances >>>>>> over 100 machines, etc? Would hate to be optimising/debugging the >>>>>> server, >>>>>> when the problem is in the client simulation! >>>>>> >>>>> >>>>> I knew this question was coming :), and clearly not a stupid one. No, >>>>> actually, did not run those many clients. Instead had a single threaded >>>>> (or >>>>> at max 10 threads) client sending over a channel pool of size 100, 500, >>>>> 10000 channels and so on. So, this client just acquires a channel, >>>>> writes, >>>>> and the handler, in its read function drains the response sent by the >>>>> server. Everything was async. The aquire of the channel from the pool, >>>>> writing to the channel, and reading of the response in the handler on >>>>> client side. In fact, if we have multiple clients (order of 3-4) each >>>>> with >>>>> a fraction of the number of connections as a single client with a large >>>>> connection pool, the situation does not change much. >>>>> >>>>> >>>>>> Also, were all the clients synchronized/sending their messages at the >>>>>> same time, or spread out? i.e. did you start them all at the same time, >>>>>> and >>>>>> have a method triggering a request at System.currentTimeMillis() % >>>>>> 10000? >>>>>> Or was there a degree of randomness in the sending of the messages from >>>>>> the >>>>>> client? >>>>>> >>>>> >>>>> So, the messages were spread out, but only slightly, since it was a >>>>> while loop on the client side that sent data on the acquired channel. A >>>>> sleep at send was done after sending around n records where n is a >>>>> function >>>>> of the number of channels in the pool. E.g, sleep for a 'millisecond' >>>>> after >>>>> sending 20000 requests or more. >>>>> >>>>> >>>>>> >>>>>> Rogan >>>>>> >>>>>> On Tue, Mar 1, 2016 at 8:48 AM samantp <[email protected]> wrote: >>>>>> >>>>> Thanks much for the reply Rogan. >>>>>>> >>>>>>> Actually, we want both (if that is achievable with asyncIO and netty >>>>>>> that is). The scenario is many clients (e.g 10000) will be sending >>>>>>> requests >>>>>>> with time in between them, say 10 requests per second, and one netty >>>>>>> server >>>>>>> should handle all these clients and give an aggregated throughput of >>>>>>> 100000 >>>>>>> requests per second. >>>>>>> >>>>>>> We tried this scenario, and found that with anything above a few >>>>>>> hundred clients, the aggregated throughput given by netty does not >>>>>>> exceed >>>>>>> 15-20K requests per second. Situation is more or less unchanged for >>>>>>> 1000 or >>>>>>> 10000 concurrent connections. But if the same traffic is sent via one >>>>>>> or >>>>>>> two channels, 100K requests per second is very much achievable. >>>>>>> >>>>>>> Therefore, was assuming that the overheads (user-kernel mode >>>>>>> switches, epoll, io, etc. not sure) of reading for each channel (and >>>>>>> that >>>>>>> too, with a yield of only one or two records each time) does become >>>>>>> prominent compared to request processing time in the worker. Especially >>>>>>> considering that the amount of processing to be done is not heavy (say, >>>>>>> a >>>>>>> simple hardcoded response is to be sent). Then, the server is generally >>>>>>> idle in terms of CPU utilization. >>>>>>> >>>>>>> Regards, >>>>>>> --samantp >>>>>>> >>>>>>> On Monday, 29 February 2016 18:18:01 UTC+5:30, Rogan Dawes wrote: >>>>>>> >>>>>>>> It makes sense to benchmark the scenarios in which you will use the >>>>>>>> app. >>>>>>>> >>>>>>>> If your clients will be making a small number of requests, with >>>>>>>> time in between them, but you are concerned about how scalable the >>>>>>>> solution >>>>>>>> is, implement a client that does the above, and then generate a whole >>>>>>>> lot >>>>>>>> and see how your server scales. >>>>>>>> >>>>>>>> If you will have a small amount of clients that will be sending >>>>>>>> large numbers of packets, and throughput is the critical factor rather >>>>>>>> than >>>>>>>> latency, create a benchmark that measures *that*. >>>>>>>> >>>>>>>> From your email, it sounds like you are testing the latter, but >>>>>>>> deploying the former in reality. >>>>>>>> >>>>>>>> Rogan >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Feb 29, 2016 at 12:55 PM samantp <[email protected]> wrote: >>>>>>>> >>>>>>> Hello, >>>>>>>>> >>>>>>>>> Greetings! I had this question about netty. >>>>>>>>> >>>>>>>>> While doing some experiments, found that if we have a simple netty >>>>>>>>> TCP server that just returns back a fixed response, and a single >>>>>>>>> threaded >>>>>>>>> client sending data over a single channel continuously as soon/long >>>>>>>>> as the >>>>>>>>> channel "isWritable" was true, we could get very high throughputs. >>>>>>>>> 100K >>>>>>>>> requests per second were normal. The read at the client side channel >>>>>>>>> handler was in this case just draining the response data returned. >>>>>>>>> >>>>>>>>> However, when we try to do this by modifying clients to use large >>>>>>>>> channel pools and such that a send over a channel is not done till >>>>>>>>> the >>>>>>>>> response of the previous is not received (by acquiring the channel >>>>>>>>> during >>>>>>>>> send, and releasing it on read), the throughput for the same input >>>>>>>>> load >>>>>>>>> goes down by almost a factor of 10. We tried to cover for the delay >>>>>>>>> in >>>>>>>>> sending successive requests by having very large channel pools. >>>>>>>>> >>>>>>>>> In this scenario, though, we can support a large number of >>>>>>>>> concurrent channels (tried with 20000 on our low end machines) >>>>>>>>> without any >>>>>>>>> problem, the throughput is very low. >>>>>>>>> >>>>>>>>> Is this expected? Especially considering that every channel now >>>>>>>>> has only sparse data, in fact, only one request at a time to be read >>>>>>>>> and >>>>>>>>> serviced by server? >>>>>>>>> >>>>>>>>> Or for such a scenario where a large number of connections send >>>>>>>>> very sparse data and behave in a synchronous style, some different >>>>>>>>> Netty >>>>>>>>> settings would be more beneficial? >>>>>>>>> >>>>>>>>> Tried to search on the web for netty benchmarks and expected >>>>>>>>> numbers, but could not find numbers in the scenario mentioned above. >>>>>>>>> >>>>>>>>> Help on this would be much appreciated. >>>>>>>>> >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> --samantp. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>> Groups "Netty discussions" group. >>>>>>>>> >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>> send an email to [email protected]. >>>>>>>> >>>>>>>> >>>>>>>>> To view this discussion on the web visit >>>>>>>>> https://groups.google.com/d/msgid/netty/b6b2033b-33a7-4e92-a088-a4686c4ec815%40googlegroups.com >>>>>>>>> >>>>>>>>> <https://groups.google.com/d/msgid/netty/b6b2033b-33a7-4e92-a088-a4686c4ec815%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>>> . >>>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>>> >>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Netty discussions" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> >>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/netty/19ff0158-91f5-414c-b859-dfd33e7386f9%40googlegroups.com >>>>>>> >>>>>>> <https://groups.google.com/d/msgid/netty/19ff0158-91f5-414c-b859-dfd33e7386f9%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Netty discussions" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> >>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/netty/d7fd4c19-943c-4c2f-adf0-e38a492c0c93%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/netty/d7fd4c19-943c-4c2f-adf0-e38a492c0c93%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Netty discussions" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/netty/e9a15760-585a-4dcd-a709-bd123782cd5b%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/netty/e9a15760-585a-4dcd-a709-bd123782cd5b%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- You received this message because you are subscribed to the Google Groups "Netty discussions" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/netty/c5f2547c-ac25-421a-94e2-c343a6fe1fff%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
