Hi Arthur, I see in your client code that you have the "delete" for the call object commented out. This could certainly cause issues with throughput.
In terms of why there is only one connection between the client and server, keep in mind that gRPC attempts subchannel sharing when the endpoints are the same. So if you have several channel objects on the client side and they are going to the same endpoint, they might share the same subchannel, and that's why you're seeing only one connection. If you would like, you can force each channel to have its own connection by passing in different channel arguments for each channel. I believe this is what our official benchmark code does. On Saturday, June 1, 2019 at 10:23:21 AM UTC-7, Arthur Wang wrote: > > After did these optimizations on windows (sorted by the their > contributions to the optimized result): > > 1. build the *released version* from the server itself and all its > dependent libraries. > 2. using different threads and assigning each of the threads with a > dedicated asynchronous client, each client with a distinguished channel. > 3. separating the client and the server from being located on one > single machine. > > And, I've compiled the released versions for all the dependencies and the >> application itself. Results are basically the same . > > > Note: this previously posted conclusion is wrong. > > I got a better result now: > > OS pool 1 thread 2 thread 4 thread 8 thread 16 thread > 1 CQ win 100 42589 45289 43610 42194 37023 > 200 45998 48053 43840 41823 40048 > 400 43610 43497 39984 41084 41928 > mac 100 42283 39416 25693 24497 24113 > 200 42408 39385 25354 24348 24691 > 400 42517 39261 25614 24491 24295 > 2 CQ win 100 42319 42918 49480 46533 41928 > 200 46189 47236 47824 48030 45310 > 400 47664 47778 42480 47058 36941 > mac 100 42265 36456 36523 26695 24838 > 200 39463 36127 35842 24570 24467 > 400 42444 39292 36995 26702 24642 > 4 CQ win 100 40436 43610 41631 45850 40799 > 200 46882 44903 50813 41034 42354 > 400 45766 49504 46189 48590 45330 > mac 100 41684 39619 34578 36805 26143 > 200 42753 36153 36140 36710 24844 > 400 42158 36710 36443 36927 26329 > [image: image.png] > > Some conclusions based on the above: > > 1. the release version is a critical factor for benchmark on windows. > 2. *scaling #CQ / #thread / #pool_size still doesn't help in improving > the throughput.* > 3. on mac, more threads even meaning worse result. > > There is still a big gap between my test and the official benchmark for > C++ > <https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5685265389584384&widget=374364983&container=1597677437>, > > and my cpu usage still being low(50~60% on windows and 100~260% on mac), I > don't know what's next. > > And I thought different channel objects on the client side may stands for > different TCP connections, but my test proved it to be wrong: even I've > created 4~8 channel objects on the client side, there was still *only one > connection between the client and the server*, and I presumed that the > #connections also being an important factor for throughput, although one > single connection can be multiplexed. > > The code files are attached. > > > > On Thu, May 23, 2019 at 12:47 AM Lei Wang <[email protected] <javascript:>> > wrote: > >> And one I thing I want to figure out is that CQ is just for event loop >> processing. More cqs does not promise you more speed. You could run cq in >> following modes: >> >> Sever >> - 1 cq in a main loop thread : many worker threads pool to process tagged >> events each of which releases itself once reading and writing finish. >> - 1 cq in many worker threads because cqs are thread safe : many worker >> threads pool to process tagged events each of which releases itself once >> reading and writing finish. >> - many cq in different loop threads : many worker threads to process >> tagged events events each of which releases itself once reading and writing >> finish. >> >> What ever the model you implement, only the number of worker threads >> affect how much you can process when they arrive. >> >> On Thu, May 23, 2019 at 12:28 AM Lei Wang <[email protected] <javascript:>> >> wrote: >> >>> Are you implementing a server for streaming reply? I have provided you >>> an implementation how to implement 'CallStreamDataReply' otherwise you >>> won't be able to use asynchronous mechanism by grpc c++ server. Also, try >>> to use c++ threads pool instead of grpc threads pool for cq. >>> >>> I have written an end to end test for pubsub services to test how >>> asynchronous server works. There is no serious problems in my tests. >>> >>> >>> On Wed, May 22, 2019 at 2:50 AM Lalit Kumar <[email protected] >>> <javascript:>> wrote: >>> >>>> Just to add I'm trying server streaming using c++ bindings. Client >>>> opens just one long lived RPC and Server starts streaming data to client. >>>> Both running on same Debian machine having 4 cores. Server is implemented >>>> using async api. I'm getting max 23-25K messages/second. Message payload >>>> is >>>> just 4 bytes. Tried multiple threads to service completionQ but its not >>>> changing anything. >>>> >>>> -Lalit >>>> >>>> On Tuesday, May 21, 2019 at 10:38:15 AM UTC-7, Arthur Wang wrote: >>>>> >>>>> And, I've compiled the released versions for all the dependencies and >>>>> the application itself. Results are basically the same. >>>>> >>>>> On Sun, May 19, 2019 at 5:19 PM Arthur Wang <[email protected]> wrote: >>>>> >>>>>> And, both the client and the server on the same machine, >>>>>> communicating with tcp, even though I don't think this could be a reason >>>>>> for poor throughput. >>>>>> >>>>>> On Sun, May 19, 2019 at 5:12 PM Arthur Wang <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> And, of course, the cpu are not fully running on both win(~30+%, >>>>>>> 100% fully load) and mac(200+%,800% fully load ). >>>>>>> >>>>>>> On Sun, May 19, 2019 at 5:08 PM Arthur Wang <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi all: >>>>>>>> >>>>>>>> I've done a benchmark for the asynchronous version of the c++ >>>>>>>> server, but the result is terribly poor compared to either the >>>>>>>> official >>>>>>>> one >>>>>>>> <https://performance-dot-grpc-testing.appspot.com/explore?dashboard=5685265389584384> >>>>>>>> or >>>>>>>> some other results like this >>>>>>>> <https://github.com/grpc/grpc/issues/6504> done by other users, >>>>>>>> I'm wondering what is wrong with my test, could someone do me a favor? >>>>>>>> >>>>>>>> There are the details for my test.: >>>>>>>> >>>>>>>> Enrivonment : >>>>>>>> >>>>>>>> - win 10pro 64bit : intel i7 4.0GHZ * 4 physical core * 2 >>>>>>>> logic core RAM:16GB >>>>>>>> - mac high serria 10.13.6: intel i7 2.2GHZ * 4 physical core * >>>>>>>> 2 logic core RAM:16G >>>>>>>> >>>>>>>> *Note: I don't have any linux machines, so there are no >>>>>>>> results for it.* >>>>>>>> >>>>>>>> Grpc version: >>>>>>>> >>>>>>>> - win : 77ec6c94ad58bedfdf8f28682914e9893e318be0, around 2018.1 >>>>>>>> - mac: e3f37b7f4320f0e016a806796e2adaed03bf23f7, around 2019.2 >>>>>>>> >>>>>>>> Compiler: >>>>>>>> >>>>>>>> - win: vs2015 Microsoft (R) C/C++ Optimizing Compiler Version >>>>>>>> 19.00.24215.1 for x86 >>>>>>>> - mac: Apple LLVM version 10.0.0 (clang-1000.10.44.4) >>>>>>>> >>>>>>>> Parameters: >>>>>>>> >>>>>>>> - number of CQs an server instance has. >>>>>>>> - number of threads totally exist, evenly distributed on each >>>>>>>> CQ. >>>>>>>> - number of pool size(which is the *CallData* instances >>>>>>>> pre-allocated for each CQ. >>>>>>>> >>>>>>>> Client & Server Behavior: >>>>>>>> >>>>>>>> - Client: Sending arond 30-50k requests to the async server in >>>>>>>> parallel, and waiting for all of them to be responded. >>>>>>>> - Server: Based on the *greeter_async_server.cc of the >>>>>>>> helloworld example* - just do some basically ignorable logic >>>>>>>> and response immediately after that. >>>>>>>> >>>>>>>> * The code are in the attachments.* >>>>>>>> >>>>>>>> The result (requests the server can deal per second in average) >>>>>>>> is : >>>>>>>> >>>>>>>> OS pool 1 thread 2 thread 4 thread 8 thread 16 thread >>>>>>>> 1 CQ win 100 3647 4016 3366 3012 3427 >>>>>>>> 200 3963 5202 3308 3411 4507 >>>>>>>> 400 3787 4494 3476 3203 3123 >>>>>>>> mac 100 38880 35803 22629 22128 22321 >>>>>>>> 200 37778 35285 24801 21805 22172 >>>>>>>> 400 39200 36231 23446 22311 22553 >>>>>>>> 2 CQ win 100 3086 4144 4575 3662 3959 >>>>>>>> 200 3334 4354 3588 3507 3536 >>>>>>>> 400 3069 4299 3565 3830 3852 >>>>>>>> mac 100 39154 32362 31948 25227 23441 >>>>>>>> 200 37792 33046 31908 25169 24142 >>>>>>>> 400 40584 33909 32446 25284 24943 >>>>>>>> 4 CQ win 100 3204 4199 4644 3988 3742 >>>>>>>> 200 3125 4097 3954 3997 3638 >>>>>>>> 400 3207 4196 3731 4008 3526 >>>>>>>> mac 100 39793 33101 28851 32310 25214 >>>>>>>> 200 38804 31545 32605 32268 24869 >>>>>>>> 400 38819 32776 31786 32133 25207 >>>>>>>> >>>>>>>> [image: image.png] >>>>>>>> >>>>>>>> Recap of the result: >>>>>>>> >>>>>>>> - number of CQs basically has nothing to do with the >>>>>>>> throughput, *which is unbelievable, in my understanding, it is >>>>>>>> an critical factor after learned how the polling-engine >>>>>>>> >>>>>>>> <https://github.com/grpc/grpc/blob/master/doc/core/epoll-polling-engine.md> >>>>>>>> works.* >>>>>>>> - number of pool size has nothing to do with the throughput,* >>>>>>>> I'm not sure whether this is normal or not.* >>>>>>>> - number of threads which are evenly distributed among the CQs >>>>>>>> *has >>>>>>>> nothing to do with the throughput on windows, but decreasing it on >>>>>>>> mac. >>>>>>>> This is somewhere wired.* >>>>>>>> >>>>>>>> Besides, I tested the synchronous version(*greeter_server.cc*) >>>>>>>> of c++ server, giving me the result of 18807 on mac and 3041 on win, >>>>>>>> which >>>>>>>> is also very poor. >>>>>>>> >>>>>>>> I've read the example of *grpc\test\cpp\qps\server_async.cc *which >>>>>>>> said to be an good example of how to write a high throughput async c++ >>>>>>>> server. I found its approach of enhancing the throughput is more of >>>>>>>> less >>>>>>>> the same with mine: >>>>>>>> >>>>>>>> - scale #CQ. >>>>>>>> - scale #thread. >>>>>>>> >>>>>>>> I haven't test the *qps example* on my machine, I haven't find a >>>>>>>> easier way to build it and hard to believe to it can achieve a better >>>>>>>> result with the same environment of mine. >>>>>>>> >>>>>>>> I know there are something must be wrong, but where are they? >>>>>>>> >>>>>>>> - Thanks a lot. >>>>>>>> - Arthur. >>>>>>>> >>>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "grpc.io" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected] <javascript:>. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/grpc-io/0bb9c0de-e91e-42ce-b99f-9ace2b44d5b7%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/grpc-io/0bb9c0de-e91e-42ce-b99f-9ace2b44d5b7%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/CAF84%3DtevjXUAcuZ-e4dHQO1S-86BxMH%3DNT0Vcy7DPd%3D50%2BZ_Dg%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/grpc-io/CAF84%3DtevjXUAcuZ-e4dHQO1S-86BxMH%3DNT0Vcy7DPd%3D50%2BZ_Dg%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/478df7d6-65c1-47ea-8f84-44e98e172595%40googlegroups.com.
