Hi Louis, Thanks, I have made the changes and I see good numbers with async client. I have committed the change in my repo.
But I still see very low data transfer rate. What I have observed is that when I have big data say ( 1024 bytes ) Then I get rate on the same machine around 900-1000 MegaBytes/seconds. Thanks Avinash Num Of Rows -> 2000000 Total Data Size -> 512000000 Time -> 7.484403756 Seconds Data Rate -> 65.20225470316008 MBps Time -> 5.776407626 Seconds Data Rate -> 84.48157256137519 MBps Time -> 6.010102214 Seconds Data Rate -> 81.19662239075524 MBps Time -> 5.401799312 Seconds Data Rate -> 90.34026845757057 MBps Time -> 5.352594307 Seconds Data Rate -> 91.17074300994655 MBps On Tuesday, October 4, 2016 at 4:48:50 AM UTC+5:30, Louis Ryan wrote: > > I see you're using a blocking stub to make the streaming calls. I suggest > you switch to using an async callback instead here > > > https://github.com/davinash/grpc-bench/blob/master/bench/src/main/java/io/adongre/grpc/formatted/ScanFormattedClient.java#L35 > > E.g. > > asyncStub.formatedScan(ScanRequest.newBuilder()..., new StreamObserver() > { ... public void onValue( ScanFormattedResponse response) { .... do > stuff with response. i.e. replace the while loop in your code } }); > .setNumOfColumns(NUMBER_OF_COLUMNS) > .setNumOfRows(NUM_OF_ROWS) > .setSizeOfEachColumn(SIZE_OF_EACH_COLUMN) > .build()); > > On Sun, Oct 2, 2016 at 3:16 AM, Avinash Dongre <[email protected] > <javascript:>> wrote: > >> Thanks Louis, For helping me here >> >> Sorry for delayed response >> >> I am sure I am doing something wrong in my code, but could not figure out >> yet. >> >> I have checked-in my benchmark project here at >> https://github.com/davinash/grpc-bench >> >> Raw Server , client and impl can be found at >> https://github.com/davinash/grpc-bench/tree/master/bench/src/main/java/io/adongre/grpc/raw >> >> Formatted Server, client and Impl can be found at >> https://github.com/davinash/grpc-bench/tree/master/bench/src/main/java/io/adongre/grpc/formatted >> >> With this above code for Raw I get following numbers >> >> Num Of Rows -> 2000000 >> Time -> 68.468212235 Seconds >> Total Data Size -> 65536000000 >> Data Rate -> 912.8323635132227 MBps >> >> >> >> And with formatted one I get following numbers. >> >> Num Of Rows -> 2000000 >> Time -> 358.549532039 Seconds >> Total Data Size -> 81952000000 >> Data Rate -> 217.97546228982097 MBps >> >> >> Thanks >> Avinash >> >> >> >> >> >> On Monday, September 26, 2016 at 10:08:47 PM UTC+5:30, Louis Ryan wrote: >>> >>> I see that you're using a steaming RPC. Can you send the code from the >>> server and client. How you interact with flow control can greatly affect >>> performance >>> >>> -louis (from phone) >>> >>> On Sep 26, 2016 3:57 AM, "Avinash Dongre" <[email protected]> wrote: >>> >>> Hi All, >>> Please help. >>> >>> Thanks >>> Avinash >>> >>> >>> On Saturday, September 24, 2016 at 12:01:02 PM UTC+5:30, Avinash Dongre >>> wrote: >>>> >>>> >>> Now I get around 130-135 MegaBytes/Seconds Speed. >>>> >>>> This result is on the Same Machine. i.e. gRPC Client and gRPC Servers are >>>> running on the same machine. >>>> >>>> >>>> >>>> On Saturday, September 24, 2016 at 11:59:53 AM UTC+5:30, Avinash Dongre >>>> wrote: >>>>> >>>>> Hi All, >>>>> >>>>> Thanks all for replying/resolving my previous doubts. I am planning to >>>>> use gRPC for our project. I have following question related to >>>>> performance. >>>>> >>>>> If I have following proto definition. >>>>> >>>>> option optimize_for = SPEED; >>>>> message ScanRow { >>>>> repeated bytes row = 1; >>>>> } >>>>> message ScanResult { >>>>> repeated ScanRow row = 1; >>>>> } >>>>> message ScanRequest { >>>>> int32 numOfColumns = 1; >>>>> int32 sizeOfEachColumn = 2; >>>>> int64 numOfRows = 3; >>>>> int32 batchSize = 4; >>>>> } >>>>> service ScanService { >>>>> rpc Scan (ScanRequest) returns (stream ScanResult) {} >>>>> } >>>>> >>>>> I get around 900-1000 MegaBytes/Seconds on Single Machine and across >>>>> two different Physical Machines I am getting 100-110 MegaBytes/Seconds >>>>> speed. >>>>> >>>>> If I change above proto definition to following. >>>>> >>>>> option optimize_for = SPEED; >>>>> message ColumnValues { >>>>> bytes columnName = 1; >>>>> bytes columnValue = 2; >>>>> } >>>>> message ScanRow { >>>>> int64 rowId = 1; >>>>> int64 timeStamp = 2; >>>>> repeated ColumnValues columnValue = 3; >>>>> } >>>>> message ScanResult { >>>>> repeated ScanRow row = 1; >>>>> } >>>>> message ScanRequest { >>>>> int32 numOfColumns = 1; >>>>> int32 sizeOfEachColumn = 2; >>>>> int64 numOfRows = 3; >>>>> int32 batchSize = 4; >>>>> } >>>>> service ScanService { >>>>> rpc Scan (ScanRequest) returns (stream ScanResult) {} >>>>> } >>>>> >>>>> Now I get around 130-135 MegaBytes/Seconds Speed. >>>>> >>>>> >>>>> Why it is slow with kind of multi-level proto files. Is there any >>>>> Serialization/De-Serialization overhead with this ? >>>>> >>>>> >>>>> Thanks >>>>> >>>>> Avinash >>>>> >>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "grpc.io" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/grpc-io. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/grpc-io/cc74ecdf-3eac-4475-bad0-23fd66ee03b2%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/grpc-io/cc74ecdf-3eac-4475-bad0-23fd66ee03b2%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "grpc.io" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/grpc-io. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/grpc-io/76dcf498-35e2-4e5a-ab50-92e708cdb808%40googlegroups.com >> >> <https://groups.google.com/d/msgid/grpc-io/76dcf498-35e2-4e5a-ab50-92e708cdb808%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/99752304-bbb3-4a6b-8e43-225629408e76%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
