Hi Carl,

That is very helpful indeed, thank you!

Best regards,

Jorn



From:   "'Carl Mastrangelo' via grpc.io" <grpc-io@googlegroups.com>
To:     "grpc.io" <grpc-io@googlegroups.com>
Date:   09/02/2018 04:51
Subject:        [grpc-io] Re: gRPC to stream binary image data from 
database: Intended use case or not?



Yes, yours is an intended use case.  gRPC has some advantages here 
compared to Websockets that would help you:

1.  RPCs can use application level flow control to control how fast data 
is sent.  For example, if you were sending 1GB of data, but the remote 
side didn't have enough memory to store all of it at the moment, gRPC 
let's the receiver push back against the client.  
2.  All RPCs on a connection are multiplexed, allowing you to interleave 
multiple RPCs at the same time.   For example, if you have multiple 
streaming RPCs for your image data, gRPC will intelligently interleave big 
images with small images.   On a given stream, the messages are in order, 
but with multiple streams (or just multiple RPCs), a small message won't 
get stuck behind a large one.   An example example would be a client 
requesting two images, one being 1000MB, and one being 1MB.   The 1MB one 
is sent at the same time as the 1000MB one, allowing it to complete in a 
reasonable time.

For REST, it means a lot of things to a lot of people, but I'll assume you 
mean HTTP/1.1 with hanging gets.  With that, you are limited to a few 
concurrent requests at a time (as a norm), and in-order responses.        
For example, if you wanted to request 100 images with hanging gets, you 
would need to pipeline the requests, and the server would have to retrieve 
them in order, even if it would be faster to serve them out of order.  
(Perhaps some of them are cached and some of them are on disk?).   HTTP/2 
does not have this limitation.


Also, gRPC has some operation advantages too.   For example, there is 
pluggable name resolution, so if you used ZooKeeper or etcd, you could 
plug those in and keep using gRPC.  There is also a solid load-balancer 
story with high quality default implementations provided (error tolerant 
round-robin and pick first), as well as stubs to a grpclb load balancer 
service.   Lastly, gRPC has well integrated monitoring and tracing, so you 
can see how long your RPCs take as well as how many.   This is in the 
works, but is used widely internally at Google.  

Oh yeah, and gRPC is quite fast as you noticed.    We intend it as a full 
replacement for all RPCs inside Google, which means it has to serve many, 
extreme usages.  (there are not many general purpose RPC libraries that 
can send and receive at line rate).  This is mainly due to using protobuf, 
but gRPC works very well with protobuf.   (also, gRPC is encoding 
agnostic, so you can send raw bytes or json, or whatever if you want to).

Hope that helps!

Carl  

On Thursday, February 8, 2018 at 2:14:10 AM UTC-8, jorn....@kisters-bv.nl 
wrote:
Dear all,

We have recently been investigating various RPC technologies that would 
help us
query and transfer data between a database server containing image data, 
and a client.

For HBase, Thrift is typically used for this.  But our benchmarks show 
that the
current version of gRPC also performs very well for this task, and we are
tempted to use gRPC instead of Thrift, REST, or WebSocket.

Hence our question:
> Is streaming moderately sized chunks of binary data part of the vision 
for gRPC?  Can we 
count on the streaming of moderately sized chunks of binary data 
performing equally well in future releases of gRPC?

We would very much appreciate your views on this.

Thanks and best regards,

Jorn Baayen
-- 
You received this message because you are subscribed to a topic in the 
Google Groups "grpc.io" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/grpc-io/MbDTqNXhv7o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/abee3d06-bec0-427e-bbb8-b14966bedb6a%40googlegroups.com
.
For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"grpc.io" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to grpc-io+unsubscr...@googlegroups.com.
To post to this group, send email to grpc-io@googlegroups.com.
Visit this group at https://groups.google.com/group/grpc-io.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/grpc-io/OF50A3DFA5.A2E19CAD-ONC1258232.0038CFFB-C1258232.00396F24%40kisters.de.
For more options, visit https://groups.google.com/d/optout.

Reply via email to