(This is also being discussed in this StackOverflow question <https://stackoverflow.com/questions/43948975/grpc-unary-streaming-continues-after-disconnect-blocks-on-interrupt> .)
On Sun, May 14, 2017 at 1:15 PM, <[email protected]> wrote: > I'm having an issue with client disconnection using a Python gRPC server > with a unary-stream procedure. I may have missed something in the > documentation, but multiple attempts of checking whether or not the client > is still active have all failed. > > My hope is to find a way for the Servicer to detect when to stop > generating values when a client disconnects. > The ServicerContext's is_active and add_callback methods are the recommended means for your code to be informed of RPC termination. A client subscribes to a stream of updates once and receives all changes to > the server state for an unbounded amount of time. The actual server > implementation should also send values regularly on a heartbeat with low > activity. A minimal example is given below that demonstrates my problem. > > class TaskService(proto_grpc.TaskServiceServicer): > def Subscribe(self, request, context): > print 'subscribed' > > while True: > time.sleep(1) > print 'sending' > yield proto.TaskState(state='IDLE') > > print 'finished' > > > def main(): > server = grpc.server(concurrent.futures.ThreadPoolExecutor( > max_workers=3)) > > proto_grpc.add_TaskServiceServicer_to_server( > TaskService(), > server, > ) > > server.add_insecure_port('0.0.0.0:3253') > server.start() > > try: > while True: > time.sleep(1) > print 'bump' > finally: > print 'stopping' > server.stop(grace=0) > print 'stopped' > > > There are two issues at play here. First, if a client subscribes and then > is disconnected forcefully, the server will continue to send updates to > that channel. > This shouldn't be the case - if a disconnection happens, the server-side run-time code should detect it and refrain from further calling into response-iterator object given to it by your application. Of course there could be a bug in that logic, but that's what *should* happen. Notice that it is not guaranteed that "finished" will ever be printed in the case of an RPC that terminates with non-OK status (as would be the case in a disconnection). This means if three clients have *ever* subscribed to this server, then a > fourth may never connect in the future (due to the low number in the thread > pool). This seems wrong, but I may be missing a setting somewhere. > You're right about there being a limit on the number of RPCs concurrently being serviced; if you've found a bug that prevents disconnected RPCs from being recognized as terminated, then you're right that their service would continue and would starve out additional "real" RPCs. Second, if a client subscribes and then disconnects, the server will never > shut down completely. The process will hang in an un-killable state. This > does not occur when a client never connects, or if a client connects but > stays connected as the process shuts down. > As you describe it this certainly smells like a defect. Are you able to demonstrate it in a single program? If so, in a single Python interpreter, or does it require a small family of communicating but separate processes? The following output occurs when the server is killed after a client > connects: > > subscribed > bump > sending > bump > sending > ^C > stopping > stopped > Traceback (most recent call last): > ... > time.sleep(1) > KeyboardInterrupt > sending > <process terminated gracefully> > > The following output occurs when the server is killed after a client > disconnects: > > subscribed > bump > sending > bump > <client is killed> > sending > bump > ^C > stopping > stopped > Traceback (most recent call last): > ... > time.sleep(1) > KeyboardInterrupt > sending > <process hangs indefinitely> > > I hope there is an easy solution as I simply misunderstood the correct use > to implement a streaming response and apologize in advance if there is. > Perhaps there's trouble of some sort - whether or not you're understanding the streaming semantics correctly <https://groups.google.com/forum/#!msg/grpc-io/JAwRhBnSDfI/DuSCxQ6ROgAJ> should have no bearing on the fact that a server told to shut down should shut down cleanly and allow the process to exit rather than hanging indefinitely. With what version of gRPC Python are you working, and with what version of Python and on what platform? -Nathaniel -- You received this message because you are subscribed to the Google Groups "grpc.io" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/grpc-io. To view this discussion on the web visit https://groups.google.com/d/msgid/grpc-io/CAEOYnAQLxGFZobHCTHYyr7F2kLwv7rMqirP207m%2Bth%2BVcsdkGg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
smime.p7s
Description: S/MIME Cryptographic Signature
