Hi all,
We're using the Python SDK with the portable Flink runner and running into
some problems integrating gevent. We're patching the gRPC runtime for
gevent as described in [0] which allows pipelines to start and partially
run. However the tasks produce a stream of gevent exceptions:
Exception greenlet.error: error('cannot switch to a different thread',) in
'grpc._cython.cygrpc.run_loop' ignored
Traceback (most recent call last):
File "src/gevent/event.py", line 240, in gevent._event.Event.wait
File "src/gevent/event.py", line 140, in gevent._event._
AbstractLinkable._wait
File "src/gevent/event.py", line 117, in gevent._event._
AbstractLinkable._wait_core
File "src/gevent/event.py", line 119, in gevent._event._
AbstractLinkable._wait_core
File "src/gevent/_greenlet_primitives.py", line 59, in
gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 59, in
gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/_greenlet_primitives.py", line 63, in
gevent.__greenlet_primitives.SwitchOutGreenletWithLoop.switch
File "src/gevent/__greenlet_primitives.pxd", line 35, in
gevent.__greenlet_primitives._greenlet_switch
greenlet.error: cannot switch to a different thread
and do not make any progress.
Has anybody else successfully used the portable python sdk with gevent? Or
is there a recommended alternative for doing async IO in python pipelines?
[0] https://github.com/grpc/grpc/issues/4629#issuecomment-376962677
Micah
