abrenaut edited a comment on issue #5447: Gunicorn workers running blocking calls URL: https://github.com/apache/incubator-superset/issues/5447#issuecomment-407216994 I reproduced locally on the master branch running superset with the following command ``` gunicorn -w 1 -k gevent --timeout 120 -b 0.0.0.0:8088 --limit-request-line 0 --limit-request-field_size 0 superset:app ``` If I load a chart based on a query that takes a long time to run, all the request I try to make while the chart is loading are hanging. I managed to work around that by configuring the pg driver to run in a non-blocking style in superset_config.py (see http://initd.org/psycopg/docs/advanced.html#support-for-coroutine-libraries). ``` import psycopg2 from psycopg2 import extensions from gevent.socket import wait_read, wait_write def make_psycopg_green(): """Configure Psycopg to be used with gevent in non-blocking way.""" if not hasattr(extensions, 'set_wait_callback'): raise ImportError( "support for coroutines not available in this Psycopg version (%s)" % psycopg2.__version__) extensions.set_wait_callback(gevent_wait_callback) def gevent_wait_callback(conn, timeout=None): """A wait callback useful to allow gevent to work with Psycopg.""" while 1: state = conn.poll() if state == extensions.POLL_OK: break elif state == extensions.POLL_READ: wait_read(conn.fileno(), timeout=timeout) elif state == extensions.POLL_WRITE: wait_write(conn.fileno(), timeout=timeout) else: raise psycopg2.OperationalError( "Bad result from poll: %r" % state) make_psycopg_green() ``` I still have an issue when loading multiple charts (6+) with long running queries at the same time and I'm not quite sure what's causing the workers to be blocked.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
