sjmeyer commented on issue #5279: Slices backed by AWS Athena load sequentially, not in parallel. URL: https://github.com/apache/incubator-superset/issues/5279#issuecomment-402870216 @m4neda My -w argument is 16. The full ENV "GUNICORN_CMD_ARGS" is: -w 16 -k gevent --timeout 120 -b 0.0.0.0:8088 --limit-request-line 0 --limit-request-field_size 0 --proxy-protocol true --proxy-allow-from * --forwarded-allow-ips * --keep-alive 75" The application is running in a container on a linux server. The container's max memory is 8GB and it has no limit on the CPU (quad core). It is not an issue with either memory or CPU, it is well within the limits, based on docker stats. I've been trying to figure out exactly where the call to Athena actually happens, to see if it's some kind of issue with that. I see that the "create_engine" method is called for each query. Should the connection not persist? Each query has a boto3 call to authenticate the query. Is that correct? Under database settings I see that you can enable/disable asynchronous queries. But asynchronous queries expect Celery workers to be setup. Would you advise using this setting and the celery workers for use in a system that should scale out/in based on demand? Or is this purely for long running queries?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
