[GitHub] john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn
john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn URL: https://github.com/apache/incubator-superset/pull/3895#issuecomment-345404884 @mistercrunch @Mogball @xrmx I'm not overly familiar with this portion of the code base, and the interactions with Gevent, I'm merely trying to fix an issue which is plaguing us in production. I can't fathom the scale @Mogball of your Druid datasources, but for us this request takes mere seconds and thus I'm unsure whether the 4x speedup (maximum) of a multi-threaded environment outweighs the additional complexity, potential thread-safeness etc. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn
john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn URL: https://github.com/apache/incubator-superset/pull/3895#issuecomment-345333095 @Mogball per @xrmx's comment in the issue this may actually be a Gevent issue. I'm not certain whether we want to disable that or go with the approach outlined in this PR, i.e., moving from multi-processing to multi-threading. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn
john-bodley commented on issue #3895: [druid] Fixing issue 3894 multi-processing w/ Gunicorn URL: https://github.com/apache/incubator-superset/pull/3895#issuecomment-345141793 Thanks @Mogball for the explanation. Are you opposed to using multi-threading rather than multi-processing? Also could you provide context on the `refresh_async` method name? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
