etr2460 commented on issue #8574: [SIP-26] Proposal for Implementing Connection Pooling for Analytics Database Connections URL: https://github.com/apache/incubator-superset/issues/8574#issuecomment-588320991 A couple questions/comments: Will this SIP also cover using connection pooling for connecting to the metadata database from Celery workers? That's a common issue that we've seen, where if too many connections get opened in Celery, we start to overwhelm the total connections available to MySQL (this may also be related to not closing connections after finishing an async query, this is uncertain). What would the user experience be if there is no available connections when making a sync query? Would that request stall and wait for a connection to be available or instantly fail? Finally, although this may be tangental to the work here, a `CappedDistributedPool` would be extremely useful for us, especially if it capped connections (essentially currently active queries) by both user and analytics db. This would let us rely on Superset to throttle/queue queries prior to sending them to dbs so we can make best use of our cache.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
