thammanenitcs opened a new issue #14414: URL: https://github.com/apache/superset/issues/14414
Hi Team, i am using superset0.36 version, have observed superset is creating more no.of db sessions when dashboards/reports are running continuously. Sometimes due to high load/no.of schedule jobs are running db sessions are breaking. I thought to see whether connection pool creation for superset will improve db sessions failures. i have followed below SIP as a reference. https://github.com/apache/superset/issues/8574 i have followed the sqlalchemy documentation and used QueuePool to created connection pool when superset is establishing db session with help of "create_engine" method. Below are the connection pool parameters and file locations where i have updated in superset. Documentation: https://docs.sqlalchemy.org/en/13/core/pooling.html#sqlalchemy.pool.QueuePool Connection pool parameters: filepath: superset/models/core.py [ function: get_sqla_engine ] params["poolclass"] = QueuePool params["pool_size"] = 20 params["max_overflow"] = 10 params["pool_recycle"]= 3600 params["pool_pre_ping"] = True params["pool_use_lifo"] = True ------------------------------------------------------------------- filepath: superset/migrations/env.py engine = engine_from_config( config.get_section(config.config_ini_section), prefix="sqlalchemy.", poolclass=pool.QueuePool, ) I can these are the main functions where superset initiating for db sessions creations. there are couple of places where i have override nullpool parameter to false, since nullpoll is for disabling connection pool. i also tried to see by varying the pool size but i can't see any difference after applying connection pool code. I need suggestion to whether i am going in right direction or not OR is there a way to test/see the connection pool usage at application level to make sure the new code customizations that i have done is working. Please give some suggestions/ if there are any implementation done w.r.to this requirement in superset before. Thanks for the attention and time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
