SbloodyS commented on issue #10918: URL: https://github.com/apache/dolphinscheduler/issues/10918#issuecomment-1182704512
> I just see the latest code, we didn't use connection pool in SQL task, so the hikari pool config isn't used for SQL task. > > Right now, for each task we will create a new connection and will close this connection after finished. I am not clear if we need to use connection pool in SQL task. > > cc @caishunfeng @SbloodyS I think we can provide an configure option to users in the data source center. But I think using connection pooling in SQL task may cause task blocking since ds is a distributed big data scheduling system. It is common to execute hundreds of tasks in parallel. The maximum number of connection supported in the database of my current production environment is 10000. As a result, it's better to expand the database or decrease the SQL task's parallelism. @fhygh -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
