When everything works smooth - the end user does not need to know whats under the hood.
However, when things begin to swing and your hands are tied to just one CP - what do you do? On Wed, Mar 14, 2018 at 10:50 AM, Khosrow Moossavi <kmooss...@cloudops.com> wrote: > Why would we want to expose this choice to administrator of Cloudstack > whose responsibility > is to keep it running and not knowing about the inner-mechanic of how it > works. right? It's not > like that we're giving them a choice of which database to connect to. > > So on that note, I would say we need to agree on any of those CP libraries > and implement, the > same way we chose for example log4j or slf4j over one another, or any other > _library_ we use. > > Khosrow Moossavi > > CloudOps > > > > On Wed, Mar 14, 2018 at 10:36 AM, Nicolas Vazquez < > nicolas.vazq...@shapeblue.com> wrote: > > > Thanks Khosrow and Rafael. You both agree on Spring Data as the best > > option, I see it would require a big effort and commitment to migrate to > > it, therefore it can take some (long) time to achieve it. > > > > As a more viable option, would you agree on supporting different > > connection pool management libraries and letting the administrator choose > > which one to use? (DBCP 1.4 as default) > > > > ________________________________ > > From: Rafael Weingärtner <rafaelweingart...@gmail.com> > > Sent: Tuesday, March 13, 2018 8:52:50 AM > > To: dev > > Subject: Re: [DISCUSS] CloudStack Connection Pools > > > > Spring data would be awesome. It is very flexible and has a very good > API. > > However, this would require commitment from our side to slowly migrate > > things to it. > > > > Regarding the connection pool management libraries; I would prefer either > > C3P0 or 2.* DBCP. The other two sound trendy, but I worry about this type > > of project in the long run. Both DBCP from Apache and C3P0 from Hibernate > > (RedHat) sound a more reasonable selection for me. They have been around > > for years, and have a solid community base already. > > > > On Mon, Mar 12, 2018 at 11:31 PM, Khosrow Moossavi < > kmooss...@cloudops.com > > > > > wrote: > > > > > Hi Nicolas > > > > > > From my past experiences, I prefer 1) HikariCP 2) Tomcat Pool 3) C3P0 > 4) > > > DBCP in that order. Although I don't have > > > any benchmark of my own to provide, and the ones you mentioned are > really > > > informative anyway. > > > > > > To me the broader subject is the _one_ who uses the pool, I mean if the > > > transactions are handled in a faster way and > > > released sooner and with shorter locks, generally speaking if it's more > > > efficient, I don't think from ACS point of view > > > there won't be much difference between the above mentioned options. > > > > > > On the same subject, it might be more interesting to use Spring Boot in > > > general and Spring Boot Data in particular > > > rather than only changing the CP functionality, and slowly > > refactor/retire > > > the DAO layer in favor of Spring Boot equivalent > > > implementation. > > > > > > > > > Khosrow Moossavi > > > > > > CloudOps > > > > > > > > > > > > On Mon, Mar 12, 2018 at 9:32 PM, Nicolas Vazquez < > > > nicolas.vazq...@shapeblue.com> wrote: > > > > > > > Hi all, > > > > > > > > > > > > I would like to introduce a topic for discussion, regarding DB > > connection > > > > pools used in CloudStack, currently Apache Commons DBCP 1.4 ( > > > > http://commons.apache.org/) is used. I've been investigating this > > topic > > > > as we are having complains of random issues on MySQL connection pool > on > > > > large environments. Please let me know if this topic has already been > > > > discussed before. > > > > > > > > > > > > First of all, DBCP 1.4 has been released on 2010 ( > > > > https://commons.apache.org/proper/commons-dbcp/changes-report.html), > > and > > > > no minor/patch version has been released since then. It seems to work > > in > > > > high performance with relatively low traffic and low load > applications. > > > > However, it is single threaded, and in order to be thread-safe, the > > > entire > > > > pool needs to be locked. It is also reported that an CPU and > concurrent > > > > threads increases, the performance gets affected. This is a serious > > issue > > > > on highly concurrent systems, such as CloudStack. > > > > > > > > > > > > I've been investigating some options to replace it: > > > > - The first option can be upgrading to version 2.x. Issues on > > performance > > > > and concurrency could be solved using this version. > > > > - Tomcat JDBC Connection Pool. Please check: > > https://tomcat.apache.org/ > > > > tomcat-7.0-doc/jdbc-pool.html. > > > > > > > > - Other replacement options found: BoneCP, C3P0, HikariCP > > > > > > > > > > > > Given these options, I've been looking for benchmarks to compare them > > > (*). > > > > Looks like HikariCP (http://brettwooldridge.github.io/HikariCP/) > could > > > be > > > > the best replacement, improving performance and stability. Another > good > > > > replacement option could be Tomcat. > > > > > > > > > > > > I've been also examining the codebase, data source initialization is > > done > > > > on TransactionLegacy class under the cloud-framework-db project. > > > > Replacement work should be done on this class. Instead of pure > > > replacement, > > > > a global setting can be introduced to make the admins able to select > > > which > > > > connection pool to use. > > > > > > > > What do you think? Any possitive/negative feedback is welcome as well > > as > > > > new ideas. As mentioned before, I don't know if it has been discussed > > > > before, sorry in advance if it has. > > > > > > > > Kind regards, > > > > Nicolas > > > > > > > > (*) Links to benchmarks and comparissons: > > > > https://www.wix.engineering/single-post/how-many-threads- > > > > does-it-take-to-fill-a-pool > > > > https://www.wix.engineering/single-post/how-does-hikaricp- > > > > compare-to-other-connection-pools > > > > <https://www.wix.engineering/single-post/how-does-hikaricp- > > > > compare-to-other-connection-pools>https://beansroasted. > > > > wordpress.com/2017/07/29/connection-pool-analysis/ > > > > https://beansroasted.wordpress.com/tag/connection-pool-comparison/ > > > > <https://beansroasted.wordpress.com/tag/connection-pool-comparison/> > > > > https://github.com/brettwooldridge/HikariCP/wiki/%22My-benchmark- > > > > doesn't-show-a-difference.%22 > > > > http://www.trustiv.co.uk/2014/06/battle-connection-pools > > > > > > > > nicolas.vazq...@shapeblue.com > > > > www.shapeblue.com<http://www.shapeblue.com> > > > > , > > > > @shapeblue > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Rafael Weingärtner > > > > nicolas.vazq...@shapeblue.com > > www.shapeblue.com > > , > > @shapeblue > > > > > > > > >
