Re: [dbcp] Handling many different user accounts over time
On 6/18/15 8:17 AM, Oksana wrote: Cameron, Scott scott.cameron at sap.com writes: Interesting that this has not come up before. You are right that PerUserPoolDataSource does not bound the number of pools that can be created and SharedPoolDataSource does not bound the total combined size of the keyed pool it maintains. Please do open a JIRA requesting the capability. An ugly but possibly adequate workaround would be to subclass either one of the datasources above and override getPooledConnectionAndInfo to enforce a global bound that you maintain in a new instance variable before calling the superclass method. Thanks for the confirmation of the issue. I've entered it into JIRA as DBCP-373. The ugly work-around you suggest is actually exactly what we ended up doing to get access the inner object pool object. We're hoping we can replace it with something nicer in the future but for now it does the trick with only a few (admittedly stinky) lines of code. Cheers, scott Hi, I was wondering if this issue was resolved. I can see that the JIRA request has been closed (https://issues.apache.org/jira/browse/DBCP- 373). However, I have looked into the source code changes (https://fisheye6.atlassian.com/changelog/commons?cs=1568055) and API documentation (version 2.0) and I don't see new attributes which would control the total number of user pools allocated by PerUserPoolDataSource. I am seeing a bunch of new attributes added to control the functionalities of individual pools, but I don't see any new options to control the total number of pools. Sorry for the slow response. You are correct that, as stated in the comment closing the original ticket, there is no direct way to limit the total number of pools. I can't think of a straightforward way to add that feature directly without significant performance impact, but that doesn't mean there is no way to do it. Feel free to either reopen the ticket or create a new one. Patches welcome, of course! Phil Thanks, Oksana - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [dbcp] Handling many different user accounts over time
Cameron, Scott scott.cameron at sap.com writes: Interesting that this has not come up before. You are right that PerUserPoolDataSource does not bound the number of pools that can be created and SharedPoolDataSource does not bound the total combined size of the keyed pool it maintains. Please do open a JIRA requesting the capability. An ugly but possibly adequate workaround would be to subclass either one of the datasources above and override getPooledConnectionAndInfo to enforce a global bound that you maintain in a new instance variable before calling the superclass method. Thanks for the confirmation of the issue. I've entered it into JIRA as DBCP-373. The ugly work-around you suggest is actually exactly what we ended up doing to get access the inner object pool object. We're hoping we can replace it with something nicer in the future but for now it does the trick with only a few (admittedly stinky) lines of code. Cheers, scott Hi, I was wondering if this issue was resolved. I can see that the JIRA request has been closed (https://issues.apache.org/jira/browse/DBCP- 373). However, I have looked into the source code changes (https://fisheye6.atlassian.com/changelog/commons?cs=1568055) and API documentation (version 2.0) and I don't see new attributes which would control the total number of user pools allocated by PerUserPoolDataSource. I am seeing a bunch of new attributes added to control the functionalities of individual pools, but I don't see any new options to control the total number of pools. Thanks, Oksana - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
RE: [dbcp] Handling many different user accounts over time
Interesting that this has not come up before. You are right that PerUserPoolDataSource does not bound the number of pools that can be created and SharedPoolDataSource does not bound the total combined size of the keyed pool it maintains. Please do open a JIRA requesting the capability. An ugly but possibly adequate workaround would be to subclass either one of the datasources above and override getPooledConnectionAndInfo to enforce a global bound that you maintain in a new instance variable before calling the superclass method. Thanks for the confirmation of the issue. I've entered it into JIRA as DBCP-373. The ugly work-around you suggest is actually exactly what we ended up doing to get access the inner object pool object. We're hoping we can replace it with something nicer in the future but for now it does the trick with only a few (admittedly stinky) lines of code. Cheers, scott - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [dbcp] Handling many different user accounts over time
On Thu, Dec 8, 2011 at 7:41 PM, Cameron, Scott scott.came...@sap.com wrote: It seems like everything we want to customize to get the behavior we want is actually at the ObjectPool level. So I was wondering why not expose deeper access to that? It seems like it would provide a lot more flexibility without necessarily changing the DataSource-specific aspects of the class. But maybe there are issues with this I'm not thinking about. I was thinking about posting something to the Developer list asking about this. But do you think JIRA is a better avenue? Yes, JIRA is the way to go with feature requests. Also, check to see if someone else hasn't already requested this and it's coming in a future release. I know that it's on the verge of a new release, but waiting on some minor things from commons-pool. Bill- - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [dbcp] Handling many different user accounts over time
On Wed, Dec 7, 2011 at 9:27 AM, Cameron, Scott scott.came...@sap.com wrote: Hi, I'm considering using either SharedPoolDataSource or PerUserPoolDataSource to do some connection pooling but I've noticed that there doesn't seem to be any way to configure an upper bound on the total number of connections in the pool across all users. For example, say I have 5,000 named users in my database. Any of the 5,000 can come in to request a DB connection at any time, but likely no more than, say, 100 of them will ever be active at a particular point in time. If I want to allow 3 connections per user, how do I ensure that I'm not eventually going to end up with 15,000 open connection over time. It looks like the maxTotal setting on the GenericObjectPool (and GenericKeyedObjectPool) can be used to control this such that when this absolute upper bound is reached on the pool the least recently used 15% of connections will be recycled. But neither SharedPoolDataSource nor PerUserPoolDataSource expose any way to configure maxTotal. Is there another recommended way to solve this problem? What do high-traffic web containers like Tomcat or JBoss do to deal with this scenario (if they do deal with it)? Interesting that this has not come up before. You are right that PerUserPoolDataSource does not bound the number of pools that can be created and SharedPoolDataSource does not bound the total combined size of the keyed pool it maintains. Please do open a JIRA requesting the capability. An ugly but possibly adequate workaround would be to subclass either one of the datasources above and override getPooledConnectionAndInfo to enforce a global bound that you maintain in a new instance variable before calling the superclass method. Thanks for reporting this. Phil Thanks in advance! scott - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
Re: [dbcp] Handling many different user accounts over time
On Wed, Dec 7, 2011 at 11:27 AM, Cameron, Scott scott.came...@sap.com wrote: Hi, I'm considering using either SharedPoolDataSource or PerUserPoolDataSource to do some connection pooling but I've noticed that there doesn't seem to be any way to configure an upper bound on the total number of connections in the pool across all users. What do you mean, across all users? Do you have different connection strings (user/pass) for each person who connects to your database? For example, say I have 5,000 named users in my database. Any of the 5,000 can come in to request a DB connection at any time, but likely no more than, say, 100 of them will ever be active at a particular point in time. If I want to allow 3 connections per user, how do I ensure that I'm not eventually going to end up with 15,000 open connection over time. It looks like the maxTotal setting on the GenericObjectPool (and GenericKeyedObjectPool) can be used to control this such that when this absolute upper bound is reached on the pool the least recently used 15% of connections will be recycled. But neither SharedPoolDataSource nor PerUserPoolDataSource expose any way to configure maxTotal. Is there another recommended way to solve this problem? What do high-traffic web containers like Tomcat or JBoss do to deal with this scenario (if they do deal with it)? Do I understand correctly, if I show up to your site as user wspeirs, then I'm connecting to your database as wspeirs? If so, then why wouldn't you simply create a connection for that user and store it in the user's session? When the session is destroyed, you close the connection. There is a bit more start-up time in creating the connection when they first show up, but it'll be live and active during the rest of the time they are there. That work for you? Bill- - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
RE: [dbcp] Handling many different user accounts over time
What do you mean, across all users? Do you have different connection strings (user/pass) for each person who connects to your database? It's a slightly mixed bag as the app is multi-tenanted. We have some connection strings used by nearly everybody for certain things. Some connection strings are assigned to an organization and will be used by all users in that org. And some connection strings are unique for each individual user. Do I understand correctly, if I show up to your site as user wspeirs, then I'm connecting to your database as wspeirs? If so, then why wouldn't you simply create a connection for that user and store it in the user's session? When the session is destroyed, you close the connection. There is a bit more start-up time in creating the connection when they first show up, but it'll be live and active during the rest of the time they are there. Even if the connections weren't sometimes shared across users, the component that manages connections doesn't have any knowledge of application-level concepts like session. It's job is just to manage connections, which can potentially have a different lifecycle than session. We're thinking that maybe we can rely on maxIdle to evict unused connections. The downside to this is that the default minimum idle time before eviction is 30 minutes and can't be configured. But maybe that will be OK. It would be nice to have more control over the object pool inside the pooled data source classes as that seems to be where most of the interesting tweakable pooling features live. But only a very small number of options are exposed to data source consumers. -- scott - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
RE: [dbcp] Handling many different user accounts over time
If you have specific ideas, open a JIRA ticket and we can look into it. Bill- On Dec 8, 2011 5:52 PM, Cameron, Scott scott.came...@sap.com wrote: What do you mean, across all users? Do you have different connection strings (user/pass) for each person who connects to your database? It's a slightly mixed bag as the app is multi-tenanted. We have some connection strings used by nearly everybody for certain things. Some connection strings are assigned to an organization and will be used by all users in that org. And some connection strings are unique for each individual user. Do I understand correctly, if I show up to your site as user wspeirs, then I'm connecting to your database as wspeirs? If so, then why wouldn't you simply create a connection for that user and store it in the user's session? When the session is destroyed, you close the connection. There is a bit more start-up time in creating the connection when they first show up, but it'll be live and active during the rest of the time they are there. Even if the connections weren't sometimes shared across users, the component that manages connections doesn't have any knowledge of application-level concepts like session. It's job is just to manage connections, which can potentially have a different lifecycle than session. We're thinking that maybe we can rely on maxIdle to evict unused connections. The downside to this is that the default minimum idle time before eviction is 30 minutes and can't be configured. But maybe that will be OK. It would be nice to have more control over the object pool inside the pooled data source classes as that seems to be where most of the interesting tweakable pooling features live. But only a very small number of options are exposed to data source consumers. -- scott - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
RE: [dbcp] Handling many different user accounts over time
If you have specific ideas, open a JIRA ticket and we can look into it. It seems like everything we want to customize to get the behavior we want is actually at the ObjectPool level. So I was wondering why not expose deeper access to that? It seems like it would provide a lot more flexibility without necessarily changing the DataSource-specific aspects of the class. But maybe there are issues with this I'm not thinking about. I was thinking about posting something to the Developer list asking about this. But do you think JIRA is a better avenue? Thanks for your assistance. -- scott - To unsubscribe, e-mail: user-unsubscr...@commons.apache.org For additional commands, e-mail: user-h...@commons.apache.org
[dbcp] Handling many different user accounts over time
Hi, I'm considering using either SharedPoolDataSource or PerUserPoolDataSource to do some connection pooling but I've noticed that there doesn't seem to be any way to configure an upper bound on the total number of connections in the pool across all users. For example, say I have 5,000 named users in my database. Any of the 5,000 can come in to request a DB connection at any time, but likely no more than, say, 100 of them will ever be active at a particular point in time. If I want to allow 3 connections per user, how do I ensure that I'm not eventually going to end up with 15,000 open connection over time. It looks like the maxTotal setting on the GenericObjectPool (and GenericKeyedObjectPool) can be used to control this such that when this absolute upper bound is reached on the pool the least recently used 15% of connections will be recycled. But neither SharedPoolDataSource nor PerUserPoolDataSource expose any way to configure maxTotal. Is there another recommended way to solve this problem? What do high-traffic web containers like Tomcat or JBoss do to deal with this scenario (if they do deal with it)? Thanks in advance! scott
