[jira] [Commented] (POOL-350) Add option for not executing "hasBorrowWaiters()" while returning objects
[
https://issues.apache.org/jira/browse/POOL-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17695074#comment-17695074
]
Gary D. Gregory commented on POOL-350:
--
My concern with more configuration parameters is adding complexity to an
already complex library leading to more gotchas, catch-22s, and
harder-to-diagnose support tickets. That said, if we have confidence in this
specific solution, let's implement it. I would hope [~NickNYU] can either
prototype this and confirm that this is really all that is needed for this use
case since once this is enabled (or disabled in this case), it could lead to
unforeseen side effects that require more configuration toggles or more code
changes.
A more extensive architectural avenue would be to make pooling more
configurable not through toggles but with lambdas. I don't know what this would
exactly look like in this specific case but I can imagine configuring this
example with a lambda that is either a no-op or one that does not call the API
in question.
> Add option for not executing "hasBorrowWaiters()" while returning objects
> -
>
> Key: POOL-350
> URL: https://issues.apache.org/jira/browse/POOL-350
> Project: Commons Pool
> Issue Type: New Feature
>Affects Versions: 2.6.0
> Environment: h5. uname -a:
> Linux VMS26239 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> *Java version:*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
>
>Reporter: zhu chen
>Priority: Critical
> Labels: easyfix
>
> h2. Phenomena:
> I'm recently leveraging commons-pool as my Redis connection pool in my
> project, however, the pain is that when my system is dealing with over
> thousands of Redises, CPU load become such high. By checking JVM through
> JFR(FlightRecorder), it turned out the hot method was
> "{color:#FF}hasBorrowWaiters(){color}", which is invoked by
> "{color:#FF}returnObject(){color}" each time.
> That means the system will go through over *thousands*(the number will grow
> as well as my system) of keys after *each* object's *return*, what's worse,
> the program is running concurrently, which, obviously cause a huge CPU load.
>
> h2. Expect:
> I was wondering if we could add a config for optionally run this
> "hasBorrowWaiters()" each time when we return an object.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (POOL-350) Add option for not executing "hasBorrowWaiters()" while returning objects
[
https://issues.apache.org/jira/browse/POOL-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653704#comment-17653704
]
Phil Steitz commented on POOL-350:
--
Any feedback on the reuseCapacityOnReturn, reuseCapacityOnMaintenance idea?
Might help in situations like the OP's. Should not be hard to implement.
> Add option for not executing "hasBorrowWaiters()" while returning objects
> -
>
> Key: POOL-350
> URL: https://issues.apache.org/jira/browse/POOL-350
> Project: Commons Pool
> Issue Type: New Feature
>Affects Versions: 2.6.0
> Environment: h5. uname -a:
> Linux VMS26239 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> *Java version:*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
>
>Reporter: zhu chen
>Priority: Critical
> Labels: easyfix
>
> h2. Phenomena:
> I'm recently leveraging commons-pool as my Redis connection pool in my
> project, however, the pain is that when my system is dealing with over
> thousands of Redises, CPU load become such high. By checking JVM through
> JFR(FlightRecorder), it turned out the hot method was
> "{color:#FF}hasBorrowWaiters(){color}", which is invoked by
> "{color:#FF}returnObject(){color}" each time.
> That means the system will go through over *thousands*(the number will grow
> as well as my system) of keys after *each* object's *return*, what's worse,
> the program is running concurrently, which, obviously cause a huge CPU load.
>
> h2. Expect:
> I was wondering if we could add a config for optionally run this
> "hasBorrowWaiters()" each time when we return an object.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (POOL-350) Add option for not executing "hasBorrowWaiters()" while returning objects
[
https://issues.apache.org/jira/browse/POOL-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16945183#comment-16945183
]
Phil Steitz commented on POOL-350:
--
How about adding two config parameters
reuseCapacityOnReturn
reuseCapacityOnMaintenance
Maybe with better names. The way it works now is reuseCapacity is called on
return only. I can see wanting to turn this off in some applications (though
it risks liveness problems). Having the option to have the evictor do it might
work for some workloads.
Another option is to make it more efficient. I played a little with
introducing a map of atomic ints maintaining waiter counts so you don't have to
walk the pool list and get the locks on the LBDs; but I could not demonstrate
better performance that way. Could be my benchmarks were misleading though and
that is another option here.
> Add option for not executing "hasBorrowWaiters()" while returning objects
> -
>
> Key: POOL-350
> URL: https://issues.apache.org/jira/browse/POOL-350
> Project: Commons Pool
> Issue Type: New Feature
>Affects Versions: 2.6.0
> Environment: h5. uname -a:
> Linux VMS26239 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> *Java version:*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
>
>Reporter: zhu chen
>Priority: Critical
> Labels: easyfix
>
> h2. Phenomena:
> I'm recently leveraging commons-pool as my Redis connection pool in my
> project, however, the pain is that when my system is dealing with over
> thousands of Redises, CPU load become such high. By checking JVM through
> JFR(FlightRecorder), it turned out the hot method was
> "{color:#FF}hasBorrowWaiters(){color}", which is invoked by
> "{color:#FF}returnObject(){color}" each time.
> That means the system will go through over *thousands*(the number will grow
> as well as my system) of keys after *each* object's *return*, what's worse,
> the program is running concurrently, which, obviously cause a huge CPU load.
>
> h2. Expect:
> I was wondering if we could add a config for optionally run this
> "hasBorrowWaiters()" each time when we return an object.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (POOL-350) Add option for not executing "hasBorrowWaiters()" while returning objects
[
https://issues.apache.org/jira/browse/POOL-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1717#comment-1717
]
zhu chen commented on POOL-350:
---
exactly,@ [~struberg].
I would appreciate it if you could considering about my idea.
I understand the idea that 'hasBorrowWaiters()' is working for the most
hot/busy keyed objects, so that the local setting (max object per key) for the
hottest key runs exhausted before the global setting(max object total). But it
might be a good idea to also make the operation configurable.
> Add option for not executing "hasBorrowWaiters()" while returning objects
> -
>
> Key: POOL-350
> URL: https://issues.apache.org/jira/browse/POOL-350
> Project: Commons Pool
> Issue Type: New Feature
>Affects Versions: 2.6.0
> Environment: h5. uname -a:
> Linux VMS26239 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> *Java version:*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
>
>Reporter: zhu chen
>Priority: Critical
> Labels: easyfix
>
> h2. Phenomena:
> I'm recently leveraging commons-pool as my Redis connection pool in my
> project, however, the pain is that when my system is dealing with over
> thousands of Redises, CPU load become such high. By checking JVM through
> JFR(FlightRecorder), it turned out the hot method was
> "{color:#FF}hasBorrowWaiters(){color}", which is invoked by
> "{color:#FF}returnObject(){color}" each time.
> That means the system will go through over *thousands*(the number will grow
> as well as my system) of keys after *each* object's *return*, what's worse,
> the program is running concurrently, which, obviously cause a huge CPU load.
>
> h2. Expect:
> I was wondering if we could add a config for optionally run this
> "hasBorrowWaiters()" each time when we return an object.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (POOL-350) Add option for not executing "hasBorrowWaiters()" while returning objects
[
https://issues.apache.org/jira/browse/POOL-350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1366#comment-1366
]
Mark Struberg commented on POOL-350:
Do you mind to ship a patch which will show what you had in mind?
> Add option for not executing "hasBorrowWaiters()" while returning objects
> -
>
> Key: POOL-350
> URL: https://issues.apache.org/jira/browse/POOL-350
> Project: Commons Pool
> Issue Type: New Feature
>Affects Versions: 2.6.0
> Environment: h5. uname -a:
> Linux VMS26239 3.10.0-229.11.1.el7.x86_64 #1 SMP Thu Aug 6 01:06:18 UTC 2015
> x86_64 x86_64 x86_64 GNU/Linux
>
> *Java version:*
> java version "1.8.0_60"
> Java(TM) SE Runtime Environment (build 1.8.0_60-b27)
> Java HotSpot(TM) 64-Bit Server VM (build 25.60-b23, mixed mode)
>
>
>Reporter: zhu chen
>Priority: Critical
> Labels: easyfix
>
> h2. Phenomena:
> I'm recently leveraging commons-pool as my Redis connection pool in my
> project, however, the pain is that when my system is dealing with over
> thousands of Redises, CPU load become such high. By checking JVM through
> JFR(FlightRecorder), it turned out the hot method was
> "{color:#FF}hasBorrowWaiters(){color}", which is invoked by
> "{color:#FF}returnObject(){color}" each time.
> That means the system will go through over *thousands*(the number will grow
> as well as my system) of keys after *each* object's *return*, what's worse,
> the program is running concurrently, which, obviously cause a huge CPU load.
>
> h2. Expect:
> I was wondering if we could add a config for optionally run this
> "hasBorrowWaiters()" each time when we return an object.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
