[jira] [Resolved] (MATH-1640) "KMeansPlusPlusClusterer": Do not change caller's arguments
[
https://issues.apache.org/jira/browse/MATH-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gilles Sadowski resolved MATH-1640.
---
Resolution: Done
commit c6b4ca908cc53c6d5d89e911391337086062e74a
> "KMeansPlusPlusClusterer": Do not change caller's arguments
> ---
>
> Key: MATH-1640
> URL: https://issues.apache.org/jira/browse/MATH-1640
> Project: Commons Math
> Issue Type: Improvement
>Affects Versions: 3.6.1
>Reporter: Gilles Sadowski
>Assignee: Gilles Sadowski
>Priority: Trivial
> Labels: api
> Fix For: 4.0
>
>
> Class
> [{{KMeansPlusPlusClusterer}}|https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=commons-math-legacy/src/main/java/org/apache/commons/math4/legacy/ml/clustering/KMeansPlusPlusClusterer.java;h=989019466d3257d87d76b81a68d74066deb21fe8;hb=HEAD]
> tries to outguess the caller by [replacing negative values with
> Integer.MAX_VALUE|https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=commons-math-legacy/src/main/java/org/apache/commons/math4/legacy/ml/clustering/KMeansPlusPlusClusterer.java;h=989019466d3257d87d76b81a68d74066deb21fe8;hb=HEAD#l198].
> Such a behaviour can hide bugs on the caller's side.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (COMMONSSITE-133) Release plugin: Reviewing checksums
[
https://issues.apache.org/jira/browse/COMMONSSITE-133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481505#comment-17481505
]
Gilles Sadowski commented on COMMONSSITE-133:
-
I've attached a patch.
The test passes but I don't see any of the supposedly created files (in the
location reported by the console output), so I couldn't check that the contents
is as expected (i.e. compliant with "sha512sum").
Please review.
> Release plugin: Reviewing checksums
> ---
>
> Key: COMMONSSITE-133
> URL: https://issues.apache.org/jira/browse/COMMONSSITE-133
> Project: Apache Commons All
> Issue Type: New Feature
>Reporter: Gilles Sadowski
>Priority: Minor
> Labels: release, review
> Attachments: COMMONSSITE-133.patch
>
>
> The plugin should generate a text file (to be stored in the "dist" area along
> the archived artefacts) so that a single command:
> {noformat}
> $ sha512sum -c checksumfile
> {noformat}
> can be used to check all of them.
> The template for generating the vote message body should contain a command to
> download all the to-be-checked files from the dist area.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (COMMONSSITE-133) Release plugin: Reviewing checksums
[
https://issues.apache.org/jira/browse/COMMONSSITE-133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gilles Sadowski updated COMMONSSITE-133:
Attachment: COMMONSSITE-133.patch
> Release plugin: Reviewing checksums
> ---
>
> Key: COMMONSSITE-133
> URL: https://issues.apache.org/jira/browse/COMMONSSITE-133
> Project: Apache Commons All
> Issue Type: New Feature
>Reporter: Gilles Sadowski
>Priority: Minor
> Labels: release, review
> Attachments: COMMONSSITE-133.patch
>
>
> The plugin should generate a text file (to be stored in the "dist" area along
> the archived artefacts) so that a single command:
> {noformat}
> $ sha512sum -c checksumfile
> {noformat}
> can be used to check all of them.
> The template for generating the vote message body should contain a command to
> download all the to-be-checked files from the dist area.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Updated] (MATH-1580) Outdated "userguide" examples
[
https://issues.apache.org/jira/browse/MATH-1580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gilles Sadowski updated MATH-1580:
--
Description:
Codes under directory {{src/userguide}} don't compile anymore.
Several examples are outdated (points marked with (/) have been resolved):
* Illustration of distributions (corresponding code is now in "Commons
Statistics") (/)
* Illustration of some geometrical algorithms (corresponding code is now in
"Commons Geometry") (/)
* Custom benchmark for class {{FastMath}} (should be replaced by JMH) (x)
* Illustration of the clustering algorithms (/)
* Usage examples of the classes in package {{o.a.c.m.legacy.filter}} (x)
* Usage examples of the classes in package {{o.a.c.m.legacy.genetics}} (x)
* Illustration of low discrepancy sequences (x)
The examples should either be deleted or fixed (and moved either into module
"commons-math-examples" or the respective components).
was:
Codes under directory {{src/userguide}} don't compile anymore.
Several examples are outdated (points marked with (/) have been resolved):
* Illustration of distributions (corresponding code is now in "Commons
Statistics") (/)
* Illustration of some geometrical algorithms (corresponding code is now in
"Commons Geometry") (/)
* Custom benchmark for class {{FastMath}} (should be replaced by JMH) (x)
* Illustration of the clustering algorithms (x)
* Usage examples of the classes in package {{o.a.c.m.legacy.filter}} (x)
* Usage examples of the classes in package {{o.a.c.m.legacy.genetics}} (x)
* Illustration of low discrepancy sequences (x)
The examples should either be deleted or fixed (and moved either into module
"commons-math-examples" or the respective components).
I've created an {{examples-kmeans}} module, with an illustration of two of the
available clustering algorithms.
I don't think it's worth porting the
[{{ClusterAlgorithmComparison}}|https://gitbox.apache.org/repos/asf?p=commons-math.git;a=blob;f=src/userguide/java/org/apache/commons/math4/userguide/ClusterAlgorithmComparison.java;h=2c631f0bc145ff98a90a703980d6ea32d6e58db8;hb=HEAD]
since it does not add anything and it would involve the same many changes I've
done for the {{ImageClusteringExample}} class. It would be more useful to think
about the refactoring (namely to allow multi-threaded usage).
Unless there is an objection, I'll thus simply remove class
{{ClusterAlgorithmComparison}} (since it does not compile anymore).
> Outdated "userguide" examples
> -
>
> Key: MATH-1580
> URL: https://issues.apache.org/jira/browse/MATH-1580
> Project: Commons Math
> Issue Type: Sub-task
>Reporter: Gilles Sadowski
>Priority: Major
> Fix For: 4.0
>
>
> Codes under directory {{src/userguide}} don't compile anymore.
> Several examples are outdated (points marked with (/) have been resolved):
> * Illustration of distributions (corresponding code is now in "Commons
> Statistics") (/)
> * Illustration of some geometrical algorithms (corresponding code is now in
> "Commons Geometry") (/)
> * Custom benchmark for class {{FastMath}} (should be replaced by JMH) (x)
> * Illustration of the clustering algorithms (/)
> * Usage examples of the classes in package {{o.a.c.m.legacy.filter}} (x)
> * Usage examples of the classes in package {{o.a.c.m.legacy.genetics}} (x)
> * Illustration of low discrepancy sequences (x)
> The examples should either be deleted or fixed (and moved either into module
> "commons-math-examples" or the respective components).
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Work logged] (MATH-1371) Provide accelerated kmeans++ implementation
[ https://issues.apache.org/jira/browse/MATH-1371?focusedWorklogId=714131=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-714131 ] ASF GitHub Bot logged work on MATH-1371: Author: ASF GitHub Bot Created on: 25/Jan/22 00:38 Start Date: 25/Jan/22 00:38 Worklog Time Spent: 10m Work Description: asfgit closed pull request #35: URL: https://github.com/apache/commons-math/pull/35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 714131) Remaining Estimate: 0h Time Spent: 10m > Provide accelerated kmeans++ implementation > --- > > Key: MATH-1371 > URL: https://issues.apache.org/jira/browse/MATH-1371 > Project: Commons Math > Issue Type: Improvement >Reporter: Artem Barger >Assignee: Artem Barger >Priority: Major > Fix For: 4.X > > Attachments: ElkanKmeansPlusPlusClusterer.java, > ElkanKmeansPlusPlusClustererTest.java > > Time Spent: 10m > Remaining Estimate: 0h > > There is an updated version of kmeans++ algorithm available, which is > published in: Elkan, Charles. "Using the triangle inequality to accelerate > k-means." ICML. Vol. 3. 2003. paper. > The main essence is to boost the kmeans iterations by avoiding computation of > distances between centers and points when there is no need for that. For > example after the update cluster center haven't moved too far from the point > therefore no change in point assignment. The accelerated algorithm avoids > unnecessary distance calculations by applying the triangle inequality in two > different ways, and by keeping track of lower and upper bounds for distances > between points and centers. > Algorithm description is available in the paper. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [commons-math] asfgit closed pull request #35: [MATH-1371] - Implementation of Elkan optimizations for kmeans++.
asfgit closed pull request #35: URL: https://github.com/apache/commons-math/pull/35 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (POOL-404) No way to close evictor thread
[
https://issues.apache.org/jira/browse/POOL-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481433#comment-17481433
]
Patrick Barry commented on POOL-404:
The connections we borrow have a wrapper around them provided by lettuce. The
wrapper implements Closeable and so when you put in a try-with-resources block,
close is called and the wrapper code returns connection to pool.
> No way to close evictor thread
> --
>
> Key: POOL-404
> URL: https://issues.apache.org/jira/browse/POOL-404
> Project: Commons Pool
> Issue Type: Bug
>Affects Versions: 2.11.1
>Reporter: Patrick Barry
>Priority: Major
>
> Using GenericObjectPool> to help with
> lettuce client/redis connection management. I have everything shutting down
> cleanly except the commons-pool-evictor. I see this problem has been
> reported many times in the past, but all the changes are still not allowing
> this thread to shut down cleanly on close. I am using version 2.11.1. I
> have tried to code around this issue, but because EvictionTimer.java is so
> locked down, there is very little that can done to change the behavior of how
> this class interacts with GenericObjectPool.
> For this thread to shutdown, the taskMap has to be empty, which it never is
> in my case. So even though we call close() on the pool, this class fails to
> shutdown the embedded executor because it thinks it has more tasks.
> Looking at this code, it did remove 1 entry from taskMap, but we had many
> more in that map. Is there a way to clear this map, so it will allow this
> thread/executor to shutdown?
> {code:java}
> static synchronized void cancel(final BaseGenericObjectPool.Evictor
> evictor, final Duration timeout,
> final boolean restarting) {
> if (evictor != null) {
> evictor.cancel(); //why does this not interrupt!?
> remove(evictor);
> }
> if (!restarting && executor != null && taskMap.isEmpty()) { //<-- How do
> you force taskMap to be empty!?
> executor.shutdown();
> try {
> executor.awaitTermination(timeout.toMillis(),
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException e) {
> // Swallow
> // Significant API changes would be required to propagate this
> }
> executor.setCorePoolSize(0);
> executor = null;
> }
> }{code}
> }
> I had all these entries in the taskMap when trying to shut down:
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@73d4066e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@3c69362a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2412a42b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@45404d5
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@29138d3a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5cbe2654
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6dbcf214
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@496a31da
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@7c251f90
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@51841ac6
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5ba26eb0
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@435e60ff
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@17d32e9b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@66f0548d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2e6f610d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@1e86a5a7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@10afe71a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@741f8dbe
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@212dfd39
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@a2ddf26
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@49ede9c7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@65d57e4e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6daf7d37
> *Method calls:*
> pool.close() ->
> stopEvictor(); ->
> startEvictor(Duration.ofMillis(-1L)); ->
> EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false); ->
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (POOL-404) No way to close evictor thread
[
https://issues.apache.org/jira/browse/POOL-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481427#comment-17481427
]
Gary D. Gregory commented on POOL-404:
--
[~patrickjamesbarry]
FWIW: In you example, you borrow objects but neither return them or invalidate
them.
> No way to close evictor thread
> --
>
> Key: POOL-404
> URL: https://issues.apache.org/jira/browse/POOL-404
> Project: Commons Pool
> Issue Type: Bug
>Affects Versions: 2.11.1
>Reporter: Patrick Barry
>Priority: Major
>
> Using GenericObjectPool> to help with
> lettuce client/redis connection management. I have everything shutting down
> cleanly except the commons-pool-evictor. I see this problem has been
> reported many times in the past, but all the changes are still not allowing
> this thread to shut down cleanly on close. I am using version 2.11.1. I
> have tried to code around this issue, but because EvictionTimer.java is so
> locked down, there is very little that can done to change the behavior of how
> this class interacts with GenericObjectPool.
> For this thread to shutdown, the taskMap has to be empty, which it never is
> in my case. So even though we call close() on the pool, this class fails to
> shutdown the embedded executor because it thinks it has more tasks.
> Looking at this code, it did remove 1 entry from taskMap, but we had many
> more in that map. Is there a way to clear this map, so it will allow this
> thread/executor to shutdown?
> {code:java}
> static synchronized void cancel(final BaseGenericObjectPool.Evictor
> evictor, final Duration timeout,
> final boolean restarting) {
> if (evictor != null) {
> evictor.cancel(); //why does this not interrupt!?
> remove(evictor);
> }
> if (!restarting && executor != null && taskMap.isEmpty()) { //<-- How do
> you force taskMap to be empty!?
> executor.shutdown();
> try {
> executor.awaitTermination(timeout.toMillis(),
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException e) {
> // Swallow
> // Significant API changes would be required to propagate this
> }
> executor.setCorePoolSize(0);
> executor = null;
> }
> }{code}
> }
> I had all these entries in the taskMap when trying to shut down:
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@73d4066e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@3c69362a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2412a42b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@45404d5
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@29138d3a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5cbe2654
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6dbcf214
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@496a31da
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@7c251f90
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@51841ac6
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5ba26eb0
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@435e60ff
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@17d32e9b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@66f0548d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2e6f610d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@1e86a5a7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@10afe71a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@741f8dbe
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@212dfd39
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@a2ddf26
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@49ede9c7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@65d57e4e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6daf7d37
> *Method calls:*
> pool.close() ->
> stopEvictor(); ->
> startEvictor(Duration.ofMillis(-1L)); ->
> EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false); ->
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Closed] (POOL-404) No way to close evictor thread
[
https://issues.apache.org/jira/browse/POOL-404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Phil Steitz closed POOL-404.
Resolution: Not A Problem
We can open an enhancement request to add global pool management, but I do not
see a pool bug here.
> No way to close evictor thread
> --
>
> Key: POOL-404
> URL: https://issues.apache.org/jira/browse/POOL-404
> Project: Commons Pool
> Issue Type: Bug
>Affects Versions: 2.11.1
>Reporter: Patrick Barry
>Priority: Major
>
> Using GenericObjectPool> to help with
> lettuce client/redis connection management. I have everything shutting down
> cleanly except the commons-pool-evictor. I see this problem has been
> reported many times in the past, but all the changes are still not allowing
> this thread to shut down cleanly on close. I am using version 2.11.1. I
> have tried to code around this issue, but because EvictionTimer.java is so
> locked down, there is very little that can done to change the behavior of how
> this class interacts with GenericObjectPool.
> For this thread to shutdown, the taskMap has to be empty, which it never is
> in my case. So even though we call close() on the pool, this class fails to
> shutdown the embedded executor because it thinks it has more tasks.
> Looking at this code, it did remove 1 entry from taskMap, but we had many
> more in that map. Is there a way to clear this map, so it will allow this
> thread/executor to shutdown?
> {code:java}
> static synchronized void cancel(final BaseGenericObjectPool.Evictor
> evictor, final Duration timeout,
> final boolean restarting) {
> if (evictor != null) {
> evictor.cancel(); //why does this not interrupt!?
> remove(evictor);
> }
> if (!restarting && executor != null && taskMap.isEmpty()) { //<-- How do
> you force taskMap to be empty!?
> executor.shutdown();
> try {
> executor.awaitTermination(timeout.toMillis(),
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException e) {
> // Swallow
> // Significant API changes would be required to propagate this
> }
> executor.setCorePoolSize(0);
> executor = null;
> }
> }{code}
> }
> I had all these entries in the taskMap when trying to shut down:
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@73d4066e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@3c69362a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2412a42b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@45404d5
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@29138d3a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5cbe2654
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6dbcf214
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@496a31da
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@7c251f90
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@51841ac6
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5ba26eb0
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@435e60ff
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@17d32e9b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@66f0548d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2e6f610d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@1e86a5a7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@10afe71a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@741f8dbe
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@212dfd39
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@a2ddf26
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@49ede9c7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@65d57e4e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6daf7d37
> *Method calls:*
> pool.close() ->
> stopEvictor(); ->
> startEvictor(Duration.ofMillis(-1L)); ->
> EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false); ->
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (POOL-404) No way to close evictor thread
[
https://issues.apache.org/jira/browse/POOL-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481366#comment-17481366
]
Patrick Barry commented on POOL-404:
After adding thread safety code to our class(synchronized methods), I cannot
reproduce my issue. So I guess it is resolved on my end. I am still scratching
my head how I had so many entries in the taskMap and it would have been nice to
have a forceClose on the objectPool that would have closed down everything.
In regards to the weird naming issue on the pool's threads, I opened a ticket
with lettuce team.
https://github.com/lettuce-io/lettuce-core/issues/1972
> No way to close evictor thread
> --
>
> Key: POOL-404
> URL: https://issues.apache.org/jira/browse/POOL-404
> Project: Commons Pool
> Issue Type: Bug
>Affects Versions: 2.11.1
>Reporter: Patrick Barry
>Priority: Major
>
> Using GenericObjectPool> to help with
> lettuce client/redis connection management. I have everything shutting down
> cleanly except the commons-pool-evictor. I see this problem has been
> reported many times in the past, but all the changes are still not allowing
> this thread to shut down cleanly on close. I am using version 2.11.1. I
> have tried to code around this issue, but because EvictionTimer.java is so
> locked down, there is very little that can done to change the behavior of how
> this class interacts with GenericObjectPool.
> For this thread to shutdown, the taskMap has to be empty, which it never is
> in my case. So even though we call close() on the pool, this class fails to
> shutdown the embedded executor because it thinks it has more tasks.
> Looking at this code, it did remove 1 entry from taskMap, but we had many
> more in that map. Is there a way to clear this map, so it will allow this
> thread/executor to shutdown?
> {code:java}
> static synchronized void cancel(final BaseGenericObjectPool.Evictor
> evictor, final Duration timeout,
> final boolean restarting) {
> if (evictor != null) {
> evictor.cancel(); //why does this not interrupt!?
> remove(evictor);
> }
> if (!restarting && executor != null && taskMap.isEmpty()) { //<-- How do
> you force taskMap to be empty!?
> executor.shutdown();
> try {
> executor.awaitTermination(timeout.toMillis(),
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException e) {
> // Swallow
> // Significant API changes would be required to propagate this
> }
> executor.setCorePoolSize(0);
> executor = null;
> }
> }{code}
> }
> I had all these entries in the taskMap when trying to shut down:
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@73d4066e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@3c69362a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2412a42b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@45404d5
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@29138d3a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5cbe2654
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6dbcf214
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@496a31da
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@7c251f90
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@51841ac6
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5ba26eb0
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@435e60ff
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@17d32e9b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@66f0548d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2e6f610d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@1e86a5a7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@10afe71a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@741f8dbe
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@212dfd39
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@a2ddf26
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@49ede9c7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@65d57e4e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6daf7d37
> *Method calls:*
> pool.close() ->
> stopEvictor(); ->
> startEvictor(Duration.ofMillis(-1L)); ->
> EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false); ->
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (POOL-404) No way to close evictor thread
[
https://issues.apache.org/jira/browse/POOL-404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481215#comment-17481215
]
Phil Steitz commented on POOL-404:
--
[~ggregory] Can you see a scenario where multiple evictors can be created for
a single pool or where either of these are accessed without holding the guard
(evictionLock)? I can't make this happen and it looks to me like the
protection is good.
> No way to close evictor thread
> --
>
> Key: POOL-404
> URL: https://issues.apache.org/jira/browse/POOL-404
> Project: Commons Pool
> Issue Type: Bug
>Affects Versions: 2.11.1
>Reporter: Patrick Barry
>Priority: Major
>
> Using GenericObjectPool> to help with
> lettuce client/redis connection management. I have everything shutting down
> cleanly except the commons-pool-evictor. I see this problem has been
> reported many times in the past, but all the changes are still not allowing
> this thread to shut down cleanly on close. I am using version 2.11.1. I
> have tried to code around this issue, but because EvictionTimer.java is so
> locked down, there is very little that can done to change the behavior of how
> this class interacts with GenericObjectPool.
> For this thread to shutdown, the taskMap has to be empty, which it never is
> in my case. So even though we call close() on the pool, this class fails to
> shutdown the embedded executor because it thinks it has more tasks.
> Looking at this code, it did remove 1 entry from taskMap, but we had many
> more in that map. Is there a way to clear this map, so it will allow this
> thread/executor to shutdown?
> {code:java}
> static synchronized void cancel(final BaseGenericObjectPool.Evictor
> evictor, final Duration timeout,
> final boolean restarting) {
> if (evictor != null) {
> evictor.cancel(); //why does this not interrupt!?
> remove(evictor);
> }
> if (!restarting && executor != null && taskMap.isEmpty()) { //<-- How do
> you force taskMap to be empty!?
> executor.shutdown();
> try {
> executor.awaitTermination(timeout.toMillis(),
> TimeUnit.MILLISECONDS);
> } catch (final InterruptedException e) {
> // Swallow
> // Significant API changes would be required to propagate this
> }
> executor.setCorePoolSize(0);
> executor = null;
> }
> }{code}
> }
> I had all these entries in the taskMap when trying to shut down:
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@73d4066e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@3c69362a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2412a42b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@45404d5
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@29138d3a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5cbe2654
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6dbcf214
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@496a31da
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@7c251f90
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@51841ac6
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@5ba26eb0
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@435e60ff
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@17d32e9b
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@66f0548d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@2e6f610d
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@1e86a5a7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@10afe71a
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@741f8dbe
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@212dfd39
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@a2ddf26
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@49ede9c7
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@65d57e4e
> org.apache.commons.pool2.impl.BaseGenericObjectPool$Evictor@6daf7d37
> *Method calls:*
> pool.close() ->
> stopEvictor(); ->
> startEvictor(Duration.ofMillis(-1L)); ->
> EvictionTimer.cancel(evictor, evictorShutdownTimeoutDuration, false); ->
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
[jira] [Commented] (COMPRESS-606) Suspected memory leak
[ https://issues.apache.org/jira/browse/COMPRESS-606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17481113#comment-17481113 ] Gilles Sadowski commented on COMPRESS-606: -- bq. I use gdb Great. Please provide more details about the method/variable that seems to eat up that memory. A unit test should also help track the issue... > Suspected memory leak > - > > Key: COMPRESS-606 > URL: https://issues.apache.org/jira/browse/COMPRESS-606 > Project: Commons Compress > Issue Type: Improvement >Affects Versions: 1.21 >Reporter: hao ying >Priority: Major > > i try to use TarArchiveInputStream decompress a *tar.gz file. After > decompressing, the memory cost of java process from 50Mb to almost 300Mb. And > full gc cannot clear this part of memory. > I use gdb to check context of this memoy block. it seemd like a buffer. It is > out of the heap memory so gc cannot clear it. I cannot confirm if it is > really memory leak. > So, besides killing java process, how can I free this part of memory? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[GitHub] [commons-math] avijit-basak closed pull request #203: Feature/math 1563 rb fix
avijit-basak closed pull request #203: URL: https://github.com/apache/commons-math/pull/203 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [commons-math] avijit-basak commented on pull request #203: Feature/math 1563 rb fix
avijit-basak commented on pull request #203: URL: https://github.com/apache/commons-math/pull/203#issuecomment-1020012571 Closing this PR. An alternative new PR(#204) is created. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (COMPRESS-606) Suspected memory leak
[ https://issues.apache.org/jira/browse/COMPRESS-606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] hao ying updated COMPRESS-606: -- Description: i try to use TarArchiveInputStream decompress a *tar.gz file. After decompressing, the memory cost of java process from 50Mb to almost 300Mb. And full gc cannot clear this part of memory. I use gdb to check context of this memoy block. it seemd like a buffer. It is out of the heap memory so gc cannot clear it. I cannot confirm if it is really memory leak. So, besides killing java process, how can I free this part of memory? was: i try to use TarArchiveInputStream decompress a *tar.gz file. After decompressing, the memory cost of java process from 50Mb to almost 300Mb. And full gc cannot clear this part of memory. I use gdb to check context of this memoy block. it seemd like a buffer in order to decompressing. It is out of the heap memory so gc cannot clear it. I cannot confirm if it is really memory leak. So, besides killing java process, how can I free this part of memory? Priority: Major (was: Minor) > Suspected memory leak > - > > Key: COMPRESS-606 > URL: https://issues.apache.org/jira/browse/COMPRESS-606 > Project: Commons Compress > Issue Type: Improvement >Affects Versions: 1.21 >Reporter: hao ying >Priority: Major > > i try to use TarArchiveInputStream decompress a *tar.gz file. After > decompressing, the memory cost of java process from 50Mb to almost 300Mb. And > full gc cannot clear this part of memory. > I use gdb to check context of this memoy block. it seemd like a buffer. It is > out of the heap memory so gc cannot clear it. I cannot confirm if it is > really memory leak. > So, besides killing java process, how can I free this part of memory? -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Created] (COMPRESS-606) Suspected memory leak
hao ying created COMPRESS-606: - Summary: Suspected memory leak Key: COMPRESS-606 URL: https://issues.apache.org/jira/browse/COMPRESS-606 Project: Commons Compress Issue Type: Improvement Affects Versions: 1.21 Reporter: hao ying i try to use TarArchiveInputStream decompress a *tar.gz file. After decompressing, the memory cost of java process from 50Mb to almost 300Mb. And full gc cannot clear this part of memory. I use gdb to check context of this memoy block. it seemd like a buffer in order to decompressing. It is out of the heap memory so gc cannot clear it. I cannot confirm if it is really memory leak. So, besides killing java process, how can I free this part of memory? -- This message was sent by Atlassian Jira (v8.20.1#820001)
