[jira] [Comment Edited] (JEXL-399) Needs support more javascript syntax

[Xu Pengcheng (Jira)]

[ 
https://issues.apache.org/jira/browse/JEXL-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741235#comment-17741235
 ] 

Xu Pengcheng edited comment on JEXL-399 at 7/8/23 3:39 AM:
---

Thanks for your explanation.


was (Author: JIRAUSER294041):
Thanks for your explaination.

> Needs support more javascript syntax
> 
>
> Key: JEXL-399
> URL: https://issues.apache.org/jira/browse/JEXL-399
> Project: Commons JEXL
>  Issue Type: New Feature
>Reporter: Xu Pengcheng
>Priority: Major
>
> Hi, I am trying ask chatgpt to generate code for some cases, JEXL, as a 
> lightweight, secure, easy to communicate with host solution, I think is the 
> best option for java world as a bridge between chatgpt and app.
> Unfortunately, chatgpt seems is not familiar with Jexl syntax, so I tried to 
> ask chatgpt to write as javascript, I had implements most of the javascript 
> build-in objects (like Array, Object, Math, console, etc) for JEXL, 95% cases 
> generated by chatgpt can run successfully.
> However, some syntax error will cause the 5% failure, for example:
>  # a === b
>  # a !== b
>  # a = b ** 2
>  # const \{a, b} = x;
>  # a = \{x: 1, y: 2,}
>  # func = () => (\{"a":1, "b": 2})
>  # for (let k in map)
>  # for (let value of map)
> These syntax looks very reasonable, It will be great to support these 
> javascript syntax to let both chatgpt and developer easy to use.
> Thanks!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (JEXL-399) Needs support more javascript syntax

[Xu Pengcheng (Jira)]

[ 
https://issues.apache.org/jira/browse/JEXL-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741235#comment-17741235
 ] 

Xu Pengcheng commented on JEXL-399:
---

Thanks for your explaination.

> Needs support more javascript syntax
> 
>
> Key: JEXL-399
> URL: https://issues.apache.org/jira/browse/JEXL-399
> Project: Commons JEXL
>  Issue Type: New Feature
>Reporter: Xu Pengcheng
>Priority: Major
>
> Hi, I am trying ask chatgpt to generate code for some cases, JEXL, as a 
> lightweight, secure, easy to communicate with host solution, I think is the 
> best option for java world as a bridge between chatgpt and app.
> Unfortunately, chatgpt seems is not familiar with Jexl syntax, so I tried to 
> ask chatgpt to write as javascript, I had implements most of the javascript 
> build-in objects (like Array, Object, Math, console, etc) for JEXL, 95% cases 
> generated by chatgpt can run successfully.
> However, some syntax error will cause the 5% failure, for example:
>  # a === b
>  # a !== b
>  # a = b ** 2
>  # const \{a, b} = x;
>  # a = \{x: 1, y: 2,}
>  # func = () => (\{"a":1, "b": 2})
>  # for (let k in map)
>  # for (let value of map)
> These syntax looks very reasonable, It will be great to support these 
> javascript syntax to let both chatgpt and developer easy to use.
> Thanks!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741221#comment-17741221
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/8/23 12:27 AM:
---

Committed to master. Because allocate is accessed in and out of a lambed, it 
must be final, hence the object reference instead of a primitive.


was (Author: garydgregory):
Committed to master. Because allocate is accessed in and out of a lambed, it 
must be final, hence the object reference instead of primitive.

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741221#comment-17741221
 ] 

Gary D. Gregory commented on POOL-411:
--

Committed to master. Because allocate is accessed in and out of a lambed, it 
must be final, hence the object reference instead of primitive.

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Phil Steitz (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741220#comment-17741220
 ] 

Phil Steitz commented on POOL-411:
--

[~ggregory]  - looks great.  Great idea to use computeIfAbsent and to make the 
read on numInterested atomic.  I wish I understood better the sequence that 
gets the counter off, but this looks like it should help.  One small comment: 
the allocate variable added in register can just be a normal boolean, since it 
is only visible to the executing thread.  I would say go ahead and commit this 
change and we can keep testing.  

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-text] garydgregory merged pull request #441: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #441:
URL: https://github.com/apache/commons-text/pull/441


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-email] garydgregory merged pull request #151: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #151:
URL: https://github.com/apache/commons-email/pull/151


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741210#comment-17741210
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/7/23 11:12 PM:
---

The following allows {{testConcurrentBorrowAndClear_JiraComment17741156}} to 
pass locally for me in Eclipse, running it over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..5500f48 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,20 +834,17 @@
 lock.lock();
 final ObjectDeque objectDeque = poolMap.get(k);
 if (objectDeque != null) {
-final long numInterested = 
objectDeque.getNumInterested().decrementAndGet();
-if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0) {
-// Potential to remove key
-// Upgrade to write lock
-lock.unlock();
-lock = keyLock.writeLock();
-lock.lock();
-if (objectDeque.getCreateCount().get() == 0 && 
objectDeque.getNumInterested().get() == 0) {
-// NOTE: Keys must always be removed from both poolMap 
and
-// poolKeyList at the same time while protected by
-// keyLock.writeLock()
-poolMap.remove(k);
-poolKeyList.remove(k);
-}
+// Potential to remove key
+// Upgrade to write lock
+lock.unlock();
+lock = keyLock.writeLock();
+lock.lock();
+if (objectDeque.getNumInterested().decrementAndGet() == 0 && 
objectDeque.getCreateCount().get() == 0) {
+// NOTE: Keys must always be removed from both poolMap and
+// poolKeyList at the same time while protected by
+// keyLock.writeLock()
+poolMap.remove(k);
+poolKeyList.remove(k);
 }
 }
 } finally {
@@ -1385,16 +1383,17 @@
 lock.unlock();
 lock = keyLock.writeLock();
 lock.lock();
-objectDeque = poolMap.get(k);
-if (objectDeque == null) {
-objectDeque = new ObjectDeque<>(fairness);
-objectDeque.getNumInterested().incrementAndGet();
+final AtomicBoolean allocated = new AtomicBoolean(); 
+objectDeque = poolMap.computeIfAbsent(k, key -> {
+allocated.set(true);
+final ObjectDeque deque = new ObjectDeque<>(fairness);
+deque.getNumInterested().incrementAndGet();
 // NOTE: Keys must always be added to both poolMap and
 //   poolKeyList at the same time while protected by
-//   keyLock.writeLock()
-poolMap.put(k, objectDeque);
 poolKeyList.add(k);
-} else {
+return deque;
+});
+if (!allocated.get()) {
 objectDeque.getNumInterested().incrementAndGet();
 }
 } else {
 {noformat}


was (Author: garydgregory):
The following allows {{testConcurrentBorrowAndClear_JiraComment17741156}} to 
pass locally for me in Eclipse, running it over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b2a35a5 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,20 +834,19 @@
 lock.lock();
 final ObjectDeque objectDeque = poolMap.get(k);
 if (objectDeque != null) {
+// Potential to remove key
+

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741210#comment-17741210
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/7/23 11:09 PM:
---

The following allows {{testConcurrentBorrowAndClear_JiraComment17741156}} to 
pass locally for me in Eclipse, running it over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b2a35a5 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,20 +834,19 @@
 lock.lock();
 final ObjectDeque objectDeque = poolMap.get(k);
 if (objectDeque != null) {
+// Potential to remove key
+// Upgrade to write lock
+lock.unlock();
+lock = keyLock.writeLock();
+lock.lock();
 final long numInterested = 
objectDeque.getNumInterested().decrementAndGet();
-if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0) {
-// Potential to remove key
-// Upgrade to write lock
-lock.unlock();
-lock = keyLock.writeLock();
-lock.lock();
-if (objectDeque.getCreateCount().get() == 0 && 
objectDeque.getNumInterested().get() == 0) {
-// NOTE: Keys must always be removed from both poolMap 
and
-// poolKeyList at the same time while protected by
-// keyLock.writeLock()
-poolMap.remove(k);
-poolKeyList.remove(k);
-}
+if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0 && objectDeque.getCreateCount().get() == 0
+&& objectDeque.getNumInterested().get() == 0) {
+// NOTE: Keys must always be removed from both poolMap and
+//   poolKeyList at the same time while protected by
+//   keyLock.writeLock()
+poolMap.remove(k);
+poolKeyList.remove(k);
 }
 }
 } finally {
@@ -1385,16 +1385,17 @@
 lock.unlock();
 lock = keyLock.writeLock();
 lock.lock();
-objectDeque = poolMap.get(k);
-if (objectDeque == null) {
-objectDeque = new ObjectDeque<>(fairness);
-objectDeque.getNumInterested().incrementAndGet();
+final AtomicBoolean allocated = new AtomicBoolean(); 
+objectDeque = poolMap.computeIfAbsent(k, key -> {
+allocated.set(true);
+final ObjectDeque deque = new ObjectDeque<>(fairness);
+deque.getNumInterested().incrementAndGet();
 // NOTE: Keys must always be added to both poolMap and
 //   poolKeyList at the same time while protected by
-//   keyLock.writeLock()
-poolMap.put(k, objectDeque);
 poolKeyList.add(k);
-} else {
+return deque;
+});
+if (!allocated.get()) {
 objectDeque.getNumInterested().incrementAndGet();
 }
 } else {
 {noformat}


was (Author: garydgregory):
The following allows {{testConcurrentBorrowAndClear_JiraComment17741156}} to 
pass locally for me in Eclipse, running it over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b67cc00 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,13 +834,13 @@
             lock.lock();
             final ObjectDeque objectDeque = 

[jira] [Commented] (IMAGING-355) Large animated GIF takes too much heap memory in getMetadata

[Andrew Khoury (Jira)]

[ 
https://issues.apache.org/jira/browse/IMAGING-355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741211#comment-17741211
 ] 

Andrew Khoury commented on IMAGING-355:
---

Hi [~ggregory], yes I tested against master as well and the same issue occurs.  

Also, I was able to fix the issue using this change 
[here|https://github.com/andrewmkhoury/commons-imaging/commit/d4a5f6a039e8ccf232750d5427f88638162db3f1]
{code:java}
readFile(byteSource, true); // instead of false to skip storing the imageData 
which isn't required for getMetadata and getImageInfo{code}
 

> Large animated GIF takes too much heap memory in getMetadata
> 
>
> Key: IMAGING-355
> URL: https://issues.apache.org/jira/browse/IMAGING-355
> Project: Commons Imaging
>  Issue Type: Bug
>  Components: Format: GIF
>Affects Versions: 1.0-alpha3
>Reporter: Andrew Khoury
>Priority: Major
> Attachments: commons-imaging-test.gif, 
> image-2023-06-29-15-18-17-076.png, screenshot-1.png
>
>
> When calling ImageParser.getMetadata on large animated gif files, the java 
> heap consumption is extremely high.
> For example, see the test project I created:
> [https://github.com/andrewmkhoury/commons-imaging-gif-test]
> When calling ImageParser.getMetadata on the attached 5MB gif 
> [^commons-imaging-test.gif], it uses ~1.5GB of heap space.  When the max heap 
> is set to -Xmx1488M or lower it fails with this exception. When the heap is 
> set to -Xmx1489M it works.
> {code:java}
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>   at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
>   at 
> java.base/java.io.ByteArrayOutputStream.toByteArray(ByteArrayOutputStream.java:211)
>   at 
> org.apache.commons.imaging.mylzw.MyLzwDecompressor.decompress(MyLzwDecompressor.java:143)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.readImageDescriptor(GifImageParser.java:881)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.readBlocks(GifImageParser.java:596)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:696)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.readFile(GifImageParser.java:680)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:485)
>   at 
> org.apache.commons.imaging.formats.gif.GifImageParser.getMetadata(GifImageParser.java:58)
>   at 
> org.apache.commons.imaging.ImageParser.getMetadata(ImageParser.java:832)
>   at Test.main(Test.java:28)
> {code}
> To generate the large gif file I did the following:
> 1. Install ffmpeg and gifsicle
> 2. Use quicktime to create a screen recording
> 3. Generate a gif out of the screen recording
> {code:bash}
> ffmpeg -i ~/Desktop/Screen\ Recording\ 2023-06-29\ at\ 12.09.21\ PM.mov 
> -pix_fmt rgb8 -r 10 commons-imaging-test.gif && gifsicle -O3 
> commons-imaging-test.gif -o commons-imaging-test.gif
> {code}
> To run the test program:
> {code:bash}
> git clone g...@github.com:andrewmkhoury/commons-imaging-gif-test.git
> cd commons-imaging-gif-test
> mvn assembly:assembly
> java -Xmx1g -jar target/IMAGING-test-0.0.1-SNAPSHOT-jar-with-dependencies.jar 
> commons-imaging-test.gif{code}
> Heap analysis via Eclipse MAT shows that the ImageDescriptor.imageData 
> storing the bytes of each frame is the cause of the problem:
> !image-2023-06-29-15-18-17-076.png|width=169,height=93!!screenshot-1.png|width=226,height=86!



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741210#comment-17741210
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/7/23 10:53 PM:
---

The following allows {{testConcurrentBorrowAndClear_JiraComment17741156}} to 
pass locally for me in Eclipse, running it over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b67cc00 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,13 +834,13 @@
             lock.lock();
             final ObjectDeque objectDeque = poolMap.get(k);
             if (objectDeque != null) {
+                // Potential to remove key
+                // Upgrade to write lock
+                lock.unlock();
+                lock = keyLock.writeLock();
+                lock.lock();
                 final long numInterested = 
objectDeque.getNumInterested().decrementAndGet();
                 if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0) {
-                    // Potential to remove key
-                    // Upgrade to write lock
-                    lock.unlock();
-                    lock = keyLock.writeLock();
-                    lock.lock();
                     if (objectDeque.getCreateCount().get() == 0 && 
objectDeque.getNumInterested().get() == 0) {
                         // NOTE: Keys must always be removed from both poolMap 
and
                         // poolKeyList at the same time while protected by
@@ -1385,16 +1386,17 @@
                 lock.unlock();
                 lock = keyLock.writeLock();
                 lock.lock();
-                objectDeque = poolMap.get(k);
-                if (objectDeque == null) {
-                    objectDeque = new ObjectDeque<>(fairness);
-                    objectDeque.getNumInterested().incrementAndGet();
+                final AtomicBoolean allocated = new AtomicBoolean(); 
+                objectDeque = poolMap.computeIfAbsent(k, key -> {
+                    allocated.set(true);
+                    final ObjectDeque deque = new ObjectDeque<>(fairness);
+                    deque.getNumInterested().incrementAndGet();
                     // NOTE: Keys must always be added to both poolMap and
                     //       poolKeyList at the same time while protected by
-                    //       keyLock.writeLock()
-                    poolMap.put(k, objectDeque);
                     poolKeyList.add(k);
-                } else {
+                    return deque;
+                });
+                if (!allocated.get()) {
                     objectDeque.getNumInterested().incrementAndGet();
                 }
             } else {
 {noformat}


was (Author: garydgregory):
The following allows the test to pass locally for me in Eclipse, running it 
over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b67cc00 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,13 +834,13 @@
             lock.lock();
             final ObjectDeque objectDeque = poolMap.get(k);
             if (objectDeque != null) {
+                // Potential to remove key
+                // Upgrade to write lock
+                lock.unlock();
+                lock = keyLock.writeLock();
+                lock.lock();
                 final long numInterested = 
objectDeque.getNumInterested().decrementAndGet();
                 if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0) {
-                    // Potential to remove key
-                    // Upgrade to write lock
-                    lock.unlock();
-                    lock = keyLock.writeLock();
-                    lock.lock();
                     if (objectDeque.getCreateCount().get() == 0 && 
objectDeque.getNumInterested().get() == 0) {
           

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741210#comment-17741210
 ] 

Gary D. Gregory commented on POOL-411:
--

The following allows the test to pass locally for me in Eclipse, running it 
over and over:
{noformat}
diff --git 
a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java 
b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
index 0f8a275..b67cc00 100644
--- a/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
+++ b/src/main/java/org/apache/commons/pool2/impl/GenericKeyedObjectPool.java
@@ -30,6 +30,7 @@
 import java.util.TreeMap;
 import java.util.concurrent.ConcurrentHashMap;
 import java.util.concurrent.TimeUnit;
+import java.util.concurrent.atomic.AtomicBoolean;
 import java.util.concurrent.atomic.AtomicInteger;
 import java.util.concurrent.atomic.AtomicLong;
 import java.util.concurrent.locks.Lock;
@@ -833,13 +834,13 @@
             lock.lock();
             final ObjectDeque objectDeque = poolMap.get(k);
             if (objectDeque != null) {
+                // Potential to remove key
+                // Upgrade to write lock
+                lock.unlock();
+                lock = keyLock.writeLock();
+                lock.lock();
                 final long numInterested = 
objectDeque.getNumInterested().decrementAndGet();
                 if (numInterested == 0 && objectDeque.getCreateCount().get() 
== 0) {
-                    // Potential to remove key
-                    // Upgrade to write lock
-                    lock.unlock();
-                    lock = keyLock.writeLock();
-                    lock.lock();
                     if (objectDeque.getCreateCount().get() == 0 && 
objectDeque.getNumInterested().get() == 0) {
                         // NOTE: Keys must always be removed from both poolMap 
and
                         // poolKeyList at the same time while protected by
@@ -1385,16 +1386,17 @@
                 lock.unlock();
                 lock = keyLock.writeLock();
                 lock.lock();
-                objectDeque = poolMap.get(k);
-                if (objectDeque == null) {
-                    objectDeque = new ObjectDeque<>(fairness);
-                    objectDeque.getNumInterested().incrementAndGet();
+                final AtomicBoolean allocated = new AtomicBoolean(); 
+                objectDeque = poolMap.computeIfAbsent(k, key -> {
+                    allocated.set(true);
+                    final ObjectDeque deque = new ObjectDeque<>(fairness);
+                    deque.getNumInterested().incrementAndGet();
                     // NOTE: Keys must always be added to both poolMap and
                     //       poolKeyList at the same time while protected by
-                    //       keyLock.writeLock()
-                    poolMap.put(k, objectDeque);
                     poolKeyList.add(k);
-                } else {
+                    return deque;
+                });
+                if (!allocated.get()) {
                     objectDeque.getNumInterested().incrementAndGet();
                 }
             } else {
 {noformat}

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-statistics] kinow commented on a diff in pull request #46: [STATISTICS-71]: Add base interfaces for all statistic implementations.

[via GitHub]

kinow commented on code in PR #46:
URL: https://github.com/apache/commons-statistics/pull/46#discussion_r1256568342


##
commons-statistics-descriptive/src/main/java/org/apache/commons/statistics/descriptive/Statistic.java:
##
@@ -0,0 +1,28 @@
+package org.apache.commons.statistics.descriptive;
+
+/**
+ * Represents a statistic being computed.
+ */
+public enum Statistic {
+MIN,
+
+MAX,
+
+SUM,
+
+MEAN,
+
+SUM_OF_LOGS,
+
+SUM_OF_SQUARES,
+
+VARIANCE,
+
+POPULATION_VARIANCE,
+
+STANDARD_DEVIATION,
+
+GEOMETRIC_MEAN,
+
+QUADRATIC_MEAN

Review Comment:
   I believe the statistic enum values can also be documented.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741196#comment-17741196
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/7/23 9:25 PM:
--

I refactored the test and added 
{{testConcurrentBorrowAndClear_JiraComment17741156}} to see how the test will 
behave on the CI. It was OK for me locally until it wasn't:
{noformat}
org.opentest4j.AssertionFailedError
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:46)
at org.junit.jupiter.api.Assertions.fail(Assertions.java:158)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.lambda$2(TestGenericKeyedObjectPool.java:1090)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.testConcurrentBorrowAndClear(TestGenericKeyedObjectPool.java:1086)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.testConcurrentBorrowAndClear_JiraComment17741156(TestGenericKeyedObjectPool.java:1034)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at 
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at 
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741196#comment-17741196
 ] 

Gary D. Gregory edited comment on POOL-411 at 7/7/23 9:24 PM:
--

I refactored the test and added 
{{testConcurrentBorrowAndClear_JiraComment17741156}} to see how the test will 
behave on the CI. It's OK for me locally so far until it did not:
{noformat}
org.opentest4j.AssertionFailedError
at org.junit.jupiter.api.AssertionUtils.fail(AssertionUtils.java:46)
at org.junit.jupiter.api.Assertions.fail(Assertions.java:158)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.lambda$2(TestGenericKeyedObjectPool.java:1090)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.testConcurrentBorrowAndClear(TestGenericKeyedObjectPool.java:1086)
at 
org.apache.commons.pool2.impl.TestGenericKeyedObjectPool.testConcurrentBorrowAndClear_JiraComment17741156(TestGenericKeyedObjectPool.java:1034)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
at 
org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at 
org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
  

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Gary D. Gregory (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741196#comment-17741196
 ] 

Gary D. Gregory commented on POOL-411:
--

I refactored the test and added 
{{testConcurrentBorrowAndClear_JiraComment17741156}} to see how the test will 
behave on the CI. It's OK for me locally so far.

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256480780


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilter.java:
##
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+import java.util.function.Predicate;
+
+/**
+ * Layered Bloom filters are described in Zhiwang, Cen; Jungang, Xu; Jian, Sun
+ * (2010), "A multi-layer Bloom filter for duplicated URL detection", Proc. 3rd
+ * International Conference on Advanced Computer Theory and Engineering (ICACTE
+ * 2010), vol. 1, pp. V1–586–V1–591, doi:10.1109/ICACTE.2010.5578947, ISBN
+ * 978-1-4244-6539-2, S2CID 3108985
+ * 
+ * In short, Layered Bloom filter contains several bloom filters arranged in
+ * layers.
+ * 
+ * 
+ * When membership in the filter is checked each layer in turn is checked
+ * and if a match is found {@code true} is returned.
+ * When merging each bloom filter is merged into the last newest filter in
+ * the list of layers.
+ * When questions of cardinality are asked the cardinality of the union of
+ * the enclosed Bloom filters is used.
+ * 
+ * The net result is that the layered Bloom filter can be populated with more
+ * items than the Shape would indicate and yet still return a false positive
+ * rate in line with the Shape and not the over population.
+ * 
+ * This implementation uses a LayerManager to handle the manipulation of the
+ * layers.
+ * 
+ * 
+ * Level 0 is the oldest layer and the highest level is the newest.
+ * There is always at least one enclosed filter.
+ * The newest filter is the {@code target} into which merges are performed.
+ * Whenever the target is retrieved, or a {@code merge} operation is
+ * performed the code checks if any older layers should be removed, and if so
+ * removes them. It also checks it a new layer should be added, and if so adds
+ * it and sets the {@code target} before the operation.
+ * 
+ */
+public class LayeredBloomFilter implements BloomFilter, BloomFilterProducer {
+private final Shape shape;
+private LayerManager layerManager;
+
+/**
+ * Creates a fixed size layered bloom filter that adds new filters to the 
list,
+ * but never merges them. List will never exceed maxDepth. As additional 
filters
+ * are added earlier filters are removed.
+ *
+ * @param shapeThe shape for the enclosed Bloom filters
+ * @param maxDepth The maximum depth of layers.
+ * @return An empty layered Bloom filter of the specified shape and depth.
+ */
+public static LayeredBloomFilter fixed(final Shape shape, int maxDepth) {
+LayerManager manager = 
LayerManager.builder().withExtendCheck(LayerManager.ExtendCheck.ADVANCE_ON_POPULATED)
+
.withCleanup(LayerManager.Cleanup.onMaxSize(maxDepth)).withSuplier(() -> new 
SimpleBloomFilter(shape))
+.build();
+return new LayeredBloomFilter(shape, manager);
+}
+
+/**
+ * Constructor.
+ *
+ * @param shapethe Shape of the enclosed Bloom filters
+ * @param layerManager the LayerManager to manage the layers.
+ */
+public LayeredBloomFilter(Shape shape, LayerManager layerManager) {
+this.shape = shape;
+this.layerManager = layerManager;
+}
+
+@Override
+public LayeredBloomFilter copy() {
+return new LayeredBloomFilter(shape, layerManager.copy());
+}
+
+/**
+ * Gets the depth of the deepest layer.
+ *
+ * @return the depth of the deepest layer.
+ */
+public final int getDepth() {
+return layerManager.getDepth();
+}
+
+/**
+ * Gets the Bloom filter at the specified depth
+ *
+ * @param depth the depth of the filter to return.
+ * @return the Bloom filter at the specified depth.
+ * @throws NoSuchElementException if depth is not in the range 
[0,getDepth())
+ */
+public BloomFilter get(int depth) {
+return 

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256480417


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayerManager.java:
##
@@ -0,0 +1,349 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.LinkedList;
+import java.util.NoSuchElementException;
+import java.util.function.Consumer;
+import java.util.function.Predicate;
+import java.util.function.Supplier;
+
+/**
+ * Implementation of the methods to manage the Layers in a Layered Bloom 
filter.
+ * 
+ * The manager comprises a list of Bloom filters that are managed based on
+ * various rules. The last filter in the list is known as the {@code target} 
and
+ * is the filter into which merges are performed. The Layered manager utilizes
+ * three methods to manage the list.
+ * 
+ * 
+ * ExtendCheck - A Predicate that if true causes a new Bloom filter to be
+ * created as the new target.
+ * FilterSupplier - A Supplier that produces empty Bloom filters to be used
+ * as a new target.
+ * Cleanup - A Consumer of a LinkedList of BloomFilter that removes any
+ * expired or out dated filters from the list.
+ * 
+ * 
+ * When extendCheck returns {@code true} the following steps are taken:
+ * 
+ * 
+ * If the current target is empty it is removed.
+ * {@code Cleanup} is called
+ * {@code FilterSuplier} is executed and the new filter added to the list 
as
+ * the {@code target} filter.
+ * 
+ *
+ * @since 4.5
+ */
+public class LayerManager implements BloomFilterProducer {
+
+/**
+ * Static methods an variable for standard extend checks.
+ *
+ */
+public static class ExtendCheck {
+private ExtendCheck() {
+}
+
+/**
+ * Advances the target once a merge has been performed.
+ */
+public static final Predicate ADVANCE_ON_POPULATED = lm 
-> {
+return !lm.filters.isEmpty() && 
!lm.filters.peekLast().forEachBitMap(y -> y == 0);
+};
+
+/**
+ * Does not automatically advance the target. next() must be called 
directly to
+ * perform the advance.
+ */
+public static final Predicate NEVER_ADVANCE = x -> false;
+
+/**
+ * Calculates the estimated number of Bloom filters (n) that have been 
merged
+ * into the target and compares that with the estimated maximum 
expected n based
+ * on the shape. If the target is full then a new target is created.
+ *
+ * @param shape The shape of the filters in the LayerManager.
+ * @return A Predicate suitable for the LayerManager extendCheck 
parameter.
+ */
+public static final Predicate 
advanceOnCalculatedFull(Shape shape) {
+return advanceOnSaturation(shape.estimateMaxN());
+}
+
+/**
+ * Creates a new target after a specific number of filters have been 
added to
+ * the current target.
+ *
+ * @param breakAt the number of filters to merge into each filter in 
the list.
+ * @return A Predicate suitable for the LayerManager extendCheck 
parameter.
+ */
+public static Predicate advanceOnCount(int breakAt) {
+return new Predicate() {
+int count = 0;
+
+@Override
+public boolean test(LayerManager filter) {
+return ++count % breakAt == 0;
+}
+};
+}
+
+/**
+ * Creates a new target after the current target is saturated. 
Saturation is
+ * defined as the estimated N of the target Bloom filter being greater 
than the
+ * maxN specified.
+ * 
+ * This method uses the integer estimation found in the Bloom filter. 
To use the
+ * estimation from the Shape use the double version of this function.
+ *
+ * @param maxN the maximum number of estimated items in the filter.
+ * @return A Predicate suitable for the LayerManager extendCheck 
parameter.
+ */
+public static final Predicate advanceOnSaturation(int 
maxN) {
+return new 

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256479009


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayerManager.java:
##
@@ -0,0 +1,349 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.LinkedList;
+import java.util.NoSuchElementException;
+import java.util.function.Consumer;
+import java.util.function.Predicate;
+import java.util.function.Supplier;
+
+/**
+ * Implementation of the methods to manage the Layers in a Layered Bloom 
filter.
+ * 
+ * The manager comprises a list of Bloom filters that are managed based on
+ * various rules. The last filter in the list is known as the {@code target} 
and
+ * is the filter into which merges are performed. The Layered manager utilizes
+ * three methods to manage the list.
+ * 
+ * 
+ * ExtendCheck - A Predicate that if true causes a new Bloom filter to be
+ * created as the new target.
+ * FilterSupplier - A Supplier that produces empty Bloom filters to be used
+ * as a new target.
+ * Cleanup - A Consumer of a LinkedList of BloomFilter that removes any
+ * expired or out dated filters from the list.
+ * 
+ * 
+ * When extendCheck returns {@code true} the following steps are taken:
+ * 
+ * 
+ * If the current target is empty it is removed.
+ * {@code Cleanup} is called
+ * {@code FilterSuplier} is executed and the new filter added to the list 
as
+ * the {@code target} filter.
+ * 
+ *
+ * @since 4.5
+ */
+public class LayerManager implements BloomFilterProducer {
+
+/**
+ * Static methods an variable for standard extend checks.
+ *
+ */
+public static class ExtendCheck {

Review Comment:
   Done



##
src/main/java/org/apache/commons/collections4/bloomfilter/LayerManager.java:
##
@@ -0,0 +1,349 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.LinkedList;
+import java.util.NoSuchElementException;
+import java.util.function.Consumer;
+import java.util.function.Predicate;
+import java.util.function.Supplier;
+
+/**
+ * Implementation of the methods to manage the Layers in a Layered Bloom 
filter.
+ * 
+ * The manager comprises a list of Bloom filters that are managed based on
+ * various rules. The last filter in the list is known as the {@code target} 
and
+ * is the filter into which merges are performed. The Layered manager utilizes
+ * three methods to manage the list.
+ * 
+ * 
+ * ExtendCheck - A Predicate that if true causes a new Bloom filter to be
+ * created as the new target.
+ * FilterSupplier - A Supplier that produces empty Bloom filters to be used
+ * as a new target.
+ * Cleanup - A Consumer of a LinkedList of BloomFilter that removes any
+ * expired or out dated filters from the list.
+ * 
+ * 
+ * When extendCheck returns {@code true} the following steps are taken:
+ * 
+ * 
+ * If the current target is empty it is removed.
+ * {@code Cleanup} is called
+ * {@code FilterSuplier} is executed and the new filter added to the list 
as
+ * the {@code target} filter.
+ * 
+ *
+ * @since 4.5
+ */
+public class LayerManager implements BloomFilterProducer {
+
+/**
+ * Static methods an variable for standard extend checks.
+ *
+ */
+public static class ExtendCheck {
+private ExtendCheck() {
+}
+
+/**
+ * 

[GitHub] [commons-ognl] garydgregory merged pull request #129: Bump h2 from 2.1.214 to 2.2.220

[via GitHub]

garydgregory merged PR #129:
URL: https://github.com/apache/commons-ognl/pull/129


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-exec] garydgregory merged pull request #110: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #110:
URL: https://github.com/apache/commons-exec/pull/110


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-daemon] garydgregory merged pull request #98: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #98:
URL: https://github.com/apache/commons-daemon/pull/98


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] garydgregory merged pull request #405: Bump guava-testlib from 32.1.0-jre to 32.1.1-jre

[via GitHub]

garydgregory merged PR #405:
URL: https://github.com/apache/commons-collections/pull/405


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-imaging] garydgregory merged pull request #300: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #300:
URL: https://github.com/apache/commons-imaging/pull/300


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Phil Steitz (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741156#comment-17741156
 ] 

Phil Steitz commented on POOL-411:
--

If I make the following changes to testConcurrentBorrowAnd clear, the test 
fails about one in three times against the code in master now (as of 
108735e2e262bad80f1f18abe9feae93b5db0624)
 # Reduce threadCount and taskCount to 2 
 # Increase borrow and clear cycles to 5000
 # Eliminate Thread.yields

 

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-statistics] ani5rudh opened a new pull request, #46: [STATISTICS-71]: Add base interfaces for all statistic implementations.

[via GitHub]

ani5rudh opened a new pull request, #46:
URL: https://github.com/apache/commons-statistics/pull/46

   (no comment)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256043841


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilter.java:
##
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+import java.util.function.Predicate;
+
+/**
+ * Layered Bloom filters are described in Zhiwang, Cen; Jungang, Xu; Jian, Sun
+ * (2010), "A multi-layer Bloom filter for duplicated URL detection", Proc. 3rd
+ * International Conference on Advanced Computer Theory and Engineering (ICACTE
+ * 2010), vol. 1, pp. V1–586–V1–591, doi:10.1109/ICACTE.2010.5578947, ISBN
+ * 978-1-4244-6539-2, S2CID 3108985
+ * 
+ * In short, Layered Bloom filter contains several bloom filters arranged in
+ * layers.
+ * 
+ * 
+ * When membership in the filter is checked each layer in turn is checked
+ * and if a match is found {@code true} is returned.
+ * When merging each bloom filter is merged into the last newest filter in
+ * the list of layers.
+ * When questions of cardinality are asked the cardinality of the union of
+ * the enclosed Bloom filters is used.
+ * 
+ * The net result is that the layered Bloom filter can be populated with more
+ * items than the Shape would indicate and yet still return a false positive
+ * rate in line with the Shape and not the over population.
+ * 
+ * This implementation uses a LayerManager to handle the manipulation of the
+ * layers.
+ * 
+ * 
+ * Level 0 is the oldest layer and the highest level is the newest.
+ * There is always at least one enclosed filter.
+ * The newest filter is the {@code target} into which merges are performed.
+ * Whenever the target is retrieved, or a {@code merge} operation is
+ * performed the code checks if any older layers should be removed, and if so
+ * removes them. It also checks it a new layer should be added, and if so adds
+ * it and sets the {@code target} before the operation.
+ * 
+ */
+public class LayeredBloomFilter implements BloomFilter, BloomFilterProducer {
+private final Shape shape;
+private LayerManager layerManager;
+
+/**
+ * Creates a fixed size layered bloom filter that adds new filters to the 
list,
+ * but never merges them. List will never exceed maxDepth. As additional 
filters
+ * are added earlier filters are removed.
+ *
+ * @param shapeThe shape for the enclosed Bloom filters
+ * @param maxDepth The maximum depth of layers.
+ * @return An empty layered Bloom filter of the specified shape and depth.
+ */
+public static LayeredBloomFilter fixed(final Shape shape, int maxDepth) {
+LayerManager manager = 
LayerManager.builder().withExtendCheck(LayerManager.ExtendCheck.ADVANCE_ON_POPULATED)
+
.withCleanup(LayerManager.Cleanup.onMaxSize(maxDepth)).withSuplier(() -> new 
SimpleBloomFilter(shape))
+.build();
+return new LayeredBloomFilter(shape, manager);
+}
+
+/**
+ * Constructor.
+ *
+ * @param shapethe Shape of the enclosed Bloom filters
+ * @param layerManager the LayerManager to manage the layers.
+ */
+public LayeredBloomFilter(Shape shape, LayerManager layerManager) {
+this.shape = shape;
+this.layerManager = layerManager;
+}
+
+@Override
+public LayeredBloomFilter copy() {
+return new LayeredBloomFilter(shape, layerManager.copy());
+}
+
+/**
+ * Gets the depth of the deepest layer.
+ *
+ * @return the depth of the deepest layer.
+ */
+public final int getDepth() {
+return layerManager.getDepth();
+}
+
+/**
+ * Gets the Bloom filter at the specified depth
+ *
+ * @param depth the depth of the filter to return.
+ * @return the Bloom filter at the specified depth.
+ * @throws NoSuchElementException if depth is not in the range 
[0,getDepth())
+ */
+public BloomFilter get(int depth) {
+return 

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256041156


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilter.java:
##
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+import java.util.function.Predicate;
+
+/**
+ * Layered Bloom filters are described in Zhiwang, Cen; Jungang, Xu; Jian, Sun
+ * (2010), "A multi-layer Bloom filter for duplicated URL detection", Proc. 3rd
+ * International Conference on Advanced Computer Theory and Engineering (ICACTE
+ * 2010), vol. 1, pp. V1–586–V1–591, doi:10.1109/ICACTE.2010.5578947, ISBN
+ * 978-1-4244-6539-2, S2CID 3108985
+ * 
+ * In short, Layered Bloom filter contains several bloom filters arranged in
+ * layers.
+ * 
+ * 
+ * When membership in the filter is checked each layer in turn is checked
+ * and if a match is found {@code true} is returned.
+ * When merging each bloom filter is merged into the last newest filter in
+ * the list of layers.
+ * When questions of cardinality are asked the cardinality of the union of
+ * the enclosed Bloom filters is used.
+ * 
+ * The net result is that the layered Bloom filter can be populated with more
+ * items than the Shape would indicate and yet still return a false positive
+ * rate in line with the Shape and not the over population.
+ * 
+ * This implementation uses a LayerManager to handle the manipulation of the
+ * layers.
+ * 
+ * 
+ * Level 0 is the oldest layer and the highest level is the newest.
+ * There is always at least one enclosed filter.
+ * The newest filter is the {@code target} into which merges are performed.
+ * Whenever the target is retrieved, or a {@code merge} operation is
+ * performed the code checks if any older layers should be removed, and if so
+ * removes them. It also checks it a new layer should be added, and if so adds
+ * it and sets the {@code target} before the operation.
+ * 
+ */
+public class LayeredBloomFilter implements BloomFilter, BloomFilterProducer {
+private final Shape shape;
+private LayerManager layerManager;
+
+/**
+ * Creates a fixed size layered bloom filter that adds new filters to the 
list,
+ * but never merges them. List will never exceed maxDepth. As additional 
filters
+ * are added earlier filters are removed.
+ *
+ * @param shapeThe shape for the enclosed Bloom filters
+ * @param maxDepth The maximum depth of layers.
+ * @return An empty layered Bloom filter of the specified shape and depth.
+ */
+public static LayeredBloomFilter fixed(final Shape shape, int maxDepth) {
+LayerManager manager = 
LayerManager.builder().withExtendCheck(LayerManager.ExtendCheck.ADVANCE_ON_POPULATED)
+
.withCleanup(LayerManager.Cleanup.onMaxSize(maxDepth)).withSuplier(() -> new 
SimpleBloomFilter(shape))
+.build();
+return new LayeredBloomFilter(shape, manager);
+}
+
+/**
+ * Constructor.
+ *
+ * @param shapethe Shape of the enclosed Bloom filters
+ * @param layerManager the LayerManager to manage the layers.
+ */
+public LayeredBloomFilter(Shape shape, LayerManager layerManager) {
+this.shape = shape;
+this.layerManager = layerManager;
+}
+
+@Override
+public LayeredBloomFilter copy() {
+return new LayeredBloomFilter(shape, layerManager.copy());
+}
+
+/**
+ * Gets the depth of the deepest layer.
+ *
+ * @return the depth of the deepest layer.
+ */
+public final int getDepth() {
+return layerManager.getDepth();
+}
+
+/**
+ * Gets the Bloom filter at the specified depth
+ *
+ * @param depth the depth of the filter to return.
+ * @return the Bloom filter at the specified depth.
+ * @throws NoSuchElementException if depth is not in the range 
[0,getDepth())
+ */
+public BloomFilter get(int depth) {

Review Comment:
   Because 

[jira] [Comment Edited] (POOL-411) NPE when deregistering key at end of borrow

[Phil Steitz (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741106#comment-17741106
 ] 

Phil Steitz edited comment on POOL-411 at 7/7/23 3:50 PM:
--

[~sebb] - thanks for looking at this.  You are right - the scenario I posted 
(and deleted) is impossible.  The jdk docs are a little vague, but I have 
confirmed you are right - the thread trying to get the write lock will have to 
block until any readers release their read locks.  The null guard that was 
added in deregister prevents the exception reported above from happening, but 
the test case that Gary added now fails sporadically (very rarely) with this in 
the stack trace:

{{java.lang.NullPointerException: Cannot invoke 
"org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getIdleObjects()"
 because the return value of "java.util.Map.get(Object)" is null}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addIdleObject(GenericKeyedObjectPool.java:308)}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addObject(GenericKeyedObjectPool.java:333)}}
{{    at org.apache.com}}

The problem with above is that addObject calls register on the key before 
invoking addIdleObject so any deregister that happens in between should not be 
able to delete the pool.

The intent is for register to signal that a thread is "interested" in a key so 
others should refrain from deleting it.  It will also create an empty pool if 
there is no pool under the given key. It returns the pool under the registered 
key.  Calling deregister says the thread is no longer interested and also 
checks to see if the associated keyed pool is empty, has no objects under 
management and has no other threads interested in it.  In that case, it deletes 
it.  This setup ensures that a) when you call register, you can be sure that 
you will get a non-null pool and that pool will not be deleted until after you 
deregister it b) empty pools with no objects under management and no interested 
threads get cleaned up. Thread interest is tracked in the numInterested counter 
attached to the pool itself, so what must be going on is that counter must be 
getting corrupted somehow.  One way that could happen is if deregister is 
called twice for one registration.}}{}}}


was (Author: psteitz):
[~sebb] - thanks for looking at this.  You are right - the scenario I posted 
(and deleted) is impossible.  The jdk docs are a little vague, but I have 
confirmed you are right - the thread trying to get the write lock will have to 
block until any readers release their read locks.  The null guard that was 
added in deregister prevents the exception reported above from happening, but 
the test case that Gary added to now fails sporadically (very rarely) with this 
in the stack trace:

{{java.lang.NullPointerException: Cannot invoke 
"org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getIdleObjects()"
 because the return value of "java.util.Map.get(Object)" is null}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addIdleObject(GenericKeyedObjectPool.java:308)}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addObject(GenericKeyedObjectPool.java:333)}}
{{    at org.apache.com}}

The problem with above is that addObject calls register on the key before 
invoking addIdleObject so any deregister that happens in between should not be 
able to delete the pool.

The intent is for register to signal that a thread is "interested" in a key so 
others should refrain from deleting it.  It will also create an empty pool if 
there is no pool under the given key. It returns the pool under the registered 
key.  Calling deregister says the thread is no longer interested and also 
checks to see if the associated keyed pool is empty, has no objects under 
management and has no other threads interested in it.  In that case, it deletes 
it.  This setup ensures that a) when you call register, you can be sure that 
you will get a non-null pool and that pool will not be deleted until after you 
deregister it b) empty pools with no objects under management and no interested 
threads get cleaned up. Thread interest is tracked in the numInterested counter 
attached to the pool itself, so what must be going on is that counter must be 
getting corrupted somehow.  One way that could happen is if deregister is 
called twice for one registration.{{{}{}}}

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Phil Steitz (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741106#comment-17741106
 ] 

Phil Steitz commented on POOL-411:
--

[~sebb] - thanks for looking at this.  You are right - the scenario I posted 
(and deleted) is impossible.  The jdk docs are a little vague, but I have 
confirmed you are right - the thread trying to get the write lock will have to 
block until any readers release their read locks.  The null guard that was 
added in deregister prevents the exception reported above from happening, but 
the test case that Gary added to now fails sporadically (very rarely) with this 
in the stack trace:

{{java.lang.NullPointerException: Cannot invoke 
"org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getIdleObjects()"
 because the return value of "java.util.Map.get(Object)" is null}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addIdleObject(GenericKeyedObjectPool.java:308)}}
{{    at 
org.apache.commons.pool2.impl.GenericKeyedObjectPool.addObject(GenericKeyedObjectPool.java:333)}}
{{    at org.apache.com}}

The problem with above is that addObject calls register on the key before 
invoking addIdleObject so any deregister that happens in between should not be 
able to delete the pool.

The intent is for register to signal that a thread is "interested" in a key so 
others should refrain from deleting it.  It will also create an empty pool if 
there is no pool under the given key. It returns the pool under the registered 
key.  Calling deregister says the thread is no longer interested and also 
checks to see if the associated keyed pool is empty, has no objects under 
management and has no other threads interested in it.  In that case, it deletes 
it.  This setup ensures that a) when you call register, you can be sure that 
you will get a non-null pool and that pool will not be deleted until after you 
deregister it b) empty pools with no objects under management and no interested 
threads get cleaned up. Thread interest is tracked in the numInterested counter 
attached to the pool itself, so what must be going on is that counter must be 
getting corrupted somehow.  One way that could happen is if deregister is 
called twice for one registration.{{{}{}}}

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256028611


##
src/test/java/org/apache/commons/collections4/bloomfilter/TestingHashers.java:
##
@@ -88,4 +90,18 @@ public static  T populateRange(T 
filter, int start, int e
 });
 return filter;
 }
+
+private static Random random;

Review Comment:
switched to ThreadLocalRandom.current()



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256027853


##
src/test/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilterTest.java:
##
@@ -0,0 +1,325 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.ArrayList;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.concurrent.TimeUnit;
+import java.util.function.Consumer;
+import java.util.function.Predicate;
+
+import org.junit.jupiter.api.Test;
+
+public class LayeredBloomFilterTest extends 
AbstractBloomFilterTest {
+
+@Override
+protected LayeredBloomFilter createEmptyFilter(Shape shape) {
+return LayeredBloomFilter.fixed(shape, 10);
+}
+
+protected BloomFilter makeFilter(int... values) {
+return makeFilter(IndexProducer.fromIndexArray(values));
+}
+
+protected BloomFilter makeFilter(IndexProducer p) {
+BloomFilter bf = new SparseBloomFilter(getTestShape());
+bf.merge(p);
+return bf;
+}
+
+protected BloomFilter makeFilter(Hasher h) {
+BloomFilter bf = new SparseBloomFilter(getTestShape());
+bf.merge(h);
+return bf;
+}
+
+@Test
+public void testMultipleFilters() {
+LayeredBloomFilter filter = LayeredBloomFilter.fixed(getTestShape(), 
10);
+filter.merge(TestingHashers.FROM1);
+filter.merge(TestingHashers.FROM11);
+assertEquals(2, filter.getDepth());
+assertTrue(filter.contains(makeFilter(TestingHashers.FROM1)));
+assertTrue(filter.contains(makeFilter(TestingHashers.FROM11)));
+BloomFilter t1 = makeFilter(6, 7, 17, 18, 19);
+assertFalse(filter.contains(t1));
+assertFalse(filter.copy().contains(t1));
+assertTrue(filter.flatten().contains(t1));
+}
+
+private LayeredBloomFilter setupFindTest() {
+LayeredBloomFilter filter = LayeredBloomFilter.fixed(getTestShape(), 
10);
+filter.merge(TestingHashers.FROM1);
+filter.merge(TestingHashers.FROM11);
+filter.merge(new IncrementingHasher(11, 2));
+filter.merge(TestingHashers.populateFromHashersFrom1AndFrom11(new 
SimpleBloomFilter(getTestShape(;
+return filter;
+}
+
+@Test
+public void testFindBloomFilter() {
+LayeredBloomFilter filter = setupFindTest();
+int[] result = filter.find(TestingHashers.FROM1);
+assertEquals(2, result.length);
+assertEquals(0, result[0]);
+assertEquals(3, result[1]);
+result = filter.find(TestingHashers.FROM11);
+assertEquals(2, result.length);
+assertEquals(1, result[0]);
+assertEquals(3, result[1]);
+}
+
+@Test
+public void testFindBitMapProducer() {
+LayeredBloomFilter filter = setupFindTest();
+
+IndexProducer idxProducer = 
TestingHashers.FROM1.indices(getTestShape());
+BitMapProducer producer = 
BitMapProducer.fromIndexProducer(idxProducer, getTestShape().getNumberOfBits());
+
+int[] result = filter.find(producer);
+assertEquals(2, result.length);
+assertEquals(0, result[0]);
+assertEquals(3, result[1]);
+
+idxProducer = TestingHashers.FROM11.indices(getTestShape());
+producer = BitMapProducer.fromIndexProducer(idxProducer, 
getTestShape().getNumberOfBits());
+result = filter.find(producer);
+assertEquals(2, result.length);
+assertEquals(1, result[0]);
+assertEquals(3, result[1]);
+}
+
+@Test
+public void testFindIndexProducer() {
+IndexProducer producer = TestingHashers.FROM1.indices(getTestShape());
+LayeredBloomFilter filter = setupFindTest();
+
+int[] result = filter.find(producer);
+assertEquals(2, result.length);
+assertEquals(0, result[0]);
+assertEquals(3, result[1]);
+
+

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256026172


##
src/test/java/org/apache/commons/collections4/bloomfilter/LayerManagerTest.java:
##
@@ -0,0 +1,225 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import static org.junit.Assert.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.LinkedList;
+import java.util.List;
+import java.util.function.Consumer;
+import java.util.function.Predicate;
+
+import org.junit.jupiter.api.Test;
+
+public class LayerManagerTest {
+
+private Shape shape = Shape.fromKM(17, 72);
+
+private LayerManager.Builder testBuilder() {
+return LayerManager.builder().withSuplier(() -> new 
SimpleBloomFilter(shape));
+}
+
+@Test
+public void testAdvanceOnPopulated() {
+Predicate underTest = 
LayerManager.ExtendCheck.ADVANCE_ON_POPULATED;
+LayerManager layerManager = testBuilder().build();
+assertFalse(underTest.test(layerManager));
+layerManager.target().merge(TestingHashers.FROM1);
+assertTrue(underTest.test(layerManager));
+}
+
+@Test
+public void testNeverAdvance() {
+Predicate underTest = 
LayerManager.ExtendCheck.NEVER_ADVANCE;
+LayerManager layerManager = testBuilder().build();
+assertFalse(underTest.test(layerManager));
+layerManager.target().merge(TestingHashers.FROM1);
+assertFalse(underTest.test(layerManager));
+}
+
+@Test
+public void testAdvanceOnCount() {
+Predicate underTest = 
LayerManager.ExtendCheck.advanceOnCount(4);
+LayerManager layerManager = testBuilder().build();
+for (int i = 0; i < 3; i++) {
+assertFalse(underTest.test(layerManager), "at " + i);
+layerManager.target().merge(TestingHashers.FROM1);
+}
+assertTrue(underTest.test(layerManager));
+}
+
+@Test
+public void testAdvanceOnCalculatedFull() {
+Double maxN = shape.estimateMaxN();
+Predicate underTest = 
LayerManager.ExtendCheck.advanceOnCalculatedFull(shape);
+LayerManager layerManager = testBuilder().build();
+while 
(layerManager.target().getShape().estimateN(layerManager.target().cardinality())
 < maxN) {
+assertFalse(underTest.test(layerManager));
+layerManager.target().merge(TestingHashers.randomHasher());
+}
+assertTrue(underTest.test(layerManager));
+}
+
+@Test
+public void testAdvanceOnSaturationWithDouble() {
+Double maxN = shape.estimateMaxN();
+Predicate underTest = 
LayerManager.ExtendCheck.advanceOnSaturation(maxN);
+LayerManager layerManager = testBuilder().build();
+while 
(layerManager.target().getShape().estimateN(layerManager.target().cardinality())
 < maxN) {
+assertFalse(underTest.test(layerManager));
+layerManager.target().merge(TestingHashers.randomHasher());
+}
+assertTrue(underTest.test(layerManager));
+}
+
+@Test
+public void testAdvanceOnSaturationWithInt() {

Review Comment:
   Removed int version



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256022467


##
src/main/java/org/apache/commons/collections4/bloomfilter/WrappedBloomFilter.java:
##
@@ -0,0 +1,143 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+
+/**
+ * An abstract class to assist in implementing Bloom filter decorators.
+ *
+ * @since 4.5
+ */
+public abstract class WrappedBloomFilter implements BloomFilter {
+final BloomFilter wrapped;
+
+public WrappedBloomFilter(BloomFilter bf) {

Review Comment:
   done



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1256018727


##
src/main/java/org/apache/commons/collections4/bloomfilter/Shape.java:
##
@@ -227,6 +227,15 @@ public double estimateN(final int cardinality) {
 return -(m / k) * Math.log1p(-c / m);
 }
 
+/**
+ * Estimates the maximum number of elements that can be merged into a 
filter of this shape before

Review Comment:
   text has been changed to read:
   
   Estimates the maximum number of elements that can be merged into a filter of
* this shape before the false positive rate exceeds the desired rate. 
 The
* formula for deriving {@code k} when {@code m} and {@code n} are known 
is:
*
* 
* k = ln2 * m / n
* 
*
* solving for {@code n} yields:
*
* 
* n = ln2 * m / k
* 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] (POOL-411) NPE when deregistering key at end of borrow

[Phil Steitz (Jira)]

[ https://issues.apache.org/jira/browse/POOL-411 ]


Phil Steitz deleted comment on POOL-411:
--

was (Author: psteitz):
The added test case has failed at least once now.  I think there is a problem 
with register/deregister protection of pools.  Very low probability, but the 
following seems like it could happen:

Thread 1 
 # Enter register
 # get the read lock
 # get a non-null, existing pool

Thread 2
 # Clear the pool
 # Deregister and remove it

Thread 1
 # Register interest in the now orphaned pool

This may be possible because deregister can escalate to write lock while others 
are holding read locks.  The Thread 2 actions have to happen between two 
instructions in register (when it gets the pool and when it increments its 
numInterested).  

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (JEXL-399) Needs support more javascript syntax

[Henri Biestro (Jira)]

[ 
https://issues.apache.org/jira/browse/JEXL-399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17741004#comment-17741004
 ] 

Henri Biestro commented on JEXL-399:


JEXL is not JavaScript and does not try to be as feature-rich. You will have to 
teach chatgpt some of its syntax. Alternatively, have you tried using/embedding 
an actual JavaScript interpreter ? (https://github.com/openjdk/nashorn-)

That being said:
6. Fat-arrow for lambda is configurable through JexlFeatures.
1. and 2. could be added as strictEqual/notStrictEqual operators.
3. exponentiation is accessible through Math.pow in Java (true for Javascript 
too).
4. destructuring assignment is a very rich feature and complex endeavor.
5. map literals using identifiers as keys without interpretation could be 
achieved through options.
7. and 8. can be expressed as for(let k : m.keySet()) and for(let v : 
m.valueSet()) when m is a map.

So 6. is here, 1.2 are easy and may be part of next minor version.

The following are unlikely to be implemented in the short term:
5. is a tad more involved in its simplest form (maps) for little gain.
3., 7., 8. are of a low value for JEXL users.
4. is a lot of work for a condensed syntax.


 

> Needs support more javascript syntax
> 
>
> Key: JEXL-399
> URL: https://issues.apache.org/jira/browse/JEXL-399
> Project: Commons JEXL
>  Issue Type: New Feature
>Reporter: Xu Pengcheng
>Priority: Major
>
> Hi, I am trying ask chatgpt to generate code for some cases, JEXL, as a 
> lightweight, secure, easy to communicate with host solution, I think is the 
> best option for java world as a bridge between chatgpt and app.
> Unfortunately, chatgpt seems is not familiar with Jexl syntax, so I tried to 
> ask chatgpt to write as javascript, I had implements most of the javascript 
> build-in objects (like Array, Object, Math, console, etc) for JEXL, 95% cases 
> generated by chatgpt can run successfully.
> However, some syntax error will cause the 5% failure, for example:
>  # a === b
>  # a !== b
>  # a = b ** 2
>  # const \{a, b} = x;
>  # a = \{x: 1, y: 2,}
>  # func = () => (\{"a":1, "b": 2})
>  # for (let k in map)
>  # for (let value of map)
> These syntax looks very reasonable, It will be great to support these 
> javascript syntax to let both chatgpt and developer easy to use.
> Thanks!
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-csv] garydgregory merged pull request #333: Bump h2 from 2.1.214 to 2.2.220

[via GitHub]

garydgregory merged PR #333:
URL: https://github.com/apache/commons-csv/pull/333


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-parent] garydgregory merged pull request #287: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #287:
URL: https://github.com/apache/commons-parent/pull/287


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-dbcp] garydgregory commented on pull request #281: Bump mockito-core from 4.11.0 to 5.4.0

[via GitHub]

garydgregory commented on PR #281:
URL: https://github.com/apache/commons-dbcp/pull/281#issuecomment-1625179769

   Closing, requires Java 11.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-collections] Claudenw commented on pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#issuecomment-1625178843

   I have made a mess of the formatting.  I am working at cleaning it up now.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [commons-dbcp] garydgregory merged pull request #287: Bump h2 from 2.1.214 to 2.2.220

[via GitHub]

garydgregory merged PR #287:
URL: https://github.com/apache/commons-dbcp/pull/287


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Sebb (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740943#comment-17740943
 ] 

Sebb commented on POOL-411:
---

To simplify testing, it might be worth temporarily adding a sleep where you 
think the window occurs.

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1255448741


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilter.java:
##
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+import java.util.function.Predicate;
+
+/**
+ * Layered Bloom filters are described in Zhiwang, Cen; Jungang, Xu; Jian, Sun
+ * (2010), "A multi-layer Bloom filter for duplicated URL detection", Proc. 3rd
+ * International Conference on Advanced Computer Theory and Engineering (ICACTE
+ * 2010), vol. 1, pp. V1–586–V1–591, doi:10.1109/ICACTE.2010.5578947, ISBN
+ * 978-1-4244-6539-2, S2CID 3108985
+ * 
+ * In short, Layered Bloom filter contains several bloom filters arranged in
+ * layers.
+ * 
+ * 
+ * When membership in the filter is checked each layer in turn is checked
+ * and if a match is found {@code true} is returned.
+ * When merging each bloom filter is merged into the last newest filter in
+ * the list of layers.
+ * When questions of cardinality are asked the cardinality of the union of
+ * the enclosed Bloom filters is used.
+ * 
+ * The net result is that the layered Bloom filter can be populated with more
+ * items than the Shape would indicate and yet still return a false positive
+ * rate in line with the Shape and not the over population.
+ * 
+ * This implementation uses a LayerManager to handle the manipulation of the
+ * layers.
+ * 
+ * 
+ * Level 0 is the oldest layer and the highest level is the newest.
+ * There is always at least one enclosed filter.
+ * The newest filter is the {@code target} into which merges are performed.
+ * Whenever the target is retrieved, or a {@code merge} operation is
+ * performed the code checks if any older layers should be removed, and if so
+ * removes them. It also checks it a new layer should be added, and if so adds
+ * it and sets the {@code target} before the operation.
+ * 
+ */
+public class LayeredBloomFilter implements BloomFilter, BloomFilterProducer {
+private final Shape shape;
+private LayerManager layerManager;
+
+/**
+ * Creates a fixed size layered bloom filter that adds new filters to the 
list,
+ * but never merges them. List will never exceed maxDepth. As additional 
filters
+ * are added earlier filters are removed.
+ *
+ * @param shapeThe shape for the enclosed Bloom filters
+ * @param maxDepth The maximum depth of layers.
+ * @return An empty layered Bloom filter of the specified shape and depth.
+ */
+public static LayeredBloomFilter fixed(final Shape shape, int maxDepth) {
+LayerManager manager = 
LayerManager.builder().withExtendCheck(LayerManager.ExtendCheck.ADVANCE_ON_POPULATED)
+
.withCleanup(LayerManager.Cleanup.onMaxSize(maxDepth)).withSuplier(() -> new 
SimpleBloomFilter(shape))
+.build();
+return new LayeredBloomFilter(shape, manager);
+}
+
+/**
+ * Constructor.
+ *
+ * @param shapethe Shape of the enclosed Bloom filters
+ * @param layerManager the LayerManager to manage the layers.
+ */
+public LayeredBloomFilter(Shape shape, LayerManager layerManager) {
+this.shape = shape;
+this.layerManager = layerManager;
+}
+
+@Override
+public LayeredBloomFilter copy() {
+return new LayeredBloomFilter(shape, layerManager.copy());
+}
+
+/**
+ * Gets the depth of the deepest layer.
+ *
+ * @return the depth of the deepest layer.
+ */
+public final int getDepth() {
+return layerManager.getDepth();
+}
+
+/**
+ * Gets the Bloom filter at the specified depth
+ *
+ * @param depth the depth of the filter to return.
+ * @return the Bloom filter at the specified depth.
+ * @throws NoSuchElementException if depth is not in the range 
[0,getDepth())
+ */
+public BloomFilter get(int depth) {
+return 

[jira] [Commented] (POOL-411) NPE when deregistering key at end of borrow

[Sebb (Jira)]

[ 
https://issues.apache.org/jira/browse/POOL-411?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17740941#comment-17740941
 ] 

Sebb commented on POOL-411:
---

Surely it should not be possible to obtain a write lock whilst others are 
holding read locks?

> NPE when deregistering key at end of borrow
> ---
>
> Key: POOL-411
> URL: https://issues.apache.org/jira/browse/POOL-411
> Project: Commons Pool
>  Issue Type: Task
>Affects Versions: 2.11.1
>Reporter: Richard Eckart de Castilho
>Priority: Major
> Fix For: 2.12.0
>
>
> There is a potential for an NPE happening in the finally block of 
> borrowObject:
> {noformat}
> Caused by: java.lang.NullPointerException: Cannot invoke 
> "org.apache.commons.pool2.impl.GenericKeyedObjectPool$ObjectDeque.getNumInterested()"
>  because "objectDeque" is null
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.deregister(GenericKeyedObjectPool.java:821)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:507)
>   at 
> org.apache.commons.pool2.impl.GenericKeyedObjectPool.borrowObject(GenericKeyedObjectPool.java:350)
>  
> {noformat}
> From reading the code, it seems this could happen e.g. if a pool is 
> concurrently cleared while a borrow is in progress.
> Not sure what a proper solution here would be. Maybe deregister should 
> silently do nothing if poolMap.get(k) returns null? 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[GitHub] [commons-collections] Claudenw commented on a diff in pull request #402: COLLECTIONS-843: Implement Layered Bloom filter

[via GitHub]

Claudenw commented on code in PR #402:
URL: 
https://github.com/apache/commons-collections/pull/402#discussion_r1255444851


##
src/main/java/org/apache/commons/collections4/bloomfilter/LayeredBloomFilter.java:
##
@@ -0,0 +1,416 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.commons.collections4.bloomfilter;
+
+import java.util.Arrays;
+import java.util.NoSuchElementException;
+import java.util.Objects;
+import java.util.function.IntPredicate;
+import java.util.function.LongPredicate;
+import java.util.function.Predicate;
+
+/**
+ * Layered Bloom filters are described in Zhiwang, Cen; Jungang, Xu; Jian, Sun
+ * (2010), "A multi-layer Bloom filter for duplicated URL detection", Proc. 3rd
+ * International Conference on Advanced Computer Theory and Engineering (ICACTE
+ * 2010), vol. 1, pp. V1–586–V1–591, doi:10.1109/ICACTE.2010.5578947, ISBN
+ * 978-1-4244-6539-2, S2CID 3108985
+ * 
+ * In short, Layered Bloom filter contains several bloom filters arranged in
+ * layers.
+ * 
+ * 
+ * When membership in the filter is checked each layer in turn is checked
+ * and if a match is found {@code true} is returned.
+ * When merging each bloom filter is merged into the last newest filter in
+ * the list of layers.
+ * When questions of cardinality are asked the cardinality of the union of
+ * the enclosed Bloom filters is used.
+ * 
+ * The net result is that the layered Bloom filter can be populated with more
+ * items than the Shape would indicate and yet still return a false positive
+ * rate in line with the Shape and not the over population.
+ * 
+ * This implementation uses a LayerManager to handle the manipulation of the
+ * layers.
+ * 
+ * 
+ * Level 0 is the oldest layer and the highest level is the newest.
+ * There is always at least one enclosed filter.
+ * The newest filter is the {@code target} into which merges are performed.
+ * Whenever the target is retrieved, or a {@code merge} operation is
+ * performed the code checks if any older layers should be removed, and if so
+ * removes them. It also checks it a new layer should be added, and if so adds
+ * it and sets the {@code target} before the operation.
+ * 
+ */
+public class LayeredBloomFilter implements BloomFilter, BloomFilterProducer {
+private final Shape shape;
+private LayerManager layerManager;
+
+/**
+ * Creates a fixed size layered bloom filter that adds new filters to the 
list,
+ * but never merges them. List will never exceed maxDepth. As additional 
filters
+ * are added earlier filters are removed.
+ *
+ * @param shapeThe shape for the enclosed Bloom filters
+ * @param maxDepth The maximum depth of layers.
+ * @return An empty layered Bloom filter of the specified shape and depth.
+ */
+public static LayeredBloomFilter fixed(final Shape shape, int maxDepth) {
+LayerManager manager = 
LayerManager.builder().withExtendCheck(LayerManager.ExtendCheck.ADVANCE_ON_POPULATED)
+
.withCleanup(LayerManager.Cleanup.onMaxSize(maxDepth)).withSuplier(() -> new 
SimpleBloomFilter(shape))
+.build();
+return new LayeredBloomFilter(shape, manager);
+}
+
+/**
+ * Constructor.
+ *
+ * @param shapethe Shape of the enclosed Bloom filters
+ * @param layerManager the LayerManager to manage the layers.
+ */
+public LayeredBloomFilter(Shape shape, LayerManager layerManager) {
+this.shape = shape;
+this.layerManager = layerManager;
+}
+
+@Override
+public LayeredBloomFilter copy() {
+return new LayeredBloomFilter(shape, layerManager.copy());
+}
+
+/**
+ * Gets the depth of the deepest layer.
+ *
+ * @return the depth of the deepest layer.
+ */
+public final int getDepth() {
+return layerManager.getDepth();
+}
+
+/**
+ * Gets the Bloom filter at the specified depth
+ *
+ * @param depth the depth of the filter to return.
+ * @return the Bloom filter at the specified depth.
+ * @throws NoSuchElementException if depth is not in the range 
[0,getDepth())
+ */
+public BloomFilter get(int depth) {
+return 

[GitHub] [commons-compress] garydgregory merged pull request #399: Bump github/codeql-action from 2.20.1 to 2.20.3

[via GitHub]

garydgregory merged PR #399:
URL: https://github.com/apache/commons-compress/pull/399


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@commons.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org