Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-21 Thread Claude Warren
After looking at the LayeredBloomFilter and the LayerManager and the way it is intended to be used I found a couple of changes that we might want to consider. 1) WrappedBloomFilter should not implement copy(), that should be left to the wrapping implementation. 2) We change LayerManager to be a

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-20 Thread Alex Herbert
On Sat, 20 Apr 2024 at 11:36, Claude Warren wrote: > The LayerdBloomFilter has a method find() that returns an array of ints > that are the indices into the layer array. This is easily reproducible > using an iterator. > There is also get() method that takes an integer argument (an index of

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-20 Thread Claude Warren
The LayerdBloomFilter has a method find() that returns an array of ints that are the indices into the layer array. This is easily reproducible using an iterator. There is also get() method that takes an integer argument (an index of the bloom filter) and returns the Bloom filter from the layer.

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-19 Thread Alex Herbert
On Fri, 19 Apr 2024 at 08:26, Claude Warren wrote: > While the Deque makes clear the idea of enqueueing and dequeueing the > layers it does not have the method to natively traverse and extract entries > from the middle of the queue. Nor would I expect it to. So I think the > Deque does not

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-19 Thread Claude Warren
While the Deque makes clear the idea of enqueueing and dequeueing the layers it does not have the method to natively traverse and extract entries from the middle of the queue. Nor would I expect it to. So I think the Deque does not accurately reflect how the collection of Bloom filters is

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-17 Thread Alex Herbert
Looks good to me. Any opinions on changing the LayerManager to keep the layers in a Deque rather than a LinkedList. I think it would only require a change to the following method: public final BloomFilter get(int depth) Performance will be the same as the Deque can be a LinkedList. This is more

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-17 Thread Claude Warren
I have an open pull request to fix this problem. I could use another review: https://github.com/apache/commons-collections/pull/476 On Tue, Apr 9, 2024 at 11:29 AM Claude Warren wrote: > Alex, > > I like your solution. To answer your question. We create a BloomFilter > that has a timestamp

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-09 Thread Claude Warren
Alex, I like your solution. To answer your question. We create a BloomFilter that has a timestamp associated with it. When the timestamp is greater than System.currentTimeMillis() the filter is removed. The custom cleanup calls Cleanup.removeEmptyTarget().andThen() I think that creating a

Re: [Collections-BloomFilter][Discuss] missing functionality?

2024-04-09 Thread Alex Herbert
Hi Claude, Q. What is your current clean-up filter, i.e. the Consumer>? I assume you are using a custom one. The current collections code only has 2 functional implementations. One will remove the newest filter if it is empty, one will remove the oldest filters until the size is below a limit.

[Collections-BloomFilter][Discuss] missing functionality?

2024-04-09 Thread Claude Warren
Greetings, I am attempting to use the Bloomfilter code in Kafka to manage PID generation. The requirement is to remove pid tracking after a period of time. This is possible with the LayeredBloomFilter but it has an edge case problem. The LayeredBloomFilter uses the LayerManager to manage the