n3nash commented on a change in pull request #666: Add support for dynamic
bloom filter to increase efficiency of bloom filter for static sizing
URL: https://github.com/apache/incubator-hudi/pull/666#discussion_r281452198
##########
File path: hoodie-common/src/main/java/com/uber/hoodie/common/BloomFilter.java
##########
@@ -37,33 +38,49 @@
*/
public static final double LOG2_SQUARED = Math.log(2) * Math.log(2);
- private org.apache.hadoop.util.bloom.BloomFilter filter = null;
+ private org.apache.hadoop.util.bloom.BloomFilter bloomFilter = null;
- public BloomFilter(int numEntries, double errorRate) {
- this(numEntries, errorRate, Hash.MURMUR_HASH);
+ private org.apache.hadoop.util.bloom.DynamicBloomFilter dynamicBloomFilter =
null;
Review comment:
Hbase ->
https://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/util/BloomFilter.html
Looked through Hbase code and they have implemented their own BloomFilters
based on the algorithms in the above class.
Cassandra ->
https://docs.datastax.com/en/cassandra/3.0/cassandra/operations/opsTuningBloomFilters.html
They don't support dynamic bloom filters, instead force rewriting the bloom
filter.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
