vinothchandar commented on a change in pull request #666: Add support for
dynamic bloom filter to increase efficiency of bloom filter for static sizing
URL: https://github.com/apache/incubator-hudi/pull/666#discussion_r281253775
##########
File path:
hoodie-common/src/main/java/com/uber/hoodie/common/util/ParquetUtils.java
##########
@@ -146,10 +146,10 @@ public static Schema readAvroSchema(Configuration
configuration, Path parquetFil
* Read out the bloom filter from the parquet file meta data.
*/
public static BloomFilter readBloomFilterFromParquetMetadata(Configuration
configuration,
- Path parquetFilePath) {
+ Path parquetFilePath, boolean enableDynamicBloomFilter) {
String footerVal = readParquetFooter(configuration, parquetFilePath,
HoodieAvroWriteSupport.HOODIE_AVRO_BLOOM_FILTER_METADATA_KEY).get(0);
- return new BloomFilter(footerVal);
+ return new BloomFilter(footerVal, enableDynamicBloomFilter);
Review comment:
I am assuming this is needed coz only then we 'd know the type of bloom
filter and deserialize properly?Can BloomFilter read a DynamicBloomFilter
serialized value? what happens when we have a mix of files written using normal
and dynamic filterS? should we resolve this using an additional footer, instead
of making this a writer side config
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
