kishoreg commented on a change in pull request #3528: Adding support for bloom
filter
URL: https://github.com/apache/incubator-pinot/pull/3528#discussion_r236502718
##########
File path:
pinot-common/src/main/java/com/linkedin/pinot/common/config/IndexingConfig.java
##########
@@ -113,6 +116,15 @@ public void setSortedColumn(List<String> sortedColumn) {
_sortedColumn = sortedColumn;
}
+
+ public List<String> getBloomFilterColumns() {
Review comment:
I agree with the concern that this is another config. Unfortunately
creating bloom filter on all dimensions will add to memory usage (at least in
the given implementation where the bitset is on heap) and additional overhead.
The right solution would be to have a component that looks at the workload and
tunes it automatically.
Currently, bloom filter is created when the segment is getting loaded. We
can add the optimization to create it during segment generation in another PR.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
