siddharthteotia commented on a change in pull request #5013: For RANGE
predicate queries touching offline segments, use sorted inverted index if the
column is sorted
URL: https://github.com/apache/incubator-pinot/pull/5013#discussion_r370883057
##########
File path:
pinot-core/src/main/java/org/apache/pinot/core/operator/filter/FilterOperatorUtils.java
##########
@@ -56,16 +57,23 @@ public static BaseFilterOperator
getLeafFilterOperator(PredicateEvaluator predic
// Use inverted index if the predicate type is not RANGE or REGEXP_LIKE
for efficiency
DataSourceMetadata dataSourceMetadata = dataSource.getDataSourceMetadata();
Predicate.Type predicateType = predicateEvaluator.getPredicateType();
- if (dataSourceMetadata.hasInvertedIndex() && (predicateType !=
Predicate.Type.RANGE) && (predicateType
- != Predicate.Type.REGEXP_LIKE)) {
- if (dataSourceMetadata.isSorted()) {
+ if (dataSourceMetadata.hasInvertedIndex() && (predicateType !=
Predicate.Type.REGEXP_LIKE)) {
+ if (shouldUseSortedInvertedIndexOperator(dataSource, predicateType)) {
return new SortedInvertedIndexBasedFilterOperator(predicateEvaluator,
dataSource, startDocId, endDocId);
- } else {
+ } else if (predicateType != Predicate.Type.RANGE) {
+ // TODO: add support for bitmap inverted index operator can be used
for RANGE predicate
return new BitmapBasedFilterOperator(predicateEvaluator, dataSource,
startDocId, endDocId);
}
- } else {
- return new ScanBasedFilterOperator(predicateEvaluator, dataSource,
startDocId, endDocId);
}
+ return new ScanBasedFilterOperator(predicateEvaluator, dataSource,
startDocId, endDocId);
+ }
+
+ private static boolean shouldUseSortedInvertedIndexOperator(DataSource
dataSource, Predicate.Type predicateType) {
+ boolean isSorted = dataSource.getDataSourceMetadata().isSorted();
+ // we can sorted inverted index if:
+ // 1. column is sorted AND
+ // 2. predicate is not RANGE OR predicate is RANGE but this physical plan
is being built for offline segment
Review comment:
If the column is sorted, we can use sorted index if there is no RANGE
predicate -- we do this today.
If the column is sorted and there is RANGE predicate, use sorted index only
for offline segments.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
