ankitsultana commented on code in PR #15350:
URL: https://github.com/apache/pinot/pull/15350#discussion_r2289402409
##########
pinot-core/src/main/java/org/apache/pinot/core/query/pruner/SegmentPrunerService.java:
##########
@@ -136,28 +136,17 @@ public List<IndexSegment> prune(List<IndexSegment>
segments, QueryContext query,
* undefined way. Therefore, this list should not be used
after calling this method.
* @return the new list with filtered elements. This is the list that have
to be used.
*/
- private static List<IndexSegment> removeInvalidSegments(List<IndexSegment>
segments, QueryContext query,
- SegmentPrunerStatistics stats) {
+ private static List<IndexSegment> removeEmptySegments(List<IndexSegment>
segments) {
int selected = 0;
- int invalid = 0;
for (IndexSegment segment : segments) {
if (!isEmptySegment(segment)) {
- if (isInvalidSegment(segment, query)) {
- invalid++;
- } else {
- segments.set(selected++, segment);
- }
+ segments.set(selected++, segment);
}
}
- stats.setInvalidSegments(invalid);
return segments.subList(0, selected);
}
private static boolean isEmptySegment(IndexSegment segment) {
return segment.getSegmentMetadata().getTotalDocs() == 0;
}
-
- private static boolean isInvalidSegment(IndexSegment segment, QueryContext
query) {
Review Comment:
@praveenc7 : we recently found out that `isInvalidSegment` was becoming a
bottleneck for some of our high qps use-cases. I see that after this patch we
don't call it anywhere. I suppose that is because Pinot can now gracefully
handle mismatching schema across segments?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
